Chris Hess: Hi there. Thanks for tuning into this video presentation about archive case study titled "Informing Housing Policy through Web Automation: Lessons for Designing Programming Tools for Domain Experts".
Chris Hess: The context for this case study is that housing has reached historic levels of unaffordability across the United States over the past couple of decades.
Chris Hess: Moreover, they are important inequalities by race, ethnicity and socioeconomic status (SES) in terms of how burdensome of these rising housing costs have become for different families.
Chris Hess: However, to be able to adequately study housing problems like an affordability social scientists need timely and spatially granular data about the conditions that households face when they're searching the market for their next dwelling.
Chris Hess: While existing public and industry data come with important trade offs in terms of their timeliness, geographic specificity and methodological transparency,
Chris Hess: data about housing opportunities in a given area are nonetheless publicly available 24/7 on websites like Craigslist and Zillow.
Chris Hess: The chief constraint is that social scientists do not typically have training and web languages like html and javascript that can be essential to creating web automation scripts by oneself.
Chris Hess: Instead of this being a persistent constraint,
Chris Hess: This training gap presents an opportunity for folks in the computer science community to work with social scientists to create tools that meet the needs of social science users who have domain expertise, but may lack particular technical skills for a task like web automation.
Chris Hess: With that context in mind, our case study is a multi year collaboration between researchers in the social sciences and computer science based on using a novel web automation tool called Helena to collect housing advertisement data.
Chris Hess: Using Helena has enabled a team of social scientists from a variety of disciplinary perspectives to work together on computational social science research focusing on housing inequalities and housing policy in the United States.
Chris Hess: Throughout the rest of this video presentation, I'll introduce the Web automation tool, I'll discuss the design guidelines we contribute to our case study and I'll briefly summarize the three phases of this project's growth over time.
Chris Hess: I'll briefly introduced the Web automation tool Helena that is at the center of this social science -computer science collaboration.
Chris Hess: Helena is a programming-by-demonstration tool for writing web automation scripts that users interact with via a Chrome browser extension.
Chris Hess: And in this screenshot on the right, you can see what this looks like in practice with the Helena chrome extension on the left hand side triggered to collect data from Seattle's Craigslist page for apartment listings on the right hand side.
Chris Hess: And within the extension window, Helena shows the fields of the user has selected as the first row of the data set that they'd like to collect in the future.
Chris Hess: Importantly, Helena allows users to demonstrate scripts for data that is distributed across a number of web pages as is commonly the case with listing data on sites like Craigslist.
Chris Hess: So carrying the example from the prior slide forward, on the right we can see that the user has demonstrated that additional fields for each individual ad be collected as part of the craigslist scraper being programmed.
Chris Hess: Finally, once a demonstration is complete, Helena synthesizes the demonstration into a script that can be edited within the same browser extension that users demonstrate their data of interest with.
Chris Hess: This is important because it gives users the capacity to tweak a script that may be close but not quite right. The script editor also allows users to program how the scraper handles navigation through multiple pages, or how it handles the scraping of duplicate data, among other things.
Chris Hess: Since our design guidelines are a key contribution of our case study, I'll now briefly review them before sharing some of the project experiences relevant to how we arrived at them.
Chris Hess: The first of our design guidelines is to design for easy first drafts and, specifically, this means prioritizing easy prototyping and enabling users to create a functional proof of concept quickly.
Chris Hess: To the extent possible, this also means letting users draft working programs first, even if they don't understand the technical complexity and even if they will eventually benefit from understanding this technical complexity.
Chris Hess: The second guideline is to design for scaling up computation. In other words first drafts should be able to be extended or "productionized" into large scale or long running projects should use our goals eventually require this.
Chris Hess: The third guideline is to design for scaling down human effort as project complexity grows.
Chris Hess: And this describes how human interaction burdens that are bearable during the prototyping stage may become overwhelming as a project grows in its scale.
Chris Hess: As a result, it is useful to design applications such that small amounts of additional programming effort can be substituted for large amounts of other human effort.
Chris Hess: The fourth guideline is to design for integration with other tools, this means to prioritize the ability to import, export and transition inputs between the tools that are already prominent in domain experts technical skill set.
Chris Hess: Finally, our fifth guideline is to design for single users who have mixed expertise levels.
Chris Hess: In practice, this means to make manual and programmatic manipulation available so users can switch freely between them at any point in a project lifecycle.
Chris Hess: This is important because, even as users develop technical skills they continue to face brush challenges that sometimes they may be able to tackle most efficiently the other interfaces designed for non technical users.
Chris Hess: I'll now give some examples from our paper about how we arrived at these five design guidelines.
Chris Hess: During the pilot stage of our project, the fact that we were able to get started with scraping based on just demonstrating the data of interest was integral to freeing up time for us to think about how we would process and analyze the data for our report.
Chris Hess: This experience informed our guideline about facilitating easy first drafts because project success at this stage hinged on being able to prepare a sample of data and analyses within a relatively short period of time.
Chris Hess: Further, the fact that hello to export a csv files meant that there was essentially no friction for being able to incorporate it within our existing workflows.
Chris Hess: The second point informed our design guideline that about prioritizing integration with other tools, since many social science users have existing technical skills and data analysis software, like our that can be complemented with tools like Helena.
Chris Hess: After our proof of concept stage we sought to expand our spring to cover all craigslist pages for the largest metropolitan areas in the United States.
Chris Hess: Crucial to our success at this stage was that we could deploy the Helena scripts demonstrated on our laptops onto virtual machines where headless Docker containers ran Helena scrapers.
Chris Hess: This experience is part of what informed our guideline related to scaling up computation, which describes tools being designed such that the first drafts can be extended to handle larger workloads to the accept that this becomes a project priority.
Chris Hess: The third and final stage of our project has expanded our data collection based on Helena to cover multiple housing platforms at a national scale.
Chris Hess: Updates to the Helena tool that were informed by our project have meant that we can now demonstrate a single Helena script for a source,
Chris Hess: parameterize the starting URL, and then deploy the scrapers for all locations of interest.
Chris Hess: This motivates our guideline related to scaling down human effort, since a small amount of additional programming capacity yielded a large return in terms of the time required to create scrapers for a given source.
Chris Hess: And while we were building out our expanded system, we did notice an unexpected case for Zillow containers never reached a completed status.
Chris Hess: And running the relevant scraper program outside of our virtual machines in the Helena GUI let us watch what was causing our programs to stall, at which point, it became clear that ads for content, other than vacancies were failing to load.
Chris Hess: This informed our guideline about designing for users with mixed expertise levels, while also providing another example of how Helena's easy integration with other tools help us overcome new obstacles encountered while growing our project.
Chris Hess: Overall, our case study illustrates how new computing tools can be an integral part of collaboration between folks in computer science and different social science disciplines.
Chris Hess: For social scientists collaboration is like the President project or an invaluable source of new opportunities to subtle studies social phenomenon like this housing search process.
Chris Hess: And for the CS Community this case study highlights design guidelines that enable applications, ranging from one off uses to ongoing projects with multiple stakeholders.
Chris Hess: And with that I just want to thank you for watching our video presentation, I encourage you to check out the full paper if you're interested in learning more about what we've done.