Hi I'm Uzma Haque Syeda and today I'll be talking to you about our paper Design Study "Lite" Methodology: Expediting design
Studies and enabling the synergy of visualization pedagogy and social good. I hope you find this useful. I
would like to start by telling you about the Chester Square neighborhood association in Boston, Massachusetts.
The historic Chester Square neighborhood in Boston is next to my institution, Northeastern University,
in the south end near Roxbury. The Neighborhood Association works to bring together everyone who lives in the community,
share information and promote the implementation of programs beneficial to the neighborhood including safety and security. The
President of this association is Carol Blair and she along with the members of the board need to tackle important issues such as
new developments in the community
transportation and pollution.
Having these
responsibilities also mean pouring over dozens of documents and gathering data to present to the city of Boston in support of the neighborhood.
However, the association lacks volunteers, expertise, and finances to fully analyze the necessary data.
Therefore they cannot present their data to the city with effective visualizations.
This scenario is not unique to the Chester Square Neighborhood Association.
Most nonprofit and community
organizations such as neighborhood associations, schools, and food banks lack these resources.
They commonly have essential operational data in the forms of spreadsheets, text documents, and maps for which important data analysis
need to be conducted which they cannot afford.
This lack of data visualization
expertise can negatively affect their goals such as increasing and broadening participation,
communicating their effort to funders, and increasing their community impact.
Now let me introduce you to Michelle Borkin, a professor who teaches data visualization to undergraduate and graduate students.
She wants to teach them about design studies. Design studies are a form of problem-driven research in which
Visualization researchers investigate and solve a real-world problem faced by domain experts through visualization of their data.
Design studies are a canonical type of research in the visualization community with hundreds of instances in the literature.
Models and theories such as the Nested Model by Munzner and the design study process by Sedmair
et al. are well established for conducting design studies. However,
Michelle has a problem...
Design studies typically take several months to years to complete. For this reason, most
visualization educators do not run full design studies as part of their curriculum.
This exclusion of design studies from visualization pedagogy creates a gap between budding visualization students and researchers.
So how could Michelle have her students experience the full design study process in just a single semester?
Let me also introduce you to Dan, an undergraduate student majoring in Data Science in Michelle's visualization course.
Dan has learned the essential components of design studies in his visualization course.
However, Dan does not know how to implement these skills in the real world to help real stakeholders like Carol.
Even though students like Dan have the ability to help out organizations like Chester Square Neighborhood Association,
their curriculum does not empower them to do so and therefore the community goals of these organizations remain unmet.
To solve these problems and to teach and execute design studies that fit the duration of a typical semester,
we present the Design Study "Lite" Methodology
(DSLM), a novel framework for implementing design studies with novice students in just 14 weeks.
Professor Michelle Borkin applied DSLM to her courses across five semesters starting from Spring 2017.
In our paper,
we present this curriculum in which students like Dan not only learn visualization design studies in 14 weeks
but can also make a positive impact on their community by helping stakeholders like Carol with their organization's needs.
This integration happens with Service-Learning
Service-Learning is a credited pedagogical model in which classroom learning objectives are aligned with community service to meet both
educational and community goals. Although DSLM is a design study framework and can be applied on its own,
implementing it with conjunction to Service-Learning provides additional
benefits to the process by helping not only the students but also the community.
Using DSLM, Dan and his classmates were able to successfully conduct an entire design study from start to finish with the Chester Square Neighborhood
Association.
The gathered data, analyzed them, and finally presented visualizations about the issues faced by the Neighborhood Association.
During the process, Dan was also able to learn valuable skills through Service-Learning.
He learnt skills which are unreproducible in class such as better community engagement and a better sense of civic responsibility
towards his community
It also helped him develop better interviewing and brainstorming skills
and he learned how to work with real-world messy data.
Now I would like to give you a brief walk-through of the 7 steps in DSLM in the context of our implementation
of it and Service-Learning.
The first step in DSLM is to choose a collaborator.
One of the biggest difference between the design study process by Sedlmair et al. and
DSLM is that DSLM truncates the precondition stage of the former by shifting it prior to the beginning of the framework.
In Sedlmair et al.’s framework, the precondition stage takes a couple of months or more in order for researchers to choose a
collaborator through thorough winnowing.
However, In DSLM, a community partner is chosen by the instructor of the course through their own connections or through the support of a Service-Learning
facility within the institution.
We advocate for using a Service-Learning facility as it simplifies the process of identifying the partners.
For instance, the Service-Learning department at Northeastern University
has partnerships with over hundreds of nonprofit organizations.
They identify and recommend suitable projects to instructors after a three-step filtering process.
Instructors then choose a subset of collaborators from this list for students to work on during the course of the semester.
This process of choosing a partner starts at least a couple of months before the start of each semester.
In our implementation of DSLM with Service-Learning, over the course of the 5 semesters the students worked 7 different partner
organizations, many of whom were recurring partners across different semesters.
In stage 2, that is the abstract stage, the domain problem is identified through
interviewing the stakeholders and then the problems are translated into tasks through tasks abstraction.
In stage 3,
which is the design stage, and is a very iterative one, where the students clean and mine their data,
draw design sketches for their visualizations and have 2 feedback sessions to iterate on their designs and finalize one.
In stage 4, which is the build stage, the students create their final web-based interactive visualizations.
Let's look at some of the demos of the final visualizations that the students created for their partner organizations.
These visualizations created by the students helped the partner organizations
to better understand their data and enabled them to present and communicate their data more
effectively to others. It also helped students to implement their classroom learning skills in the real world
Coming back to the DSLM timeline.
The next stage is stage 5, where a full-scale evaluation is not feasible to complete within the time-frame of the semester.
So a qualitative usability study is performed in the class by each team to improve their visualizations
In
stage 6, the students present their final project to the entire class, share the websites with their stakeholders and solicit feedback.
in
stage 7 the students submit their design study project report, which is written in a conference style paper format. And
there you have it! design study in just 14 weeks!
Now, how did DSLM perform over the 5 semesters? In each semester,
students and partners were asked open response questions on their learning experience with DSLM through an online survey.
We then open coded the student and partner responses to identify the high level themes in their responses using thematic
analysis techniques.
Let's see what they had to say...
Four high-level themes were identified from the students' comments.
Firstly the most common theme was students agreeing that the framework helped them better learn the concepts of the course.
One of them said "The service helped in applying course concepts in a real-world setting. We were solving real problems
using what we learned in class."
Many of them also felt that it helped them make an impact on the community.
They echoed similar comments
like "My service has made me interested in the idea of using data science skills to promote social good."
The systematic structure of DSLM helped the students finish project deliverables in time and not leave them for the last minute.
For example one student commented: "This model not only gave me good grades, but I never got flustered."
On the other hand,
we also received some comments that requires improvements to the framework in the future.
Few students felt that there was a disconnect between the Service-Learning goals and their personal goals.
One of them said: "A more technical project would have been in sync with what we learnt in class and hence a better application
of the concepts."
Now let's see what the partners had to say
All the partners agreed on the effectiveness of the course and felt that the student projects were useful for their
organizations. Many of them echoed similar comments like: “It provided us with treasure trove of data and insights.”
They also agreed on the success of the projects and left comments such as: "The students bring great
technologies and fresh eyes to our challenges."
Many felt that the projects were also a learning experience for them.
For example, one of them said "It brought out trends in our data in ways we had not seen before."
Therefore, overall both students and partners agreed on the successful integration of design study project in the course with objectives
being fulfilled at both the classroom and community levels.
This was reflected from the fact that they cared enough to leave positive comments on a voluntary survey.
Over the course of the iterations of DSLM, we too learned some valuable insights.
Firstly,
even though DSLM can be applied without
Service-learning, we advocate for the use of Service-Learning with DSLM because it provides real-world data science
experience to students.
It provides professional development and future opportunities and most importantly
students make a positive impact on the community.
Now, if you are interested in implementing DSLM, here are a few things to keep in mind...
Adjust prerequisites to adapt to different semester lengths. If you have a longer semester, reduce the technical
prerequisites as students will get enough time to develop those.
If you have a shorter one, then increase the prerequisites so that the students are technically well equipped to tackle the problems.
Set goals that are feasible to accomplish within the limitations of the course.
Plan ahead. For example, if you're not using a Service-Learning facility to identify partners, make sure you're doing it way ahead of the semester.
Have more tightly defined guidelines because a strict guideline to follow will make the projects more streamlined.
Define clear rules and guidelines on IP and data privacy because you don't want to get into trouble there, and
finally, maintain effective communication with both the students and the partners.
In the future, we look forward to implementing DSLM in larger classes with more than 50 students.
We are also excited for the use of DSLM to create short expedited research projects.
Thank you so much for listening. I hope you'll give DSLM a try and empower your students to step into the shoes of visualization
researchers and make an impact in the community.
If you would like to know more about DSLM or the student projects,
please read our paper and visit our website that will be shown on the screen.
Thank you!