Hello, my name is Duri Long I'm an Assistant Professor in the
Department of Communication Studies at Northwestern University and the director
of the Creative Interfaces Research and Design Studio.
Today, I am going to be talking about my case study on conducting
remote design research on embodied collaborative museum exhibits.
This work uh sort of started many years ago, I was working on a series of museum exhibits
that were intended to foster family groups' AI literacy—so helping families better understand
AI technologies and how they work, and I was working with my museum partner at the Museum
of Science and Industry in Chicago to develop a series of embodied—so physically interacting
with these exhibits—and collaborative—so families collaborating together as a group—exhibits that
families could interact with to learn about AI. But we all know what happened shortly
after February 2020. COVID came into the picture, and so I really had to shift the
direction of this research because we were developing exhibits for museums and we had
to shift that to a different context. At the point in time that this happened, I was in the
sort of early stages of an iterative design research project so I had engaged in a lot of
ideation of different exhibits which you can see on this slide; so I had created design sketches,
I had also created low fidelity prototypes, and I had conducted a co-design workshop with
stakeholders to gather their feedback on exhibit ideas and for them to generate ideas of their own.
At the time of February 2020, I was sort of shifting to developing higher fidelity prototypes
and testing those out on the museum floor, and the advent of the COVID-19 pandemic led me to
sort of inadvertently explore this question of how can we conduct iterative design research on
embodied collaborative learning interventions when we are unable to conduct in-person user studies.
And there were a number of considerations or questions that this invited. Obviously,
the first was how to make the study contact free so it was safe for both the researchers
and the participants. The second was how do we collect data if we're not in the same room as
our research participants. Perhaps the most challenging question that was unique to this
particular study is how to keep the interaction embodied and collaborative. These were two key
design considerations that were really central to the designs that I had been exploring and
uh didn't would not have been supported by a Zoom environment. um so This raised sort
of some new questions about how do we conduct remote design research that's not just on Zoom—um
people couldn't engage with the exhibits tangibly or physically in that setting. And
finally I had the question of how to keep the exhibits fun and playful and not make
them a burden to user participants. This was a challenging and stressful time for everybody,
and I was also studying informal learning I wanted families to have fun and enjoy themselves as a
part of the activity. So these were some key considerations that I was keeping in mind when
I was considering how to shift my study from an in-person museum environment to a remote setting.
And I talk about this more in the paper, but what I did was I took some of these initial exhibit
ideas that I had had—the design sketches and the low fidelity prototypes which were designed for
museum spaces—and I iteratively developed those into activity boxes for families to
engage with in the comfort of their homes. So this involved scaling down the exhibits; part
of the process involves selecting exhibits that I thought could be scaled down appropriately. So,
I'm just showing one example, I had a series of exhibits that I did this with, but I'm showing
one example on this slide of an exhibit called uh Creature Features which was intended to teach
learners about bias in datasets by allowing them to sort of put weight on particular features in a
dataset which would then impact the outcome of an algorithm that distinguished between birds
and non-birds. So you can see here an image of the sort of museum exhibit which was very like large,
collaborative, like you interacted with it with your body, and then the final exhibit,
er, activity that I ended up developing for at-home learning spaces which is more of a
game board-like activity. It utilizes a card deck and an iPad interface to allow learners
to engage with the algorithm. So, again, more details in the paper but this is sort
of the process I went through to adapt each exhibit idea to an activity box in actuality.
And I want to jump now to talking about sort of some of the considerations and Inspirations that
came up for me for adapting design research to remote contexts. These are sort of considerations
that I think can be useful to other researchers based on my experience adapting these embodied
collaborative museum exhibits to a remote context. So I'm going to talk a little bit
more about these here. These are not intended as like a list of "must do's" or like a prescriptive
list of things that you have to do, but more as a set of sort of Inspirations to allow
researchers to think about how to adapt embodied collaborative activities to a new context.
The first consideration I want to highlight is to examine the artifact—so, considering
the affordances of the activity or artifact you have designed or are planning to design,
and thinking about whether you can scale it down or use smaller more portable components
and easy to use technology. So, for example, some of my exhibit ideas lent themselves more
easily to translating to like a game board type activity than others, which
involve more expensive technology or are really centered around the idea of like a large scale
installation. So I selected the artifacts that I felt would adapt most easily to this new context.
I also returned to early prototypes. So, I suggest that designers consider whether they developed
low fidelity or paper prototypes of their design and returning to those prototypes as inspiration
for a version of the activity that encapsulates the sort of main elements of that activity but
um is uses low-cost material and is easy to transport.
The next consideration is to make it easy, so I asked designers to consider:
can you find technology with easy to use interfaces and simple setup or are there aspects
of the study that you can have participants complete in advance to sort of simplify and
reduce the burden on participants? I spent a lot of time considering what types of data collection
technology to use to enable learners and families to easily set this up and collect their own data.
My next consideration is to make it portable—so thinking about how you can scale down the size of
your designs to make them portable, considering how you package your designs. In my case,
I packaged each of these activity boxes in a pizza box and I delivered them to families
doorstops around the Atlanta area, but others might be able to use the same method and mail
a set of boxes to people, for example, so considering the packaging and transport.
The next consideration is to prioritize fun, excitement, and mystery—so,
thinking about how you can make your user study fun. Can you use study methods that
do not feel like tests or extensions of the school or work day? and How can the
packaging and presentation foster excitement and mystery surrounding the activities? For example,
a lot of the families that engaged with these activity boxes reported that finding the boxes
on their doorstep and the unboxing activity of opening them up together really provided a lot
of anticipation and excitement for them and their kids during a difficult time.
The next consideration is—what if we didn't use Zoom? So, consider how you might conduct
your study without or with minimal use of video conferencing. In my case, I was really concerned
with how to make this a fun activity. Families and kids had already been spending like eight
hours a day on Zoom with work and school during COVID, and so I gave participants the option of
participating in a video call or just engaging together with their family and recording their
data on their own. So, thinking about how you can provide such options is important in this process
the next consideration is redundancy redundancy redundancy. So, thinking
about how you can introduce redundancy in your data collection process without
overburdening participants. So for example I collected video data, audio data, as well as
had participants complete a survey after their interaction, in order to ensure that I gathered
some data even if one of the data collection technologies malfunctioned due to user error.
The next consideration was parallel play—so, thinking about making multiple copies of your
design so you can conduct several studies in parallel and thinking about how long it will
take for participants to complete your study and how long delivery and pickup will take,
because this method of conducting design research takes longer than
just conducting uh one study in a museum space.
The next consideration is to diversify recruitment methods this is something
that's important you know beyond this context, but it was particularly challenging to recruit
participants to engage in a user study during the pandemic, so thinking about who is using the
audience and the platforms you're recruiting on and what reach your recruitment methods
have is really important and we iterated on this throughout our study to reach a broader
audience that was more representative of the metro area where we were conducting the study.
And finally the last consideration—to end on a positive note—is to find a silver lining. So,
thinking about what some of the possible positive outcomes of scaling down your
designs or making your study remote are, and what lessons you can translate to your
research more broadly. In my case, I was able to create a set of low-cost portable
activities which you can access at this QR code here to teach family groups about AI
that extends beyond the museum and allows a larger group of people to engage with these
AI literacy learning activities than I would have been able to do in a museum.
Of course, in the past few years we have been lucky to return to a lot of in-person studies
um due to vaccines, but uh this has implications for a lot of researchers beyond the pandemic.
It has implications for interacting with at-risk populations during the pandemic,
as it is still ongoing. It has implications for design research with communities who may not be
able to easily visit the study location such as physically distant communities,
disabled communities, or people without the ability to take time off work to
visit a site location. And this also has implications for future local or
global crises and the possibility of reducing carbon emissions in research.
Thank you so much to everyone who has supported this work over the years,
and I would be happy to take any questions my contact information is on the slide.