Hi, my name is Angela Mastrianni.
I am a PhD student at Drexel University.
Today, I will be talking about how we transitioned
our user-centered design activities
from in-person to remote during the COVID-19 pandemic.
When the pandemic struck, we were in the process of designing,
creating, and evaluating
features for a digital checklist
at a level 1 trauma center.
We had to transition all of our user-centered design activities
from in-person to remote
to be able to continue doing this work.
We ended up transitioning four user-centered design activities:
participatory design workshops,
interviews & design walkthroughs,
near-live simulation sessions,
and usability evaluation sessions.
While transitioning these sessions, we had the following research questions:
How do we design remote user-centered design activities
to obtain useful data and results?
How did these remote activities affect
our interactions with participants?
And how did remote activities affect
the interactions between research team members?
First, how do we design remote user-centered design activities?
It was important that we still collected high quality
data and obtain valid results.
Since we had been located in a different part of the country
from our research site,
we had remote access to hospital systems
and IRB protocols that quickly allowed
us to setup additional remote access.
Before running the sessions, we ran pilot tests
and had backup plans that we used
when facing technological issues during the sessions.
We did observe differences in data collection
between in-person and remote
and these differences varied based
upon the user-centered design activity.
We found that we collected more data from
interviews and design walkthroughs,
near-live simulations and usability evaluations
when we ran these sessions remotely.
For example, in the near-live simulation sessions,
when we ran the session remotely
we remotely accessed the tablet at the hospital
so we could see all the interactions
the participant was doing during the session
and record them.
This was extra data that we received.
We did find it harder to collect data
from participatory design workshops when running them remotely.
When these sessions were held in-person,
users would sketch on pieces of paper
that we could take back with us.
But during the remote sessions, we had to just ask
participants to hold their sketches up
to the camera so that we could see them.
We also found that when running the sessions remotely
we lacked contextual details
about the site that we would have recieved if we ran the sessions in-person.
We also explored how our remote activites affected interactions with participants
We did experience challenges hearing participants
especially in sessions where they would also be watching videos.
In those sessions, we had to rely
more on retrospective questioning than the think aloud method.
We also faced issues building trust
as we had to establish connections with participants remotely..
Technological and scheduling issues also caused stress
and these issues had to be resolved remotely as well.
Finally, we experienced self-presentation tensions
as we were conducing these research,
professional sessions from our home environments
and having these two environments mix together.
Finally, how did remote user-centered design activities
affect our interactions with other research team members?
Since there was no travel cost and lower time cost,
multiple researchers were able to join the sessions.
As they were being held remotely,
we could message each other during the sessions
over platforms like Facebook messenger.
This created a back-channel discussion
and the messages from those discussions became
a source of field notes.
We observered that because of these back-channel discussions
we were having two sets of discussions:
one during the session with our real, live-time feedback
and then a set of discussions after the session.
Note taking became a collaborative process as
researchers used Google sheets to take notes together during the session
We were also able to discuss any questions about the protocol
with other researchers during the sessions.
However, these different channels of discussions could
influence the power dynamics in participatory design workshops.
From our experiences, we came up with a set of lessons learned
and best practices.
For designing remote user-centered design activities,
we found it was helpful to have remote access
to as many systems as possible
and back-up plans in case of technological issues.
In terms of interacting with participants,
we found it useful to inform participants ahead of time
of any technical requirements
and to use technologies that
participants would already be familiar with.
For interacting with other researchers,
we defined the roles for each research member before the session
and introduced all research team members
and their roles to the participant at the beginning of the session.
Thank you for attending this talk.