Today we’ll be presenting on our CHI 2020 paper titled “The DELAY framework: Designing
for Extended Latency”.
My name is Derek Hansen and I’ll be presenting along with Amanda Hughes. We’re both faculty
at Brigham Young University. In addition, we worked with a group of interdisciplinary
students who are part of our Studio UX group at BYU.
Imagine living for 8 months in a 140 square meter habitat like the one pictured here,
which is part of the HI-SEAS Mars Habitat Simulator in Manoa, Hawaii. You may be surprised
that perhaps the most significant social challenge you would face would be caused by the 20 minute
one-way delay in communication to and from the habitat. HI-SEAS Crew members, such as
the recently married Laura Lark, vividly described the challenges of being “cut off” with
“no live communications” making it hard to “connect to the people you love on Earth”
and ”just be together.”
While this example may seem a bit extreme, people frequently deal with communication
time delays – whether they are caused by time zone differences, incompatible schedules,
or intermittent Internet access.
Our project is an exploration of how to think about and design communication systems for
these high-latency communication situations.
Our paper makes two main contributions. The first is the creation and validation of a
new framework intended to help design and evaluate high-latency communication technologies.
The second is the presentation of 3 proof-of-concept prototypes of novel systems designed to increase
“social presence” and “connectedness” in delayed communication situations. The literature
review in our paper describes these psychological constructs, why they are important, and why
they are so challenging to reproduce in high-latency situations. For now, think of “social presence”
as the feeling of “being with another person”. And think of “social connectedness” as
a feeling that a person feels loved, cared for, and valued by other people. Basically,
the opposite of “social isolation.”
We used a research-through-design approach in this project, where insights about delayed
communication and possible solutions were gained by applying design methods, practices,
and processes.
The first project phase was exploratory. We discussed the problem of supporting delayed
communication and conducted a literature review. Three experts on the topic talked with the
class and critiqued our developing ideas. We also conducted a survey of existing solutions.
As we better understood the design space, we started to iteratively develop our framework.
We often discussed the current framework, classified existing solutions using the framework,
and debated variations of the design dimensions.
In parallel, we created and evolved three prototype designs to explore aspects of the
DELAY framework. Prototypes were critiqued and refined based on feedback from experts,
class members, and other potential users.
Lastly, we conducted two participatory design workshops with 21 participants to evaluate
the usefulness of the DELAY framework and to collect feedback on our prototypes.
In the workshops, we demoed the three prototypes to participants. Afterwards, we discussed
the prototypes and what they liked or would change about them. We also asked them to classify
the prototypes into the DELAY framework categories. Next, participants worked in groups to create
and sketch a low-fidelity prototype of a system inspired by the DELAY framework. We concluded
the workshops with a discussion about the framework, how they might modify it, whether
they found it useful, and why.
Please refer to the paper for more details around the method.
This table summarizes the final iteration of our DELAY framework. I’ll be walking
through the various component parts next. For now, notice that we consider 6 different
design dimensions, each of which is further broken down into more specific categories.
The Goal dimension characterizes the intended purpose of the system, considering the users’
relationships with each other, as well as the overall context.
Since these categories are not necessarily unique to delayed communication, I’m not
going to spend a lot of time delving into them. I do want to point out two things. First,
contextual factors may include the reason for the time delay (as illustrated on this
slide), or other factors that relate to the context in which people use the system whether
that is a home environment, a nature trail, or a confined habitat on Mars.
Second, the examples shown on this slide are not meant to be exhaustive. There’s really
is no limit to the purposes, relationships, or contextual factors at play. However, as
designers it is important to clearly identify these elements of any system. And, of course,
recognize that systems will inevitably be used in unexpected ways.
The Communication Genre describes the platform the system is built upon, as well as the specific
types of media that are shared. For example, a novel system may use an enhanced virtual
reality platform to convey images, audio, and tactile feedback.
Sequencing describes the exchange patterns in communication between users. Throughout
this framework, we assume a significant delay in communication, so “synchronous” communication
is not really an option. One-way sequencing would allow Alice to send a package on a spaceship
to Bob who is on Mars, and has no avenue for sending packages back to Earth. Two-way sequencing
would allow Alice and Bob to send asynchronous messages to each other, such as via email
or voicemail.
Veiled communication describes when Bob and Alice both create content independent of one
another – and that content is brought together at a later time. For example, they might each
video themselves dancing to the song “Somewhere Out There” at exactly 2pm. They wouldn’t
see each other creating the video at the same time, due to the delay, but once their videos
are sent to each other, the system could put them next to each other into a collage. This
is similar to games such as Rock, Scissors, Paper, where two players independently take
an action that is simultaneously revealed – though with delayed communication they
would have to wait for the reveal since the delay acts as a veil separating their ability
to see each others’ choices immediately.
Cardinality indicates the number of people that send and receive messages. Most phone
calls are one-to-one cardinality because there is one person on each end of the line. Emails
are one-to-many cardinality, when a single sender copies in multiple recipients. My twitter
feed is a good example of many-to-one cardinality, since it aggregates tweets from all of my
followers. And systems such as Facebook can be thought of as Many to Many when considered
as a whole, because many people are creating content that many others are digesting. Notice
that these are not mutually exclusive. A single system may include many of them.
Responsiveness measures how the content itself responds to a receiver’s actions, independent
of the sender’s immediate involvement. You can think of it as the level of interactivity
of the content itself. At the low extreme, are messages that do not change at all based
on the receiver. Letters, text messages, images, and even videos simply display content when
a recipient chooses to view it. An example of a system with medium responsiveness would
be GamePigeon’s Shuffleboard texting game, which shows your disks reacting to pre-recorded
moves by your opponent. At the High responsiveness end, you could imagine a chatbot that responds
in real-time to many different textual prompts anticipated by the sender.
Our last dimension is mutability, which captures the ways in which the recipient can change
the content that they have received. If the recipient cannot modify the content, then
they have read-only access. Imagine a typical video that they can watch, but can’t change.
If, however, the recipient could add emoji reactions to the video, they would have Layer
Mutability – since they can add to, but not modify, the original content. Finally,
if they can directly modify the content, such as changing the soundtrack or shortening the
video, then the system would be classified as Modify.
With that, we wrap up the description of the DELAY Framework. Amanda will now describe
some of the prototypes our team developed alongside the framework. They were inspired
by the emerging framework, while also helping us to refine and improve the framework itself.
We now describe our three prototypes, how they relate to the DELAY framework as summarized
in this table, and key insights about the nature of delayed communication from workshop
feedback.
InSync is a mobile app that connects people asynchronously through activities, such as
dancing, working out, or creating music. A key way to use InSync is to schedule a “joint”
activity such as a Zumba workout for a specific time. Each person performs the activity and
video records it. A composite video would then be created and sent to each participant,
an example of Veiled Sequencing.
The DELAY framework led to our team realizing that the original design could also support
One-way and Two-way Sequencing. For example, the system may allow a person watching a movie
to add simple reaction emojis or audio commentary, which would be sent with the movie for the
recipient to experience “live” as they watch the video. This One-way sequencing could
be turned into Two-way if the second person records their own reactions and sends it back.
Many workshop participants reported that they would use the app and they liked that the
experiences were archived. One participant said that “the best feature is the memory
that the activity makes.”
The After the Beep prototype allows users to leave messages for others that are triggered
by the recipient’s interaction with an Internet of Things device. For example, Alice may pre-record
an audio message that will be played when Bob opens the garage door.
This prototype led to the inclusion of the Communication Genre dimension and helped us
refine the Responsiveness dimension in the DELAY framework.
Workshop participants liked the concept and reported the kinds of triggers and messages
they would create. Common triggers included opening the fridge or turning a light on.
People imagined creating messages that served as reminders, such as take your medicine,
or send a birthday card. They also imagined “timing triggers based on a person’s schedule”,
such as going to bed or waking up. Messages could also be triggered with instructions
on how to complete a particular task, such as the laundry or making dinner.
Surrogate is a battle video game similar to Super Smash Brothers, that lets you to play
battles with an artificial intelligence “surrogate” version of your friends. Each player creates
a custom avatar that mimics their gameplay style and sounds. Machine learning would then
be used to learn and update the gameplay style of a player.
This prototype inspired us to include the responsiveness category in our framework.
Also, the framework helped us realize that the prototype could work well as a Many-to-Many
experience—one that supports players who are synchronously battling, as well as Surrogates
of players in a delayed scenario.
During our workshop, participants played against a computer player that was an AI version of
one of the research team members. The player name matched the researcher’s name and all
player sounds were voice recorded from the researcher. The overall response was very
favorable with participants stating that the concept was a “really cool idea.” A few
people had concerns, stating that it “feels weird playing against someone not there”
and it “could just emphasize the loneliness.” We recognize this as a potential problem for
delay systems.
The workshop co-design sessions helped validate the framework’s usefulness as a generative
tool for novel designs. As further evidence of the framework’s value, most participants
classified the three prototypes into the DELAY framework categories that the design team
also classified them in. This suggests that most participants understood the framework
and prototypes, and they were able to apply this knowledge in a consistent way. The framework
was also useful for refining existing ideas. Participants felt the framework was comprehensive
and did not have suggestions for new dimensions or categories.
We hope the DELAY framework will help inspire new systems designed specifically for high-latency
communication scenarios. Several areas of the design framework are ripe for exploration.
The InSync app showed one possible way to use veiled sequencing, but there are many
others. For example, many popular social games such as Psych or those made by Jackbox could
be designed to facilitate time-delayed play using a veiled sequencing approach.
Another area ready for innovation is the creation of highly responsive artifacts. Our Surrogate
app helped illustrate the potential for gamers, but it could also be used to serve other purposes,
like promoting intimacy by creating a personalized chatbot that anticipates the responses of
a partner. Finally, the CHI community is already pushing the boundaries of using novel platforms
and media, such as haptic feedback, which could be used to enhance a feeling of social
presence in some interesting ways.
Designers may consider relaxing the requirement for delayed communication that we built into
the DELAY framework, and instead make Synchronicity a design dimension. Although in this paper
we wanted to draw attention to the issues that arise with delayed communication, our
Surrogate prototype helped us realize that blending synchronous and delayed communication
could be valuable in many cases. We are anxious to see how others extend our initial ideas
and excited to experience the systems they help inspire.