﻿Hello, I’m Inchan Jung, a master student in the department of industrial design at KAIST.
Today, I’m gonna present our recent work : Understanding how users experience the
physiological expression of non-humanoid voice-based conversational agent in
healthcare services. Nowadays, interactions
with voice-based conversational agents in not human-like forms are becoming more common.
Since these VCAs primarily use natural speech-based conversation
users often expect them as humans although they are non-humanoid.
Accordingly, researches on these VCAs engaging diverse human-like
traits have been conducted. However, we wondered what if the
VCA was newly endowed with the physiological states that every human has. For instance,
we imagined a virtual scenario where the VCA coughs or tells a user that VCA’s eyes hurt.
In detail, our VCA’s physiological states model after the human health conditions
since expressing these states could have potentials in the domain of health management,
where VCA primarily supports users’ self-care and delivers health-related information.
Therefore, in our study, physiologically expressible VCA,
which manifests its physiological states solely through the voice,
can virtually sense and react to external stimuli such as light or sound
can suffer from diseases and express symptoms,
and can even manage its physiological states for recovery.
Regarding this novel area, we set out to explore how a user might interact with a physiologically
expressible VCA in a healthcare scenario. We first conducted design workshops to discover
physiologically expressible VCA’s design opportunities and design considerations for
their healthcare services based on which we plan the following main Wizard-of-Oz user study.
We conducted workshops on 12 design students,
who were familiar with VCAs and had designed user scenarios incorporating new technologies.
And 3 students participated together in each workshop as a team.
After the introduction, For diverse designs, each team
designed two different healthcare scenarios applying VCA physiology.
They first defined user persona’s health-related problems, and then developed healthcare solutions
from a physiologically expressible VCA. At last, interviews were conducted on design
ideas and rationales, and then transcribed and thematically analyzed, resulting in 2 themes.
First findings are 3 design opportunities for healthcare services.
First, physiologically expressible VCA might be able to convince users to adopt the similar
health-related behaviors by sharing its prior healthcare experiences.
Second, VCA with conditions similar to a user’s poor health may cause the
user’s behavioral changes by suggesting and practicing common healthcare together.
Lastly, VCA that experienced similar health problems
will be able to empathize with the emotions of the user struggling with the concerns.
In addition, we discovered 2 design considerations for physiologically expressible VCA.
First, the VCA should describe what physiological states
the onomatopoeias refer to in words for the user’s clear understanding. In addition,
the VCA needs to reflect human-like paralanguage expressions in its voice moderately.
Second, self-introduction for physiology that instills a mental model in the user that VCA can
possess all the physiological states like humans, should be made at the first encounter.
Applying these findings from workshops, we designed a single healthcare scenario
of physiologically expressible VCA to explore the user experience. To help the participants
to realistically experience the scenario, we applied the Wizard-of-Oz method.
In addition, we set up a common VCA that does not engage physiological expressions
to enable articulating the experience from the physiologically expressible VCA
more clearly by comparing two VCAs. All the VCA’s speeches were pre-structured
so that each speech of the two VCAs could be compared one-on-one.
Thereafter, we designed the options of a user response
for each VCA speech through the survey. Finally, we developed the video prototypes by
placing the VCA speech videos and creating the optional buttons for the user responses.
Then, we conducted online experiments on 18 participants who were similar to user persona
and had used smart speakers. After the introduction,
participants experienced interactions with two VCAs.
When the wizard played the speech video of the VCA,
I hurt my neck, too. [sound of popping joints]
Let’s do stretching together. then, participants responded by reading the most
preferred user response among the options. Then, we conducted in-depth interviews.
To explain the healthcare scenario, the user persona that participants would immerse in
were suffering from neck pain. As the healthcare services, both VCAs delivered
stretching and medication information and managed the user stretching.
Unlike the general VCA, the physiologically expressible VCA provided these services
utilizing its physiological states. And it also expressed its states with
non-verbal sounds by utilizing the onomatopoeias “crunch” and “ouch”.
All Interviews were transcribed and thematically analyzed, resulting in 2 themes. User experiences
on its healthcare services and user perceptions of physiological expressions themselves.
Regarding the experiences on healthcare services, we first found that the
physiologically expressible VCA’s experiential knowledge is much friendlier and gives users
the motivation to follow its advice. In addition, we found that accomplishing
the healthcare activities together with VCA could help users create connections and companionship,
and enhance their willingness to accept its healthcare proposal.
And, to explain their perceptions, most were very awkward about human-like
physiology of VCA at first. However, we found two types of
users’ mental models, which enabled them to naturally imagine the virtual concepts :
a human-like VCA mental model that allows the user to think of a VCA as a complete human, and thereby
considers given physiological states natural,
and a robot-like VCA mental model that makes the user perceive a VCA as a robot which can virtually only possess physiological
states of the external body parts, and thereby considers taking medicine very unnatural.
On the non-verbal sounds, user perceptions emerged in two patterns.
Some participants reacted positively to the sound of popping joints
as it helped them to virtually experience that the VCA can perform stretching.
However, the other participants considered such sound unnatural
since it did not practically have the form factor to produce this sound itself.
Interestingly, all the participants with the robot-like
VCA mental model reacted like this. For the grunting sound expressing pain,
all participants accept it as an ordinary sound that will occur in speech.
Based on these findings, we suggest the first design implication : designers should
assign acceptable physiological states and apply appropriate non-verbal sounds to
a VCA depending on the user’s mental models. And based on the findings that the physiologically
expressible VCA could provide social support in health-related conversations, we suggest
utilizing it to the behavioral health intervention where its support could be meaningful.
For the possible areas, there could be treatment of chronic illnesses where
patients seek health stories from peer patients for self-care. Through sharing experiences,
the peer VCA will aid patients socializing and reflecting its health decisions.
And there could be regular exercise where group setting encourages to complete the activities.
Through forming a companionship with the VCA, users may feel less isolated in their journey,
and show more perseverance. To conclude, we explored user
experiences of physiological states in the healthcare domain.
And our design implications can aid designers and researchers initiate understanding how to design
physiologically expressible VCAs considering user mental models and what healthcare domains
such VCAs could be meaningfully applied to. Thank you very much for watching and goodbye!