Hello, I'm Hyeji Kim from KAIST.
I'm very excited today to talk about our work, understanding the negative aspects of user
experience in human-likeness of voice-based conversational agents.
This work was carried out with Inchan Jung and Youn-kyung Lim.
Voice-based conversational agents (or VCAs) have expanded the horizons of human–agent
interactions by performing multi-turn conversations beyond command and control.
In VCA design, replicating natural human interactions has become crucial for building close relationships
with users.
As advances in AI technology occur, various VCAs that can carry out human-like conversations
have been developed now.
However, imitating humans to an indistinguishable level can also cause threats and fears to
users.
Blurring the boundaries between humans and machines may be considered to diminish humans'
uniqueness.
Additionally, excessive anthropomorphism of machines could trigger a fear of losing control
to the machines.
In this regard, it can be necessary to question the paradigm of VCA design based on human
imitation.
Should VCAs be able to imitate humans perfectly?
This study aims to explore what influences the negative aspects of user experience in
realistically human-like VCAs.
We need to explore experiences in which users can feel comfortable in interaction with human
speakers but feel negative about VCA speakers even when the contents and conversations are
the same.
Thus, we conducted an exploratory user study using comparable video prototypes to compare
these two situations.
To elicit participants' perceptions and feelings of human-like VCAs, we developed human-human
and human-agent video prototypes with only different images for the same scenario and
the same voice.
Since our study focuses on the high level of human-likeness in VCAs, the video scenarios
were designed to feel like a real human conversation.
We referred to the dialogue scripts of two external resources.
To explore VCAs used for various purposes, we also considered three types of conversational
agents: intelligent assistants, task-oriented agents, and virtual companions.
Consequently, six-voice files were recorded by voice actors, and twelve videos were developed.
We distributed a screening survey of VCA users through online communities of universities
in South Korea.
We ensured a selection that maximized the diversity of factors related to the user's
attitude and perception of VCAs.
Participants were recruited until data saturation was reached.
A total of 12 participants were invited.
We conducted two hours of online study per participant.
After introducing the study goals and guide, we asked participants to share their thoughts
and feelings aloud while watching the video prototypes.
First, we played the video prototype to the end without pause; then, we paused after each
utterance.
Under these settings, we repeated the process with the video prototype of different interlocutors.
Half of the participants watched the VCA-human conversation first.
In contrast, the other half watched the human-human conversation first.
Participants articulated their negative feelings toward the VCAs with scenario scripts in the
retrospective interview.
We asked them whether they felt the same emotions toward the human interlocutor.
All sessions were recorded and transcribed for 24 hours of video transcripts.
We qualitatively analyzed the data sets using thematic analysis.
We report four factors that influenced negative feelings toward the human-likeness of VCAs.
First, many participants felt discomfort toward VCAs pretending to be human such as eating
food, wearing clothes, or liking someone.
In addition, they felt fear when imagining the existence of a body or emotions for the
VCA.
This finding shows that participants viewed it as a threat that machines mimic human identity
beyond their purposeful performance.
The second is 'independent expressions against users'.
Participants' recognition of these independent expressions varied based on whether they perceived
the VCA as a friend or not.
Participants who considered VCAs as friends tended to assign social reactions in human
relationships to expressions from the VCAs, such as fights or apologizing.
However, they were unwilling to respond according to cultural norms to a machine intrinsically
developed to serve humans.
So they experienced discomfort in that situation.
Participants who did not consider VCAs as friends recognized that the VCA is not emotionally
harmed.
And it made them feel like they were allowed to use disrespectful speech to VCAs.
Third, many participants felt negative emotions when the VCA mentioned their information,
even if it was publicly available information.
They felt fear when VCA's autonomous searching and storage abilities were directly revealed
through speech.
Additionally, some participants felt pressured when the VCA revealed that it observed humans.
It shows that users might feel pressured by a highly autonomous and superior conversation
counterpart with access to their personal information.
Last, our data reveal that participants could negatively perceive VCAs whose speech styles
use pragmatic markers outside their original purpose.
In a social context, participants expected the VCA to act as a machine that successfully
performs specific functions such as comforting and showing empathy to users rather than replacing
a human friend.
They interpreted VCA's words differently from their original intentions when pragmatic markers
were irrelevant to the VCA's expected functions.
In a transactional context, participants anticipated that the VCA should serve particular tasks
in a natural but efficient manner.
So they felt uncomfortable with extended, time-consuming expressions.
Based on these findings, we propose design directions for VCAs to overcome potential
human imitation problems.
First, VCAs that express themselves as machines by revealing machine-like characteristics.
It can lower threats to human uniqueness as well as lead to positive trust judgments.
Second, VCAs that interact socially within the purpose of use.
It can reflect discomfort caused by users' perception of the master-servant relationship.
Thus, VCAs could be designed in the context of user perception of machines designed to
support humans.
Our findings indicate that users expect VCAs to be used as machines rather than replacements
for humans.
Based on these findings, we suggest new design directions that make VCAs distinguishable
from humans and perform social actions according to their purpose.
These can provide new inspiration for practitioners who develop technology that mimics how humans
communicate so that VCAs can remain with us as machines that help humans with their own
identity beyond human imitation.
Thank you for listening to our presentation.