Hi, I am Sunok Lee from KAIST next interface lab which is located in South Korea.
Thank you so much for attending this session.
My research title is What If Conversational Agents Became Invisible?
Comparing Users' Mental Models according to Physical Entity of AI Speaker.
This study is about understanding the user's mental model of visible and invisible conversational agent.
Moreover, we explored how this physical entity impacts user experience.
To begin, let me give an overview of the background and purpose of this study.
Due to the unobtrusive advantage of the voice user interface (VUI), such as eye-free and hands-free interaction,
the potential of VUIs has increased in the ubiquitous environment.
To control the various, IoT devices and smart homes,
an increasing number of users purchase multiple AI speakers and place them throughout their homes.
When we observed these users' interactions with AI speakers, as you can see on the right,
users often naturally face the device, physically, for better communication
because of the sound direction, visual cue, and microphone location of the device.
However, this visual and auditory directed interaction can cause limitations
on VUI's unobtrusive advantage such as multi-tasking.
For example, when cooking, users are interrupted by having to look at an AI speaker
to ask for recipes as illustrated in the figure.
Therefore, some researchers have proposed that technology for smart homes should blend into the background without visual interference,
such as invisible intelligence and ambient intelligence,
and that such forms of technology can support the user to focus on the main task without interruption.
In this study, the research questions were as follows:
How do users perceive and interact with invisible devices?
What other advantages do invisible CAs have compared to CAs with physical entities?
How can sensor-based, invisible interfaces be designed for implicit interaction with users?
To answer those questions, this study compared users' perception, mental model, and interaction between visible and invisible CA
through qualitative methods such as a drawing study and interview.
For the study environment, we set up two different home-like environments in our lab.
In the first setting, for the visible CA, 3 AI speakers were placed in 3 different locations.
In the second setting with the invisible CA, the AI speakers were located in the same place as in setting one,
but they were hidden from the user’s view.
We recruited 30 novice users (avg. age = 22.5 years old);
half interacted with the visible CA, and the other half interacted with the invisible CA.
Participants' task consisted of 3 steps.
First, they were allowed to interact freely with the agent.
After interacting with the CAs, participants were asked to draw a CA or CAs based on their perception.
Lastly, participants rated on a 6 overall user experience scale
and interviewed based on the drawings and observations.
Based on our results, we concluded that physical entity impacts users' perception, mental model, and interactions.
For perception, as we can see in the drawings, for the visible CA,
users expressed the 3 identical AI speakers as 3 different agents with different appearances and roles.
However, the invisible CA was drawn as 1 agent that controls the entire space.
For the mental model, participants expected every 3 different agents played a role
with its own expertise depending on the space.
For the invisible CA, they expected 1 agent could detect detect a user's behavior and status without space limitations.
Through the quantitative data, we found that there was no significant difference
in overall user experience in terms of interaction.
However, during the interaction task,
they naturally faced the AI speaker and maintained “eye contact” during interaction like human communication.
For the invisible CA, participants did not seem to be conscious of the agents without physical entities,
and they talked and listened to the agent from anywhere in the home without interruption.
Based on the findings, we can conclude that the physical entity makes the user feel comfortable like human communication
and can support learnability for an initial user or people who are not familiar with interacting with an agent.
Therefore it would be appropriate to the agent for social interaction.
For invisible CAs, ambient interaction leads to natural multi-tasking at home;
users can focus on the main task and agents can support users' subtasks in a seamless way.
As a result, we thought visible and invisible agents should trade-off depending on user and context.
In terms of functional perspective, the user may prefer an interaction
without visual interference while multitasking.
However, related to emotional and social interaction, users prefer some “eye contact” with the speaker
and the listener which is necessary for social interaction.
This is the content of our research, please refer to the paper for details.
Thank you so much for your time. Please feel free to contact me with any questions.