Hi, I’m Yunjae Choi from the Next Interface Lab in KAIST,
Today I would like to present a research titled ‘Toward a multilingual conversational agent: challenges and expectations of code-mixing multilingual users,’
co-authored with Minha Lee and Sangsu Lee.
Code-mixing is one of the most commonly observed linguistic strategies of multilingual speakers,
which is to alternate two or more languages in words, phrases, clauses, or sentences.
This research studies frequently code-mixing multilingual users and consider them as an individual user group with unique speech behaviors.
And this work explores the challenges and expectations the users have with the conversational agents, CAs for short,
and discusses the design considerations to enhance the user experience.
So, this work is motivated by the discovery of multilingual users facing difficulties in using CA
because of the agents’ inability to comprehend their natural communication, which includes multiple languages.
These difficulties have become increasingly prominent in recent years.
We also spotted some communication failure cases that our multilingual relatives had with their CAs,
as shown in the pictures on the right.
Therefore, through this study, we aim to make three contributions,
which are to understand multilingual users’ CA usage experience and the impact of their code-mixing on it
and to propose the design considerations and possible design directions to enhance the experience of multilingual users in CA usage.
We came up with two research questions and the corresponding research methods to succeed in each of the questions.
The focus group interview was conducted to explore the factors that need to be considered to enhance the user experience of code-mixing among CA.
On the other hand, the participatory design workshop was used to understand what multilingual users expect and how they intend to utilize code-mixing capable CA.
The flow of the entire user study proceeded as shown.
The study began with an introduction to the research and an ice-breaking activity.
The main procedure of the study consisted of two phases, Phase 1, the focus group interview, and Phase 2, the participatory design workshop.
The study was followed by discussions about participants’ thoughts and impressions of code-mixing CAs.
The experiment was conducted as an online study to recruit participants from various regions.
Participants were recruited as eligible candidates who met the following three criteria through an online survey.
and they were divided into four groups, with average of four participants in each group.
All groups conducted discussions through screen sharing.
Our results are grouped into four findings.
The findings are related to multilingual users’ code-mixing patterns and their thoughts about code-mixing itself,
how the users perceive their current CAs,
their difficulties with using CA with their code-mixing patterns,
and their expectations for the code-mixable CAs.
Due to time constraints, we will only cover a few interesting findings for each topic in this presentation.
So we encourage you to refer to the paper for the entire findings and explanations.
The first finding encompasses the multilingual users code-mixing,
the challenges and difficulties they face in code-mixing communication,
and the things they need to code-mix effectively in everyday interactions.
They said that because each word is ingrained in the language they first learned,
they have to translate the word themselves and then use it when communicating in a situation where only one language is allowed.
Therefore, in moments where code-mixing is possible,
some participants felt they could freely express words in the language that best captures the nuances,
and were relieved to be in a safe zone.
In the second finding, we analyzed the common perceptions of code-mixing users with the current CAs they are using or have used.
Interestingly, despite having difficulty using existing CAs due to their poor language abilities and natural conversations,
multilinguals tended to personify their CAs.
They treated each CA as if it were a person who only spoke one language,
the language of the country where the CA was developed.
We also examined the difficulties that arose while using existing CAs due to the perceptions mentioned
and identified five situations in our third finding.
For example,
participants mentioned that they often take time and effort to translate words into a language that can be understood by the CAs,
which results in delays in conversation.
However, they have experienced instances where the CAs did not wait
and ended the conversation or urged them to hurry up,
leading the users to frustration and giving up on using the CA.
And if CAs fully support code-mixing, how would the users want to utilize it?
We explored this through a participatory design workshop
and constructed the fourth finding solely based on its results.
The most prominent finding that emerged was the commonality among the use cases of such CAs.
There were three major cases to be categorized.
Users thought of CA as a companion to hang out with while casual talking,
assuming that CA can code-mix and understand memes and slang from all cultures,
or they even considered it as a tool to understand their work context,
aid them, and help them maintain and improve their language skills.
They also wished it to understand and utilize code-mixing by incorporating familiar words and languages they usually use.
Based on these interesting findings, we proceeded to discuss major discussion points,
including the four points to consider in designing multilingual-friendly CAs.
The first is that CAs should provide code-mixing similar to how the users do,
because similar code-mixing can help create intimacy and relationship building through comfortable communication
and make the users to feel fully accepted.
Second, users expect different code-mixing and characteristics depending on their social and transactional roles and contexts,
which are commonly found in their participatory workshop scenarios.
Therefore, we mentioned the possibility of considering code-mixing and its level as one element that shapes CA’s complex and relational nature.
Third, we pointed out the importance of recognizing that multilingual users can use various accents
and that they are capable of mix-using it while code-mixing.
Lastly, to design a more personalized communication-enabled CA,
it is important to maintain a consistent persona of the CA, even if the language and cultural settings change.
In summary, the main contribution of this study can be highlighted as examining
how the code-mixing multilinguals use the current CA and what their expectations for the future CA is.
Also, presenting design insights and considerations to improve their user experience beyond their preference for code-mixing functionality,
and lastly, attempting to establish design guidelines for CA for multilingual users.
That’s the end of the presentation.
And for more detailed information, please refer to the paper.
Thank you.