Hi everyone, my name is Eunkyung Jo. I'm a PhD student at UC Irvine.
I'm excited to present our work on understanding the benefits and challenges of deploying conversational AI
leveraging large language models for public health intervention.
Before I start, I'd like to thank my collaborators, Daniel Epstein, Hyunhoon Jung, and Young-Ho Kim.
Recent advances in large language models have brought breakthroughs in open-domain dialogue systems, which perform free-form conversations.
A good example is ChatGPT, which was built on top of OpenAI's large language model, GPT-3.
Such systems can be beneficial for public health interventions in providing empathetic interactions
for populations going through difficult health experiences and reaching out to broader populations who've been underserved.
However, few studies have explored how LLM-based based chatbots can be leveraged in public health interventions in real-world settings.
We thus examine the case of CareCall, an open-domain LLM-driven chatbot
that aims to support socially isolated individuals via check-up phone calls.
CareCall was developed by a tech corporation named NAVER to collect data about the individual's general health
and serve as a conversational partner to mitigate their loneliness.
The system was deployed to 20 municipalities in South Korea as of May 2022.
Note that the scope of our study was conducting interviews and observations of focus group workshops;
thus, the development in the pilot deployment of CareCall was outside the scope of the study
To understand the benefits and challenges of LLM-driven chatbots in public health interventions,
we observed focus group workshops with 14 CareCall users
and conducted interviews with 20 people from three groups of main stakeholders around the CareCall system,
including users, teleoperators to monitor the user's conversation logs,
and developers who designed and implemented the system as well as communicated with local governments.
The CareCall users were low-SES, middle-aged adults who were living alone.
The teleoperators were part-time workers hired by the government of Seoul.
CareCall developers were IT professionals at NAVER, consisting of UX designers, machine learning engineers, business managers, and a quality manager.
From the study, we identified the benefits and challenges of leveraging CareCall in public health interventions.
In this talk, we'll only cover the highlighted findings for the sake of time.
For the full findings, please refer to our paper.
The teleoperators felt that CareCall's open-ended conversations provided rich contextual information
to help them gain a holistic understanding of each user's circumstances.
T4 noted: "Some users are leading a satisfying life, typically people who have jobs, regularly go to a community welfare center, and have friends to meet.
I'm not too worried about them; I'm more worried about those who are mostly lying in bed all day and have depression."
This information helped them figure out whom they needed to prioritize monitoring.
The users perceived that the open-ended nature of the dialogue helped mitigate loneliness
by covering conversation topics beyond health, such as their hobbies and interests.
P5 enjoyed having conversations about his habits and sketching with CareCall:
"When it asked what I was doing, I said I was drawing something he then responded 'That sounds fun! I want to learn how to draw too.'
I really liked it when I said that. I wanted to talk more about my work."
Despite the benefits, we observed various challenges in leveraging CareCall for public health interventions,
largely due to the inherent challenges of LLMs.
We noticed several mismatches between different stakeholders' needs and LLM-driven chatbot's challenges.
For example, developers mentioned that several local government officials asked to ensure
that the system asks questions about medication adherence during the calls.
However, due to the inherent uncertainty in controlling the dialogue flows, the developers could not accommodate the municipality's requests.
D5 stated:"What we can do is to fine-tune the model with more datasets that ask certain questions
so that the probability of asking such questions becomes higher, but we cannot guarantee that; such tasks are performed just indirectly.
In addition, the open-ended nature of LLM-driven chatbots made it challenging for CareCall
to manage expectations around the emergency and social service needs.
The users wished that the system offered a direct connection to emergency services
due to their anxiety resulting from living alone, getting older and having chronic conditions.
P3 noted: "I had a stroke last year which left my right side of the body paralyzed.
I'm worried about having a stroke again when alone."
However, the developers were not confident about the reliability of the emergency detection, making them hesitant to support such a feature.
CareCall users also expected that the systems would help provide access to social services
but the developers and the teleoperators felt it was out of scope.
Teleoperators noticed that the user asked to join the food assistance program during the calls
but they did not have the power to accommodate those requests.
T4 said: "They ask for lunchbox deliveries, but all we can do is just empathize with them and report it to their municipality.
We don't have any power to connect to such social services."
Despite the user's needs related to social services, the developers were concerned about the potential burden on the public health officers
and wanted to keep the system specifically for regular check-up calls that inform the public health workers of concerning cases.
Through this study, we found that LLM-driven chatbot's uncertainty in control led to challenges in supporting different stakeholders' needs in public health interventions.
Our findings highlight that expectation management about open domain, LLM-driven chatbots can be challenging, particularly in public health settings.
Interactions with LLM-driven chatbots performing open-ended conversations are likely to lead various stakeholders to assume that
the chatbots can take on the maximal, most flexible set of tasks.
Users may assume that the chatbot is a conduit for all things government-related.
Government agencies can similarly assume that chatbots can take on a whole suite of public health tasks based on the promise of natural conversations.
Our findings point to the significance of understanding multiple stakeholders' needs when designing chapels in complex public health settings.
In our study, both the governments and the users had some informational needs that could have been better served by more traditional task-oriented systems.
For example, task-oriented chatbots could more easily support asking specific health questions that fit governments' needs, such as medication adherence.
Task-oriented chatbots could also more reliably respond to a user's request to connect to emergency or social services.
In contrast, while open-domain chatbots faced challenges in serving those needs,
they demonstrated clear benefits in providing a holistic understanding of care recipients to facilitate care and emotional support.
This suggests that currently, the choice of model puts informational and emotional support in tension with one another.
Designing resources that transparently communicate the capabilities and limitations of open-domain and task-oriented chatbots
could help different stakeholders figure out what types of technology they need and negotiate their needs with each other.
In addition, it'll be beneficial to create opportunities to hear multiple stakeholders' perspectives before developing a system for public health intervention.
This opportunity will help developers better recognize what tensions might exist among different stakeholders
and what misconceptions they might have toward the system.
Thank you so much for listening. Please feel free to contact me if you have any questions about this work.