Hi everyone, my name is Elliot Mitchell and I'll be presenting research for my dissertation
at Columbia. I'm currently a data scientist at geislinger.
And this work is titled examining AI methods for micro coaching dialogues.
Jumping in. In the face of the growing burden of chronic diseases, health coaching interventions
are a promising approach. Conversational interaction, for example, through chatbots is
one particularly well suited approach to designing health coaching interventions. There have been
recent advances in conversational AI driven by advancing NLP methods and openly available corpora
to produce more dynamic and flexible chatbots. Until recently, though, chatbots and health
have been predominantly scripted or rule based, which while potentially effective,
can be limited and result in a repetitive user experience. There are open questions
and pursuing more advanced approaches in health, as well as a lack of openly available corpora.
Taking a look at common architectures for chatbots
many approaches separate components for natural language understanding, parsing and understanding
the users utterance from dialogue management, which is determining how the system should respond
to natural language generation, which is actually constructing that natural language response.
Dialogue management is the focus of this work, particularly rule based or data driven approaches,
for example, with data driven with reinforcement learning, where responses are selected to optimize
the overall effectiveness or flow of the conversation over time learning from examples.
So with that background out of the way, I'll next talk about the focus area for this research,
which is in health coaching. I'll then overview the four chatbots,
with different approaches to dialogue management being compared and the research,
followed by the evaluation methods and results and a brief discussion.
So taking a look at how health coaching typically unfolds, there's an initial coaching session,
followed by the client working on their own and then a potential follow up session. Now in this
interim time, the client may have questions about how to achieve their goals. And so, in prior work,
we argued that this represents a potential opportunity for automated approaches to coaching
through chatbots to offer in the moment feedback and advice potentially multiple times throughout
the day. And we call this idea of micro coaching. And here's a bit of an overview of how it might
look in the context of nutrition goals. Based on some initial description of a planned meal.
The micro coach would need to ask follow up questions to determine whether or not the meal
meets a goal to then offer positive reinforcement or offer suggestions to further refine the plan.
The focus of this work is on these initial components of understanding whether a
meal is consistent with the goal which itself presents considerable complexity.
As a bit of a prelude, and this is all covered in more detail in the paper. We engaged health
coaches to learn what are the kinds of questions that might be appropriate to ask and these kinds
of micro coaching scenarios, which are shown here as a relatively limited set of question types. In
addition, we selected three nutritional goals to use as sort of working examples or case studies
throughout this work. Now let's talk about the four different chatbots. So first, is the
scripted approach. And this is the simplest thing you could possibly do. For each goal always ask
the same questions in the same order. The other systems incorporate some level of intelligence
through natural language understanding with the system we designed called Food NLU.
Returning to this overview of Chatbot.
We designed food NLU to process user utterances and generate a set of possible responses.
More details on the system and evaluation are in the paper. But the key point is that it produces
a computational representation of the meal is a discrete set of follow up questions. So dialog
management can then choose the next appropriate action. So looking at the first example here,
rule based this uses if then logic, based on the presence of certain foods that are related to a
goal based on the attributes identified by food, and I'll you The second approach is data driven,
in this case, through reinforcement learning, and specifically Q learning to estimate the potential
value of asking a question in a given state. And so to give some intuition for how this works,
on the left is a sort of a common game scenario where you might see reinforcement learning,
you're trying to get to the star with the shortest path and micro coaching.
Through dialogue, we're trying to get to the answer about whether somebody meets their goal
with the shortest dialogue possible. And so the state is not where we are in the grid world,
but it's sort of what do we know about the meal and the actions are not moving through the grid
world, but they're asking different kinds of follow up questions that we learned from,
from the health coaches. So this is sort of the intuition here.
More details on the setup and evaluation of Q learning are also in the paper. But in general,
we created a corpus for learning through crowdsourcing of 300 total dialogues,
which we've also made openly available, open source on GitHub.
We did a validation experiment with simulated and also that crowd sourced data,
and found that reinforcement learning did result in shorter dialogues.
And also that it learns a logical next action when there's situations where
there's sort of a clear cut thing that you think it should be doing. So that's reinforcement
learning the last chatbot here is a control condition where it's just randomly choosing
from the possible follow up questions. So we would hope that this performs performs worst.
Now jumping into the evaluation methods and results, we wanted to compare
these different approaches to dialogue, management and the chatbots in terms of
the length of conversations they produced, as well as their perceived quality in terms of
the strategy employed by the coach, how natural the conversations felt, and how
coherent the coach's responses were, as well as the perceived user experience. And so to do this,
we crowdsource additional dialogues for unseen meals, and showed them to other crowd workers
to evaluate these dialogues in pairwise comparisons for each of those questions.
Looking at the results, we found that reinforcement learning did indeed result in
significantly shorter conversations than the other conditions. So that was its intention and
examining the quality of conversations, the results were somewhat mixed.
Surprisingly, the scripted dialogues, which were sort of the simplest, least intelligent approach
performed pretty well on these quality ratings, including strategy. But there's
an important caveat that because there was no intelligence involved, they didn't always
collect the necessary information to proceed with the following steps of micro coaching.
So there's some trade offs there. And performance varied by conditions where
sometimes reinforcement learning did seem to to win out in these in these pairwise comparisons.
To dig into this a little bit more, we looked at the
overall length and so we found that shorter dialogues did
end up being perceived as more natural but counter
intuitively longer dialogues were perceived as having a better question asking strategy.
There was no difference in perceived user experience between the groups.
So what does this all mean? The reinforcement learning?
Dialog management chatbot did succeed in leading to shorter dialogues. So that is, in one hand
successful but shorter did not necessarily equate with higher quality. It's possible that the
reinforcement learning question asking strategy in order was perceived as counterintuitive or
illogical. And there's also some potential trade off between how comprehensive dialogues felt,
for example, a longer dialogue might be perceived as more comprehensive
versus a shorter, more efficient one. It's also potential that these
differences would play out differently with one off versus extended use.
We do contribute a corpus here for open use from others,
though there are some important limitations and caveats to this work as well.
With that, I want to thank you all for listening. Thank the collaborators
and funding sources and please reach out if you have any questions