Hello. I am Ekaterina Svikhnushina,
a PhD student at HCI Group at EPFL.
In this presentation I will talk about
the PEACE model which describes the key
social qualities
of conversational chatbots. I will start
by briefly discussing the background and
motivation for our work.
Then, I will explain how we developed and
evaluated the PEACE model.
And finally, conclude with the resulting
implications.
To begin, let's define what a chatbot is.
By chatbot we understand an artificial
intelligence-empowered computer program
that interacts with the user via
natural language.
Chatbots are broadly classified in two
types.
Task-oriented chatbots help users
achieve a specific goal, for example,
order a pizza.
On the contrary, open-domain chatbots do
not pursue any functional aim
and try to become virtual companion to
their users by engaging in small talk
with them
and socializing. These chatbots are yet
minimal on the market.
Interestingly, human role metaphor of a
chatbot
triggers social behavior from the users
even when they interact with the
utilitarian task-oriented agents.
In some domains, such as online shopping,
the percentage of social user input to
these agents
was shown to reach remarkable 80 percent.
Clearly, social skills are becoming
imperative for chatbots.
Many researchers maintain that future
agents should be developed following the
hybrid framework
combining both task-oriented and social
chatbots' properties.
However, previous work doesn't provide a
consolidated view
on what social qualities are most
crucial for chatbots to ensure
enhanced user experience leading to
technology adoption.
To address this question we draw
inspiration from existing user
experience and chatbot adoption studies.
Specifically, we thoroughly reviewed
works on user experience and
expectations of chatbots
to create a model of their most
prominent social qualities.
And then we assessed their influence on
user intention to interact with the
chatbot
using psychometric techniques, typically
employed in adoption studies.
Based on previous research on
Expectation-Confirmation Theory and
technology adoption,
we decided to structure the constructs
of our model into three high-level
layers:
user attitudes, expected chatbot
qualities,
and behavioral intentions.
User attitudes refer to a set of their
beliefs and behaviors toward a
particular thing or event.
This layer consists of several
constructs.
Openness to technologies describes user's
personal disposition towards
new IT products and services.
Vulnerability,
defined by social and monetary harms,
refers to users' concerns about their
security and privacy.
Finally, empathy propensity illustrates
user's disposition to understand and
respond adaptively to others' emotions.
For expected chatbot qualities we
highlighted the following constructs.
Politeness - chatbot's ability to
establish respectful and non-judgmental
behavior.
Entertainment - a promise of playful
interactions, diversity in chatbot's
responses,
sense of humor, and small talk.
Attentive curiosity -
chatbot's ability to adapt to the users,
anticipate their needs, and play a role
of an active listener.
And empathy - ability of an agent to
recognize and respond to user emotions
appropriately.
Lastly, behavioral intentions describe
users' intention to use the chatbot and
their desire to introduce a chatbot to
their friends.
According to the Theory of Planned
Behavior, these indicators can be a
strong predictor of
actual behavior. To evaluate our model we
form a set of hypotheses
about how various constructs relate to
each other.
We hypothesize that openness to
technologies and empathy propensity
have a positive effect on users
expectations.
At the same time, a sense of
vulnerability would influence them
negatively.
We also expect a significant positive
effect from users' expectations to their
intention to use
socially and emotionally aware chatbots.
To validate the conceptualized model
and the hypothesis we developed a
structured online survey where each
section of questions measure different
constructs of our model.
We used existing Toronto Empathy
Questionnaire scale
to measure users' empathy propensity and
devised other Likert-type question items
ourselves. We ran our survey on Amazon
Mechanical Turk platform,
requiring our participants to be based
in the US.
The main reason for that was to
eliminate the possibility of question
misunderstanding
caused by language-related issues.
Initially we collected 1000 of
responses and then
followed a number of filtering steps to
ensure the data quality.
The selected 536 workers adequately
represent
different demographic variables and
provide a reasonable sample
of laypeople users. Further, we conducted
data screening for all question items to
check that they satisfy Structural
Equation Modeling assumptions of
normality.
In the following analysis we first
verified that the relationship between
multiple observed variables and the
underlying constructs
exist by applying Confirmatory Factor
Analysis.
As suggested by the results, our model is
validated as satisfactory in terms of
internal consistency reliability and the
convergent and discriminant validity.
Next, we conducted path analysis in
structural equation modeling to examine
the causal relationship between the
variables with respect to our hypothesis.
The model fit indices surpassed the
recommended values
and the resultant path loadings and
R-squared coefficients
proved the hypothesis relating the
constructs and produced the final PEACE
model.
Let's have a look at it in the next
slide.
So, the PEACE model comprises the
essential social and emotional qualities
for conversational chatbots.
It also allows us to extract several
important implications.
First, it demonstrates that individual
differences in people's attitudes
play a considerable role in forming
their expectations about future chatbot
technology.
Therefore, chatbot designers and
developers should use this initial
knowledge about their users
to tailor user experience. Second,
politeness proved to be an essential
prerequisite for people to consider
engagement with social conversational
agents. Therefore,
it is critical for developers to
meticulously control the quality of the
training data for their conversational
models
and regulate unethical chatbots replies
by other means.
Third, in line with previous studies, the
PEACE model supports that entertainment
and humor demonstrate superior leading
effect on user adoption intention,
indicating that these qualities should
be provisioned as a part of the
chatbot's social intelligence.
Next, our findings suggest that users
would be willing to share their thoughts
and information with the agent
once the politeness requirement is
satisfied.
Thus, developers should incorporate
active listening and question-asking
capabilities into their chatbots.
Finally, our results prove chatbot's
empathy to be one of the decisive
factors
influencing users' desire to accept it. So,
it's vital for future agents to combine
both the ability to detect the user
emotion
accurately and generate an emotionally
appropriate response.
To conclude, our work presented an
overview of recent user experience
research devoted to conversational
chatbots.
Careful examination of the results from
related studies
led to the conceptualization and
validation of the PEACE model.
It captures essential social and
emotional qualities of chatbots,
predicting user desire to accept this
technology.
Future research should further validate
the PEACE model with other populations
and by employing alternative research
methods.
This could offer additional validity
assessment for the findings originating
from the self-report study
as well as account for potential changes
in user expectations
following their actual interaction
experience with the technology.
Furthermore, additional studies on
chatbot personality are needed
to evaluate its influence on user
adoption.
We presume that personality failed to
segregate as a standalone construct in
our analysis
as the traits defining one's
personality are considerably dispersed
across
the other four established constructs.
Thank you for your attention! And now I
would be happy to take
any questions.