Hi everyone, I'm Cataldo Musto from University of Bari, and in this presentation I'll introduce
you our idea of building Queryable User Profiles, and in particular I will show you how we have
tried to move forward this idea by introducing conversational agents in a platform for Holistic
User Modeling.
So, in this presentation we will mainly talk about user modeling, and, in particular, we will
talk about user modeling in the 'egosystem' era.
What are we talking about? The main idea behind the vision of the so-called 'egosystem' is that a profile
of a person, or of a user, can be built by acquiring and aggregating personal data that
are spread on social networks, such as YouTube, Facebook, Twitter, LinkedIn and so on.
In our previous work, that has been recently published on User-Modeling and User-Adapted
Interaction journal on this year, we have tried to introduce our 'holistic user modeling'
strategy, that has a good overlap with the ideas of the so-called 'egosystem', because
we extracted user's personal data from several sources, such as social media and personal
devices, such as smartphones and smartwatches and so on, we processed these data and we
mapped these data to a set of eight different facets (that you can see in this slide), so
Demographics, Interests, Affects, Social Relations, Physical States, and so on, that represent - I mean
- the main characteristics that can describe a person.
This is just a conceptual model, but we have also built a platform that allows to concretely
build these holistic user models that is called Myrror.
Here you can see, in this slide and in the following ones, you can see some of the screenshots
of this platform.
Each user, when she connects to the system, she can connect her own personal identities
to the system and she can enable the extraction of the data.
Once the user has enabled the extraction of the data, personal data are extracted and
processed by Myrror, and they are used to build this holistic user model of the user.
Once the profile is built, it is made available through this kind of dashboard and the user
can check the personal information that are stored in the profile by clicking on the different facets
that she can see on the left side of the screen.
This dashboard includes checking the trends of mood over time, checking the amount of
activity the user has done day by day, checking the places visited based on the geo-tags shared
on social media or the personal interests.
So, this is our starting point.
We have a conceptual model for building such a rich and comprehensive representation of
the person, and we also have a platform that allows to concretely build these user models.
But there is also room for improvement of the current strategy, because in our opinion,
first, the overall transparency may be improved because we think that the user in every moment
aware of the information which is encoded in her own profile, but the interaction with
a dashboard like the one that I have just shown you can be difficult - especially for
beginner users - so it can be difficult for example to check what are the current interests that
are encoded in the profile.
Similarly, with the current implementation of the system it is also difficult for the
user to get some 'valuable' insight by checking the information that is stored in the profile.
So, for example, we have the information about the amount of time the user has slept, but
we can't say the user whether she has slept enough or not.
So, what we have tried to do in this work is improve the way people can access to the information
that is encoded in their holistic user profiles.
Our idea is to exploit conversational agents for this goal.
Conversational agents have been popularized by Alexa, Siri, Google Now, etc. and they
spread a very intuitive idea because they acquire a natural language request, they understand
the request and they provide an answer - in natural language, typically - and this is
done for very general purpose services, such as listening to some music.
Our idea is to replicate this strategy in user profiling, so we want to make a first
step towards queryable user profiles.
Our idea is that a user should express her information need in natural language and the
platform should be provided with the ability of answering to the question that the user
has expressed in natural language, as well.
In this way, in our opinion, we can make easier to access to the information that is encoded
in the profiles, because this kind of interaction is much more 'natural' and simpler to use,
by also beginners users as well.
So, the main contribution of this work is represented by MyrrorBot.
MyrrorBot is a conversational agent that is built on top of Myrror, so our platform
for holistic user modeling.
On the left side of this figure you can see the general workflow and I just want to highlight
you the main features that we have implemented.
The first one is represented by personalized services, because we allow the user to access
to services similar to those that are available in Alexa, so music recommendation, news recommendation,
recipe recommendation and so on.
But, differently from Alexa and other similar devices, the user can also inspect - in order
to have more transparency - the information that is encoded in the profile.
This is the focus of our presentation, actually.
In order to trigger this kind of usage of the system, as you can see from the workflow,
the core of the whole system is the Intent Regognizer.
The goal of the Intent Recognizer is to implement a Natural Language Understanding strategy
because the goal of this module is to acquire the request of the user and to understand
the request the has expressed.
The output of this module is the so-called 'Intent', which is a representation of the
informative need of the user.
If we refer again to the previous example concerning Amazon Alexa, if the user asks
"play Africa by Toto', the intent which is recognized by the Intent Recognizer module
which is implemented in Alexa is that the user wants to play some music, so the goal
of the Intent Recognizer is to understand what the user wants when she formulates a
particular request.
In our case, we have implemented this module by exploiting Google Dialogflow.
So, the main question we have to answer now is"What kind of questions can we handle?"
through our system?
Well, as Amazon Alexa recognizes intents that are directly related to the services she can
provide, so MyrrorBot has intents that are directly mapped to the facets of the holistic
user model.
So, for each facet of the holistic user model, as those I have previously showed you, we
have one intent in MyrrorBot.
To be more precise, we have two intents for each facet of the holistic user model because
users can express simple 'Informative' queries, for each of the intents, for example:"How
much did I sleep today?", but she can also express an informative need that is oriented
to get some insights from the platform, so, "Did I sleep enough today?".
So, we have 16 intents in total, that needs to be encoded in our conversational agent.
The intent recognizer is basically a machine learning module, so in order to allow this
module to correctly recognize all the possible way the user can express her own informative
need, we have to train this module.
So, for each intent that we have designed for the intent recognizer, we defined around
20-30 sentences that are used to feed and train the algorithm.
In particular, we have tried to encode all the utterances that can be used to express
a particular information need.
In the figure you can see some of the expressions we have used to train our "Behaviors" intent.
I don't have time to provide more details of this strategy, but at a very high level
idea of this module is to carry out the task in a way similar like this, so, we have
questions of the user and these questions are correctly classified in the intents they
basically refer.
So, if the user asks "tell me about my personality traits", the module understands that the user
is looking for some information about her psycholocial traits.
Finally, the last module of the workflow is the Generator.
The goal of the Generator is to produce the output that the user receives, based on the
intent that was previously recognized.
Here, on the left side of the figure, you can see some of the outputs and some example
of the interaction with the system, so if the user asks for the age, the system replies
with the age, and so on and so forth.
You can see the examples there.
What I want to emphasize here is that when the user expresses an 'informative' query,
there is no computation which is carried out.
So, the intent is recognized, we query the holistic user model, we receive the information from
the holistic user model and we use this information to fill in a natural language template which
is presented to the user.
Conversely, when the user expressed an insight-based intent, there is some computation to be carried
out.
In particular, we make for example some comparison between the values that are stored in the
user profile and some 'threshold' values, so for example the average amount of time
to be slept per night.
So, as a recap, this is the pipeline that we implemented.
We basically carry out these steps: we acquire an holistic user model, we recognize the user
request which is formulated in natural language and we generate a natural language answer
related to user informative need.
In the experimental evaluation we tried to answer basically to two different questions.
The first one is a more qualitative question, since we aimed to understand the opinion of
the users in terms of usability, ease of use and learning curve.
The second one is much more a 'quantitative' analysis because we evaluated the time needed
through a conversational interface with respect to a classical web-based interface to complete
some tasks.
So, we calculate - on eight different tasks related to checking the information stored
in the user models - and we calculated the time needed to complete this task.
To this end, we carried out a user study with 76 subjects, with a low experience with
chatbots.
For the first research question we defined a questionnaire that was based on the System
Usability Survey, while for the second one, as I said, we defined some tasks and
we calculated the time in second.
As for the first research question, as you can see, in the columns you have the metrics
as well as the questions and the percentage of users who completely agree or agree with
the statement.
As you can see, most of the users had a positive opinion of the system, so they liked the general
idea behind the system.
As for the second research question, which is more interesting, you can see that for
each task - generally speaking we had one task for each facet of the user profile - we
had a significant reduction of the task completion time, so the adoption of a conversational
interface leads to a reduction of the time needed to check the information which is stored
in the profile.
This is confirmed also for the other tasks to be carried out, so users had generally
a positive opinion and they can access to the information in a very effective way.
In general, we had a 40.3% of reduction of the time needed to complete the tasks, 7 seconds
on average, which is quite good.
So, as a recap, through this paper we made a first step towards queryable user profiles,
we started from Myrror - a platform for holistic user modeling - implemented through a classical
web interface, and we provided as a contribution MyrrorBot, a conversational agent built on
top of Myrror.
As we have shown in our experiments, our system had a good impact on users, and we had also
a significant reduction of task completion time.
We have a demo available, if you want to try or to interact with the system, and - as future
work - hopefully at UMAP 2021 - we will also evaluate personalized services, we will try
to integrate also different interaction methods, such as social robots, and we will also try
to train more sophisticated intents, such as those connected to personal lifestyle.
That's it.
Thank you.