hi i'm gabriel and i'm presenting keep it short a comparison of voice assistance response behavior
i did this work together with michael rietzler matt jones and enrico rukzio voice assistants
such as amazon's alexa or the google assistant became very popular in smartphones cars and most
importantly in the living room so they help us with simple tasks and retrieve information
for us they are designed to be interacted with in natural language and emulate human behavior
there's a lot of research on the human likeness and personification of voice assistance
such as the computer's are social actors paradigm that basically says that we apply social rules to
computers when they use human features such as voice output experiments showed also that voice
output affects the anthropomorphism even more than the physical appearance of computer systems
and that human-like speech improves the compliance and evaluation of computer systems
but we're also not quite there yet as there's research that shows that voice assistance um
are seen as more formal and less authentic than human conversation partners and often fail to meet
users expectations in practice to get a better understanding of those commercial voice assistants
we choose eight typical tasks and perform them with three most used voice assistants
and those are amazon's alexa apple siri and the google assistant so what we found is
that they always try to emulate human behavior by responding in full sentences
so based on this observation we wondered if this is the best approach to emulate human behavior
and as ben schneider means that in one of his recent articles that successful robots
will use features of machines and be more tool-like instead of using human features
so therefore wanted to explore different response styles to often use requests and
see how they are perceived by users so for queries where the user requests information
we started with the baseline which is in full sentences such as in commercial voice
assistants we then added version which contains only the absolutely necessary information that
was requested so when uh the participants or the user ask about the weather tomorrow
uh we'll just give them the information sunny five to minus three degrees
uh then we added a third version which is in between
those two others but it's more data oriented and not based on human grammar so this gives you
uh feedback about the request which was the weather tomorrow and then the information
those response styles are also similar
when using commands so this is where we don't request information from the assistant but want
the assistant to perform a task for us so here the minimal response is just a confirmation
so if you want music to be played back the music just starts and this is enough feedback
if you want to create a reminder the confirmation is only okay in the keyword you also get the
information of the request so if you want to play rafales ukulele song you get exactly this
information in the keyword and last but not least there's also the full sentence baseline
so with those three response styles we then created an online study
where we used the browser-based voice assistant that we implemented in javascript
our participants had to perform those eight tasks that we also used for analysis
and four of them were queries and four of them were commands each of those participants had to
do three times and they were answered in the three response styles in a randomized order
so we had 72 participants in the study between 18 and 67 years they all had prior experience with
voice assistants because we thought that it helps them to evaluate those response styles as they
are already used to have a voice assistance and interact with them on a daily basis as measures
we collected the response quality response behavior comprehensibility and efficiency we
have the affinity for technology questionnaire a brief measure of personality
and also some single items and the preference for response styles and some open feedback
so regarding the individual tasks you can find detailed results and discussion in the
paper but in this talk i want to focus more on the general preferences so here you can
see the percentages of which response style was most preferred by participants for all tasks
and you can see that for all but two of those tasks the keyword condition was
actually the most preferred this is most likely because this is an efficient response style
but also provides enough context and feedback if the assistant understood everything correct
so it's not only the necessary information or confirmation but also some feedback
in the news condition full sentences were most preferred um probably because this is close to
how a news reporter would report the news and how we are just used to to get the news and in the
home command condition the confirmation only was preferred so here the action that the
system performs uh already provides enough feedback for the participants
when we asked our participants how they see voice assistants
significantly more saw them as technical systems and not so much as social companions
and also the efficiency was slightly more important than human likeness to them
so for general observations we thought there's an overall trend towards shorter answers such
as the the keyword and the minimal response styles that it's good to not only include
the absolutely necessary information but also provide feedback either through the
action that the assistant performs or through actually saying this to the user these are that
the preferences are user and context specific especially in the open feedback so when users
are home alone they like to converse more with the assistant but not so much when others are around
and then also human likeness is a two-sided sword so if a assistant is very human-like there's also
human-like behavior and intelligence expected by the users so the implications for this design
of voice assistance we thought that short and well-defined tasks don't require long responses
as they are already provide enough feedback through the tasks that are performed these are
the responses to queries should be specific so this means that you
should listen to the user and don't just bin many requests to a generic response and also
it's important to make the voice assistant configurable or adaptive because there are users
that like to converse with the assistant but users that like to use this as a very efficient tool
so it's good to give both of those user types the assistance they want so that's the end of
my presentation please see the paper for more details and if you like to give the different
response styles a try you can find an interactive demonstration on my website thanks for listening