PETER HODGSON: I'm Pete Hodgson.
I'm an interaction designer working
on the Google Assistant.
Over the next few minutes, I'm going
to give you a brief overview of one of the key topics core
to conversation design.
That's conversational repair.
People historically don't much like voice interfaces
because of the feeling it gives them
when the conversation takes a misstep.
Research shows that 60% of all users
feel either angry or confused, or awkward and self-conscious,
in general, because they feel like they did something wrong.
After all, our instinct is to assume
that the other party in the conversation
is being cooperative.
So it must be us that's doing something wrong.
The good news is that 69% of the same group
are likely to give the voice interface another go even when
something has just gone wrong.
So if we can make a conversation work well
when it takes a misstep, then there's
a great opportunity to build user trust and confidence
in the assistant.
It's tempting to describe these misstep situations
as error cases, which suggests that the user has
done something wrong.
But it's important to remember that the users know
what they want.
There's no such thing as a voice query without an intent.
For example, the user might be being ambiguous.
Perhaps their intent is unspoken,
and they've implied an intent, which we haven't covered
in the recognition grammars.
Maybe they're just poking around and trying things out.
Remember, discovery is an intent.
Poking around is an intent.
And figuring out the boundaries of a piece of technology
is also an intent.
But mostly, people are being co-operative
and want to do something, even if they didn't say so
explicitly.
And it's our job to figure out what
they're trying to do, and to move the conversation forward.
Just because their query doesn't match our expectations,
doesn't mean it's an error.
For example, let's talk about a no input state.
When the assistant prompts for a reply
it opens the microphone for a short period of time,
listening for a response.
If there's no spoken input detected
within a number of seconds, the assistant
closes the microphone again and will re-prompt the user.
It's easy to label this situation
as a single type of error based on a technical description.
But there are many reasons why the user might not
reply to a prompt before the microphone is closed.
They might have been distracted or changed their minds.
Or perhaps the options presented don't match their expectations.
Or they simply hesitated too long.
In each of these cases, the assistant
might say something slightly different,
depending on the context.
So it's important to consider the possible context
of every repair case, since they can change the language
and structure of re-prompts.
Conversational repair is a big topic.
And we can't cover everything in this short video.
So in the next few minutes I just want
to focus on three main types of repair cases.
First, there's no match inputs.
This is when the assistant has heard something
that doesn't match the recognition
grammars for a specific state.
Then there's no inputs, as I just
described, when the assistant has prompted for a response,
but heard nothing back from the user.
And then there's disambiguation, where the system
has a confident but partial match
and needs more specific information to progress.
So here's an example conversation with these three
repair cases shown.
[INAUDIBLE] just sending a message.
Let's run through it.
SPEAKER 1: Hey, Google.
Send a message.
GOOGLE ASSISTANT: Sure.
Who do you want to message?
SPEAKER 1: Hey, Hon.
Can you turn down the TV, please?
GOOGLE ASSISTANT: Who do you want to message?
SPEAKER 1: David.
GOOGLE ASSISTANT: Which David?
Is it David Singh, David J, or David Brown?
SPEAKER 1: David Brown.
PETER HODGSON: The first misstep is
when the Assistant has the mic open,
and the grammars are listening for a specific contact name.
GOOGLE ASSISTANT: Sure.
Who do you want to message?
SPEAKER 1: Hey, Hon.
Can you turn down the TV, please?
GOOGLE ASSISTANT: Who do you want to message?
PETER HODGSON: This is a no match case.
Since it's the first time in this conversation
that the assistant hasn't understood the user,
we will try the lightest touch repair in much the same way
as we would a natural human to human conversation.
It's the equivalent of saying, huh, say that again.
We call this a rapid re-prompt.
The second case is when the assistant hits a disambiguation
state.
GOOGLE ASSISTANT: Who do you want to message?
SPEAKER 1: David.
GOOGLE ASSISTANT: Which David?
PETER HODGSON: It has a confident match,
but it's ambiguous.
The input matches more than one contact name,
so prompts for more specific information.
As this is the first repair in a disambiguation,
we might also treat it quite lightly.
After all, the user knows their contacts and who they meant.
The assistant just needs a bit more information.
And the third repair is a no input time out.
After the last prompt, the user hasn't said anything.
Or the assistant hasn't picked up the user's speech.
They may have left the room or missed the question.
But the system is still in a disambiguation state.
So the strategy here is to help the user make
a choice by escalating the detail in the prompt.
In this case, it's by presenting the specific Davids
from [INAUDIBLE] contacts list.
GOOGLE ASSISTANT: Is it David Singh, David J, or David Brown?
SPEAKER 1: David Brown.
PETER HODGSON: So here are those cases again.
In a multi-step conversation, every time
the assistant prompts for a response,
the conversation designer needs to consider all
of these possible repair cases.
The goal is to always move the conversation forward
as naturally as possible.
But we can't keep reprompting forever.
So we've got to consider the maximum number of attempts that
are reasonable in each conversation,
and provide ways out of the conversation
when we reach that limit.
And remember, if we have to keep on asking,
then the chances are we've got something wrong.
The user is most probably trying to be cooperative.
It's up to us to help them out.
So depending on each case, the conversation designer
and engineers need to decide the maximum number of tries
the assistant will prompt the user before ending
the conversation.
And at that point, how it will end the conversation.
SPEAKER 1: OK, Google.
Book a ticket to the ISS.
GOOGLE ASSISTANT: Sure.
Which carrier are you orbiting with?
SPEAKER 1: Honey, what carrier were we looking at?
GOOGLE ASSISTANT: Which carrier?
SPEAKER 1: Honey?
GOOGLE ASSISTANT: To make a booking,
I'll need to know which carrier you're orbiting with.
For example, NASA or SpaceX.
Or you can say cancel.
SPEAKER 1: What?
GOOGLE ASSISTANT: Since I'm having trouble understanding,
go ahead and tap a carrier name on the screen.
PETER HODGSON: So in this case, the Assistant
has tried three times to get the name of the carrier
without getting a recognizable answer.
That means the system had to manage three
no match cases, one after the other.
What you've heard is one example of how
you might construct a repair strategy for this case.
And we can break it down like this.
No match one is treated as a rapid re-prompt.
In no match two, we're escalating the detail
in the prompt to clarify the type of things we're
expecting the user to say.
We've also provided the user with an escape option,
in case they've changed their minds
or just want to stop the conversation.
By no match three, we could just end the conversation.
Or alternatively, we can give the users
the option to make an explicit input
by using the touch screen.
But remember, for all conversational repair
prevention is better than cure.
An iterative design process and formative user research
around the user's expected language
will inform your design of the most likely things
that users will say at any point.
And, as a result, a highly relevant set
of grammars for each step of the conversation
can significantly improve the user experience
and reduce the number of repair cases.
I've just touched on a couple of examples here.
And I want to reiterate, it's important to recognize
that every conversation will warrant its own strategy.
If you want to find out more about conversational repair,
I highly recommend you visit Google's new conversation
design guidelines.
And check out the other videos on conversation design
in this series.
Thanks for listening.
[MUSIC PLAYING]