LAURENCE MORONEY: Welcome to the Googleplex.
This is an incredible place with lots of great stuff being
worked on every single day.
Before I worked here, I always wondered
what it would be like to come to the Googleplex,
meet up with a Googler, and have coffee with them,
and just chat about what they do, how they do it,
and why they do it.
And today, we're going to do exactly that.
Welcome to "Coffee with a Googler."
I'm Laurence Moroney.
And I'm here today chatting with Sunil, who's
from our Voice Actions team.
And he's going to be talking a lot about Voice Actions
and how they work in Android and some of the great things
that you can do as a developer with them.
So welcome, Sunil.
And thanks, for joining us.
SUNIL VEMURI: Thanks, for having me.
LAURENCE MORONEY: So tell us all about what you do.
SUNIL VEMURI: So I am a product manager on the Voice Actions
team.
And the main thing that I do is enable
Voice Actions for developers.
But let me first say, what is a voice action?
LAURENCE MORONEY: Yeah, can we have a demo?
SUNIL VEMURI: Yeah.
LAURENCE MORONEY: Oh, cool.
SUNIL VEMURI: This is the part in Google
where you can say, OK Google.
Navigate to San Francisco.
ELECTRONIC VOICE: Navigating to San Francisco.
SUNIL VEMURI: And then--
LAURENCE MORONEY: Cool.
SUNIL VEMURI: --you can get directions and things
like that.
But there are other things you can
do, like I can say, call Laurence or start my run.
All of these things are various voice actions
that Google supports right now.
LAURENCE MORONEY: OK.
Nice.
Now I know you've done a DevByte recently
explaining some of this, right?
SUNIL VEMURI: Yes, we have.
LAURENCE MORONEY: OK.
Let's run a clip from that DevByte now.
[VIDEO PLAYBACK]
-I love listening to the radio on my commute.
And the TuneIn app gives me great choices.
OK Google.
Play music on TuneIn.
-Which genre would you like to listen to?
-Classical.
-Playing classical.
[END PLAYBACK]
LAURENCE MORONEY: Pretty neat stuff.
SUNIL VEMURI: Thank you.
LAURENCE MORONEY: Now, one of the things I've always
been curious about was-- because I remember in the past, when
I've been doing voice interaction stuff,
I used to have to train the machine.
And I'd sit with a microphone close my mouth,
and I'd read weird sentences, so it
could understand all the different things
that I was saying.
But now, I can just pick up my phone and say, OK Google,
navigate to San Francisco.
What's happened?
How has that come about?
SUNIL VEMURI: Yeah.
That's been a really awesome progress in the speech field.
What really has changed is the amount of data that we've had.
So in the early days, when we only had a little bit of data,
it only worked for a few circumstances.
But later on, as we got more and more data,
it got better and better and better.
So take, for example, in 2013, we had about a 23% word error
rate.
And that's how we measure the quality of speech.
So 23% of the words were wrong.
And that's not very good.
LAURENCE MORONEY: Yeah.
SUNIL VEMURI: At I/O, we recently announced
it's down to 8%.
LAURENCE MORONEY: Wow.
SUNIL VEMURI: And that makes a big difference,
especially when you're talking about applications like this.
And in many ways, it has shown that voice is really possible
and is a viable candidate for a lot of different interactions
right now, because the error rates have gotten down
so significantly.
LAURENCE MORONEY: Right.
And even like earlier, you said, San Francisco,
which are technically Spanish words, right?
But it recognized them as the place.
And I thought that was really cool.
SUNIL VEMURI: Yeah.
I mean, there's a lot of treatment that
goes into it that tries to understand,
what is the actual right interpretation of this sentence
in this context, as well?
So that's a lot of the work that goes on in this team that's
helped to really make it a better experience for users
and now developers.
LAURENCE MORONEY: Yeah.
And I have a foreign accent, and it recognizes me.
SUNIL VEMURI: Yeah.
LAURENCE MORONEY: Right?
And it's like, how is it that you're
able to come up with-- to be able to recognize
different voices, not just the classic monotone,
straight English voice type of thing?
What type of work has been done for that?
SUNIL VEMURI: Yeah.
You know, back in the early days,
you had to be a white male between the ages of 25 and 40
for this thing to work.
And you had to be in a quiet room.
LAURENCE MORONEY: OK.
SUNIL VEMURI: So what really changed
is we started to include a much more diverse set of samples
in here, so a broader set of accents, a broader
set of speakers, older, younger speakers.
Even nowadays, it works really well
with children, which has long since been a difficult problem.
So as we included more and more of these samples
and a diverse set, it created this virtual cycle
where more people started to use it.
And we can improve the system with that.
LAURENCE MORONEY: Cool.
Cool.
Now developers-- we have a DevByte that we just
showed a moment ago-- now, how do developers
get started with doing this?
What kind of things can they do?
SUNIL VEMURI: So believe it or not, a lot of developers
are already voice-enabled.
And the way this has happened is we've leveraged the Android
intent system here.
So first thing we do is we analyze or we look at,
what did the user say?
And if they say a number of things
that we already recognize, like start a run, set an alarm,
play some music, Google has already
developed the language to recognize these things,
and we fire an intent.
Now, lots of applications out there
already support these intents.
LAURENCE MORONEY: OK.
SUNIL VEMURI: So we're already firing these things
to these applications that support these intents.
So many applications are already supported.
And this is what we call system actions.
LAURENCE MORONEY: OK.
SUNIL VEMURI: Google recognizes the language
and fires an intent over to an application.
And then the application is the one
that can give the response to the user.
And this, in many ways, is how we are driving traffic
to applications.
LAURENCE MORONEY: Oh, nice.
OK.
Now, the system intent-- system actions.
Sorry.
There's other types of actions?
SUNIL VEMURI: Yeah.
So Google has created a bunch of these things
to handle the common situations that we
see users use on phones, watches, and other devices.
But what if it's something that Google hasn't created?
That's where we introduce custom actions.
LAURENCE MORONEY: OK.
SUNIL VEMURI: This is where a developer can tell us
the language that they would like us to trigger on and then
send to their applications.
Take, for example, turn on the lights.
We don't do that right now.
Or Shazam, this song.
This isn't something that we would build by ourselves.
But developers have asked us for grammars like this.
And then, when they do that, we trigger the intent
that they asked for for their application.
So the custom actions APIs are the ones that
allow developers to do this.
LAURENCE MORONEY: Cool.
Cool.
Now, there's also the ability to interact over voice, right?
So follow up questions, that type of thing.
SUNIL VEMURI: Yes, exactly.
LAURENCE MORONEY: How does that work?
SUNIL VEMURI: So we talked about two
of these things, system and custom.
And this has to do with triggering.
Now, just imagine you're on the Google home screen,
and you say, OK Google, turn on the lights,
or whatever it might be.
But let's take that one query for example.
That can be ambiguous.
LAURENCE MORONEY: Right.
SUNIL VEMURI: Because the home automation application
might wonder, what lights?
Which room?
LAURENCE MORONEY: OK.
SUNIL VEMURI: Or if I said, unlock the door,
it might want to say, are you sure?
Because I wouldn't want you to unlock my door, would I?
LAURENCE MORONEY: Absolutely.
SUNIL VEMURI: Well, no.
You seem like a nice guy.
But, yeah.
LAURENCE MORONEY: Appearances can be deceptive.
SUNIL VEMURI: They can.
[LAUGHTER]
So what we have is another API, called Voice Interaction API.
And this is a new API in Android M that
allows for the follow-on questions that are very
typical in a voice dialogue.
LAURENCE MORONEY: OK.
SUNIL VEMURI: So in this case here,
a developer would still implement the custom action
to do something like turn on the lights.
But in their application, they would
ask the follow-up question.
And what happens here is, at the bottom of the screen,
we will maintain all the challenge and UI
for a voice interaction.
And at the top of the screen is the developer's UI.
So we split the screen with the developer.
So the developer only needs to care about the question
and what are the possible answers.
We'll take care of all the challenges
that go along with voice, including
managing the microphone, what if the user didn't say
exactly what was on the list.
All of these things are Google's responsibility.
So we've made it much simpler for a developer
to implement these types of voice interactions.
And for voice, it's particularly important,
because there's always ambiguity,
and there's always a lack of clarity.
Well, I shouldn't say always, but there's often
lack of clarity.
And we want to give developers the chance
to build very interesting and neat applications,
by dialoguing with their users.
LAURENCE MORONEY: So help me to understand that.
So if I did something like, OK Google, turn on the lights,
and so then this API would say, do you
want the bedroom lights turned on?
Or do you want the kitchen lights turned on?
That type of thing?
SUNIL VEMURI: Well, it's the application developer
who would ask that question.
They know what's the best way to respond to that.
LAURENCE MORONEY: OK.
SUNIL VEMURI: So I mean, in a home automation application,
you might label your lights as Laurence's Lights,
or Laurence's Office, or something like that.
So that's where the application can ask the question,
which room would you like?
Or it could say something like, would you
like Laurence's Office?
Or Laurence's Room?
LAURENCE MORONEY: Got it.
Got it.
SUNIL VEMURI: It's totally up to the developer.
You know, we want the developer to have
the control of that dialogue, to ask the questions that
make sense for them, and also put up
the nice UI that they might want to put up with that, as well.
Like they might have an animation
or a beautiful display of different rooms on the screen.
You know, we really want our developers
to unleash their imagination with this.
And so that's why we wanted to take the burden of handling
the voice, and let them put their creativity on the device
at the same time.
LAURENCE MORONEY: Now, are there any examples
of this that are out there that are working right now?
SUNIL VEMURI: So right now, at I/O, we showed the TuneIn
to an implementation like this.
So they did the following.
OK Google, play some music on TuneIn.
LAURENCE MORONEY: OK.
SUNIL VEMURI: And then what TuneIn
would ask is, what genre?
LAURENCE MORONEY: I see.
SUNIL VEMURI: Because the music wasn't clear,
or what kind of music wasn't clear.
So they ask, which genre?
They present a list of the TuneIn channels.
User can select that, and then the music plays.
LAURENCE MORONEY: Cool.
SUNIL VEMURI: So that's one example
that we've already seen.
LAURENCE MORONEY: Nice.
SUNIL VEMURI: And we have more coming, too.
LAURENCE MORONEY: Nice.
And I assume the conversation can take multiple steps, right?
It's not just a single question.
There can be further clarifying ones?
SUNIL VEMURI: Absolutely.
And we're looking forward to the day
when we can order a pizza with this thing,
with toppings, and sizes, and things like that.
LAURENCE MORONEY: Yeah.
I'm sure it's not too far off.
SUNIL VEMURI: I hope so.
LAURENCE MORONEY: So thanks, Sunil.
That was awesome.
And thanks, everybody, for watching this episode
of "Coffee With a Googler."
If you're interested in more about the Voice Interactions
API and all the kind of things that you
can do with Voice Interaction on Android,
check out the links underneath the video
here in the description.
And if you have any questions for me
or any questions for Sunil, just leave them
in the comments below.
Thank you.