Remember 1998?
This is when Search started.
It was a smart but simple algorithm which was trying to understand what -- our goal
from that moment was trying to understand what the user wants with things which he is
typing or she is typing in that box and try to find the best link for that.
Things has changed a lot since then, but our mission has not changed.
The mission remains the same: Organize the world's information and make it universally
accessible to everybody and in a useful way.
So we have expanded the definition of the boundaries of this mission but the mission
remains the same.
In fact, by 2020, there is going to be 5 billion mobile users.
Every day, there are more than 3 million new people getting connected to smartphones.
So when we are trying to build the future, we have to really take this into consideration
and design, in this case, the Assistant to really be with you on your phone, on these
smartphones for the future.
So let's go directly to, again, the Assistant.
I said conversation is a big part of that, and the reason we think of conversation as
a core element here is that because this is the most universal interface.
Everyone can do conversation.
Everyone can talk natural language.
From the moment that we are a young kid with -- kids can use words and try to say what
they mean, to the people who might be very old.
So it's basically independent of education, race, literacy, and age.
You can just use natural language and try to ask what you need.
So that's why conversation is very fundamental here.
So we also added an extra sauce on top of the conversation, which is personality, to
make it more delightful, fun, more engaging, but also it makes it more acceptable when
there are failures.
Technology is not yet at the level at which you can have a full conversation in which
everything is understood, so this is why personality is helping a lot here.
And what we are very excited is that these conversational experiences and this conversational
UI are really scaled -- can be scaled very easily.
So we have launched Google Assistant on multiple devices already.
I will show you on phone and later we will have demos of -- demos of Android Wear also,
but there is Google Home, but there is many more devices that we want to get connected.
And we also want to make sure that these natural interactions with this natural speaking, you
can also get connected to the apps and services and other types of -- and other types of -- apps
and services and devices that you might have.
So I personally am so excited about working on this team.
The reason is that because I feel that we are in the business of saving time for users,
and this is very important for me, to feel that we are doing something which basically
frees time from people so that they can spend time on things which matter really in their
life.
So with no more slides, let's move to the more risky part of my talk, which is going
to be fun for you, but -- So these are all live demos.
I'm going to show you different aspects of the Assistant, so what you will see is many
of -- [ Laughter ]
Yeah.
My password is "G."
You all know it now and I have to take -- [ Laughter ]
So I will be showing different types of things that you can do with the Assistant
today, but also, I will show you some examples of the work which has not -- is not yet live
and launched but we have them in our experimental labs in Google.
So let's get started.
Hi, how are you doing, Google?
Great, thanks.
What can I do for you?
Can you please tell me how is the weather tomorrow in Paris?
It will be sunny, with a high of 17 and a low of 5 degrees Celsius.
So note that I'm rather using language and longer sentences and not
like key words how we were in the search boxes.
Let's ask another question.
How can I get to Manchester by public transport?
From Grove Mill Lane, take 500 towards Watford departing in 41 minutes.
Get off at Beechen Grove North, Stop C, walk to Watford Junction and transfer to London
Midland.
Very good.
So we want to basically answer all the types of questions that users are asking us.
Those are like day-to-day questions, but the range of questions that people are asking
is really big.
So, for example, one type of more longer questions could be something like: Show me list of rides
in Europa-Park.
Okay.
This went wrong.
Here's a matching video.
Now, here you also understand that this real and that speech recognition
failed here.
List of rides in Europa-Park.
Here are the top search results --
Okay.
This is going wrong.
-- starting with -- What I wanted to show you
is -- let's change that a little bit.
What is the height restriction of Blue Fire?
You must be at least 1.31 meters tall to ride Blue Fire.
So Blue Fire is actually a ride inside an attraction park, and we actually
have to even try to answer what is the height restriction of such a thing in an attraction
park.
So it's really like -- the main -- the reason I'm showing you an example is that the range
of questions is really big and we want to answer all those types of questions.
So let's go with a more newer ones type of a more complex type of questions that we are
becoming capable of answering.
Something like: What is the name of the movie where Tom Cruise acts in it and he plays pool
and, while playing pool, he dances?
[ Laughter ] : Here is some information about "The
Color of Money."
Former pool hustler Felson decides he wants to return to the game by taking a pupil --
So now let me show you what type of experiences you can have if you
connect these types of voice interactions with other types of services which we could
have.
For example, translate.
Be my Vietnamese translator.
Understood.
I will translate all your following messages to Vietnamese.
To stop, just say "Stop translation."
Hey, I have a problem.
Can you help me?
lost my luggage in the
train.
Stop translation.
So you can -- Okay.
you can see how useful this could be in some important scenarios
like this one.
This happened to one of my friends, so that's why I used that particular example.
So another type of connection could be, for example, something like: I want to be on the
top of Eiffel Tower.
Can you bring me there now?
Opening Street View.
So you can see how powerful experiences you get from natural language
and voice to the actual applications that you would have done in some other case.
In this case, Street View.
Okay.
So this was about answering and things about public information, but the Assistant is my
personal assistant.
It should answer things which matters to me which are not necessarily public.
For example, it can answer things like: When is my flight back to Zurich?
Swiss Flight 325 from IHR to ZRH leaves at 5:10 p.m. tomorrow.
How is my team doing?
[ Laughter ] Barcelona are first in La Liga with
a total of 84 points.
[ Laughter ] Now you know what's my
team.
There isn't -- the way, actually, the Assistant knows my team is: What is my favorite team?
You told me that you liked Barcelona.
I explicitly actually told the Assistant.
And this is a new type of interactions where you can actually teach things to the Assistant,
it remembers for you, and you can personalize and improve the Assistant for you.
Now, let me show you some other examples of personal aspects when you marriage this also
with the vision and photo and object recognition and the pictures.
Something like: Show me pictures of mountain -- sorry.
Show me my pictures of mountains from last month.
Take a look at these pictures from your Google Photos.
So a few weeks back, I was here, and now I have this picture and
I have a question about it because I don't remember the name of this mountain.
Oops.
So sorry.
Show me pictures of mountains from last month.
These pictures should -- Show me my pictures of
mountains from last month.
I found this in your Google Photos.
What is the name of this mountain and how tall is it?
Jungfrau is 4,158 meters tall.
So that's exactly detecting what's in the pictures and trying to combine
this, again, with conversation.
Let me show you another one.
Show me pictures of my nephew and me drinking tea.
Here's what I found in your Google Photos.
Isn't he very cute?
[ Laughter ] So -- so note that I did
not tag anything on these pictures.
It's all automatic true machine learning, multiple machine learning, natural language
understanding, speech recognition, and vision connected to each other.
Let me now move to the next part of my demos, which is about context.
Humans, when they talk to each other, they always use context.
This is how we understand each other.
We don't repeat things.
We use the context all the time.
So let's -- let's show, as an example, what does that mean.
Show me pictures of Thomas.
Showing pictures that match -- If I don't provide any
context, this is the best guess one could do, maybe.
Thomas the Train.
Now, if I say just before that: Bayern Munich team roster --
The roster for FC Bayern Munich includes Robert Lewandowski, Arjen Robben,
Thomas Muller, and others.
So now if I ask the same question: Pictures of Thomas --
Pictures of Thomas Muller.
This time the Assistant is actually understanding the context and
it understands that Thomas is actually that person, that football player.
Bad that Carlo is not here because this was his team.
[ Laughter ] So let's -- let's know
-- let's show you another type of context, which is how conversation can happen, like
follow-on types of questions.
We can do things like: Where is the Empire State Building?
The address for Empire State Building is Empire State Building, 350 Fifth Avenue,
New York, New York 1- -- I want to see pictures.
Pictures of the Empire State Building.
Sure.
How tall is it?
The Empire State Building is 381 meters tall.
Who built it?
The Empire State Building was designed by William F. Lamb, Yasuo Matsui, and Gregory
Johnson.
When?
Construction for the Empire State Building --
So you -- -- started in March 17th, 1930.
So you can see that I have multiple follow-on questions, and at each
stage, the Assistant understands exactly what I mean by using the context.
It's exactly like the same way a human would do.
We can try to be -- it understands that "when" in this context means when was the Empire
State Building built.
So we don't need to repeat.
We can talk more naturally, and this conversation is going to be easier to be done that way.
So context can be different types.
For example, location is another context which should also these ambiguous things.
If I talk about "spa" at this moment, because we are in this hotel we would think of the
spa in this hotel, so let's try that, too.
Show me pictures of the spa.
These pictures should match.
So these are actually the pictures of the spa in our hotel, The Grove.
So, again, this is very important, to use context everywhere possible to make these
interactions to be easier.
As my last demo, I want to show you that we also have -- I mean, you meant -- you noticed
that speech recognition failed one time or two times.
Now, I want to show you that we've actually made lots of improvement on trying to improve
speech recognition in a noisy environment.
For that, I will need your help, so at some point, I'm going to ask you that all of you
make as much noise as you can.
Be creative.
Shout.
Scream.
Clap hands.
Whatever.
And I will ask also that my mic gets caught and I will ask a question like: When is Barcelona
playing next?
It might fail or it might work.
We can -- we can see.
But let's finish it with that.
So please be creative and make as much noise as you can.
You should try to make me fail.
Thank you.