SPEAKER 1: We're here at Google I/O '19, and I'm Jessica.
I'm standing right in front of the Google Assistant Sandbox,
where there's tons of cool demos and people we could chat with.
Let's go inside.
We're inside the now, and we're here with [INAUDIBLE]..
[INAUDIBLE],, tell me about Google Assistant.
SPEAKER 2: Hi.
So the Google Assistant is an AI-powered digital assistant
that is kind of your own little assistant
with the power of Google.
And the purpose of it is to help users get things done.
SPEAKER 1: That's awesome.
So how can users interact with the Google Assistant?
On what mediums?
SPEAKER 2: So right now, the Google Assistant
is available across over 1 billion devices
and over 1,500 types of devices.
So this is everything from voice-activated speakers,
smart displays, wearables, headphones.
SPEAKER 1: Oh my gosh, that's super exciting.
So here in the Sand Dome today that we have,
what are we showing off?
SPEAKER 2: So in this space, we're
showing different device types that developers
can build for-- anything from voice-only
to voice and screens.
But around the entire sandbox, we're
actually showing new integrations
for everything for Android developers, web and video
developers, device developers, and
Assistant-specific developers.
SPEAKER 1: Oh my gosh, that's super cool.
Thank you so much, [INAUDIBLE],, for hanging out with us.
Really appreciate it.
We're here you in the App Actions
section of the Sand Dome, and we're here with [INAUDIBLE]..
[INAUDIBLE],, what are app actions?
SPEAKER 3: App Actions let you shortcut your users
into different parts of your Android app
using the power of the Google Assistant.
SPEAKER 1: Oh my gosh, that sounds so exciting.
So as an Android developer, what do I
have to do in order to connect my Android
app to Google Assistant?
SPEAKER 3: Yeah, so the steps are really simple.
So first, Google has these things called built-in intents.
And what they do is they take care
of all the natural language understanding for you,
and they'll extract different parameters on your behalf.
Next, all you do is create one file called actions.xml.
And in that file, you take different parameters
that we extract from language for you
and you map them to Android deep links.
Just like that, Google Assistant can open different parts
of your app and prefill data for users, which saves them
time and saves them taps.
SPEAKER 1: Oh my gosh, that sounds super exciting,
especially when it comes to not having to handle
natural language processing.
Google Assistant handles that for you.
That's fantastic.
So can you tell me about--
how does that compare the Android experience
compared to a voice experience?
What are the benefits to that, for a user?
SPEAKER 3: So for a user, it allows you
to get where you want faster.
So essentially, App Actions is going
to save you time and effort.
So something that might have taken 10 taps--
for example, sending a payment to someone--
by sending things in natural language in just one sentence,
you'll have to avoid saying who you want to send the money to,
how much you want to send, and so on.
SPEAKER 1: Oh, that's fantastic.
I know we have a demo of S'mores,
and so I know there's multiple steps.
Can you tell me a little bit more about that?
SPEAKER 3: Yeah.
So over here we set up a demo for a s'mores ordering app,
where you can customize a delicious s'more, choosing
things like graham crackers, chocolate, et cetera.
And you'll notice that going through the entire ordering
flow takes about 10 to 15 taps.
But using App Actions, you can try again and say something
like-- order an Oreo s'more from the S'mores app--
and it takes you straight to the final screen without having
to go through the full ordering flow, saving you time and taps.
SPEAKER 1: Oh my gosh, I could definitely see for the users
this being really exciting.
And then for developers, this just
means that users are more likely to come back to their Android
app, which is great.
Thank you so much for hanging out with us.
Really appreciate it.
So we're here in the [INAUDIBLE] section for how-to guides,
and I'm here with [? Eileen. ?] [? Eileen, ?] can you tell me
all about this space?
SPEAKER 4: This space is for content creators who
wants to bring rich experiences for Search and Assistant
by using structured data program.
And we have two methods of building structured data.
One is web markup, and then the other is templates.
SPEAKER 1: That's exciting.
So if I'm a developer, and I have a website,
and I have content on there, how can I connect that content
to Google Assistant?
SPEAKER 4: Great question.
So basically, all you need to do is to use a web markup
and structure your content, and you would get rich experiences
on Search and Assistant.
And specifically for how-to, we are so happy to announce
our support of [? how ?] [? to ?] [? structure ?]
[? data ?] program as of yesterday at I/O. And now,
as a web developer, you can go ahead and mark up your website
with text, images, and videos, and get these rich experiences
on Search and the Assistant.
SPEAKER 1: Oh my gosh.
So you do this markup language once,
and it's available not just on the system,
but you get rich displays on Search too?
SPEAKER 4: Exactly.
SPEAKER 1: That's fantastic.
So what if I'm a content provider or creator
and I don't have access to my website,
or I have a YouTube video and I want
to connect all that information to Google Assistant,
can that happen?
SPEAKER 4: Yes, actually.
Amazing question.
For that, we have a solution for templates.
So specifically, for the how-to tutorials,
we build a how-to video template.
If you maintain your content mostly on YouTube,
you can come and fill in basic spreadsheet.
By following the actions in Google how-to template,
you can put your YouTube URL, anchor it by the timestamp,
add the steps, and out of the box,
you would get to rich Assistant action experiences.
SPEAKER 1: That's great.
And so do I have to worry about the natural language processing
when it comes to this, or does Google handle that for me?
SPEAKER 4: Actually, Google handles it for you.
Both for markup and the templates,
we take care of natural language understanding,
and also, we handle the discovery of your content.
You don't need to worry about anything like NLU or anything
to do with machine learning.
SPEAKER 1: That's great.
And can you tell me a little bit more
about the demo that we have here today?
SPEAKER 4: Sure, of course.
So for the demo today, we built a sample websites,
how to make an origami.
For the first method, we created the website
and we marked it up, and for the second method,
we built a YouTube video and created a spreadsheet.
So all the users have to do is naturally ask how
to make origami, and they would get
these rich experiences without doing any additional custom
work.
And users can engage with this in a system with voice
interactions, and this gives you out-of-the-box speech
recognition.
SPEAKER 1: That's fantastic.
Thank you so much, [? Eileen, ?] for hanging out with us.
Really appreciate it.
SPEAKER 4: Thank you so much, Jessica.
SPEAKER 1: OK, let's keep going.
We're at the Interactive Canvas section of the Dome,
and I'm here with Jacob.
Jacob, can you tell me more about this space?
SPEAKER 5: Yeah, absolutely.
So Interactive Canvas is a powerful new developer
capability that lets our developers get
full control over what they're putting
on the screen of the device.
You can do this using existing web technologies,
and we think it's really great for building games
for the Assistant.
SPEAKER 1: OK, when you say existing web technologies,
what type of tools are we talking about?
SPEAKER 5: Absolutely.
So anything that you have used on the web before--
[INAUDIBLE] any high-level frameworks, like Angular--
all of that works here.
So if you've built anything on the web,
you'll feel right at home building for Canvas.
SPEAKER 1: That is super exciting.
So if I already have an Action, and it's
all about playing games, can I add this onto that?
SPEAKER 5: Absolutely.
So you can think of Interactive Canvas as a visual layer
on top of your conversation Actions.
So to incorporate it, it's just a matter
of adding another type of response
in your existing webhook.
SPEAKER 1: All right.
And when it comes to designing this, is this just touch only,
is this voice, is it a combination?
What's the goal of this?
SPEAKER 5: Yeah, so as you may have seen in the demo,
it offers really rich visual experiences to our users.
And we're really excited to see what you can build,
in terms of combining both voice and touch into your Actions.
SPEAKER 1: Awesome.
Can you tell me more about this specific demo?
Because it's all jungle-themed, and seems really exciting.
Can you tell me more about the design?
SPEAKER 5: Yeah, absolutely.
So we built this game to showcase all the fun
things you can use Canvas for, and we did it
in the context of a jungle-themed adventure game.
So you go through the jungle.
You're trying to navigate through this temple
and escape eventually.
But it all turns out it was a dream all along.
SPEAKER 1: Oh my gosh, that's super exciting.
Well, Jacob, thank you so much for hanging out with us.
Really appreciate it.
SPEAKER 5: Thank you so much.
SPEAKER 1: OK, let's keep going.
So we're here in the Local Home SDK section,
and I'm here with Carl.
Carl, can you tell me about this space?
SPEAKER 6: Sure.
So this is the Local Home SDK area,
and we're showing a visual representation
of the life of a local query using the local home
SDK versus the normal cloud query with the smart home API.
SPEAKER 1: So what's different for developers
when it comes to the local SDK compared to before?
SPEAKER 6: Great question.
So with the smart home API, that's
the primary way developers integrate their smart devices
to the Google Assistant.
And that's a cloud-to-cloud integration.
Where after a system processes that command,
we send it to the developer to, ultimately,
fulfill it to the end device.
With the local home SDK, now we give them
the ability to do that same fulfillment over
on the user's local area network,
providing huge increases in speed and reliability.
SPEAKER 1: Oh my gosh.
I can definitely see from the user's perspective,
it'd be a lot faster, right?
SPEAKER 6: Totally.
Yeah, because now we don't need to go to multiple clouds.
We can do this a lot faster on the local network.
SPEAKER 1: OK, Carl.
Can you show us a demo?
SPEAKER 6: Absolutely.
So this train represents the life
of a local query and this train represents
the one that goes from the cloud-to-cloud integration
using the smart home API.
So what we see happens is when a user says a command.
For example, turn on the lights.
Both go to the Assistant server to be
processed in the same way.
But if we know the device is locally controllable,
we then send that JSON payload back down to the Google Home
device where we pull up the developers
JavaScript to process it, and ultimately fulfill it.
And there it is.
The light is on.
And what we see with the other train is it
the cloud-to-cloud integration still works.
However, it just takes longer to get to the same point.
So this really demonstrates the increase in speed
that we can get with the local home SDK.
SPEAKER 1: Oh my gosh.
That's awesome, Carl.
Thank you so much.
That was really cool.
Thanks for chatting with us.
SPEAKER 6: Thanks for coming by.
SPEAKER 1: OK.
Let's keep going.
We're at the Smart Home section and I'm talking to [? Omar. ?]
[? Omar, ?] can you tell me about this space?
SPEAKER 7: Yeah, absolutely.
So what we're showcasing over here
is the new functionality that we have come out
with for our partners to utilize to enable an Assistant
experience for their products.
SPEAKER 1: So this is all about marrying kind of hardware
and Google Assistant, right?
SPEAKER 7: Absolutely.
So this is about us being able to-- or as a user,
being able to control the devices that we
use on a daily basis in our households.
SPEAKER 1: And so if I'm a hardware developer
and I have this stuff, how do I connect my hardware to Google
Assistant?
What things should I keep in mind?
SPEAKER 7: Definitely.
So what we have is we have device types
and we have traits.
Right.
So you can think about it from the perspective
of a light bulb.
A light bulb is the identity of the device, which is the type.
And you have traits, which is functionality
for that light bulb.
You have brightness.
You have on/off.
You have color.
Once you marry the two together, you
have a fairly robust product.
So what we're asking our developers to do
is bring certain hardware that enables certain functionality
like that.
And then using the traits that we've offered,
create an Assistant experience around it.
SPEAKER 1: That's awesome.
But as a hardware developer as I connect everything,
do I have to worry about how the user is going
to interact with my action or is that something Google Assistant
handles for me?
SPEAKER 7: The Google Assistant will handle that for the user.
As new languages are supported, as new locales are enabled,
all of that is inherited as part of the smart home track.
Which is the great thing about it.
SPEAKER 1: Oh my gosh.
I know we have a cookie demo here and I have a sweet tooth.
So I'd love for you to talk more about the cookie demo.
SPEAKER 7: Absolutely.
The cookie demo has actually been very popular today.
But what we're trying to showcase with this
is a very key feature, which is two-factor authentication.
Now two-factor authentication is really
needed for security devices like doors, garage doors.
Anything that could be a security thing
needs to have this additional layer of security on top of it.
And what we do is we give you the ability
to add 2FA to any type of trait that you've implemented.
SPEAKER 1: That's amazing.
Well thank you so much, [? Omar. ?] I
really appreciate you sharing all this with us.
SPEAKER 7: Absolutely.
Thank you.
SPEAKER 1: Oh my gosh.
That was amazing.
If you're excited like me to start building
for Google Assistant, check out our docs
and I'm looking forward to seeing you guys next year.
[MUSIC PLAYING]