Hi everyone.
I’m Rebecca Nathenson, director of product for the Google Assistant Developer platform.
I'm so happy to be here today to talk with you about Google Assistant.
Now we’re really into the swing of this virtual event thing,
I'm especially happy to welcome all of you who've joined us for other virtual events
like Google Assistant Developer Day and VOICE Talks.
I’ve gotten to appear in two VOICE Talks so far,
and my goal for 2021 is to make it a hat trick.
Today, I’m going to share a bit about the background
of Google Assistant and our vision for where we’re headed,
talk about the crucial role that our developers play in our vision,
and finally, I get to share some exciting product announcements.
So let's get started.
Back in 2016, we first introduced Google Assistant
because we wanted to help people get things done.
We knew that letting them just ask for what they wanted
in plain language was the best way to do that.
Since then, Google Assistant has grown to be available
in over 30 languages, 90 countries, and on more than one billion devices.
Hundreds of millions of people every month use the Assistant to get things done
on all kinds of surfaces, from mobile phones and smart displays
to TVs, cars, and more.
My four-year-old talks to everything with a screen, expecting it to answer.
And our developers are a crucial part of this growth.
The Google Assistant team can’t build every imaginable capability that our users need.
Those hundreds of millions of users have millions of interests
that developers just like you can deliver on.
From my son's favorite games to our families video chats
to that perennial favorite in our household, “Hey, Google, tell me a monster truck story.”
In fact, queries fulfilled by developers’ apps
have more than doubled in the past year.
And we know that this is the wave of the future.
As the platform grows, the variety of queries will grow too.
And that's where our ecosystem truly shines.
People don't just want a more natural way to get what they need.
They want to get more of what they need in that natural way.
It’s natural in that users can just say what they want.
Natural because it’s not tied to a single device or service.
Natural because, with deep understanding of context,
we and you can be smart about how we step in and help users
before they even ask for the help.
Today, I want to share the changes and enhancements we have coming
for every place that Google Assistant appears,
from App Actions for Android developers to Conversational Action tools.
Let’s start with App Actions,
our solution for Android developers to easily enable their apps
to fulfill queries from Google Assistant users.
App Actions is the easiest way to integrate your Android app with Google Assistant,
letting your app join in fulfilling millions of Assistant user queries
from booking a ride to posting a message on social media.
It lets your users jump right to the most interesting and useful points in your app
with simple voice commands.
You can enable App Actions today in Android Studio
by mapping user intents to specific features
or steps within your apps.
Last year, we made App Actions available to all Android apps by introducing
over 60 built-in intents.
We introduced common intents
like opening your app or searching within your app.
Here's an example on Discord that lets the user see their mentions.
We also introduced vertical specific intents covering common use cases for communication,
transportation, health, and many more, intents like send money or start an exercise.
I can even tweet. “Hey, Google, tweet Virtual I/O is exciting.”
And there you go.
But if your app does something unusual,
something not covered by our built-in intents we've got you covered.
We also introduced support for custom intents.
So you can build App Actions that maps your app’s unique functionality.
Let’s see an example from Walmart
where the user’s booking a time with Walmart to pick up their order.
With both built-in and custom intents,
you can easily integrate Google Assistant to your mobile apps
with your favorite Android development tools.
This lets you reach your users using a simple set of Android APIs,
which can surface your app’s actions and content across Google surfaces.
And now we’re surfacing your apps functionality in more places,
from voice queries on mobile to Android Launcher and search suggestions.
Let's take a closer look at how this works.
I'm excited to introduce Capabilities,
a new framework API that lets you declare support
for your common and vertical intents.
You can describe parameterized Android intents,
which you then map to user query semantics via the Assistant built-in intent catalog.
Ooh, that’s a mouthful.
Honestly, it's much easier to do than to say.
Capabilities API provides an Android-friendly way
of declaring the built-in intents your app supports.
It's available in beta starting today.
Here’s a demo from Snap showing one of Under Armour’s AR shopping lenses.
Isn't that cool?
While we’re making things easier for you, let's also make it easier for your users
to discover that Google Assistant queries are supported.
With Android 12, we're doing exactly that.
And the best part is. it’s all a part of the APIs already in Android,
shortcuts and widgets. Let's start with shortcuts.
Android shortcuts are already used by app developers
to provide a quick entry point to users via the Android Launcher.
Last year, we introduced the ability for users
to create personal voice shortcuts into apps with Google Assistant.
Now, you can join the voice shortcuts party.
If you build Android shortcuts, they’ll automatically be suggested to users
in our voice shortcuts gallery.
So users can set up a personal voice command
that takes advantage of the Android shortcut in your app.
To make this possible, you can use our new shortcuts Jetpack module
to push shortcuts whenever users take the corresponding action in your app.
Make sure to also connect your shortcuts to the right app capabilities in shortcuts.xml.
When you follow these best practices,
Google Assistant can suggest relevant shortcuts to users
in all sorts of ways and help drive traffic to your app.
To recap, you can describe the set of deep links
supported by your app directly in their shortcuts.xml via the new Capabilities API.
You can use our Jetpack module to push unlimited shortcuts
to the Android system and to Assistant.
Next, let's talk widgets.
Widgets have been a great way for users to access glanceable, evergreen content.
They’re a great way for users to quickly monitor information, complete tasks
or just get inspired directly from their home screen.
But a great thing can get even better.
Let's talk about how.
First, with Android 12,
widgets are transforming into an interactive, customizable view into your app
with a consistent design and look and feel,
so users can get more done within the flow of their day.
Here's Strava implementing a widget
to track how many miles a user ran in a week.
Next, we're making it easier for users
to use widgets across all sorts of new surfaces.
Widgets will be accessible via voice with Google Assistant
on mobile, lock screen, and Android Auto.
Once built on mobile, widgets will be available on other surfaces automatically
and accessible to users while driving in a safe and optimized way.
The integration with Assistant enables multi-step interactions
across all these surfaces with the same widget.
So it’s not only one shot answers and quick updates that you can enable.
Users can have a full conversation with a widget with help from Google.
This integration lets you build single widgets
with multi-step flows like selecting and ordering a drink.
Check out this example from Dunkin.
Oh, now I want a donut and some coffee.
Developers can map specific built-in intents to widgets
using the Capabilities API,
which lets users invoke widgets from Assistant and optimize for Voice.
Whew, that’s a lot.
We've introduced all of these capabilities with the goal making it easier than ever
to integrate your Android app with Google Assistant.
And the greatest thing is that this is all fast.
With just a few day’s work of enabling deep links
and enabling the Capabilities API,
developers can take advantage of everything that App Actions has to offer.
And since discoverability is baked into the product,
we look forward to developers seeing real value from these integrations.
We want to thank some of our early developers shown here
who helped us test and refine these features.
And we're so excited to welcome more of you
on this journey towards making Assistant on mobile
more useful with every passing day.
Now, I love all of the ways that Google Assistant
helps me on my phone, but when my hands are full at home,
I turn to my smart display, like so many other people.
And we’re making it easier for all of you to build the ever increasing number of people
using their smart displays every day.
Last year, we introduced all sorts of improvements
to the Assistant platform for smart displays.
We introduced the Actions Builder,
a web-based IDE that lets you build, debug, test,
and release your Actions all in one place.
I used it to build my first action when joining the team
and I went from clueless to finished in less than two days.
We also released enhancements to the Actions SDK,
a new Actions API, client libraries, and a new testing API.
And finally, we worked with partners like Jovo and Voiceflow
so you could use your favorite DevTools to build for the Assistant.
We also made improvements on the user experience side
to help users easily discover what developers have built.
We introduced an updated UI for our smart display
to help users discover games experiences more easily.
And we opened two new built-in intents for public registration
in the education and storytelling verticals,
in addition to games, which expanded the reach of easy discoverability still further.
But wait, there's more.
Today, I want to share further efforts towards improving
both the developer experience and the user experience on the smart display.
First, let’s start with the improvements on the developer side,
beginning with Interactive Canvas.
As you know, Interactive Canvas helps you build
touch and voice-controlled experiences for the Assistant
using web technologies like HTML, CSS and JavaScript.
We’ve had some early developers use Canvas to build rich visual games experiences
for the smart display.
Cool Games, Zynga, and GC Turbo are just a few.
But Canvas isn't just for games.
Last year, we expanded Interactive Canvas to Actions in education and storytelling,
which let many partners in those verticals build incredibly engaging new experiences
for Google Assistant.
The day the Wonder Woman Story Experience by Capstone launched
was the day that magic lassos became common in our house.
And the ABCmouse Education Experience by Age of Learning
is responsible for my four-year-old doubling the number of letters he knows.
Speaking of the alphabet, we have some great enhancements
to manage our own collection of letters: TTS and NLU.
Text-to-speech and natural language understanding,
as well as storage, have historically been managed through webhook calls.
This approach can make coordination between the webhook and the client pretty complex,
and it also slows down the user experience. We're fixing that.
The Canvas API will now allow for client-side TTS, NLU, and storage.
This API will be optional for developers to use
and will be available soon in a developer preview.
Once you build amazing new Actions,
we’re also giving you a wider set of options around how to release.
Coming soon, developers will be able to manage
their releases in the Console by launching in stages,
starting with just one country or launching only to a percentage of users, for example.
On the user experience side, we're announcing more improvements
to help your users have full immersive experiences on the smart display.
First, we’ve removed the interruption to TTS when users tap on the smart display.
Next, we’ve launched full-screen Canvas capabilities.
So your users can get the most out of your immersive experiences.
And we’ve made improvements to the Media API to help support long-form media sessions.
Now, you can start playback from a specific moment,
resume where a user dropped out of their previous session,
and adapt conversational responses based on the media playback context.
Actions can also leverage the new playlist media response
to present user’s batches of related content at the same time
with the Assistant automatically handling the UI
and the voice controls that users need to skip entries or go back.
And finally, we know that another big area
the developers have been asking for improvements in is around monetization.
In October of last year, we made a commitment to make it easier for users
to transact with your connected home experiences.
We're happy to announce upcoming availability
of on-device CVC entry on your smart displays,
and we’ll have on-device credit card entry available soon.
Both of these features make on-device transactions
much easier and reduce the chance that users will have to be redirected
to their mobile devices.
It's been great fun working with the team to put together all of these new features
both on the developer side and on the user experience side.
I am so much looking forward to seeing
how all of you use them to create amazing experiences.
And soon, I look forward to hearing about all of them in person.
Donuts and coffee will be on me. Thank you.