(upbeat music)
- Hi all, I'm Shikha Sehgal,
I lead Partner Technology Management for Google Assistant.
My team directly works with developers and partners like you
to help you build high quality Assistant experiences
for mobile and Auto.
With me, I have Matt.
- Hi everyone. I'm Matt Carroll.
I work as a Developer Relations Engineer on the Google Assistant team.
- Thanks for joining us for this virtual I/O.
Matt and I are so excited to talk to you
about how you can voicify your app using App Actions.
Today, I will give you an overview of App Actions
and why it's valuable for you.
Matt will then go through the actual integration steps.
And finally, I will share our latest product features.
Let's jump right in.
As you heard in our keynote, with App Actions,
you can bring the magic of Google Assistant
to your Android apps.
Our goal is for users to have a conversation
with your app in a natural way.
We take care of all the heavy natural language understanding
and fast forward users directly
into your app's key functionality with voice.
This makes the user's journey on their phones faster and easier.
Let's see how MyFitnessPal does this.
Instead of finding their app
amongst hundreds of apps that you may have on your phone,
then identifying where to log food, which might be five plus tabs
and some typing, you can simply say,
"Log a berry smoothie on MyFitnessPal"
and go right in.
This integration is simple and fast.
You can use Android tools that you are already
familiar with, such as Android Studio.
Simply map user Intents to specific features or steps
within your existing Android app.
This can be done with just a few days of development work.
Feedback from our early access partners
and developers like you has helped us to optimize this experience.
We have created 60 plus Intents for you,
to cover most Android apps,
and are continuously adding to this list based on your feedback.
We cover common user journeys suggest opening an app
or directly searching within the app.
You can see an example from Google Maps here.
Directly search for Italian restaurants on Google Maps.
We also support 10 different product verticals,
such as finance and shopping.
So you can enable users to accomplish more specific tasks.
Now, if your app is not covered by any of these 10 product verticals,
you can even customize your own intent to start a specific use case.
Matt will cover the specifics of these shortly.
Once you integrate with Assistant,
your app automatically becomes eligible
for re-engagement across Google Assistant on mobile,
and even Android Auto.
I'm particularly excited about voicifying apps for Android Auto
and how important it is
so they can have their hands on the wheel eyes on the road,
still access your app's content through voice,
and make their journeys delightful.
Here is an example with PlugShare.
You can find charging stations on PlugShare,
get back the search results from Google Assistant,
and navigate right in.
All this, you can do hands-free while driving.
How cool.
Now let's see how you can add App Actions to your app.
- To get started integrating App Actions into your app,
you tell Google Assistant what queries your app supports,
and how to open your app for each type of query.
First, tell Assistant what voice queries are supported
by your app with built-in Intents, or BIIs.
Then describe how those supported queries
should open to the relevant places in your app.
For example, Etsy supports the BII named Get Thing
to support searching for items across Etsy,
directly from Google Assistant.
By asking Google Assistant something like, "Find hanging planters on Etsy,"
it has enough context to open the Etsy app
directly to Etsy's search page,
pre-filled with the search results for the query hanging planters.
By implementing App Actions, your users can get to a specific state
within your application using their voice,
and with the context of what they just said.
A built-in intent, or BII, is a pre-defined trigger
for a specific kind of action that the user wants to do.
Some common BIIs are useful for most apps,
like searching and opening a feature of an app.
Others are specific to certain verticals,
like ordering a menu item, starting a workout,
or creating a reservation.
For example, the Yahoo Finance App uses the vertical BII
for getting a stock quote to support queries like,
“Show me Verizon stock on Yahoo Finance.”
There are also custom Intents, which we'll cover later.
Google Assistant has a growing number of BIIs for all sorts of activities.
Check out the BIIs documentation to find the right BIIs for your app.
By the way, the reason why BIIs are called built-in intents is because Google builds
and maintains the language models for them.
Meaning, you don't have to.
Our models already support many different languages.
And we are always improving our models.
Because of this, as Google Assistant improves its language support,
your app can automatically benefit.
Google Assistant needs to know how to open your app
to the right place for each BII your app supports.
To do this, you need to tell App Actions how to launch your app
when the user triggers a BII with their voice.
This could be an intent with a target class
or an existing deep link that already launches the app to the right place.
Some BIIs contain parameters.
These parameters are contextual information
the user has provided, like the contents of a search query,
the ticker symbol for a stock quote, or the name of a menu item.
These parameters can be passed through to your app when it is launched.
This lets you perform actions based on the user's query,
like pre-filling a user’s cart with a specific item
or starting a specific workout.
Some BIIs allow you to define your own parameter values.
Parameter values can be defined statically in your app with inline inventory
or you can dynamically describe the availability of your items
with web inventory.
To tell Google Assistant how to map from BIIs,
to how to launch your app,
you use the Android XML configuration file for shortcuts.
In this file you'll add a capability for each BII that your app supports.
Capabilities also need to describe how to launch your app,
given the context of the user's query.
For example, the Etsy app uses the BII named Get Thing.
Their shortcuts XML file contains the capability
with the name of the Get Thing Intent.
This capability also describes a format for a deep link
that will open when the user’s query to Google Assistant indicates
they would like to search on Etsy.
When the user indicates a wish to search Etsy they can also include what they’re searching.
For instance, the user might say "Find hanging planters on Etsy."
The contents of the user search query is available
as an intent parameter called Thing Name.
The value of the Thing Name Intent parameter
will be what the user is searching for, which in this example is hanging planters.
The capability for the Intent named Get Thing
describes how to use the Thing Name intent parameter,
and the deep link to perform the search the user requested.
This way, when the user tells Google Assistant
to search for hanging planters on Etsy,
it arranges for Etsy to immediately display the search results for hanging planters.
For unique functionality that isn't supported by a BII,
you can create Custom Intents.
Custom Intents are similar to BIIs,
except that you need to provide query patterns.
Query patterns are lists of phrases you'd expect users to say
when they want to trigger your Custom Intent. For example, Walmart uses Custom Intents
to allow users to schedule times for a delivery
or to pick up their order from a Walmart store.
They use query patterns like "Book a time"
and variations on that phrase, to get this task done.
So when users ask Google Assistant to book a time with Walmart,
the Walmart app opens directly
to where the user can book a pickup or delivery.
Similar to built-in intents, capabilities are used to tell Google Assistant
what Custom Intents you support and where and how to send the user
to the right place in your app.
You’ll need to specify the query patterns in the capability for your Custom Intent.
To test your App Actions implementation,
you can preview your App Actions on a test device.
The Google Assistant plugin for Android Studio allows you to do this
quickly and easily on a physical device or an emulator.
After testing your app, upload it to the Play Store
and request an App Actions review.
Once it's been reviewed, you're done. Now, I'm going to hand it back to Shikha
to talk about the latest features we created to help you voicify your Android App.
- Now that you know how to add App Actions to your app,
let me share some exciting new product announcements
to help you enhance this integration.
Let’s start with capabilities.
You just saw capabilities in action,
with examples from Etsy and Walmart.
I will reiterate that this is a new extension
to the Android platform
that Google Assistant is building on top of.
You do this through shortcuts.XML,
which is a native Android concept
to define Android app shortcuts.
We have extended it to support App Actions Built In Intents,
as Matt explained.
This is available in beta today for you to start using.
Moving on to the next enhancement. Android Shortcuts.
We are extending Assistant to be more closely aligned
with the way Android Shortcuts work.
Android already displays app shortcuts and supported launches.
You can see some examples here.
These allow your users to quickly start common
or recommended tasks within your app
right from their home screen.
In Android S, we are removing the limit
on the number of shortcuts you provide.
So you can now push dynamic shortcuts
for every user action in your app as they occur.
Use our new Jetpack module,
called the Google Shortcuts Integration Library,
for backwards compatibility and to ensure your shortcuts are available
on Google Assistant surfaces.
You can also now connect static and dynamic shortcuts
to the capabilities that you defined in shortcuts.XML.
This will increase the likelihood
that Google Assistant will suggest personalized shortcuts
to your users and drive re-engagement to your app.
To learn more, check out the I/O session
Android Shortcuts for Google Assistant.
Next, we have In-App promo.
With In-App promo, you can proactively prompt the user
to explicitly add an Assistant shortcut within your app itself
and quickly accomplish tasks using voice.
Shortcut In-App promo lets your app call
into an Assistant API that asks the user
to set up a custom shortcut
based on their common usage pattern.
The API allows developers to query
whether a certain shortcut already exists
and branch into the Assistant dialogue to create it only if needed.
This new flow ensures that users don't leave your app
when they create the Assistant shortcut
that's providing a much more frictionless experience
to your users and even the app.
Here's an example
on how a user can create Assistant shortcut
from within the app.
And finally, we have Android widgets.
You already use Android widgets
to reach users outside your app,
to give them an at-a-glance view of your app's data,
right from their home screen.
Now widgets have been around for more than a decade now
and many developers are already familiar with them.
Very soon, you will be able to voicify these widgets,
connecting them to app Capabilities and shortcuts.XML.
You will also be able to add speech output to these widgets.
This will provide a hands-free experience
for users to open your widgets through voice
and even speak back to them in a natural way.
Once you integrate, the same experience will work on mobile
as well as Android Auto.
To learn more, check out the I/O session on Refreshing Widgets.
Now that was a lot of info, right? Let's do a quick recap.
You can voicify your app
and integrate it with Google Assistant using App Actions.
This allows your users to quickly access your app's
key journeys using voice.
This integration is simple and can be done within just a few days.
Once done, your app becomes eligible for re-engagement
across voice queries on mobile and Android Auto.
You can use several of our new product announcements
like Capabilities, Android Shortcuts, Widgets,
and In-App promo to further enhance your App Actions
integration and drive re-engagement to your app.
We have multiple offerings about App Actions
at I/O this year to help you get started.
Thanks for joining Matt and I today. We are so excited about App Actions
and the potential innovation
that we can do together with you in this space.
Thank you.
(upbeat music)