[MUSIC PLAYING]
CRYSTAL HUANG: Welcome, everybody.
I'm Crystal Huang, and I'm on the Google Assistant Product
Partnerships Team.
I'm here today with Toni and Madhukar.
We're super excited as Google Assistant continues
to deepen its collaboration with Android
to create a more seamless experience for developers
and users.
Today, we're going to highlight how
users can employ Google Assistant
for conversational, personalized help
to get things done across different devices
and how developers are able to fulfill
those requests with integrations based on Android platform
features and APIs in order to minimize work
across Android device types.
Before we dive into those details,
let's take a brief look at what it means for Assistant
and Android to be better together
and why it matters both for users,
who are looking for seamless experiences
across different devices, and for developers, who are looking
to lower development costs for different device types
by using common development and testing tools.
Let's start with users.
We can do so much in technology today.
So it's no wonder that many of us
are adding more and more computing devices to our homes.
For example, in 2019, the average US household
had 11 connected devices.
In 2021, that went up to 25.
With this proliferation of devices,
it's more important than ever that they
work better together so users don't
have to learn a bunch of new interfaces,
or remember which capabilities work on which devices,
or pull up the same content or task over and over again
as they move from one device to the next.
Google Assistant is a key ingredient
that enables devices to work better together because it
provides this really natural, intuitive interface
for accelerating a user's tasks across their apps and devices.
Think of Assistant as the glue that
provides a consistent, voice-forward user experience
and enables a user to jump straight
to the most relevant or useful point in their apps.
Assistant does this by understanding
which capabilities apps support and which apps are best
able to fulfill users' requests.
This applies whether users already have the app installed
or not, whether they make the requests through voice
or by touch, and whether those requests
are simple commands or complex, multi-step journeys.
Now that we've talked about benefits
for users, let's talk about what this means for our developers.
It's never been easier to integrate your Android apps
with Google Assistant by taking advantage of Android shortcuts
and media APIs.
These enable fast, natural entry points to content
and features in not only your mobile app but also Android TV
OS, Android for cars, and Wear OS apps.
We're working to provide a consistent developer experience
across different device types to ease
the effort of voice enabling your app wherever it runs.
And next up is Madhukar.
I'm going to hand it over to him to go through the details.
MADHUKAR SRIVASTAVA: Thanks, Crystal.
I'm Madhukar Srivastava, and I'm a product manager
on the Google Assistant team.
We introduced Assistant in 2016 on voice-activated smart
speakers, then brought Assistant to many other devices.
We heard from our developers that you appreciated
being able to build experiences for new types of devices,
such as smart speakers and displays, tablets,
and other smart home devices.
Many of you were looking for ways
to integrate wise, proactive, and other assistive
capabilities into your existing Android content
and experiences without having to learn a new platform
or build from scratch.
That's why we launched App Actions, enabling developers
to extend Android apps to Assistant on smartphones
so simple voice commands can open key app functionality.
We have continued to harmonize the App Actions development
experience with the Android platform
so it takes less effort to implement
App Actions while adding support for App Actions
to surface on more device types.
So there are greater benefits from
the same co-implementation.
Let's explore how App Actions work then
examples of how the implementation can extend
to different Android devices.
Google Assistant improves the Android user experience
by enabling users to naturally invoke
apps with voice or touch, whichever modality
is most convenient at that moment,
like voice commands when your hands are full
or you're multitasking.
To apply this functionality to an Android app,
developers can use App Actions to get started with voice
journeys in their apps.
Let's think about two main steps required for users
to access your app's features through voice commands
to Assistant.
First, we need to process the user's spoken input
and understand the request.
Second, we need to fulfill the user's request.
There are many variations on how people
might express the same intent using conversational language.
Assistant uses natural language understanding technology
to identify different speech patterns that
correspond to the same particular functionality.
We've already modeled speech patterns
for more than 60 use cases, which we call built-in intents,
or BIIs.
These include use cases that apply broadly to most apps,
like opening a feature or updating an account.
They also include use cases that are primarily
applicable to apps in a specific category,
like ordering a menu item for food and drink apps.
For functionality unique to your app that's
not covered by a built-in intent,
App Actions also support the creation of custom intents.
To tell Assistant about your app's functionality,
you can update your app bundle to declare
which built-in or custom intents your app can handle.
You can also define any parameters to expect
in the user's queries.
Once Assistant has matched a user's request
to the functionality that you have told us
your app can handle, we need to fulfill the request.
Assistant can do this in two ways
based on how you define fulfillment instructions
for each intent you support.
The first launches your app with an Android intent
that contains parameters extracted from the user's
query, which fast-tracks the user to the appropriate spot
in your app to complete the rest of the journey.
The second displays an Android widget
from your app with an assistant which provides a quick inline
response.
You can also add a spoken response
to your widget to give it a more conversational feel.
Because Assistant handles natural language modeling
to understand users' requests then uses existing components
of your app, like Android intents and widgets,
to fulfill those requests, developers can enable App
Actions and the added benefit to users quickly.
Adding voice-first functionality to your app through App Actions
creates a more simple, streamlined interaction,
minimizing the friction for users.
These App Actions that are available
across multiple devices where Assistant exists today,
and users expect Assistant features to work seamlessly
in every context.
Let's take a closer look at how Assistant leverages your App
Action implementation to provide hands-free or proactive
experiences across multiple types of Android devices,
like phones, cars, watches, and others,
plus a few examples of what developers have created.
Over to you, Toni.
TONI KLOPFENSTEIN: Thanks, Madhukar.
I'm Toni Klopfenstein, a developer relations engineer
for Google Assistant.
Madhukar highlighted how Assistant
can make the experience of using apps on phones
better by accelerating tasks that might otherwise
be hard to find or require multiple steps.
However, we heard from our developers
that it's tricky to communicate to your users
that a helpful new way to access in-app functionality
now exists.
For example, if a user asked Assistant for functionality
that your app can handle, but didn't say the name of your app
in their query, or didn't have your app installed,
Assistant wasn't able to help them.
To improve this, we've launched more ways for Assistant
to get users to your app so you don't
need to worry about providing precise instructions
to your users on how to use your app with Assistant.
Two new features-- brandless queries and app install
suggestions--
are now available immediately for all App Actions developers.
When a user queries Assistant without using a specific app's
brand name, also known as a brandless query,
Assistant can now infer what app would best
fulfill the user request and effectively route the user
to that app.
Let's see an example of this with Strava.
If a user attempts to start logging
their run without identifying Strava by name,
Assistant will still identify the desired functionality
and launch Strava to begin tracking a run.
If a user triggers the proper Assistant
query for the functionality of your app,
even if the app isn't installed yet,
then Assistant can provide app install suggestions.
Assistant can automatically direct users to your Play Store
listing and encourage them to install the app and access
full functionality.
Let's see how this looks when a user doesn't have
the requested app installed.
If a user says, "Show me recipes on TikTok,"
Assistant will open directly to the TikTok app Play Store page,
encouraging installation.
We've also improved custom intents as well.
For apps using custom intents, such as
"Schedule an appointment," machine learning-based grammar
expansion can now automatically increase the number of queries
Assistant identifies that can trigger an app,
meaning developers don't have to manually model
as many query patterns.
These features are automatically available to users
without any additional development work
required on your mobile app.
Now let's look at how an App Actions implementation
you launched for smartphones can be easily duplicated
to apps developed for other types of Android devices.
We're excited to announce that we are also extending
App Actions to Wear OS.
When you're going for a run or bike ride,
chances are that you want to leave your phone in your pocket
and keep your hands free.
Your smartwatch is easily accessible,
but the small screen makes it hard to touch and type
or to digest a lot of information.
That's where Assistant comes in, enabling users
to issue voice commands to their Wear OS device
and get glanceable information in return.
The health and fitness BIIs used in your existing mobile apps
will be enabled on Wear OS devices,
meaning users can trigger the same expected integrations.
Let's take a look at how this will
work for a health and fitness app that has integrated
App Actions on Wear OS.
Users will be able to launch the Assistant, say something like,
"Start my run," to trigger the BII,
and Assistant will launch that app feature automatically.
Wearables aren't the only type of device
where users may want or need a voice-first experience.
Let's take a look at automotive applications.
If you have a use case for your app
that's also appropriate for use in cars,
App Actions works there as well.
Developers can add voice functionality to car apps
by expanding their mobile App Actions implementation
with the Car App Library.
Once done, these apps can be used on either Android Auto
or Android Automotive OS.
Android Auto enables users to connect their Android phones
to compatible vehicles and use their favorite apps
and services right on the car's display.
Android Automotive OS, with Google built in,
allows users to access Google Assistant, Google Maps,
and apps from Google Play directly from the car display
without the need for a phone.
The Car App Library provides templates
that can be used to build parking, charging,
and navigation apps.
These templates are designed with driver safety
in mind, minimizing distractions with
glanceable, voice-forward UIs that adapt
the vehicle they're running in.
At this time, App Actions in autos supports BIIs that are
relevant to on-the-go use cases--
finding parking and charging.
Let's take a look at how those BIIs work
with our partner, PlugShare.
PlugShare allows users to find charging stations on the go.
Here's a demo of the Android Automotive OS user experience
where the user says, "Hey, Google, find charging stations
nearby on PlugShare."
For developers who haven't built a specific Android Auto app
but still want their Android mobile content
to be available to their users in the car,
you can use App Actions and Android Widgets
to provide quick answers and information on Android
Auto and Android Automotive OS.
This is great for on-the-go situations,
like when you're driving home from picking up the kids,
have a pizza coming for home delivery,
and want to check how far out that pizza delivery is.
App Actions are helpful for enabling voice commands
and apps that find parking locations, charge
stations, and more in the car.
But what if users want to listen to your media app?
If you've already built a media app,
Google Assistant can also be used
to bring voice-first functionality to your users.
When you implement a media session for your audio or video
app using the Media Session API, Assistant
can access the transport controls
for playback, allowing users to play, pause,
skip, or discover what's playing, all through a voice
query to their phone, tablet, automobile, or television.
Developers can add this functionality to existing media
sessions by setting flags for media and transport controls
inside their media session.
By using the existing media session transport controls
within your app, Android will handle displaying controls
for play, pause, seeking, or skipping
to an adjacent piece of content, depending on which actions
your media session supports.
To learn more about media sessions,
check out the TV space in IOAdventure.
Wow.
That's a lot of devices we just covered.
You've now seen how developers can easily
integrate voice functionality with the Android shortcuts
and media APIs to create a seamless user
experience with Assistant on different Android devices,
including mobile, wear, TV, and auto.
Don't forget to watch our other sessions, including
our deep technical dive into Google Assistant's
architecture, as well as our deeper dive into Assistant
for Android Auto.
If you're looking to get started building out
your own assistant integration on an Android device,
check out the workshop that will walk you
through integrating Android Widgets with Google Assistant.
Check out these additional resources to get started today.
And don't forget you can follow us on Twitter
or sign up for "The Developer" newsletter.
Thanks for watching.
[MUSIC PLAYING]