We made it through another I/O, and what an exciting year it was.
Let’s look at some of the highlights
and new product announcements for App Actions,
Conversational Actions, and smart home Actions.
We’ve got a lot of new updates coming with Android 12:
First, we’re releasing a new beta API
to help with integrating your App Actions into Android more easily.
I’m excited to introduce Capabilities, a new framework API
that lets you declare support for your common and vertical intents.
You can describe parameterized Android intents,
which you then map to user query semantics via the assistant built in intent catalog.
Woo, that’s a mouthful.
Honestly, It's much easier to do than to say.
Capabilities API provides an Android-friendly way
of declaring the built-in intents your app supports
it's available in beta starting today.
Widgets are also getting some upgrades, thanks to the Capabilities API
allowing you to build widgets for mobile and automatically leverage these widgets
onto other Assistant-enabled surfaces on Android.
We’re making it easier for users to use widgets across all sorts of new surfaces.
Widgets will be accessible via voice with Google Assistant
on mobile, lock screen, and Android Auto.
Once built on mobile,
widgets will be available on other surfaces automatically
and accessible to users while driving in a safe and optimized way.
The integration with Assistant enables multi-step interactions
across all these surfaces with this same widget
We know that the ability to engage with more users is important.
That’s why we introduced some new features
to automatically help you feature your Action Shortcuts more effectively.
Last year, we introduced the ability for users to create personal voice shortcuts
into apps with Google Assistant.
Now, you can join the voice shortcuts party.
If you build Android shortcuts,
they’ll automatically be suggested to users in our voice shortcuts gallery.
So users can set up a personal voice command
that takes advantage of the Android shortcut in your app.
To make this possible, you can use our new Shortcuts Jetpack module
to push shortcuts whenever users take the corresponding action in your app.
Make sure to also connect your shortcuts
to the right app capabilities in shortcuts.XML.
When you follow these best practices
Google Assistant can suggest relevant shortcuts to users
in all sorts of ways, and help drive traffic to your app.
We also gave you a sneak peek on the upcoming Interactive Canvas APIs
for handling natural language processing
and text to speech in a faster, more efficient manner.
Text to speech and natural language understanding
as well as storage have historically been managed through webhook calls
but this approach can make coordination
between the webhook and the client pretty complex.
And it also slows down the user experience.
We’re fixing that. The Canvas API will now allow
for client-side TTS, NLU, and storage.
This API will be optional for developers to use
and will be available soon in a developer preview.
There are also some exciting updates coming to help improve the monetization flow
on smart displays for your Actions.
We’re happy to announce upcoming availability of on-device CVC entry on your smart displays
and we’ll have on-device credit card entry available soon.
Both of these features make on-device transactions much easier
and reduce the chance users will be redirected to their mobile devices.
Following closely on the updated smart home policy launch earlier this year,
we’ve continued to push updates to tools like the Test Suite for smart home
and the Analytics dashboard to help you build quality integrations.
Umair’ll show you how to use Google’s Smart Home Test Suite to deploy Actions
that meet our quality guidelines and monitor that quality before and after launch.
Our new analytics tools provide insight into how your devices are performing
and let you dive deeper to resolve issues
To additionally help you build high-quality integrations,
we are also enabling Bluetooth Low Energy seamless setup for all developers
Seamless setup for BLE devices solves this first impression issue
by enabling users to set up their device
in a couple of taps directly in the Google Home app.
So they can experience your device within seconds
with a path to add your app or saving you from building an app
with a cloud connection in the first place.
Proactive notifications and follow-up responses
to keep users informed about their smart devices
can now be implemented for select traits.
Notifications will continue to launch per select traits.
We’re starting with ObjectDetection for devices like cameras and doorbells,
RunCycle for appliances like washers and dryers,
SensorState for sensors that can be around in the home.
We also support asynchronous notifications
for select traits as well; for example, lock, unlock.
These types of notifications are great
when immediate feedback is needed after performing an Action.
We also announced some important updates
for supported communication protocols for smart cameras
To support a better user experience,
we are now adding the support for WebRTC for camera devices.
This one is a super exciting feature.
Now our users can have the best-in-class streaming experience
on Nest Hub and on their phones over WebRTC.
WebRTC is an HTML5 specification
that you can use to add a real-time media communication
between a Nest Hub and your camera.
Enterprise developers can also now launch integrations
with Nest devices through the Device Access console.
The Device Access console provides the documentation
and tools to enable partners to develop, test,
and launch their integrations with Nest devices.
This program isn’t only for enterprise-level partners.
It is also for the developer community
that are looking to create solutions within a private environment.
Developers can experience
the full set of features of this program within a sandbox
without needing the added requirement
of doing a security assessment on a yearly basis.
And finally, we’ve got some exciting updates
for the upcoming Matter launch formerly known as Project CHIP.
We’re proud to announce that we will be certifying existing Nest devices
with Matter when the standard is formally launched.
That means when you build devices with Matter,
they can be easily set up with the Google Home app
and controlled by millions of users already using Google speakers and displays,
Google Assistant and Android power controls.
Plus, the newest Nest Thermostat will get updated to support Matter,
meaning, for the first time, it can be controlled on other platforms
that have certified with Matter.
We also have a couple new codelabs as well as technical demo videos
to help you build out your Actions.
Check out the docs and codelabs linked in the description below
to get started using these new tools and capabilities for your Actions
and let us know what you're building on Reddit or on Twitter.
Thanks for watching.