[MUSIC PLAYING]
KODLEE YIN: Hi.
My name is Kodlee.
RASEKH RIFAAT: And I'm Rasekh.
And we're both engineers on the Android Auto team
here to talk to you about what's new for app
developers on Android Auto.
Now we're incredibly excited at Google
about the automotive space right now because we
see it going through a huge transformation
in connectivity, electrification, interfaces
and sensors, sharing, and autonomy.
Cars are rapidly turning into full-blown computers on wheels.
They've got high-speed mobile connections, cameras,
microphones, and screens of all shapes and sizes everywhere.
Android Auto is an effort from Google
and our automotive partners to bring these advances together
and create a safe and seamless connected experience
for drivers everywhere.
Of course that's easier said than done.
There are dozens of different car platforms
today, many different input types from touchscreens
to touchpads to rotary controllers,
many different screen shapes, sizes, and resolutions.
Today, you can see that vision at work in any Android
Auto-compatible car.
Drivers have access to their favorite apps
right from their car's display, and developers build their app
once without worrying about different makes and models,
input controls, and screens.
Today, we'll talk about two of the most important
app categories, messaging and media.
KODLEE YIN: Great.
So first up is messaging.
Messaging has come a long way in both Android Auto and Android
the OS.
When Android Auto started supporting messaging,
there wasn't really a good way for messaging apps
to get their messaging information over to the car.
That's where CarExtender came into play.
CarExtender allowed a way for messaging apps
to provide conversation details and a way
to reply to conversations to Android Auto.
But since Android N, apps could stylize their notifications
with something called MessagingStyle.
MessagingStyle is a huge step up from CarExtender
as it allows messaging apps to provide conversation
information directly into the notification.
Not only does it provide a nicer UI for conversation details
but it provides affordances like replying and liking directly
in line to the notification.
Android Auto now fully supports the use
of MessagingStyle and actions without the need
for CarExtender.
This also means Android Auto and the Assistant both fully
support group messaging.
So for the price of implementing MessagingStyle,
apps not only gain a richer mobile user experience
but also gain the benefit of automotive support.
So let's see how Android Auto interfaces with this, starting
on the messaging app side.
From Android Auto's point of view,
messaging apps have three core functions--
notifying users of messages, marking those messages as read,
and replying to those messages.
Working backwards, apps can implement reading and replying
with services.
These services can be triggered internally
with intents or externally, like via Android Auto, with pending
intents.
Notifying is done via an Android notification,
and the messaging information is provided
with the MessagingStyle.
The mark is read, and reply-pending intents
are wrapped in actions and both provided in the notification
as well.
Note here that the reply action has a remote input that's
added that acts as a sort of input field for the reply.
And that's the messaging app's architecture.
Moving on to the other side of the notification,
we can see how Android Auto leverages these objects.
Android Auto will first post an in-car notification
and once tapped on will read aloud
the messages contained within.
The mark as read pending intent is then fired.
The user is given the choice to respond and, if taken,
a transcription of that response is set in that remote input.
The reply-pending intent is then fired.
And that's the entire Android Auto flow,
so let's see how we can put that into code.
First, the app needs to declare support for Android Auto.
To do that, it needs to create a new XML file that's
linked in the Android manifest.
This XML file says that it has notifications that Android
Auto should take a look at.
Note that for messaging apps that support SMS, MMS, or RCS,
this uses SMS bit also needs to be added.
So now Android Auto is taking a look at our messages.
We can build up the MessagingStyle.
So we can't really have a conversation without people,
so the first person we have to add is the user of the device.
To do that, we create this new Person object.
Person is used to set things like the user's name,
their icon, and a unique key in the event
that multiple people have the same name.
So we create this deviceUser, and we create
the messagingStyle with it.
We can then add our conversation information.
So I'm from Seattle and I love skiing,
so I'm setting the conversation title to ski group.
Because I'm taking multiple friends
this is a group conversation, so the messaging app
needs to set it as such.
Note here that conversation title and whether
or not the conversation is a group can be set independently.
This is new in Android P and has been back ported
to older Android versions in the Compat library.
And finally, we can add all the messages
in this conversation in the order they were received.
In this case, my friend wants to coordinate breakfast,
so the messaging app provides the text, the timestamp,
and the sender in the form of a person.
With this conversation set up, it's time to add the actions.
For the reply action we instantiate an Action.Builder
and set the semantic action to SEMANTIC_ACTION_REPLY.
That must also tell the OS that firing the reply-pending intent
won't show any extra UI.
This is especially important in Android Auto
because we don't want to be distracting drivers
with extra pop ups.
Finally, the reply action is supplied with that remote input
I talked about earlier.
On the mark-as-read side, things are done about the same way.
This time the semantic action is set
to SEMANTIC_ACTION_MARK_AS_READ, and again we
tell the OS that firing that pending intent won't
show extra UI.
Note here that the mark-as-read action does not
need a remote input.
So that's all three pieces.
The notification can now be built.
For reference, here are the three elements we created--
messaging style, which holds all our conversation information;
our reply action; and our mark-as-read action.
To build a notification, some boilerplate is provided,
and then we set the messaging style.
We can then add our actions.
Here is where the messaging app has some options.
Note that the reply action is added
as a regular visible action, and the mark-as-read action
is added as invisible.
This is purely stylistic.
One can add both actions as visible or invisible.
This will just change how it shows up in the mobile UI.
On Android Auto, actions are never shown,
but Android Auto will be able to read both visible and invisible
actions.
And finally, the messaging app can post the notification.
And there we have it.
My friends and I have planned breakfast on the road,
and our ski trip is under way.
RASEKH RIFAAT: And now that we've
coordinated with everybody, let's find something
to listen to on the drive out to the mountains.
Media in the car is one of our core user experiences,
and getting drivers access to their content
should be front and center.
I'm going to talk about several new features we're introducing
today to enhance the abilities of media apps
to provide content within Android Auto.
In particular, we want to make content more visually
pleasing by adding additional content-style hints
and enabling additional search results provided by the app.
To start off, let's go over the architecture
that an app has when communicating
with Android Auto.
The first thing a media app needs is a MediaBrowserService.
It provides a tree of playable and browsable items.
Browsable items are basically folders to organize app content
instead of returning a giant list of playable items.
The media apps implement the onLoadChildren method
which loads a particular level of the tree.
Here in our first call to onLoadChildren,
our example service would return home,
recently played, recommended, and playlists.
Now, since this is running in a car,
we recommend that media apps only
provide two levels in a tree to avoid
distracting drivers and making them
click through multiple levels while they're driving.
Now once the users pick something playable
from the browse tree, the MediaSessionService
is used to start playing music and to provide
metadata and controls to show what's currently playing.
For example, our media app that we're showing here
supports play/pause, skip forward, and skip back,
and we show that in the Android Auto UI.
There's also the ability for media
apps to provide their own custom actions, maybe something
like 30-second skip.
And obviously we want to get the user away from touching
or doing things while they're driving,
so we bring in the Assistant.
It might say something like, hey Google, play my ski jams.
The Google Assistant performs speech recognition
and can request that the MediaSessionService play
the query, and music starts playing.
We're going to take it one step farther today.
We're giving the ability for media apps
to implement an additional function
on the MediaBrowserService, onSearch.
And once the music has started playing from a Google Assistant
query, we'll provide that query to the media app,
and they can provide additional results.
Here in this case, the media app provided a ski trip playlist
from this year as well as one from last year.
So let's take a look at the code needed to make this happen.
For apps which already support Android Auto,
this should look pretty familiar.
This is the onSearch method.
It takes the query string, an extras bundle,
and a result object which the app fills in and sends back
to Android Auto.
First off, apps should return an empty list
if they get a query they don't support.
Second, for queries that can't be answered synchronously,
apps detach from the results object,
and that lets the media framework know
not to wait and not to send anything back to Android Auto
right away.
This gives a chance for apps to do extra work on a background
thread before sending the results to Android Auto.
And finally when the results are ready,
they can send the result and the result object,
and Android Auto will be notified and show the results
on screen.
Now all these code snippets come from the Universal Music
Player, an open-source media app published by Google on GitHub.
It can be easily cloned, compiled,
and used as a great reference building your own media app.
So voila.
Our media app returns a list of items from the ski jams query.
Notice it returned two playlists and an album.
It would be really nice if Android Auto
could group those items and show them to the user as groups.
Fortunately, we're introducing a way
to do that in the onSearchResults.
Here's an example function which your media app might
use to convert from an internal representation of a media item
into the MediaBrowserCompat media item
that Android Auto needs.
We can annotate items with a category extra,
and Android Auto will group any adjacent items
with the same category and show a heading.
For the two ski trip playlists we can annotate with playlist,
and Android Auto will group them together
and add the heading for you.
We're also adding some additional annotations
on media items that would be really useful on our trip.
For example, I may be heading out to the mountains
with my family.
I worry that maybe a song comes on that
has some explicit content.
We now add the ability to say, OK, this particular playlist
or song has explicit content, and Android Auto
can show that in the UI.
Similarly, out in the mountains, I
might not have great bandwidth.
I'd love to know if the playlist or songs have already
been downloaded, or maybe I don't
want to burn my data on music that I'm playing.
We can also annotate or not media items
have been downloaded and are already on the device.
Great.
Looks like the Ski Trip 2018 is already downloaded,
doesn't have any explicit content.
Great choice for my trip out to the mountains.
There's one more function that needs updating.
The MediaBrowserService onGetRoot
is called when a media app is first
connected to by Android Auto.
In order for search and for the additional styling
hints to be enabled, you'll need to add a couple of extras
to let Android Auto know that you support those features.
As I mentioned, we're introducing the concept
of additional content styling, and Android Auto
will be interpreting the browse tree returned
by apps in a much more visually pleasing way.
By default, items which are browsable, like folders,
will be interpreted as lists.
This is how we do things today.
But for playable items, things like songs or albums
or playlists, we're going to be showing them now as grids.
Most of these items have much richer visual content
that users can identify by seeing much easier than reading
and much safer when you're in the car.
There are, however, times when a list is better than a grid.
For example, in a podcast app each of the individual podcasts
would probably have individual art that is much more visually
representative while the episodes, instead, they'd
have all the same art but different episode titles
and lengths and status, and it would be much
better to show them as lists.
In the onGetRoot function, the media app
can provide a hint to Android Auto
to say I prefer my browsable items
to be grids and my playable items
to be lists or vice versa.
So they have full control over how we're showing the items.
I already mentioned the Universal Media Player.
I just want to reiterate, it's a great, comprehensive sample
media app that's available.
It gives you a canonical implementation of a media app
that actually plays music, and it also
supports Android Auto as well as other services
like Wear and Android TV.
And if you are developing a media app,
I also encourage you to check out the Android Media
Controller, another open-source app hosted on GitHub.
It will connect to your app's MediaSession
and MediaBrowseService, and it shows you information
that your app is presenting to Android Auto
in a clear, semantic format.
If you're using whitelisting to block apps other than Android
Auto from accessing your browse tree,
it's probably a good idea to either add the Media Controller
to the whitelist or disable the whitelist while testing.
So to sum up, we've shown code samples
for MessagingStyle, notification actions,
providing search results with the MediaBrowserServ
iceCompat.onSearch, attaching new extras for Media Items
metadata, and declaring support for content browse and search
in rootHints.
KODLEE YIN: So great.
We look forward to seeing all of your messaging and media apps
in the car.
Rasekh and I will be available tomorrow morning
at office hours to answer any questions
you have about Android Auto.
Thank you all for watching.
[MUSIC PLAYING]