KARTHIC VEERA: Hello, welcome to the What's New in Android Media
KARTHIC VEERA: Hello, welcome to the What's New in Android Media

session at Google I/O. We are excited to share
session at Google I/O. We are excited to share

some new media features coming to the Android platform
some new media features coming to the Android platform

and supporting libraries that will enable our developers
and supporting libraries that will enable our developers

and OEM partners to build great media experiences.
and OEM partners to build great media experiences.

I'm Karthic Veera and I'm the Product Manager
I'm Karthic Veera and I'm the Product Manager

for Android media at Google.
for Android media at Google.

NEVIN MITAL: Hello, I'm Navin Mital,
NEVIN MITAL: Hello, I'm Navin Mital,

and I'm the Developer Relations Engineer for Android Media.
and I'm the Developer Relations Engineer for Android Media.

Today, we'll cover three main topics--
Today, we'll cover three main topics--

new platform capabilities added to Android 13,
new platform capabilities added to Android 13,

including improved support for high dynamic range
including improved support for high dynamic range

videos and spatial audio; our recently introduced Media3
videos and spatial audio; our recently introduced Media3

Jetpack APIs and updates to ExoPlayer;
Jetpack APIs and updates to ExoPlayer;

and the latest on our Performance Class
and the latest on our Performance Class

device-tiering solution that launched last year.
device-tiering solution that launched last year.

KARTHIC VEERA: Let's kick things off
KARTHIC VEERA: Let's kick things off

by introducing new platform capabilities
by introducing new platform capabilities

coming to Android 13.
coming to Android 13.

In Android 13, we wanted to focus
In Android 13, we wanted to focus

on elevating the media experience for our users
on elevating the media experience for our users

by enabling OEMs to deliver premium features,
by enabling OEMs to deliver premium features,

like HDR video and spatial audio.
like HDR video and spatial audio.

HDR provides a richer video experience
HDR provides a richer video experience

by allowing for more vibrant colors, brighter highlights,
by allowing for more vibrant colors, brighter highlights,

and darker shadows.
and darker shadows.

Android already has support for some HDR use cases
Android already has support for some HDR use cases

like playback.
like playback.

And Android OEMs have flagship devices
And Android OEMs have flagship devices

that can support HDR capture.
that can support HDR capture.

But this capture support is typically available only
But this capture support is typically available only

to the native camera app.
to the native camera app.

And the lack of support in the platform
And the lack of support in the platform

for use cases like editing has made
for use cases like editing has made

it difficult for app developers to use this feature.
it difficult for app developers to use this feature.

We believe apps should have the ability to capture, edit,
We believe apps should have the ability to capture, edit,

share, and play videos in HDR.
share, and play videos in HDR.

So in Android 13, we are standardizing support
So in Android 13, we are standardizing support

for HDR in the platform and exposing the capabilities
for HDR in the platform and exposing the capabilities

to developers to build great HDR experiences.
to developers to build great HDR experiences.

To enable this end-to-end experience,
To enable this end-to-end experience,

the platform has added support for foundational elements
the platform has added support for foundational elements

of HDR video, like 10-bit of support, metadata,
of HDR video, like 10-bit of support, metadata,

and colorspace handling.
and colorspace handling.

We have also introduced new developer APIs
We have also introduced new developer APIs

for capture and accessing HDR buffers.
for capture and accessing HDR buffers.

We understand transcoding and editing of HDR videos
We understand transcoding and editing of HDR videos

can be difficult, so we are also working
can be difficult, so we are also working

on adding support for these use cases
on adding support for these use cases

to our Media3 Jetpack library.
to our Media3 Jetpack library.

We expect this to simplify operations
We expect this to simplify operations

like format conversion or composing HDR and SDR
like format conversion or composing HDR and SDR

assets for developers.
assets for developers.

Note that rendering of HDR video requires SurfaceView.
Note that rendering of HDR video requires SurfaceView.

Developers looking to support HDR using TextureView
Developers looking to support HDR using TextureView

should migrate to SurfaceView to enable HDR use cases.
should migrate to SurfaceView to enable HDR use cases.

Providing a high-quality HDR video experience
Providing a high-quality HDR video experience

is complex and heavily depends on the underlying hardware
is complex and heavily depends on the underlying hardware

capabilities.
capabilities.

Android has been working closely with our OEM and silicon
Android has been working closely with our OEM and silicon

partners to optimize their integration
partners to optimize their integration

and ensure reliable and consistent performance.
and ensure reliable and consistent performance.

Later this year, updates to Media3 Jetpack libraries
Later this year, updates to Media3 Jetpack libraries

will make it easier for developers
will make it easier for developers

to support HDR and take advantage of the underlying
to support HDR and take advantage of the underlying

hardware, including HDR capabilities of premium
hardware, including HDR capabilities of premium

devices using Snapdragon.
devices using Snapdragon.

Multiple formats are available to represent HDR videos.
Multiple formats are available to represent HDR videos.

The diversity of the Android ecosystem
The diversity of the Android ecosystem

means our users, developers, and OEMs
means our users, developers, and OEMs

have different points of view when it
have different points of view when it

comes to selecting HDR formats.
comes to selecting HDR formats.

Standardization of HDR support in Android 13
Standardization of HDR support in Android 13

allows our audience to integrate HDR formats of their choice.
allows our audience to integrate HDR formats of their choice.

While this approach allows for customized experiences on OEM
While this approach allows for customized experiences on OEM

devices, we want to ensure developers
devices, we want to ensure developers

have consistent support.
have consistent support.

To address this, we are requiring OEM devices
To address this, we are requiring OEM devices

supporting HDR to support certain base
supporting HDR to support certain base

formats in addition to additional formats.
formats in addition to additional formats.

To support playback of professional content,
To support playback of professional content,

we require HDR10 as the base format.
we require HDR10 as the base format.

For capture and playback of user-generated content,
For capture and playback of user-generated content,

we recommend using HLD10 for the most consistent experience
we recommend using HLD10 for the most consistent experience

when sharing videos across devices.
when sharing videos across devices.

Other formats, including Dolby Vision,
Other formats, including Dolby Vision,

can be additionally enabled by OEMs.
can be additionally enabled by OEMs.

We expect more and more OEMs to offer HDR video capture
We expect more and more OEMs to offer HDR video capture

support in their native camera app
support in their native camera app

due to the growing popularity of HDR videos.
due to the growing popularity of HDR videos.

Hence, it is important to ensure your apps are well
Hence, it is important to ensure your apps are well

tested to ingest HDR content that are present in the user's
tested to ingest HDR content that are present in the user's

video gallery.
video gallery.

We recommend your app pipelines are
We recommend your app pipelines are

updated to support 10-bit HDR content to maintain
updated to support 10-bit HDR content to maintain

HDR quality.
HDR quality.

Or use the transformer APIs that will be added to the Media3
Or use the transformer APIs that will be added to the Media3

Jetpack library to properly tone-map HDR videos to SDR
Jetpack library to properly tone-map HDR videos to SDR

for handling by your app.
for handling by your app.

In addition to interesting HDR content from the user's
In addition to interesting HDR content from the user's

gallery, we also recommend apps that support capture
gallery, we also recommend apps that support capture

use cases to take advantage of HDR
use cases to take advantage of HDR

capture on supported devices to capture and share
capture on supported devices to capture and share

content in HDR to improve the user experience.
content in HDR to improve the user experience.

Another premium feature we are excited to introduce
Another premium feature we are excited to introduce

Spatial Audio.
Spatial Audio.

Following upon our announcement at CES,
Following upon our announcement at CES,

the Android audio framework is adding
the Android audio framework is adding

support for Spatial Audio.
support for Spatial Audio.

This includes both static Spatial Audio
This includes both static Spatial Audio

and dynamic Spatial Audio with head tracking.
and dynamic Spatial Audio with head tracking.

OEMs looking to enable this feature
OEMs looking to enable this feature

can take advantage of standardized platform
can take advantage of standardized platform

architecture to integrate multichannel codecs
architecture to integrate multichannel codecs

of their choice.
of their choice.

The new architecture allows for lower latency head tracking
The new architecture allows for lower latency head tracking

and integration with the codec agnostic spatializer.
and integration with the codec agnostic spatializer.

The architecture also allows for extensibility
The architecture also allows for extensibility

to new use cases in the future.
to new use cases in the future.

App developers will add public APIs
App developers will add public APIs

for device capability detection and multichannel audio control.
for device capability detection and multichannel audio control.

Content is critical to enable a great experience for our users.
Content is critical to enable a great experience for our users.

So we are working closely with content providers
So we are working closely with content providers

to ensure these APIs meet their needs.
to ensure these APIs meet their needs.

NEVIN MITAL: With Android 12L, we released the new Spatializer
NEVIN MITAL: With Android 12L, we released the new Spatializer

class.
class.

This API allows you to check the current device's ability
This API allows you to check the current device's ability

to output specialized audio.
to output specialized audio.

In this code sample, we're verifying three criteria.
In this code sample, we're verifying three criteria.

First, we check if the device has
First, we check if the device has

specialization capabilities.
specialization capabilities.

Next, we check if specialization is enabled on the device.
Next, we check if specialization is enabled on the device.

This is controlled by the user through a system-level setting.
This is controlled by the user through a system-level setting.

And lastly, we check if the spatializer
And lastly, we check if the spatializer

is able to spatialize the currently playing audio track.
is able to spatialize the currently playing audio track.

This, for example, can be affected
This, for example, can be affected

by whether the audio is being routed
by whether the audio is being routed

through a supported audio output device.
through a supported audio output device.

Android 13 will include further improvements
Android 13 will include further improvements

to the developer experience with both a standard spatializer
to the developer experience with both a standard spatializer

included in the platform and a standard head-tracking protocol
included in the platform and a standard head-tracking protocol

to allow for interoperability with a large number
to allow for interoperability with a large number

of headsets.
of headsets.

ExoPlayer makes it simple to try Spatial Audio out for yourself.
ExoPlayer makes it simple to try Spatial Audio out for yourself.

The 2.17 release of ExoPlayer includes a couple updates
The 2.17 release of ExoPlayer includes a couple updates

to configure the platform for multichannel Spatial
to configure the platform for multichannel Spatial

Audio right out of the box.
Audio right out of the box.

This means that, by default, ExoPlayer
This means that, by default, ExoPlayer

enables specialization behavior and configures the decoder
enables specialization behavior and configures the decoder

to output a multichannel stream on Android 12L
to output a multichannel stream on Android 12L

and up when possible.
and up when possible.

All you have to do is include a multichannel audio track
All you have to do is include a multichannel audio track

in your media content.
in your media content.

Future updates to ExoPlayer are planned
Future updates to ExoPlayer are planned

to bring further improvements, such as optimizations
to bring further improvements, such as optimizations

in the multichannel audio track selection logic.
in the multichannel audio track selection logic.

KARTHIC VEERA: Another new capability in Android 13
KARTHIC VEERA: Another new capability in Android 13

is support for MIDI 2.0, a new standard that
is support for MIDI 2.0, a new standard that

brings huge advancement to music technology.
brings huge advancement to music technology.

Users are able to connect their MIDI 2.0 hardware directly
Users are able to connect their MIDI 2.0 hardware directly

to an Android device through USB.
to an Android device through USB.

The new standard offers many improvements over MIDI 1.0,
The new standard offers many improvements over MIDI 1.0,

such as increased resolution for controllers,
such as increased resolution for controllers,

better support for non-Western annotations,
better support for non-Western annotations,

and more expressive performance using per-note controllers.
and more expressive performance using per-note controllers.

Next, we have some updates to share
Next, we have some updates to share

regarding Media3 and ExoPlayer.
regarding Media3 and ExoPlayer.

You may remember, we announced the alpha release
You may remember, we announced the alpha release

of Media3 Jetpack libraries at the Android Developer Summit
of Media3 Jetpack libraries at the Android Developer Summit

in 2021.
in 2021.

This is a collection of support libraries
This is a collection of support libraries

for implementing media use cases on Android, including ExoPlayer
for implementing media use cases on Android, including ExoPlayer

for playback.
for playback.

Media3 will also be the home for functionality beyond playback.
Media3 will also be the home for functionality beyond playback.

For example, we are working on giving developers
For example, we are working on giving developers

better tools for transcoding and video editing.
better tools for transcoding and video editing.

To get a preview of this, you can try out
To get a preview of this, you can try out

the media3-transformer library.
the media3-transformer library.

NEVIN MITAL: Stay tuned to our media developer documentation
NEVIN MITAL: Stay tuned to our media developer documentation

on developer.android.com for more guidance
on developer.android.com for more guidance

on migrating to Media3.
on migrating to Media3.

For ExoPlayer, this will primarily
For ExoPlayer, this will primarily

involve updating the package name, as most of the APIs
involve updating the package name, as most of the APIs

have stayed the same.
have stayed the same.

You can get started today by updating to the latest
You can get started today by updating to the latest

version of ExoPlayer, which will make the Media3 migration as
version of ExoPlayer, which will make the Media3 migration as

simple as possible.
simple as possible.

We're currently in a transitional period
We're currently in a transitional period

where we're publishing parallel releases of ExoPlayer
where we're publishing parallel releases of ExoPlayer

and Media3.
and Media3.

Once Media3 is released to stable later this year,
Once Media3 is released to stable later this year,

Jetpack will be the home for the latest ExoPlayer updates.
Jetpack will be the home for the latest ExoPlayer updates.

Updating to the latest version of ExoPlayer
Updating to the latest version of ExoPlayer

will give you access to a number of new features
will give you access to a number of new features

that we've rolled out in the past few releases.
that we've rolled out in the past few releases.

ExoPlayer 2.14 starts requesting DRM keys
ExoPlayer 2.14 starts requesting DRM keys

when the content is buffering rather
when the content is buffering rather

than at the point of playback.
than at the point of playback.

This eliminates a short pause in playback
This eliminates a short pause in playback

when transitioning to protected content.
when transitioning to protected content.

ExoPlayer 2.16 introduces key updates
ExoPlayer 2.16 introduces key updates

to the WorkManager extension and DownloadService
to the WorkManager extension and DownloadService

necessary to work correctly with Android 12.
necessary to work correctly with Android 12.

And ExoPlayer 2.17 added support for server-side ad insertion,
And ExoPlayer 2.17 added support for server-side ad insertion,

in addition to the client-side ad insertion that ExoPlayer
in addition to the client-side ad insertion that ExoPlayer

already supports.
already supports.

We first discussed performance class last year,
We first discussed performance class last year,

a device property that reflects a device's level of performance
a device property that reflects a device's level of performance

and overall capabilities to help developers classify devices.
and overall capabilities to help developers classify devices.

The requirements to meet a certain performance class
The requirements to meet a certain performance class

rating are specified in the Android compatibility
rating are specified in the Android compatibility

definition document and verified by the Android compatibility
definition document and verified by the Android compatibility

test suite.
test suite.

Recently, we introduced the new Core Performance Library
Recently, we introduced the new Core Performance Library

as part of Jetpack.
as part of Jetpack.

This library will be the best way
This library will be the best way

to obtain a device's performance class
to obtain a device's performance class

in order to tailor your app's user experience accordingly.
in order to tailor your app's user experience accordingly.

For devices running Android 12 and above,
For devices running Android 12 and above,

the library will first check for the media performance class
the library will first check for the media performance class

advertised by the build API.
advertised by the build API.

For devices running Android 11 or those not advertising
For devices running Android 11 or those not advertising

a performance class, this library
a performance class, this library

will instead report the property based on device certification
will instead report the property based on device certification

test results or additional manual testing done by Google.
test results or additional manual testing done by Google.

To start using this library, include the core performance
To start using this library, include the core performance

dependency in your build.gradle file.
dependency in your build.gradle file.

Then, to retrieve the device's performance class,
Then, to retrieve the device's performance class,

first create an instance of device performance
first create an instance of device performance

in your application's onCreate life cycle event.
in your application's onCreate life cycle event.

With that done, you can get the media performance class
With that done, you can get the media performance class

property wherever you want to tailor
property wherever you want to tailor

your app's experience based on the device's capabilities.
your app's experience based on the device's capabilities.

In this example, we're using the reported performance class
In this example, we're using the reported performance class

to adjust a video's resolution for the best user experience.
to adjust a video's resolution for the best user experience.

KARTHIC VEERA: Building off of Performance Class 12,
KARTHIC VEERA: Building off of Performance Class 12,

Performance Class 13 introduces a number of new requirements
Performance Class 13 introduces a number of new requirements

and updates.
and updates.

These include increased memory requirement,
These include increased memory requirement,

higher requirements for concurrent codec sessions,
higher requirements for concurrent codec sessions,

lower limits for codec latency and frame drops.
lower limits for codec latency and frame drops.

We are also adding requirement for AV1 hardware decoder,
We are also adding requirement for AV1 hardware decoder,

secure decoders in hardware, and round trip audio latency.
secure decoders in hardware, and round trip audio latency.

For camera, we are adding requirements
For camera, we are adding requirements

around preview stabilization, slow-mo recording,
around preview stabilization, slow-mo recording,

minimum zoom ratios for ultra wide, and concurrent camera.
minimum zoom ratios for ultra wide, and concurrent camera.

We hope these additions will help developers improve media
We hope these additions will help developers improve media

experience for their users.
experience for their users.

For instance, we expect the addition of AV1 hardware
For instance, we expect the addition of AV1 hardware

decoder to increase the adoption of AV1
decoder to increase the adoption of AV1

codec to deliver better video quality for users.
codec to deliver better video quality for users.

Netflix especially has been an early adopter of AV1,
Netflix especially has been an early adopter of AV1,

taking advantage of AV1's greater compression efficiency
taking advantage of AV1's greater compression efficiency

to deliver videos with better quality on Android devices.
to deliver videos with better quality on Android devices.

In addition to content playback use cases,
In addition to content playback use cases,

AV1 usage can significantly improve visual quality
AV1 usage can significantly improve visual quality

when sharing videos captured on the device.
when sharing videos captured on the device.

Google Duo has used AV1 codec for both encode and decode
Google Duo has used AV1 codec for both encode and decode

in low-bandwidth conditions to improve video call quality.
in low-bandwidth conditions to improve video call quality.

To learn more about these features, please
To learn more about these features, please

visit the following links.
visit the following links.

That's all we have for you today.
That's all we have for you today.

NEVIN MITAL: Thanks for joining us,
NEVIN MITAL: Thanks for joining us,

and have a great Google I/O.
and have a great Google I/O.

[MUSIC PLAYING]
[MUSIC PLAYING]