[MUSIC PLAYING]
KARTHIC VEERA: Hello, welcome to the What's New in Android Media
session at Google I/O. We are excited to share
some new media features coming to the Android platform
and supporting libraries that will enable our developers
and OEM partners to build great media experiences.
I'm Karthic Veera and I'm the Product Manager
for Android media at Google.
NEVIN MITAL: Hello, I'm Navin Mital,
and I'm the Developer Relations Engineer for Android Media.
Today, we'll cover three main topics--
new platform capabilities added to Android 13,
including improved support for high dynamic range
videos and spatial audio; our recently introduced Media3
Jetpack APIs and updates to ExoPlayer;
and the latest on our Performance Class
device-tiering solution that launched last year.
KARTHIC VEERA: Let's kick things off
by introducing new platform capabilities
coming to Android 13.
In Android 13, we wanted to focus
on elevating the media experience for our users
by enabling OEMs to deliver premium features,
like HDR video and spatial audio.
HDR provides a richer video experience
by allowing for more vibrant colors, brighter highlights,
and darker shadows.
Android already has support for some HDR use cases
like playback.
And Android OEMs have flagship devices
that can support HDR capture.
But this capture support is typically available only
to the native camera app.
And the lack of support in the platform
for use cases like editing has made
it difficult for app developers to use this feature.
We believe apps should have the ability to capture, edit,
share, and play videos in HDR.
So in Android 13, we are standardizing support
for HDR in the platform and exposing the capabilities
to developers to build great HDR experiences.
To enable this end-to-end experience,
the platform has added support for foundational elements
of HDR video, like 10-bit of support, metadata,
and colorspace handling.
We have also introduced new developer APIs
for capture and accessing HDR buffers.
We understand transcoding and editing of HDR videos
can be difficult, so we are also working
on adding support for these use cases
to our Media3 Jetpack library.
We expect this to simplify operations
like format conversion or composing HDR and SDR
assets for developers.
Note that rendering of HDR video requires SurfaceView.
Developers looking to support HDR using TextureView
should migrate to SurfaceView to enable HDR use cases.
Providing a high-quality HDR video experience
is complex and heavily depends on the underlying hardware
capabilities.
Android has been working closely with our OEM and silicon
partners to optimize their integration
and ensure reliable and consistent performance.
Later this year, updates to Media3 Jetpack libraries
will make it easier for developers
to support HDR and take advantage of the underlying
hardware, including HDR capabilities of premium
devices using Snapdragon.
Multiple formats are available to represent HDR videos.
The diversity of the Android ecosystem
means our users, developers, and OEMs
have different points of view when it
comes to selecting HDR formats.
Standardization of HDR support in Android 13
allows our audience to integrate HDR formats of their choice.
While this approach allows for customized experiences on OEM
devices, we want to ensure developers
have consistent support.
To address this, we are requiring OEM devices
supporting HDR to support certain base
formats in addition to additional formats.
To support playback of professional content,
we require HDR10 as the base format.
For capture and playback of user-generated content,
we recommend using HLD10 for the most consistent experience
when sharing videos across devices.
Other formats, including Dolby Vision,
can be additionally enabled by OEMs.
We expect more and more OEMs to offer HDR video capture
support in their native camera app
due to the growing popularity of HDR videos.
Hence, it is important to ensure your apps are well
tested to ingest HDR content that are present in the user's
video gallery.
We recommend your app pipelines are
updated to support 10-bit HDR content to maintain
HDR quality.
Or use the transformer APIs that will be added to the Media3
Jetpack library to properly tone-map HDR videos to SDR
for handling by your app.
In addition to interesting HDR content from the user's
gallery, we also recommend apps that support capture
use cases to take advantage of HDR
capture on supported devices to capture and share
content in HDR to improve the user experience.
Another premium feature we are excited to introduce
Spatial Audio.
Following upon our announcement at CES,
the Android audio framework is adding
support for Spatial Audio.
This includes both static Spatial Audio
and dynamic Spatial Audio with head tracking.
OEMs looking to enable this feature
can take advantage of standardized platform
architecture to integrate multichannel codecs
of their choice.
The new architecture allows for lower latency head tracking
and integration with the codec agnostic spatializer.
The architecture also allows for extensibility
to new use cases in the future.
App developers will add public APIs
for device capability detection and multichannel audio control.
Content is critical to enable a great experience for our users.
So we are working closely with content providers
to ensure these APIs meet their needs.
NEVIN MITAL: With Android 12L, we released the new Spatializer
class.
This API allows you to check the current device's ability
to output specialized audio.
In this code sample, we're verifying three criteria.
First, we check if the device has
specialization capabilities.
Next, we check if specialization is enabled on the device.
This is controlled by the user through a system-level setting.
And lastly, we check if the spatializer
is able to spatialize the currently playing audio track.
This, for example, can be affected
by whether the audio is being routed
through a supported audio output device.
Android 13 will include further improvements
to the developer experience with both a standard spatializer
included in the platform and a standard head-tracking protocol
to allow for interoperability with a large number
of headsets.
ExoPlayer makes it simple to try Spatial Audio out for yourself.
The 2.17 release of ExoPlayer includes a couple updates
to configure the platform for multichannel Spatial
Audio right out of the box.
This means that, by default, ExoPlayer
enables specialization behavior and configures the decoder
to output a multichannel stream on Android 12L
and up when possible.
All you have to do is include a multichannel audio track
in your media content.
Future updates to ExoPlayer are planned
to bring further improvements, such as optimizations
in the multichannel audio track selection logic.
KARTHIC VEERA: Another new capability in Android 13
is support for MIDI 2.0, a new standard that
brings huge advancement to music technology.
Users are able to connect their MIDI 2.0 hardware directly
to an Android device through USB.
The new standard offers many improvements over MIDI 1.0,
such as increased resolution for controllers,
better support for non-Western annotations,
and more expressive performance using per-note controllers.
Next, we have some updates to share
regarding Media3 and ExoPlayer.
You may remember, we announced the alpha release
of Media3 Jetpack libraries at the Android Developer Summit
in 2021.
This is a collection of support libraries
for implementing media use cases on Android, including ExoPlayer
for playback.
Media3 will also be the home for functionality beyond playback.
For example, we are working on giving developers
better tools for transcoding and video editing.
To get a preview of this, you can try out
the media3-transformer library.
NEVIN MITAL: Stay tuned to our media developer documentation
on developer.android.com for more guidance
on migrating to Media3.
For ExoPlayer, this will primarily
involve updating the package name, as most of the APIs
have stayed the same.
You can get started today by updating to the latest
version of ExoPlayer, which will make the Media3 migration as
simple as possible.
We're currently in a transitional period
where we're publishing parallel releases of ExoPlayer
and Media3.
Once Media3 is released to stable later this year,
Jetpack will be the home for the latest ExoPlayer updates.
Updating to the latest version of ExoPlayer
will give you access to a number of new features
that we've rolled out in the past few releases.
ExoPlayer 2.14 starts requesting DRM keys
when the content is buffering rather
than at the point of playback.
This eliminates a short pause in playback
when transitioning to protected content.
ExoPlayer 2.16 introduces key updates
to the WorkManager extension and DownloadService
necessary to work correctly with Android 12.
And ExoPlayer 2.17 added support for server-side ad insertion,
in addition to the client-side ad insertion that ExoPlayer
already supports.
We first discussed performance class last year,
a device property that reflects a device's level of performance
and overall capabilities to help developers classify devices.
The requirements to meet a certain performance class
rating are specified in the Android compatibility
definition document and verified by the Android compatibility
test suite.
Recently, we introduced the new Core Performance Library
as part of Jetpack.
This library will be the best way
to obtain a device's performance class
in order to tailor your app's user experience accordingly.
For devices running Android 12 and above,
the library will first check for the media performance class
advertised by the build API.
For devices running Android 11 or those not advertising
a performance class, this library
will instead report the property based on device certification
test results or additional manual testing done by Google.
To start using this library, include the core performance
dependency in your build.gradle file.
Then, to retrieve the device's performance class,
first create an instance of device performance
in your application's onCreate life cycle event.
With that done, you can get the media performance class
property wherever you want to tailor
your app's experience based on the device's capabilities.
In this example, we're using the reported performance class
to adjust a video's resolution for the best user experience.
KARTHIC VEERA: Building off of Performance Class 12,
Performance Class 13 introduces a number of new requirements
and updates.
These include increased memory requirement,
higher requirements for concurrent codec sessions,
lower limits for codec latency and frame drops.
We are also adding requirement for AV1 hardware decoder,
secure decoders in hardware, and round trip audio latency.
For camera, we are adding requirements
around preview stabilization, slow-mo recording,
minimum zoom ratios for ultra wide, and concurrent camera.
We hope these additions will help developers improve media
experience for their users.
For instance, we expect the addition of AV1 hardware
decoder to increase the adoption of AV1
codec to deliver better video quality for users.
Netflix especially has been an early adopter of AV1,
taking advantage of AV1's greater compression efficiency
to deliver videos with better quality on Android devices.
In addition to content playback use cases,
AV1 usage can significantly improve visual quality
when sharing videos captured on the device.
Google Duo has used AV1 codec for both encode and decode
in low-bandwidth conditions to improve video call quality.
To learn more about these features, please
visit the following links.
That's all we have for you today.
NEVIN MITAL: Thanks for joining us,
and have a great Google I/O.
[MUSIC PLAYING]