Hello, everyone, my name is Francois Goldfain,
and I am the engineering director for Android Media.
Hello, I’m Karthic Veera, and I’m the product manager for Android Media.
We’re pleased to introduce some new media features
in the Android platform and supporting libraries.
In this video, we cover four main topics: video quality and how to capture, share,
and play the best quality video content for your users.
A new device performance class,
which enables you to customize the user experience,
based on the capabilities of the device.
We'll also cover what's new in ExoPlayer.
And lastly, give an update on real-time audio for apps that need low-latency audio.
Let's start with an area,
which has been a key area of focus for us over the past year, video quality.
Users deeply care about the quality of videos they post.
The videos users record and store on their devices are typically captured
in high resolution and bit-rate, resulting in great video quality.
But when users try to share these videos,
the visual quality is often significantly reduced
because of the stringent conditions applications have to operate under.
We understand the media pipeline in most apps is quite complex.
Applications have to manage capture, upload,
and delivery of content as it passes from the source device
to the application backend and finally the destination device.
As content passes through this pipeline,
it is subjected to a number of operations like encoding, decoding, and scaling.
The operations highlighted here in yellow,
like encoding and scaling, can be lossy and impact the quality of the final video.
It is very important for developers to have an end to end view of this pipeline
and carefully choose operational parameters that result in minimal quality loss.
Some best practices to ensure high-quality video capture and sharing include:
using higher bit-rates for encoding for both capture and upload.
This can be achieved using a single line of code
when creating a mediaRecorder object.
Minimizing the number of scaling operations
by ensuring the capture resolution matches the upload resolution,
or maintaining the aspect ratio of the video when it is not possible to avoid scaling.
Using higher profiles like main profile
instead of baseline profile to improve the efficiency when using AVC coding.
Here is how this looks in code.
We just call setVideoEncodingProfileLevel when setting up our media recorder object.
If you’d like to learn more on best practices,
please go to the developer.android.com site for our guide on video quality.
Talking about best practices, using new compression formats
is a great way to improve visual quality.
New formats offer better quality at lower storage and bandwidth cost.
In Android 11, we introduced support for AV1, a royalty-free video codec
and in Android 12, we added support for AVIF, a new image format based on the AV1 Codec.
AVIF images have similar quality at 1/3 the size of JPEG
and we expect this format to gain rapid adoption in the coming years.
Long-term, we see AV1 as the best codec for video and image content.
However, while hardware availability is still limited,
we recommend using HEVC as it offers significant quality
and file size improvements over older codecs.
In this example, a video encoded with HEVC has the same quality at half the bit-rate
of the same video encoded with AVC.
Decoding HEVC has been supported in Android for some time now.
And HEVC encoding is widely supported by premium devices.
Also beginning with Android 12, more OEMs will be using HEVC
as the default format for video capture
to improve the video quality at lower storage costs to users.
Hence it is very important to ensure your apps work well
injesting content that has been encoded in HEVC.
We understand some apps may not be ready to support HEVC
by the time Android 12 devices launch.
For those apps, we introduce a transcoding feature in the platform.
This will transcode HEVC and other formats into older, more compatible AVC format.
It is important to note that transcoding doesn't happen instantaneously.
So if you're using this feature, make sure you test your app
to account for the extra delay when opening HEVC files.
In particular, make sure you do not do any I/O operations
from the app’s main thread as this may lead to ANRs.
To use it simply declare which media formats
your app doesn't support using a media capabilities file.
Android Studio can automatically create this file for you by running lint checker.
In Android 12, we are introducing a new way for developers
to classify devices based on their capabilities.
We are calling this the device performance class.
We know that many of you already optimize your apps
so that users with flagship handsets enjoy the premium experience they expect,
while others may receive an experience that is a little scaled down to remain functional.
Using Performance Class API, you no longer have to rely on unscalable solutions
or on arbitrary criteria, such as device RAM and CPU speed
to act as unreliable proxies for the overall device performance.
We define performance criteria based on input from app developers.
Initial feedback from Snap was very positive.
We believe using Device Performance Class will help developers better optimize
their apps for the wide range of Android devices.
For a device to obtain a particular performance class,
you must meet requirements laid out
in the Android Compatibility Definition Documentor, or CDD for short.
Those requirements are verified by corresponding tests that we added
in the Android Compatibility Test Suite or CTS.
This means that as a developer you can rely on devices which report
their performance class who have met all requirements set out in the CDD.
It also means, for every Android release,
the performance class requirements will be updated.
The performance class is not tied to the Android version.
Let's look at a couple of examples.
A device launching was Android 12 can meet the performance class criteria of Android 12
and remain at Performance Class 12 when it upgrades to Android 13.
Another device function with Android 12 and Performance Class 12
can claim Performance Class 13 on upgrade if it meets the updated requirements.
For Android 12, the Performance Class definition
is focused on the needs of applications with media use cases.
It includes generic requirements, such as total device RAM,
read-write performance, screen resolution and density,
but also media-specific requirements, such as the number of concurrent codec session,
the latency to initialize a codec,
the amount of frame drops during playback and the quality of encoded videos.
As well as camera requirements such as resolution initialization
and capture latency, support for video stabilization and HDR image capture.
Here's how it works in code.
There is a new media performance class API available as part of the build class.
You use it to check device performance class,
the same way as you currently check for device API level.
We also plan to have this API available as part of Jetpack
so you can use it for devices running Android 11.
As you probably know,
our recommended way to play media content is to use ExoPlayer.
We released significant updates to this library over the past year.
Let’s go over some of the new features.
Starting with the ExoPlayer 2.12,
you no longer have to use concatenating media source to build a playlist.
Instead, you can do this directly with the player’s top-level APIs.
You need to be on media items and add them to the player.
The player will automatically preload for the right item at the right time
and seamlessly transition to the next item.
You can also build the playlist at any time with a move and remove APIs.
The player will discard, and pre-build for the right content all automatically.
Here’s a code snippet that illustrates this.
You can directly build a media item from a URI.
Or you use its builder class if you need to build more complex items.
Once you have media items created, you just add those items to the player.
If needed, you can also edit the playlist anytime,
even if the player is already playing.
You can also monetize your content by adding a different AdTagURI to each media item.
The ad transitions will be seamless just like any other playlist transitions.
We also added two UI components to ExoPlayer.
StyledPlayerView and StyledPlayerControlView.
These have similar APIs to the existing player view
and player control of components but provided more polished user experience
including animation effects when hiding and showing the controls,
built-in support for subtitles and audio track selection,
built-in support for playback speed selection and many more features.
If you do not need to customize the player view or control,
then just migrate your existing code to use these two new classes.
ExoPlayer 2.13 includes a new transformer library module.
Unlike the other modules,
the main purpose of this transformer API is not media playback.
Instead it takes an input media stream,
applies changes to it as configured by the app,
and produces a corresponding output file.
In this initial version, the available transformations
are transmuxing between container formats, track removal
and the flattening of slow-motion video files
which allows applications to use them as normal videos
while retaining the slow-motion effects.
We have plans for other transformations in the future, so stay tuned.
Another new feature of ExoPlayer 2.13
is that it now supports low-latency live streaming protocols
Low-Latency DASH and Apple’s Low-Latency HLS extension.
If you play low-latency streams in ExoPlayer,
just provide the manifest to the player via MediaItem.
There is no special configuration required.
You can choose to fine tune the behavior
by using the various setLive methods when building the menu item.
Talking of low latency, our final topic is real-time audio.
We’ve done a lot of work in the Android platform
and with OEMs to bring down audio latency.
Over the past four years,
we have seen a significant reduction in the average latency
of popular Android devices.
Dropping from over a hundred milliseconds in 2017
to now under 40 milliseconds.
We're working hard to bring this down further
and to provide the tools which you need to build
great real-time audio apps.
As you may be aware, Oboe is now the recommended library
for building apps which need low-latency audio.
It’s an open-source C++ library, which is focused on performance
and is backward compatible to API 16.
The latest release includes a bunch of new features and bug fixes including:
sample rate, format and channel count conversion
which allows you to obtain a low-latency audio stream
in a wide variety of scenarios.
We have also made the callback architecture simpler
by having separate callbacks for audio data and error conditions.
Lastly, we know that compiling from source code
is sometimes an extra hurdle you don't need.
So Oboe is now distributed via Maven
and can be included in your app by adding a single line to your build configuration.
That's about all we have time for.
You can find more information about all these features using the following links.
You can provide feedback on any Android 12 features by following the first link.
We’d love to hear from you. Thank you.