[MUSIC PLAYING]
SASHA KAPUR: Hi, everyone.
I'm Sasha, product manager of Android Camera.
Welcome to our session where we'll
discuss how to build a high quality Android camera
experience.
Before we dive in, I want to highlight the importance
of camera experiences.
In fact, by the end of this 20-minute session,
34 million people will use their Android camera.
It's a staggering statistic and highlights
the scope and potential impact of your work
as a camera experience app developer.
What's more, we know camera to be a number one factor,
driving both initial purchase and ongoing satisfaction
with a device.
As a result, whether you're building
a payments app that needs to scan QR codes,
a marketplace app that allows users to capture a profile
photo, or a social media app that offers rich video creation
experiences, we want to make sure you're
supported in bringing a high quality
experience to your users.
Specifically, in this talk, we'll focus on three pillars.
First, we'll discuss how to create polished camera
previews built on top of Camera2 and CameraX,
including new API additions that make it easier than ever
to handle orientation changes on phones, tablets,
and even foldables.
Next, we'll discuss how to bring premium visual quality
to your app, including bringing HDR video, low light capture,
and battery savings to your users.
Then we'll end with an overview of everything else that's
new on Android 14 so that you can
leverage the most of Android in your apps.
With that, let's get started.
I'll pass it to my colleague Donovan
to discuss polished camera previews.
DONAVAN MCMURRAY: Thanks, Sasha.
My name is Donovan.
And today I'm going to talk to you
about best practices for polished camera
previews on all screen sizes and device types.
The camera preview is often the very first interaction
your users get with the camera in your app.
And there are a number of common problems we see in some apps
that I'm here to help you avoid.
First, a distorted preview can instantly
make an app feel broken and less premium.
With a number of different camera sensors and screen
sizes in the Android ecosystem, it's
hard to test every combination.
Another problem is when the preview
doesn't match what's actually captured in a photo or video.
App users expect a what you see is what you get experience.
This talk will help you with both of these issues and more.
First, I'll talk about preview stabilization.
We've had video stabilization since Android 5.
And now on devices running Android 13,
you can set video stabilization to preview stabilization
so that the final recording matches what was
shown in the camera preview.
As you can see in the diagram on the right,
if stabilization is applied to the video
but not to the preview, which is the case when
video stabilization is simply set to On,
then the preview isn't necessarily
showing what is being recorded since the device needs
to crop into the frame to perform stabilization
on the video stream.
Let's start off looking at the Camera2 code
to configure preview stabilization.
First, get the list of available video stabilization modes.
Then if that list contains preview stabilization,
set the video stabilization mode accordingly
on the capture request builder.
The CameraX code looks very similar.
But instead of a capture request builder,
we have a camera object from the call to bind to life cycle.
Just like in the last example, we first
get the available video stabilization modes
from the Camera2 camera info object.
Then if previous stabilization is available,
we use a Camera2 camera control object
to set the underlying Camera2 capture request option like so.
Next, let's take a look at some convenience classes
in our Jetpack Library offering that help with camera previews.
Our two Jetpack preview classes can help your camera app deal
with screens of all shapes and sizes.
For CameraX, you can use the preview view class.
For Camera2, the class you can use is camera viewfinder.
These classes can be configured so
that your preview will display properly
no matter what screen it's on.
Dealing with every screen size can be tricky.
If the camera stream aspect ratio
differs from your preview's aspect ratio,
naively putting the image on the screen will lead to stretching.
You have to make a decision to crop some of the image or show
bars on one dimension of the preview.
Both Jetpack preview classes give you six scale types
for properly displaying a preview.
The top row are fit scale types where the entire image is fit
into the viewport, resulting in empty space in the longer
viewport dimension.
You can specify fit start, fit center,
or fit end, represented by the left, middle, and right columns
respectively.
The bottom row are fill scale types
where the image will fill the viewport
but that means that some of the image
will be cropped out of view.
You can specify fill start, fill center, or fill end,
again going from left to right.
The default value for both camera viewport and preview
view is fill center.
Before jumping into the code, let's take a look
at a company that had success with using CameraX's preview
view.
When Evernote decided to rewrite their document scanner using
CameraX, they went all in.
As a result, they saw a number of benefits
like the ones we've already discussed.
First, CameraX's easy-to-use API made handling the camera logic
and displaying the preview simple.
Also, CameraX's ML kit analyzer seamlessly integrated
Google's ML kit library and mapped the document dimensions
to the preview.
Finally, the Evernote team was able to achieve
a blazingly fast development pace, writing and testing
their CameraX rewrite in under three weeks.
In CameraX, the code to set up a preview view is really minimal.
First, get a reference to your view layout's preview view.
Then create a lifecycle camera controller
and bind it to your activity's lifecycle.
Then set the controller on the preview view and you're done.
CameraX also provides advanced preview functionality
on top of the basics.
For example, using the built-in ML kit integration,
you can run each frame through an ML kit model
and CameraX will map the frames coordinates to your preview
coordinates automatically.
And newly added in the 1.3.0 alpha that's currently out,
you can run graphics libraries effects on your preview
and the same effects can be applied to photos and videos
to, again, give users the consistent experience
they expect.
Now let's see how this is done with camera
viewfinder in a Camera2 app.
First, create a camera viewfinder,
passing in a context.
Then call request surface async and save a reference
to the surface listenable future.
Then in the onSuccess function of the surfaceListenableFuture,
you create a camera capture session
like a typical Camera2 app but instead of providing a texture
view or surface view as target surfaces,
you'd use the resulting camera viewfinder surface.
Finally, I'm going to go over some considerations
for showing previews on large screen devices like tablets
and foldables.
The Jetpack preview classes were built
with large screens in mind so using camera viewport
or preview view as we've discussed,
your app will work well on large screens, too.
When LinkedIn switched to CameraX,
they saw a number of benefits relating to large screens.
For example, they saw improvements
in aspect ratio and rotation management on large screens.
They also benefited from CameraX's built-in multi-window
support, which handled this common user
flow on large screen devices.
Finally, they saw an overall code reduction
resulting in better code quality and maintainability.
With these tips, you can now navigate your way
through displaying a camera preview on many display sizes
and types.
Now I'll hand it over to my colleague
Mozart who will talk about premium visual quality.
MOZART LEWIS: Thanks, Donavan.
Hi.
My name is Mozart Lewis.
And let's talk about achieving premium visual quality
in Camera2 and CameraX.
We've taken feedback from developers on exactly what they
needed from Camera.
And we've landed major changes and several improvements
to the framework to improve visual quality
and fidelity of the images created by Android Camera.
Let's start with HDR video.
In Android 13, we have introduced HDR video capture,
which provides higher contrast and more colors
to encoded video.
Your camera capture will be able to produce
much brighter videos and more saturated
colors compared to SDR.
And with HDR video capture comes 10-bit color capture as well,
which enables you to capture a little over 1 billion colors
instead of shy of just 17 million.
This should make colors in video smoother and more consistent
and avoid the dreaded banding that
can occur when subtle shifts in color
are represented incorrectly.
You can use a function like this to check if the device supports
10-bit output.
Using the Camera Manager, call the getCameraCharacteristics
function with the key REQUEST_AVAILABLE_CAPABILITIES.
From there, we can check if the key REQUEST AVAILABLE
CAPABILITIE DYNAMIC RANGE TEN BIT exists
in the camera's capabilities.
To check which dynamic range profiles are supported,
we can use a function like this that
uses the camera manager to query for REQUEST AVAILABLE DYNAMIC
RANGE PROFILES.
From there, you can check, for instance,
if HDR10 Plus is supported.
Note that we are requiring that all 10-bit output
capable devices support HLG10 to assure consistency
across devices.
Other formats are at the discretion
of the device manufacturer.
With that done, we'll set up each output configuration
with a selected dynamic range profile.
For instance, here we set each configuration's dynamic range
profile to HDR10 Plus.
The last thing is to set up media format for HDR encoding.
We'll use the format.setInteger function
to set the following parameters, COLOR_FORMAT
to COLOR_FormatSurface, PROFILE to HEVCProfileMain10,
Color_Transfer to HLG or HDR10 Plus,
and COLOR_STANDARD to BT2020.
Great.
That's how you get HDR video running in your app.
Next, let's talk about stream use cases.
Before stream use cases, the camera device
was unaware of the reason you are requesting the stream.
This means it will give you the default unoptimized stream
forever.
Now with stream use cases, the stream
is tagged and will let the camera device know why
the stream is being created.
Here is a list of supported stream use cases.
DEFAULT, which covers all existing behavior, PREVIEW,
which is recommended for viewfinder or in-app image
analysis, STILL_CAPTURE and VIDEO_RECORD, video_call,
which is recommended for long-running camera
use where power drain is a concern,
and PREVIEW_VIDEO_STILL, which is
a multipurpose stream recommended
for social media apps or single stream use cases.
Let's see a demonstration of how this looks like in practice.
The preview use case can optimize for performance
and usability as a viewfinder but not necessarily for image
quality while the still capture use case will likely
optimize for high quality image capture
but it's not expected to maintain preview-like frame
rates.
You can check if the stream use case is supported
using a function like so.
Query the camera manager for the key SCALAR AVAILABLE STREAM USE
CASES.
You can then configure each output configuration
and set the Config.streamUseCase to the selected use case.
Finally, let's talk about extensions.
Device manufacturers can expose special capabilities
like night mode and bokeh to third party developers
through camera extensions.
For instance, Snap has seen success
using the night extension within their app.
Their use of the night extension dramatically
improved image quality during tough lighting conditions.
Here's a list of extensions currently available.
EXTENSION_NIGHT, useful for low light conditions;
EXTENSION_HDR, high dynamic range for still capture,
EXTENSION_AUTO automatically selects
another extension based off of OEM parameters;
EXTENSION_BOKEH, focused subject with a blurry background,
and EXTENSION_FACE_RETOUCH.
Like everything else in Camera, we
must query to assure the camera is able to support extensions.
We make a method like this to check
if a specific extension is available
based on the camera ID.
Let's see an example of creating a session
with the bokeh extension.
We would first check if the extension is supported.
Then we would implement our CameraExtensionS
ession.StateCallback.
And we will have an onConfigured and onConfiguredFailed
functions.
Instead of creating the normal session configuration object,
we create an extension session configuration.
From there, we call device.createExtensionSession.
I'm also pleased to announce that all of these features that
improve image quality are available in CameraX
as of 1.3.0-alpha.
With that, I'll pass it back to Sasha
to talk about what's up and coming
for Camera in Android 14.
SASHA KAPUR: Thanks, Mozart.
Before we close out, I want to share
three more new features that are launching in Android 14.
In Android 13, we launched HDR video capture.
Now we're adding to our HDR experiences
by launching Ultra HDR for still captures.
Ultra HDR lets you bring 10-bit HDR capture
to your users for more vibrant content.
We know AppCompat is important to developers like you
so we've designed a backwards compatible JPEG
format for ultra HDR.
Android 14 also offers P3 color space,
which means your Android app can now
display content at par with digital cinema standards
with just a few lines of code.
Lastly, we've made several zoom optimizations.
For example, Android 14 now allows
you to prioritize zoom control setup latency
and offers in-sensor zoom to increase resolution of zoomed
in viewfinders.
Together this gives users faster, crisper zoom.
Now that we've covered polished camera previews, visual quality
enhancements, and all else new in Android 14,
we'd like to end with a thank you.
We appreciate your collaboration and look forward
to seeing what you deliver to Android cameras this year.
[MUSIC PLAYING]