Hi, my name's Scott, and I'm engineer on the CameraX team.
Today I am very excited to help you get to know the CameraX beta release.
For those who don't know what CameraX is
CameraX is a Jetpack library
designed to make camera app development easier.
As developers, we know writing a camera app is not an easy task.
First, you have to deal with Camera2 API
which is powerful and flexible
but it is challenging to do things right.
Second, given the large number of Android devices in the market
it's time consuming to achieve a consistent experience
across all the device types.
Often this requires a long running if/else section
within your code.
And finally, if you're publishing your app
on the Play Store, testing on a real device is a must
and testing across a large number of devices
can be both costly and time consuming
as your code may require device-specific phases
to allow for a consistent experience.
With CameraX, we fixed all of these problems for you.
We provide an easy-to-use API that is easy to understand
and significantly reduced the total amount of code you must write.
It integrates the flexibility of Camera2
and abstracts away the difficult bits in favor of use case best approach.
Currently, three use cases are supported.
Preview is for displaying the camera preview.
ImageCapture is for high quality still image capture.
ImageAnalysis is for frame buffer access
for analysis like object detection.
And we PreviewView
to handle the complexity of the preview transform for you.
Second, CameraX ensures applications have
consistent experience across 94% of Android devices
starting from API 21.
We add a lot of Camera2 API-specific
Android version-specific, and device version-specific phases under the hood
so developers get a seamless experience
regarding software device.
And finally, we invest a lot in our test lab
so you don't have to.
Our CameraX test labs runs 24 hours a day, 7 days a week.
It has devices across a wide range of OEMs and Android versions.
With CameraX we take care of the device compatibility issues for you
so you can focus more on the application you want to write.
Now, we know what benefits CameraX offers
let's talk about what's new in the first half of 2020.
Device compatibility is always our first priority
and we've put a lot of effort into fixing device issues
and improving test lab infrastructure
in terms of scale and the test coverage.
And for the preview, we have added more API to the PreviewView
in order to satisfy developers' needs.
And for the needs of advanced developers
we added an OpenGL sample and an AOSP repository.
This is a guidance for developers who want to use OpenGL on top of CameraX.
And for the ImageAnalysis
we have heard from developers that two of the most common functions they need are
first, convert from YUV to RGB.
And second, use ML Kit or TensorFlow Lite
to analyze the image retrieved from ImageAnalysis.
We have introduced samples for both of these needs
and hopefully
it can help you implement a functionality you need quickly.
Let's dive into the details of each item.
Ever since CameraX reached beta
we have been focusing on the stability and the device compatibility.
And thanks to the feedback from developers and partners
we are about to address many device issues
like wrong preview aspect ratio
fail to switch camera or tap-to-focus not working.
These kind of issues are very frustrating
because you wish it would just work on all devices.
This is why we are committed to facing any device issue we have found
to provide you a consistent API that just works on all devices.
And we have learned that our test cases are not enough for some scenarios.
So we have added more test cases for the new scenarios
that could potentially cause the issues.
For example, we have enhanced our rotation
UI widget integration, and aspect ratio test.
And we are happy to announce that
that the capacity of our CameraX test labs has grown to three times the size
compared to what we had last year.
We now have 88 devices
representing 400 million active Android devices running in our lab
and we will keep adding more in the future.
PreviewView is a very useful and easy UI widget
for application to display a preview.
It requires no transform
and what's better
PreviewView makes your application more power-efficient and low-latency.
It does this by using SurfaceView under the hood.
And if the device is not compatible with SurfaceView
it will fall back to tetra view automatically.
It also supports scaleType
so you can choose how to scale a preview instead of PreviewView.
It also makes it very easy to implement a tap-to-focus.
And today we added two APIs to the PreviewView.
First, getPreviewStreamState API.
This API allows application to be notified
when a preview is streaming or idle.
This is useful for applications
who want to avoid displaying a temporary rest screen
while a preview is starting up.
For example, they can show a placeholder image
on top of PreviewView when the preview is idle
and hide the placeholder when the preview is streaming.
Second, getBitmap API.
This API allows application to get a snapshot
of current preview displayed in the PreviewView.
Getting a bitmap is not as easy as it looks
because, first
the underlying view could be a surface view
or tetra view, which has different ways to get a bitmap.
Second, the bitmap you get from the view
is not properly transformed or cropped like the PreviewView does.
So the new getBitmap API allows the application
to get a bitmap 100% identical to what you see in the PreviewView.
PreviewView is easy and painless.
But why we want to do more?
Maybe you want to draw some advanced effect
on top of camera output
or perhaps you would like to integrate the camera preview
into your existing OpenGL bucket or game engines that are in the app.
Let us see how we can do this through the preview service provider API.
So here you have your own hardware or server
to render insert in your app.
The first step is you create a SurfaceTexture
in your render insert.
And then you create a Surface using the SurfaceTexture.
And then you provide the Surface to the Preview SurfaceProvider.
The SurfaceProvider will also provide a signal to your application
when it is safe to release the Surface.
And after that
our application can now easily receive the camera frames
in the SurfaceTexture
and your render insert can do anything you want
on the SurfaceTexture using OpenGL.
For example of using OpenGL with CameraX
see our sample app on AOSP, which we link here.
And don't worry about a long URL.
We will provide a link, as well as other links below this video.
The goal of ImageAnalysis is to make it super easy
to integrate libraries that are great at analyzing things
such as TensorFlow Lite, or ML Kit, or your own image library.
Here we provide you a list of sample calls
that can make integration easier.
First, YUV to RGB converter.
The image you retrieve from ImageAnalysis is in YUV format.
This is a native format in the Android camera system.
But we know that many of the libraries require the image to be RGB format.
So these utilities provide an easy and fast way
to convert from YUV to RGB.
It uses Renderscript under the hood
so it is efficient and highly accelerated.
It is able to achieve 30 FPS with 640x480 resolutions
across devices like Pixel 3 XL, Samsung S10, and Huawei P20.
The power of Image Analysis is not so obvious because it just provides
frame buffer acces, but once we combine it
with state-of-the-art machine learning libraries
it suddenly becomes a super powerful toolkit.
It allows the application to do something very interesting
things like text recognition
image labeling or detect your own object, like a hot dog
using your own trained model.
To give you a quick start of this, we introduce two samples for you.
First, ML Kit sample.
ML Kits bring Google's machine learning expertise
to the mobile developers.
These samples showcase how to do real time test recognition
and translate in real time using ML Kit and CameraX.
And if you prefer to use your own trained model
then you are probably more interested in TensorFlow Lite.
So we provide you a second sample
showing how to do real-time object detection
using your own model with TensorFlow Lite.
Now, we have a better understanding of CameraX beta release.
Here's what we're going to do in the near future.
First, we are committed to fixing as many issues as we can
and we take your reported issues very seriously.
So please, do file issues in our issuer trackers
or join our discussion group.
We are more than happy to resolve your problems.
And we will keep investing in our CameraX test lab
to improve the device compatibility continuously.
And also, we have some work in progress
to improve the CameraX performance
most importantly
to reduce the latency of camera opening and taking pictures.
And finally, we will keep investigating the pain points of running CameraX app
and bring you more samples, tools
and utilities to make CameraX easier to use.
Here are some useful resources.
You should check our documentation, samples, guides
if you want to learn the basics of CameraX.
And we also provide a step-by-step collab
if you want to quickly see what it takes to write a camera app.
The CameraX discussion group
is a place where we primarily get feedback from developers.
We'd love to hear the many different ways
in which developers are using CameraX.
And finally, an issue tracker.
Please file any issues that you think are caused by CameraX.
It not only helps your project
but also benefits the whole CameraX community.
And all the lists mention above, including the samples
are also listed below this video.
Thanks for your time.
We hope CameraX can help you create more and more exciting
and unique camera apps.