[MUSIC PLAYING]
DONG CHEN: Hello, everyone.
My name is Dong Chen.
I'm an engineer on Android.
I'm going to share the latest on Android machine learning
and show you how you can benefit from our newest offerings.
Machine learning continues to enable a new user experience
across vision, speech, and language tasks.
We see increasingly powerful mobile devices.
We see tremendous growth in processing happening on device.
On-device machine learning has some unique benefits
over server-based ML, such as improved latency, minimized
exposure to users private data, reduced power consumption,
and the ability to provide a consistent experience,
even with no network connectivity.
In addition, it can also save server costs.
Many of the latest features on the Pixel 6
are enabled with on-device machine learning.
For example, Magic Eraser in Google Photos
lets you easily remove distractions,
such as an accidental photo bomb.
Powered by on-device instant segmentation ML technology,
Magic Eraser can detect distractions in the photo
and [INAUDIBLE] use machine learning
to predict what it would look like if the distraction weren't
there.
Another great example is the Pixel 6 Direct My Call.
When you call a business, it can help transcribe the call
in real time and present you with the call menu options,
saving you the hassle of remembering
which option to press.
These features all run on device.
If you want to benefit from on-device machine learning
your own Android apps, there are two great options.
First is MLKit.
MLKit is a great starting point, because it provides
ready-made solutions through simple APIs for ML powered
features, like real-time pose detection, offline translation,
and digital ink handwriting recognition,
all optimized to run on mobile devices.
The best part is you don't need any machine learning expertise
to use MLKit.
If you do have some ML expertise and either your use case is not
covered by MLKit or you need more control
over the experience, you can go custom.
In order to help with this, Android provides a custom
ML stack, which consists of a set of supporting components
that make it easier to deploy custom ML.
This year, we have exciting updates on both MLKit
and the supporting components provided by Android.
Let's start with MLKit.
Since launching in 2018, MLKit has been growing rapidly.
Combining Android and iOS, its usage
has more than doubled since last year
to over 1 billion monthly active users and 80,000
monthly active apps.
Over the past year, we have been making major updates
across our existing APIs, improving accuracy, latency,
and SDK size.
First of all, we push the limit of performance
through hardware acceleration.
By enabling GPU acceleration for pose detection,
we reduced the pipeline latency by up to 43%
based on internal benchmarking.
On Pixel 6, we also enabled TPU acceleration for object
detection and tracking as well as pose detection
and reduced their pipeline latencies by as much as 28%
and 50% respectively.
Additionally, we never DSP acceleration
for object detection and tracking on qualified devices.
This sped up model inference by three times,
while consuming only one seventh of energy
based on our internal benchmarking.
We also made some big updates to text recognition
with a beta version of Text Recognition V2.
In addition to the Latin script, we added support for Chinese,
[INAUDIBLE],, Japanese, and Korean scripts,
dramatically expanding our language coverage.
We introduced a deep paragraphing technology
to provide better structural grouping.
We also added the support for text
written in all orientations.
And even with all these improvements,
we managed to boost recognition accuracy
and improved latency by as much as 30% on Android devices.
Thanks to the new script support, Text Recognition V2
beta can now covered 37 languages,
representing over 4.9 billion people worldwide.
On top of these improvements, we looked for ways
to make it easier to integrate MLKit into our app.
The new Google Code Scanner API, powered by Google Play
services, utilizes the latest MLKit scanning technology.
Using this API, apps can request that Google
scans the barcode, QR code, and other codes on their behalf,
and returns without needing to request a camera permission
or process the image within the app.
This makes it quicker and easier to integrate code
scanning features into your app while reducing users
concern about having to grant camera permissions that
can limit future usage.
Finally, I want to let you know that MLKit
has an ongoing early access program.
This can be a great opportunity to find out
about upcoming MLKit APIs or even
influence their design by partnering with our engineering
team.
You can apply by following the link on the screen.
Now, you may have a use case that's not yet
supported by a market, or maybe you need more advanced control
in deploying your own custom ML models.
Last year, we announced a set of supporting components
to make this easier.
I want to hand it over to Ali to talk about the latest
updates on those.
ALI CEVIK: Thank you, Don.
Hi, I'm Ali, a Product Manager IN Android.
I'm going to talk about Android's custom ML
stack, which consists of a set of supporting components
that make deploying custom ML easier.
As a recap, last year, we highlighted three areas
where Android's custom ML stack would deliver
significant improvements.
First, making essential components
for running on-device inference built-in
so that you don't need to bundle them into your app
separately, starting with TensorFlow Lite.
And second, helping you get optimal performance
across the ecosystem by using hardware acceleration
wherever available.
And third, providing regular updates
to hardware drivers outside of Android desert
releases with consistent support across OS versions.
I'm proud to say that we made great progress in all
of these areas.
Let's start with the essentials.
Last year, we announced the Play Services API
for TensorFlow Lite.
This API is now publicly available in beta.
By switching over to this API for inference,
you can immediately shave off up to 2 megabytes
from your app size.
Moreover, your app is likely to benefit
from regular performance improvements.
This is because the TF Lite team keeps optimizing performance
in their latest releases, and these improvements
will be picked up and delivered by Google Play services
automatically.
In fact, during our early access program
last year, some teams have observed as much as 20%
performance improvement in CPU by migrating to the new Play
Services API.
Test for Lite in Google Play services
is already being used by Google teams, including MLKit,
and serving over 400 million monthly active users
running over $20 billion daily inferences.
If you're interested in adopting,
the API is almost the same as regular TensorFlow Lite.
In most scenarios, we expect migration to involve
making minor code changes, such as adding an initialization
call and waiting for it to be completed.
You can see that we use the suspending function here.
This is a cool Kotlin feature that
allows writing sequential code without blocking the UI thread.
And once initialized, you can run inference as usual.
And that's it.
You can get started with this API
today by following the link below.
And now comes my favorite part--
performance and hardware acceleration.
Mobile devices nowadays are armed
with powerful chips that can provide up to 10x latency
improvement for ML inference.
However, it has been hard to benefit from these capabilities
due to variations in runtime latency, stability,
and even accuracy across different devices and driver
versions.
I'm excited to announce Acceleration Service,
a new API that will dramatically improve the experience for you.
This API uses a technology called local benchmarking that
helps determine optimal hardware on which to run a ML
inference safely at runtime.
This technology is already used for pose detection in MLKit,
where it's helped make it up to 43% faster.
Acceleration Service is currently
in early access preview, and you can apply
by following the link below.
And here's how local benchmarking works--
at runtime, you call into Acceleration Service
with your ML model and a few golden samples.
The service then spins up new processes
for testing each different hardware available.
Running them at separate processes
ensures your app doesn't suffer from driver stability issues,
such as crashes or hangs.
And after benchmarking is completed,
you get a report on latency, stability, and accuracy
of different accelerators.
You can then choose the accelerator
you want based on your business needs.
And here's another cool part--
the more devices you use this technology on,
the better it gets, as it collects telemetry
and automates some of this decision-making on the cloud.
And by the way, you have control over when
to run these local benchmarks.
For example, you can choose to complete all of it
in the app initialization time and get it cached.
This way your app will be ready to benefit from performance
at future execution time.
Now, earlier we talked about how we
made TensorFlow Lite updatable with Google Play services
in order to enable you to get access to the latest
optimized inference engine.
But getting the best performance also
requires access to the latest optimized hardware drivers.
This is why last year we announced
updatable neural networks API, which is now live in production
under private beta.
And this year, we're starting to make additional Tensor Lite
delegates updatable as well with Play services.
This means you will get access to the latest drivers
and delegates without having to bundle them into your app.
You can benefit from updatable drivers using the Activation
Service or target them directly.
To find out more, please follow the link on the slide.
Let's put it all together for Android's custom ML stack.
TensorFlow Lite in Google Play services
is now publicly available in beta for use.
It will help you save on binary size
and benefit from regular performance improvements.
Acceleration Service is accepting applications
to the early access preview.
It will allow you to benefit from hardware acceleration
without having to worry about latency, stability,
and accuracy variations in user devices.
Updatable hardware drivers are starting
to roll out with neural networks API and private beta.
Soon, we will be making additional TensorFlow Lite
delegates updatable as well, so that you
can benefit from the latest optimized drivers and delegates
seamlessly.
You can access all of these APIs by following the link
on the slide.
Finally, we want to acknowledge that ML can be challenging.
This is why the Android ML team is planning
to do one-to-one office hours with Android developers, who
are actively working on an on-device machine learning
related project for their apps.
This can be a great opportunity to get direct help
from Googlers about APIs, hardware acceleration, model
conversions, and anything on device machine learning
related.
You can apply by following the link below.
I want to thank you on behalf of Dong, myself, and the Android
ML team.
We have links on the screen, which will lead you
to everything we announced today and our latest
early access programs.
You can also find them in the video description.
Thank you for watching.
[MUSIC PLAYING]