[MUSIC PLAYING]
THOMAS EZAN: Hello.
My name is Thomas Ezan, and I'm a Developer Relation Engineer
in the Android team working on machine learning.
Earlier last year, we announced the TensorFlow Lite
with Google Play Services in beta,
and in September we released the stable version.
Fast forward to today.
TensorFlow Lite in Google Play Services
is used by thousands of apps and more than a billion users
monthly.
You will still be able to bundle TensorFlow
Lite with your Android application
if your user's device doesn't have Google Play.
But moving forward, TensorFlow Lite with Google Play Services
is Android's official machine learning inference engine.
It will let you easily integrate with the future APIs
and services from the Android custom ML stack
as they will be built on top of it.
Google Play Services will take care
of providing the latest version of TensorFlow Lite
to your application so you won't have to bundle it.
In addition to saving several megabytes-- actually,
up to five megabytes if you are using the GPU delegate
for hardware acceleration--
this will enable your app to benefit
from runtime optimizations and bugfixes
as they become available without any changes to your code.
OK, let's see how to add the Interpreter
API to your application.
In this video, we will focus on the Interpreter API for Java
and Kotlin, but keep in mind that you can also use
the native Interpreter API using C or C++.
But as of now, the native API is only
supported in the bundled version of TensorFlow Lite.
Start by adding new dependencies.
Add the new Play Services TensorFlow Lite import
to the build.gradle file of your app.
Optionally, you can also add the TensorFlow Lite Support Library
which provides utility methods and data
structures, helping you with data processing and data
conversion.
If you are already using TensorFlow Lite,
you should replace your existing TensorFlow Lite dependencies.
Then in your Kotlin code, create a task to initialize
your TensorFlow Lite runtime.
Here the task class is the same API
you may have already met in other Google Play Services
integrations.
The task lets you make asynchronous method calls.
You should then add the OnSuccessListener
to the initialized task.
That will create your interpreter
once TensorFlow Lite runtime is initialized.
Once the initialization is successful,
create an InterpreterApi.Options object
and specify that you want to use the TensorFlow Lite with Google
Play Services runtime by passing FROM_SYSTEM_ONLY.
Then create your interpreter by calling InterpreterApi.create
and by passing your TensorFlow Lite model and your options.
If you are already using TensorFlow Lite
with the Interpreter API in your Android application,
you'll notice that we use the Interpreter API class.
The Interpreter class still used in the bundled version
of TensorFlow Lite will be deprecated,
and implementations using the bundled version
can already use the Interpreter API class.
You can see here that we are using the Google Play Services
Task API for asynchronous method calls.
The tasks are a key to Google Play Services APIs.
You can learn more about it in the Google Play Services
documentation.
What you have seen so far enables
you to run inference on device with TensorFlow
Lite with Google Play Services using the device CPU.
But running the CPU isn't always enough to enable
a low latency and high accuracy.
Delegates enable hardware acceleration of TensorFlow Lite
models by leveraging on-device accelerators such as GPU chips
or graphical processing units.
Often a GPU, which is a chip designed
to handle graphical rendering, enables better ML inference
performances, so let's see how to take advantage of it.
First, update your app's dependency
to import GPU delegate dependency for TensorFlow Lite
with Google Play Services.
To run inference with the GPU, you
need to ensure that the device your app is currently
running on has a GPU that is compatible with the TensorFlow
Lite GPU delegates.
Once TensorFlow Lite is initialized,
you can verify the availability of the GPU delegate
by running the task provided by each GPU delegate available.
Then launch the task asynchronously
by calling continueWith, and in the callback,
check the task results to confirm
if a GPU delegate is present.
If a GPU delegate is available for your device,
add it to the Interpreter API options
by calling addDelegateFactory and passing a GPU delegate
factory object.
Keep in mind that the GPU delegate isn't supported
on emulators, even if you created
an emulator with the hardware graphics rendering enabled.
The Android ML team is dedicated to make
TensorFlow Lite with Google Play Services the best platform
to deploy on-device machine learning on Android,
and we are working on more tools and solutions
to let you build great machine learning features on Android.
For example, we are building an on-device functionality that
will let you identify the optimal acceleration hardware
for your model between CPU, GPU, DSP, digital signal processor,
or NPU--
neural processing unit.
We are also working on making the delegates and drivers
updatable so your model is always
running on the latest and greatest without any changes
on your end.
It is still early, but if you are interested in testing out
this API, go to the early access page
on the machine learning section of developer.android.com.
You can learn more about TensorFlow Lite in the Google
Play Services by visiting the TensorFlow documentation page,
and you can share feedback through the TensorFlow GitHub
project.
We can't wait to see what you are going to build with it.
Thank you for watching.
[MUSIC PLAYING]