[MUSIC PLAYING]
PAUL RUIZ: Hey, y'all.
I'm Paul Ruiz, a developer advocate with Google.
This is the first video in the ML on Android
with MediaPipe series, where you will
learn about the basics of machine learning,
along with how you can use Google's newest
on-device machine learning tool, MediaPipe, to add
useful features to your Android apps.
This series will aim to introduce some general concepts
and vocabulary behind ML, how to create models,
and how to implement some common ML task, such as hand gesture
recognition, audio classification, and text
similarity, into your Android apps.
By the end of this series, you should
have enough of a foundation in Android machine
learning to continue learning and making great apps
without getting too off track with all
of the math and underlying details.
If you are interested in a lot more
of the details behind machine learning,
I've included a link in the video description
to our TensorFlow Zero to Hero video
series, which is similar enough to MediaPipe for our purposes.
So let's get started by talking about what
machine learning even is.
With traditional programming languages,
you provide an input and a function,
then the program produces an output.
So for example, you might have a function that
adds 2 to any input number.
So 1 outputs 3, 8 outputs 10, and 40 outputs 42.
With machine learning, you provide a set of known inputs
and a set of known outputs, and then the computer
tries to figure out what the function is
that connects those two.
This process is called training, and it's
one of the core steps in machine learning.
Without getting too far into the details,
training involves solving a set of complex problems,
including optimizing models to work on relatively restrictive
hardware, using specific device hardware for acceleration,
and ensuring your models work across multiple platforms,
which is why we've created MediaPipe
as a way to make these challenges a whole lot easier
for our developers.
Once training is done and your model is ready to go,
the next step is implementing the model in your mobile apps,
though we'll cover that in a lot more detail in later videos.
At this point, you might even be wondering
what use cases machine learning can help you with.
We can separate appropriate ML tasks
into four categories-- natural language processing, vision,
audio, and finally "other" for more one-off tasks.
As you can see here, we're either already supporting
or plan to support a wide variety of tasks
with MediaPipe.
Vision tasks are those that involve images or video
to recognize content, track objects, or even provide
a modified version of visual inputs.
These tend to be the most seen use cases for mobile apps,
and a few common tasks in this domain
include barcode scanning, object tracking,
and item classification.
Natural language processing, or NLP,
includes anything text related.
You might use NLP to translate text, provide answers
from a contextual paragraph, or understand some dialogue.
Audio machine learning can accept inputs
from a microphone or audio file in order
to classify sounds, input spoken language,
or check the similarity between two separate audio files.
Finally, we have the somewhat ambiguous category of "other."
These one-off task include content recommendation,
tabular classification, or even speech output.
So what about on-device machine learning?
As the name implies, on-device machine learning
is a sub-field of machine learning.
And it is a way to run machine learning inference directly
on the edge device.
This includes mobile apps, IoT devices, and even web pages.
Because of this single attribute,
developers gain three specific advantages.
First, machine learning is able to run quickly
using models that have been specifically designed
for edge devices.
Next, your apps will be able to use machine learning features
regardless of whether or not the device has connectivity,
for example, when a user is in a subway or in a rural area.
Lastly, because data is never leaving your user's device,
that data is more secure and private for the user.
This has opened up the door for a variety of new products
and features for developers, including
the ones that were mentioned just a few moments ago.
Great, so before we move on to the next video,
I just want to mention what makes
MediaPipe unique in the on-device machine
learning ecosystem.
MediaPipe consists of a selection of tools,
such as MediaPipe Tasks, which lets developers easily
deploy ML tasks to their apps; MediaPipe Model Maker, which,
as you can probably guess, helps you create custom models;
and then MediaPipe Studio, which lets
you try a task out to create your own solutions
from a browser without needing to write any code.
This video series will primarily focus on MediaPipe Tasks,
as it's the most relevant to Android development.
Though it's worth mentioning that MediaPipe Tasks also
works with Python and web, plus at the time of this recording,
iOS will be out soon.
I will include links to content focused on MediaPipe Model
Maker and MediaPipe Studio in the video description for those
of you that are interested in learning more about those two
tools.
In the next video, we will take a look at computer vision
and how it works at a high level.
So I'll see you there.
[MUSIC PLAYING]