[MUSIC PLAYING]
PAUL RUIZ: Hey, all.
I'm Paul Ruiz, a developer advocate with Google.
And this is a series introducing machine learning
and its application on Android with the MediaPipe framework.
In this lesson, I'm going to introduce you
to the concepts behind computer vision at a very high level.
That way, you can have a better understanding
of what your code is doing in your apps
without needing to be a data scientist.
There are three main tasks that you'll learn about today--
image classification, object detection,
and gesture recognition.
Let's start by taking a look at image classification.
The goal of image classification is
to assign a label or multiple labels to an image.
For example, with a single label classification model,
the image of a cat will likely appear as "Cat"
since it's the most pronounced item in the image.
However, if you use a model for multiple labels,
then you might see "Cat," "Rug," or "Toy" as possible labels.
Cool.
So let's take a quick look into how
this classification happens.
The underlying model that I'll cover
is using something called a convolutional neural network.
This model's main way to approach image classification
is to look for patterns within images during training,
then look for those same patterns in new images.
Getting back to the cat image, let's say
that we have thousands of cat pictures, which
obviously no cat owner has just laying around on their phone.
But we can start noticing trends in things like ear shape, eye
shape, and nose shape.
Now, if we take those patterns and look at a brand new image,
we should be able to identify those items, or at least most
of them, so the model can say that there
is a cat with some level of confidence.
The next thing to know is that this classification
happens across multiple layers of an image.
For example, Android may provide an ARGB image.
Since the alpha layer doesn't really
give us much information, we can just
go ahead and drop that, but then the rest of the colors
are looked at to try and perform classification
with various patterns noticeable across different layers.
Finally, images may need to go through a few preclassification
steps to scale, shape, or rotate them
so that they match the image parameters that the model was
trained with.
For example, a model may have shrunk images to 32 pixels
by 32 pixels, so your Android app will
need to do the same to perform classification.
This is known as preprocessing.
Now that we have a better understanding
of image classification, let's move on to object detection.
This not only classifies objects in an image or video,
but it also attempts to identify where
those objects are to construct a bounding box area around them.
So the first thing to understand is
that the term "object detection" is a little misleading
since we're not just detecting the presence of an object
but we're actually performing three unique steps.
First, we're going to do what you would normally
call object detection, which is determining
where an object might be based on statistical analysis.
In this image, we can see that there
are multiple potential objects, including
cup, potted plant, and laptop.
Once our model has an idea about where some objects may exist,
it attempts to recognize those objects
through a classification process similar to what we previously
saw.
And the last step is localization.
This is where the bounding boxes are determined
for the detected objects.
The other thing to know about object detection
is that it can be incredibly difficult to get great results
when using a custom model.
So you'll want to keep an eye out
for edge cases that may need to be addressed
when you're given a model.
Some of these edge cases include overlapping objects,
different rotations and sizes for objects compared
to the training data, or only partial objects
existing within an image.
The final task that you'll learn about is gesture recognition.
As you might guess from the name,
gesture recognition can look at one or more hands
and determine what predefined gestures are
being made by the user.
For example, you may have an Android TV
app that supports rating content based on a thumbs up or thumbs
down provided by the user.
What's interesting about gesture recognition
is that, like object detection, it's made up of multiple steps.
But expanding on that a bit, gesture recognition
is also made up of multiple different models.
Because of the way MediaPipe Task is designed,
multiple models can be changed together
to do even more complex operations.
To give you a better idea, gesture recognition
consist of three separate models.
The first detects the presence and location of a hand.
The second finds specific landmarks
on the hand, such as fingertips and various joins.
And the third performs the primary step
of determining the gesture.
Later on in the series, you will see
exactly how you would perform gesture recognition
on a live camera feed with Android,
as well as how you would take it a step further by training
your own model for the game "rock, paper, scissors."
So now that you have some of the theory behind computer vision
down, it's time to go over how it can be used.
In the next video, we'll look at how these topics can be applied
using Android's MediaPipe Task.
So I'll see you there.
[UPBEAT MUSIC]