YOLO is state of the art object
detection algorithm
and it is so fast that it has become
almost a standard way
of detecting objects in the field of
computer vision.
Previously people were using sliding
window object detection
then more faster versions were invented
such as
RCNN, fast RCNN and faster
RCNN but in 2015
YOLO was invented which
outperformed all the previous object
detection algorithms
and that's what we are going to discuss
today. We will go over the theory
on how exactly YOLO works and in the
future video we will also do coding so
this video is just
about the theory behind how YOLO works
and we'll
try to see why it is faster. Full form of
YOLO is
You Only Look Oce. Let's say you're
working on
an image classification problem where
you want to decide if the
image is of a dog or a person. In this
case the
output of neural network is pretty
simple. You will say dog is equal to one
person is equal to zero but when you
talk about object localization
you're not only telling which class this
is you're also telling the bounding box
or the position of an object within the
image.
So here in addition to dog is equal to 1
and person is equal to 0
you are also telling about the bounding
box. Now how exactly you do that?
So in in terms of neural network output
you can have a vector like this
where pc is the probability of a class.
So here if there is a dog or a person
then this number will be one. If there is
no dog or no person
this number will be zero then the
bounding box
so bx bi by is the coordinate
of the center which is indicated in
yellow
circle here and 1670 is the
width and height of this red box.
C1 is class one that is for dog.
So here it will be one c2 is for
person and it will be zero if you have a
different image like this.
There is a person here. This is my
picture in my
high school. The pc probability of any
class is 1
because there is some object and these
are like bounding box
coordinates and c1 is 0 because it's
not a dog and
c2 is 1 because it's a person and when
you have
no object in the image the pc will be
zero
and rest of the values don't matter. So
now you can train
a neural network to
classify the object as well as the
bounding box. So you can have I am just
showing
three images here but you can have less
than ten thousand such images
and for each of these images since it's
a supervised learning problem
you need to give the bounding boxes. And
the way you give bounding boxes to
neural network..
understand neural network only
understands numbers so you
have to convert this into
this kind of vectors. So you will have a
vector of size
7 for each corresponding image so that
will so image is x strain
and y train will be a vector of size 7.
you can have 10 000 such images you can
train a neural network in a way
that
if you input a new image now it will
tell you
that particular vector and now this
vector is telling you
that this is a dog because c1 is set to
1 and it is also telling you the
bounding box so basically it's
essentially giving you the
answer for your object detection or
object localization rather.
This only works for a single object if
you have a multiple objects what do you
do?
Here there is person and a dog in the
same image.
One might say that okay
you know in my image there could be
n number of object there could be two
dogs three people there could be five
dogs one person you don't know
how many objects are there in the
picture. So it's
hard to determine the dimension of your
neural network
output if you have one one object.
It's pretty fixed right but if you
have
n number of objects and you don't know
then
determining the size of the output of
neural network is hard.
You can say upper max is 10 let's say
there will be only 10 objects
and you can have 10 into 7 which is like
a
70 size vector but what if there are 11
objects?
See so that doesn't work so you have to
do something
else. All right! So let's say you have
this image
and there are two bounding boxes that
this image has.
What yolo algorithm will do is it will
divide this
image into this kind of grid cells. So
I'm using
four by four grid here. It could be three
by three it could be 19 by 19.
There's no fixed rule that it has to be
four by four.
And for each of the grid cells for
example this grid cell
you can encode or you can come up with
that vector that we saw previously
which is pc bounding box c1 and c2
there are no objects here so probability
of class will be zero
and then rest of the values don't matter.
But for
this particular grid cell,
So I have highlighted here the dog is
there in the picture
see when dog is expanding to multiple
grid cell you try to
find the central place of that dog and
the dog belongs to that particular grid
cell.
So I'm in this particular cell here and
when I
look at the coordinates you can think
about this per point as a
zero and this point has one
coordinate.
And now you can create this vector where
p c is one which means you have
some object then c one and c two. C One
is for dog so it is one
c two is per person it is 0. There is
person's head here but the person's
center is here so this person object
belongs to this cell
and then 0.05. Like this particular
distance is 0.05.
This is 0.3 because see this whole thing
is 1
and then your bounding rectangle can go
out of your grid cell. It is fine
that's why these values are more than
one. So 1.3 and 1.
oh sorry 2 and 1.3 so that is the width
so 2 is this width
and 1.3 is height. So it is this height
and now talking about this particular
grid cell.
So there is a person center here so we
can say person
is in this grid and therefore c2 class
value
1 is 1 c1 is 0 because there is no dog
and these are like bounding boxes so
0.32 is see 0.32 is this much.
0.02 is this this particular height and
it is 3 because
the rectangle with this yellow line is
equal to almost 3.
The size of c the width of this grid
cell and if you compare this
this is three times this. That's why i
have three here
and now you can have
for remaining all the cells
the vector will be this. So pc will be
zero remaining will be
don't care so now you have four by four
by
seven volume. Why?
Because you have four by four total grid
cells. 16 cells
each cell is a vector of size seven.
That's why I'm saying four by four by
seven so if you're talking about this
top left cell and if you expand it in a
z
direction that will be
this vector of size 7. So I hope you're
getting an idea. If you don't please
pause the video and just think about
what I just said.
So now you have the image and then the
bounding rectangles.
Now you can form your training data set.
So your training data cell will have
so many such images. Let's say I am
showing only three
four example, but you will have 10 000
such images.
Each image will have bounding rectangle
and based on that rectangle
you will try to derive. You will first
form this kind of grid 4x4 grid or 3x3
or 19x19. It varies.
It doesn't have to be four by four and
you will come up with the
y or a target vector which will be
for each cell there will be one vector
so there will be 16
such vector per training sample
or per training image. Using this
now you can train your neural network
and
after you have trained it it can do
prediction.
So when you now give this type of image
it can produce 16 such vectors and
y 16 because this is like 4 by 4 grid
which will basically tell you the
bounding rectangle for each of these
objects.
So this is the YOLO algorithm. It is
called You Only Look Once
because we are not repeating it. See we
are not doing something like okay
we have 16 cells. So it's not like we are
inputing it 16 times and doing 60
iteration
in one forward pass. You can make all
your prediction
that is why it is called You Only Look
Once. Now this is a basic algorithm
we need some tweaks because there could
be few issues with this approach.
First issue is the algorithm might
detect
multiple bounding rectangles for a given
object.
It is possible so how do you tackle that.
So let's think about this-
Let's say for a person it detected all
these two yellow and this one white
rectangle and we know by visual
observation that this white one
is the most accurate one and the
algorithm will also throw out the
probability.
It will say this is point nine percent
you know the pc. The pc
class. It will say this is point nine
percent matching with
person and
the other rectangles have less
probability.
So maybe we can look at all the
probabilities for a person class and
take the max right?
Well we cannot do this okay?
If you just take a max and if there is
another person
what happens to that you don't know
where that person is right. So as a
neural network, as a computer you don't
know
so you can't take a max you have to use
different approach.
So we use this concept of IOU. so IOU is
basically
intersection over union- which is you
take this
rectangle which is 0.9 this is that
white rectangle
and then for that same class which is
person
you will take all other rectangles and
try to find
overlapping area and to find
overlapping area you use IOU. So here in
this case
see this is that yellow box okay?
So this is that yellow box here and this
is the white box
and the area indicated in this orange
color
is intersection area. Area indicated in
purple colors is union area. So you find
division of these two and if the objects
are overlapping this value will be more.
So let's say if the value is more
than 0.6 or 0.7
we can say these rectangles are
overlapping, if they are completely
overlapping the value will be 1.
If they are not overlapping at all value
will be 0.
So now we find that
these two yellow boxes are overlapping
because their
IOU is let's say greater than 0.65
and then you discard those rectangles.
So I discarded all the rectangles which
had IOU greater than 0.65
and kept the rectangle which has class
probability as max.
Okay. So
this so I do this for a personal object
then I do the same thing for a dog
object. So for dog
I find that okay point 81 this is the
max probability.
I find all other rectangles in this
image
again there could be two more dogs here
and there will be rectangles for those
also.
So you will try to find overlap?
Okay so let's see if there is a dog here
you will not find overlap so you will
not discard that
particular rectangle but this rectangle
you find it to be overlapping and since
point 81 is max point seven
is less you discard this and you get
final bounding boxes. This technique is
also called
nomex operation. So after
neural network has detected all the
objects you apply no max suppression and
you get
these unique bounding boxes there could
be another issue is
what if a single cell contains the
center of two objects?
In this case the dog and the person both
are in the middle's middle
grid cell. Now we use this vector to
represent the grid cell but
see this vector can represent only one
class.
So how do you represent two class? Well I
have
this value for dog. I have this value for
person
so instead of having a seven dimension
vector
how about we have a vector of size 14
where you're just concatenating these
two
vectors. Okay so this is said to have
a basically it has two anchor boxes so
this is one anchor box this is second
anchor box.
So here you have two anchor boxes and
you can actually have more than two
anchor boxes. Let's say if there are
three objects
which has the same center, Then you can
have
three anchor boxes, you can have five
anchor boxes but
if your grid sales are small enough then
in real life
it's hard to have. You know many objects
belonging to one grid cell
so now cnn with two anchor boxes will
look something like this so instead of
a vector of size the only change is now
you have a vector of size
14. If you want to have three
anchor boxes you'll have a vector of
size 21- 7 into three
okay? And that will give you your final
output.
So that was all about you only look once
or YOLO algorithm
It's a very very fast algorithm even on
a video clip
which is let's say at 40 frame per
second it can detect objects
really fast and it is the most modern
way of detecting objects so if you are
in computer vision fields
if you want to do object detection you
have to use
YOLO because it is very fast and
accurate
in the next video we will be looking at
some code we will do a real object
detection
in image and in video using YOLO
framework.
I hope you're liking this series so far.
If you do give it a thumbs up and share
it with your friends. Thanks.