welcome to the code mental channel if
you don't know me
my name is armindo and my channel is all
about software development and python
programming today i'm going to
be trying to get a tensorflow object
detection api to work
within the raspberry pi there's a google
collab notebook which might be useful
for us to try first
so without further ado let's get started
so we have one collab for training we're
not going to train for now and for
inference
so let's give it a go so first thing i
do install tensorflow but i know
google cloud will probably already have
it unless it's uh it needs to upgrade
and it's going to ignore it so since i'm
actually running the object detection uh
google collab notebook uh i'm curious
actually what is the runtime with
pre-selects right now it's not using a
gpu i don't think you need a gpu to run
this notebook so i'm going to leave it
like that
so it seems like it downloads the models
from github or it checks if they're
already there
and they're not then it's this is where
it actually installs the object
detection api
something we need to follow for the
raspberry pi so what it's actually doing
is an object-oriented message format to
hold information so we can see here an
example
this looks almost like a class in java
if you have seen one
and this is a pattern to basically
create an instance of a person object
using a builder okay nothing very
difficult here this is actually just
compiling a bunch of entity classes that
are needed to hold information for a
variety of things
so easy enough so now okay seems to be
complaining about some requirements not
being met
so it's for example it has a requirement
for requests
2.23.0 but you have requests 2.25
which is incompatible so this collab
maybe it's
not it's not up to date maybe well let's
see how far we go
it might still work so now it's
utilities and then
here is actually downloading a model it
defaults to this one
so we can actually have a look in the
documentation
they should have a table with all the
different models
[Music]
okay so you can see centernet is like
one of the latest
models so there's different models with
different capabilities
and different speeds different
accuracies so as you can see
map is a measurement for accuracy
[Music]
so mean average precision right uh it's
related to how accurate is the the
boundary box that he adds
the bounding box is the box that you put
around the objects that you detect
so for example there's different ways to
look at it there's like a measurement
basically where you even check how much
of the box is out of the area that's
supposed to be
you check the overlap let's say you're
trying to detect
some giant i don't know what this is you
have the blue box is the actual
bounding box that we're expecting if the
ar model was a hundred percent uh
accurate right you would have to draw
the blue bounding box that's the ground
truth
but the models themselves they're not
that
great right so they're not like that
precise so you expect to have
a little bit of deviation of the actual
object for example here you can see the
orange box is what the ai model is able
to predict
in order for you to actually measure the
accuracy of the model
and don't forget in ai you basically end
up creating
lots and lots and lots of models with
different levels of precision
and accuracy you need to be able to
measure which one is best
so that's why you end up with different
levels of
different terms you know for basically
determining
how good the model is iou is one of them
so basically we calculate how much
overlap there is between
the prediction and the ground truth we
divided by the actual union of the two
boxes
yeah so the smaller the area for overlap
smaller is the iou so average precision
is uh
there's image classification image
classification is very simple
you you basically say if an image has an
apple or not right you normally have
classes the other type of object
detection is basically
using bounding boxes it's kind of simple
as you say
it's here but it's just that you end up
confusing yourself
quite a lot so typically just to give
you an idea
how a model is created you basically do
the training of the model
against some training data and then you
basically
once you finish training the model for a
few iterations
you use a set of test data to test
to assert how good the model is
and this will drive the for example
training itself to keep going or not
because depending on you know that he
doesn't talk about there's a loss
function there's a wealth measurement
there's many different
indicators that we use to basically if
our training is making any
headway basically the idea is to make
sure our training is going somewhere
because we don't want to be you know
wasting computer resources and we end up
with something which was worse than what
we started with
and that can happen because a training
itself doesn't necessarily
always improve things sometimes it makes
it worse because maybe you are testing
against some
faulty data or the data itself is not
like good enough
anyway let's get some action
so i need to run this so it's going to
load the model
and this is the model that we're going
to use now
let's run so basically what this is
going to do is download
oh no okay so this box here didn't
download the model you just
basically configure a property uh
variable name with the model that we
want to use and anytime we want to
change to something else we basically
have to change this
what i can already see here is that
actually it's making a reference to a
tpu
and i am not on a tpu so i might
want to change to another runtime okay
so
downloading the model okay it's finished
now i'm just going to all these
functions
so it's complaining already about a file
that's not there
so it's missing the configs i'm going to
just uh google this to see what
what people are saying this can happen
so there's a bar open for this
so it's saying that there are two other
configs but they don't have the label
map
defined maybe i need to just try another
notebook
okay fantastic so let's see if this one
is any better
so let's try this example instead i i
need to change runtime anyway
online so let's try a tpu
okay so same process we need to
install everything again right so it's
installed
so hopefully this notebook works so this
one is using
another model okay so it has downloaded
the model
seems to be quite lightweight because it
took like a second to download
now it's loading the model itself from
from disk
into memory just in case i'm going to
run this as well
just it's next in line and we can see
some of the stuff here like for example
there's a method here
that holds an image into a numpy array
and it uses
a tensorflow class for that
g-file so it tells me that g-file
is like an api basically a class that
is used to interface with io objects
input output
it's basically it seems like it's a
wrapper for a tensorflow c plus plus
file system api okay right so we've
executed the model
already and you can see here some
results so it's
detecting dog so you can see also a
person
so this model can detect different types
of objects
yeah i can say that the object detection
api can
detect small objects but the smaller the
object is from my own experience the
less
likely it will get the the right results
one thing you'll notice here
is that it's missing people right so it
didn't it it doesn't capture everyone
okay another thing that you have to bear
in mind this
lighting is probably perfect for how
this model was trained
so as soon as you basically take
pictures with different
lighting conditions you will get
completely different results
in fact let me just uh find an example
of an image that i can
use and you'll see that i'm not going to
be so lucky so this is a good example
right the idea is we're going to run
this picture using this model
and see what results we get so don't
just think that because
this model itself can detect people in
that
picture here that if you try to use it
at home
or yeah wherever you are that is
definitely going to work
most likely whatever scenario you're
trying to apply
this model to you probably will have to
improve the model and do training
and things like that that's something i
probably will have to do is all for my
wildlife camera but in general
of course google when they create these
models they try to make them
generic enough so they give it as much
training data as possible to get better
results
so another thing that might go wrong is
my image is a high resolution and when
you train a model
you normally like always decrease your
resolution to a standard size
much smaller size so it shouldn't be a
big issue but the resolution of the
image
will also affect how good the partition
is so let's run it
okay wow okay but you know
the point is look there's few people you
missed but
i honestly think this is actually not
not a bad result pretty good
i wonder how easy is for me to change
the models
[Music]
it's not very good this notebook for
only one reason
it doesn't allow me to easily swap
models so i
if i want to swap to another model i
have to replace all these
instances here so i have to change this
i have to change this
and this as well so i'm destroying what
i already have here
but you know this notebook is on github
so i can always reload it again
then i need to basically copy this here
then hope that this is like this
let's try it so you see this model is
much bigger i don't know what's
internet so the key word is objects
as points what uh the these center nets
seems to be doing and now i understand a
little bit more why they call it center
net
they basically look at the point for
example like the center point of an
object
the bounding box itself so basically
rather than trying to figure out where
is the whole object they just calculate
the center point and then they know that
there should be a box around it
so the only time i remember using
something similar to this is when i was
trying to do tracking
of an object when you try to track an
object across
in a video for example you need to
somehow create a center point
so that you know what the position of
the object is and then you can also
try to estimate the direction in which
they go right so let's see
if i can just replace this with this
and hopefully it will just work oh looks
a bit better right it has more
more results look i think he almost got
all the people it missed maybe
this one here but um wow it's better
definitely look but the there's a
trade-off right for this model
okay it took nine seconds to basically
return a result for this image
but just think about this right if it
takes nine seconds per image to
return a result if you're trying to do
real time detection it's it's not going
to work right