what is going on everybody and welcome
to an update to the Python plays series
in this video I'm gonna be going over
just some of the updates that I've found
with the object detection and kind of
what what I'm thinking moving forward so
this is one of the out-of-the-box cocoa
models it detects cars pedestrians
traffic lights although there are really
no traffic lights around here for me to
show you but it's pretty good the
problem is it still just doesn't it
doesn't detect as far like the distance
wise potted plant over there in the
corner so quite detect out as far as I
would like it to and then also like see
that car still not being picked up
really that card - ever got picked up -
like the very very end and really all I
want is cars in people to be picked up
as good as possible so probably gonna
wind up having to train my own custom
model just because this one's not
picking up people as quickly as I would
like so the plans for the on foot model
or to you know play GTA in the way that
one plays GTA so you should be able to
engage with the police here and the way
that we can do that is you know when you
find out hey where's the police or
whoever you want to engage with you just
find the center of the cube or square
rather and that's how you engage now
same thing with stealing cars the way
that we did it before you would find the
bounding box for the car and then
basically you would move the mouse - to
force the car to be basically center
screen run towards it and then when
you're close enough press F to enter the
car
that's what we had done before and I
think that's probably what I'm going to
do again as far as like stealing cars is
concerned but I think it'd be cool to
let let the AI kind of do its own thing
on it on foot rather than purely go for
cars all the time what did that say I'm
not sure what that was calling that
yellow line so so with the models all I
really did was just let me see if I can
get away
yeah so this is the grand theft auto
object detector basically we're just
grabbing screen as usual I do also have
so it's also I can run them both at the
same time I just set the GPU fraction I
keep clicking on this two to play so I
can unpause and that's the other thing
is how do we get Charles to like walk
around in the environment so on the one
hand when he's in third person he
actually can walk around pretty well
because he's just gonna walk as if he
was driving so he should pretty much
always walk along the roads which should
provide decent enough play the problem
is generally if you walk the police are
always gonna appear behind you so that's
kind of awkward so probably still gonna
have to figure out some way to engage
with police the other option that we
have is the on foot version could be
completely retrained using similar
methods that we use to train you know
the self-driving car only I'm thinking
that you could probably do transfer
learning on foot so start with the car
model and then from there go to do
transfer learning and I think we should
be able to do that then you have some
sort of like state that you're just
tracking like where is the player is he
in a car is he on foot and then later
you know is he in a plane or all that
stuff so like as you can see he pretty
pretty well can just walk along the
street but yeah you're always gonna have
police behind you which is kind of weird
but they always do seem to end up being
behind you which is kind of lame so
anyway I gotta do something about that
maybe you know you could just use OpenCV
figure out where the police are by
looking for like the little dots or
something like that or you could train a
neural network model to always like kind
of turn towards please so no not really
any different than how we trained the
self-driving car to kind of follow the
Purple Line I don't think it would be
too difficult to get a AI to you know
look at all those like you know red or
blue dots and try to always aim towards
the red or blue dots so that should be
totally possible and hopefully require a
lot less data if we're able to get away
with using transfer learning because
it's the same exact keep
that we'd be pushing and then you would
use the object detection script itself
to actually when you when it comes time
to aim to aim so anyway I think that's
kind of my ideas for how we're gonna get
a decent on foot version but we'll see
how that goes surfboard
nice nope anyway yeah so some of the
ways like if you wanted to play with
this yourself anyways you could come
into here and all you have to do is if
you pull up let me just bring it up real
quick it's like the ger GTA tensorflow
object detection API models ooh so that
T key is for pause so that's what
happened there really the github there
we go cool so if we scroll down these
are all the models that we can use and
then here is the the milliseconds and
again I still don't actually know
unfortunately I didn't publish the other
video before I film this one so
someone's gonna be able to tell us
hopefully at some point what what this
is I don't know if it's like a score or
what this is definitely how quick it
runs though so the one I'm using I want
to say is probably like maybe this one
the or CNN inception let's see which one
I'm using yeah so the RCN inception v2
cocoa I tried some of these other ones
so this was the best one I could find so
there's always this kind of a you know a
balance between accuracy and the speed
so like for example some of these are
really slow like this is 600
milliseconds so way too way too slow you
know really even like 100 milliseconds
is is just too slow we need something
faster than that so anyway obviously the
faster the better especially because
we're trying to find that bounding box
and then in theory shoot towards the
center of that box well if that box is
actually moving in game then you're
probably always going to be like lagging
that box so that'll be a problem anyway
you can try all these and in if you want
to you can't just take the name but if
you like right click and copy the link
address and then you come down here just
anywhere really paste it out because
they all have like dates
to them and then this is the thing that
you want to pass here if you don't do
that you'll get like a 403 forbidden so
anyway so you can try throwing those in
there this is basically the exact same
script as in the tutorial that I've
already done so I'm not really groups
I'm so used to being let's just type
Python and it defaults to Python
programming that net cgta
and if we scroll down to object
detection it's basically this script
here so really the only difference is
just bringing in the newer version of
object detection and a different model
you can still try out this model but
this is quite old so you definitely want
to use the newer versions if you can I
think that's it so that so the next
thing I'm going to work on is probably a
transfer learning I'll pry transfer
learn from from this model since this
one seems to be the best one but I am
curious to try something more like
anything that has like the slowest are
the quickest speed so I don't even
actually know this one looks pretty
quick
you know 26 I don't know if anything is
quicker than that 26 milliseconds so so
in fact what I could do is copy link
address pay whoops what happened here
paste and then come over here and we
could paste in that mobile net and then
we come down here break this rerun it
and we can just see like how quickly
does everything run the other thing I'm
not sure about is like right now this is
a 1280 by 720 but I can make this size
anything I want so like for example two
before I send it to process I could make
that a you know of 480 by 270 or
whatever sized image so as you can see
this is much quicker than the other one
although it doesn't seem to be there
cooks
yeah so look how like the frame rate on
this is pretty good so the one that's
top left this is the one that's actually
running it's pretty good but you
see it doesn't have any distance to it
so like the other model was would
definitely have detected that card
probably get that pedestrian there look
how close I gotta get to this car
we're close oh my gosh come on I can't
believe it doesn't get that car or the
Pinet okay so anyway they're sort of
goes so this one unfortunately doesn't
quite work and I and I'm not totally
sure why like maybe the resolution is is
too high I'm not really sure like oh
maybe if I bring it down because I think
that matters like how big the input
images I am kind of surprised that we're
not detecting that car let me let's
break this real quick I'm just kind of
curious some of this stuff I have the
answers to as I'm filming and some of
this stuff I really don't so if I come
down here I want to say okay so this is
so I'm gonna just try 480 by 270 oh and
then when I pop it up I'm gonna do is
I'm gonna throw in a resize so cbt-i
resize so just before it shows the image
because 40 by 270 is gonna look really
really small on this screen because this
is 1280 by 720 and you can see it's
already quite small so 480 by 270 is
going to be like super small so see what
you got resize it got rid of my quote
there and then you resize 0 0 and then
the function that we want to resize for
X is just two times and then function Y
will also be at two times gonna save
that come over here run it and I'll come
back in here we'll see if it detects
that car hopefully the mods probably
gonna teleport me any second now let me
just wait a minute not sure I don't
think escaping helps the mod I think the
mod uses its own staying a long time so
we'll see let's see if right at the end
it gets it no so I don't think that's it
either as far as well we mmm we think
here
let me pause this real quick let's see
if I go back over yeah so I think this
model just see that's the unfortunate
thing is that it's it's fast but it's
not detecting the things that we want
but we might find that via transfer
learning it does and we can get that
this frame rate which is you know I
wouldn't want to play it this frame rate
but I can you know so anyway those are
the things that we're gonna have to
definitely be checking out but like I
said the best one that I've found so far
has been this one here so probably all
you do something like this like wick
when I go to train a custom model my
next step is gonna be a train of custom
model transferred from this model here
and then maybe later transferred from
like this one just because this one runs
so fast I'm actually just gonna make a
note of this myself detects poorly paced
okay so anyway those are the next steps
I'm gonna be working on if you got
questions comments concerns suggestions
feel free to leave them below hopefully
at some point someone will tell me what
that I already closed oh no I didn't
it's over here what this is Coco ma pee
or if I google it what does that mean
what is the Coco here we go could
someone just tell can we just be told
what it is yeah so it's like speed to
accuracy trade off maybe but is a high
score good or bad
okay anyway um I guess off to read about
that later someone can comment below
though like this is the one that we were
running and I've almost wonder like
speed / like if accuracy was high then
this would be very low so I'm not really
sure what we're looking for to be honest
anyways that's it for now that's what
we're gonna be winding up heading
towards the next thing is just getting
the AI to play well enough on foot so I
will see you all in another video