what's going on everybody and welcome to
part 2 of the karla self-driving car
likely with reinforcement learning
series in this video we're gonna just
carry on with what we were doing the
last video we just made sure a car that
was working connections were being made
scripts were running and now what we
want to do is actually start working on
our own Python code so my goal for this
series is to use reinforcement learning
to train a self-driving car now in order
to do that like any big problem we have
to break these problem down into little
itty-bitty chunks so the next small
chunk that we need to be able to do is
we need to be able to figure out how to
spawn this car in the environment how to
control this car in the environment and
then how to get sensory data back from
that car we need basically like at least
to start two types of sensor data the
first is like a hood camera basically so
just like some sort of forward-facing
camera and then we also need to sense
whenever that car has been in an
accident so to do reinforcement learning
pretty much all we want to do is drive
around if we get an occlusion that's bad
if we drive around at decent pace that's
good cool done so this isn't gonna be a
it's kind of gonna kind of be like the
self-driving car and Grand Theft Auto
I'm not actually gonna try to honor Road
rules or speed limits or anything if
anything I just find it much more
entertaining to watch a car drive really
really fast so so take note if you want
later you can add more rules in how you
just determine that that reward function
and you can get the car to do whatever
you want it to do so anyway what we're
gonna do here is figure out how do we
actually interface with Carla using
Python so if you go to the get started
bit and scroll down to the Python API
tutorial we've been on this page briefly
in the previous video if I recall
correctly it's been a while for me it's
probably just a few moments ago for you
but so a bit and we basically don't
forget we've got actor blueprint world
actor this is both your vehicle other
vehicles pedestrians as well as your
sensors these are all actors then
blueprints the
are the attributes of those actors of
the vehicle pedestrian sensors in the
world this is your environment basically
the server we need to connect set a time
out this way we don't end up hanging
forever and ever and ever we grab the
world once we have that information
basically these will be your first few
lines of code every time as well as this
one you're pretty much gonna always
reference the blueprint library and then
here right away we have at least this
one sensor right this is for your
collision sensor which we will
definitely be using and then this
basically is going to show you maybe how
to attach that sensor I don't really see
it here transform spawn actor blueprint
transform this is probably doing it this
little bit is kind of confusing to me as
far as like how we actually add things
to things but I'll show you guys
hopefully it'll make sense when I get
through it uh okay so I think we're
ready to rhumble what I'm gonna do is we
can leave this up for reference I think
and what I'd like to do is I'm gonna
write everything in the in that examples
directory if you want you can move what
would it be Carla and then dist
yeah you can move this into your site
packages of Python 3.7 and import it you
could do that I'm just gonna keep it
nice and simple and just keep it in the
same package here so what I'm gonna do
is just take like any one of these files
I'm just going to copy pasta just so you
can not write a few lines and we'll call
this tutorial video - what a great name
and then we will open in sublime text
move this aside and now we are ready
cool so what I'm gonna do is basically
delete everything from there to here and
then import Carla that's fine and then
everything else I'm just going to delete
from this script basically I just don't
want to write that like a little this
this bit it just changes that system
path so we can import that Carla
easy enough so now what I want to do is
begin talking about some fundamentals of
what's going on here so first of all we
anytime you've got a client a server we
need to also probably clean up what
we're doing so the first order of
business that we're going to attend to
is going to be we're gonna have this
actor underscore list and that'll be an
empty list and then what we're gonna do
is attempt to basically do our
connection to the server all that fun
stuff and then when we're all done we
need to clean up and what the way we
clean up is for actor in actor
underscore list actor dot destroy and
then print all cleaned up okay hmm
so this will probably be the starting
script for pretty much any Carla code
that you're gonna write honestly well
actually well we would first need to
connect so let's do that as well so
we're gonna say client equals Carla dot
client and then we're connecting
localhost
again that could be some sort of remote
public IP which is super awesome in the
case of reinforcement learning one of
the hardships of doing it that way
would be the latency to and from so with
reinforcement learning it's a
particularly challenging one because
each step needs to know what the result
of that step was so if you have a server
and you've got like a hundred
millisecond ping it's not like
everything's just kind of shifted over a
hundred milliseconds instead you have to
take a step and then send that out and
then wait for it to come back so you
would get 200 milliseconds per step so
five frames per second which probably
isn't good enough so we're gonna keep
everything local running on so in theory
this would all have to run inside the
car luckily you would only need to run
the model though the model would
actually predict pretty quick like you
probably wouldn't be doing exploration
in reality anyways I digress so
continuing on client dot
underscore timeout we're gonna set this
to two seconds in the docks they set a
timeout I think of ten I hope we don't
take that long if we take more in two
seconds to do something there's probably
something wrong so anyway we'll keep two
there then we're going to say world
equals equals client get underscore
world and then we're gonna do that
blueprint library equals world war world
dot gets underscore blue print
underscore library okay this is probably
more along the lines of the code that
will start every Karla endeavour then
we're gonna pick a car this might also
be pretty common but we're gonna say BP
for basically blueprint is going to be
equal to that blueprint library dot
filter and we're gonna filter by things
that say model three so we're gonna grab
a Tesla Model 3 here and I'm gonna say
the item at the zeroeth index that's
what we want so there's only one Model 3
so this will load us in a blueprint that
is the blueprint default for the Tesla
Model 3 so while I'm thinking about it I
probably should just fire up Carla and
then while that's firing up I'll
continue hopefully nothing glitches out
on me well that's doing it but we'll
find out so while we wait on that we're
gonna set and once we've got the
blueprint for the vehicle the next thing
I can hear the GPU we've must have it
already yep okay so we've got our world
and what I'm gonna do is I'm just gonna
look down and scroll up again
kind of center over here this thing
looks like a strange smiley face with
the goofy nose I wish I could scroll out
a little quicker but anyway this way
when we do spawn a car we'll be able to
actually see it in the environment
that's good enough the only place the
car could spawn and we don't see it is
if it spawns in this like tunnel area
so cool all right now okay coming over
here basically we've got blue pride okay
so what we want to do is if we spawn a
car we want to actually to do that we
need to spawn it somewhere so we're
gonna say spawn point equals we're gonna
use random choice and we have not yet
imported random so let's go ahead and
import random and come back down here
spawn point equals random choice for
world get underscore map dot get spawn
points so get spawn points as I believe
like 200 spawn points that we we can
choose from so we're just choosing
randomly one of those spawn points and
at least one is in the tunnel so just
take note sometimes you want to see it
but that's okay again we're gonna only
look down upon our world here a handful
of times before we're we're done looking
at it this way
except for debugging purposes only so
okay so once we've done we've got the
spawn point what we want to do is we
want to say our vehicle is equal to
world dot spawn underscore actor the
actor that we want to spawn is the
blueprint that we just set up here again
we can change attributes of that
blueprint before we spawn it if we so
choose in this case we're just gonna go
with whatever the default is so we're
spawn the blueprint where do we want to
spawn the spawn the blueprint at the
spawn point okay so that will spawn our
car now one thing that we can say
there's kind of two options here so if
you want to spawn NPCs or just
computerized vehicles what you could do
is vehicle dot set underscore auto pilot
to true now this is funny because even
in their docks they they're like work
and we're stressing that this is not a
self driving car code this is just
rule-based because it's connected to our
environment and to me it's just crazy
that we have to explain this but it was
super common in this self-driving car in
Grand Theft Auto series where people
were like the taxis drives so much
better
than this and the the other you know
cars in Grand Theft Auto Drive better
than this why don't you just use that
code well that's hard-coded that's it's
connected to the game engine so it knows
where the things like lanes are it knows
where other drivers are it's all
rule-based
because it's connected to the game
engine whereas in reality we're not
connected to some game engine otherwise
self-driving cars would be so simple so
anyway I wish I didn't have to make that
clear and I wish they didn't either but
you do so if you just want the car to
drive around an act sort of like a
self-driving car or not a self-driving
car just a car that's driving on the
road you can do this I don't think they
take erratic actions or do anything
unexpected though so I wouldn't use this
to replace human drivers or like
simulate human drivers I suspect it's
nothing like a human driver anyway we
don't want to set autopilot we actually
want to control the vehicle ourselves so
to do that although one thing I would
say is you probably could set autopilot
and really quickly train a model to
drive like autopilot drives we could do
that again I don't want to do that
because auto it just happens to be the
case that we have access to the game
engine I don't really necessarily want
to train it that way but you probably
could anyway
vehicle set autopilot okay so rather
than setting the autopilot instead we
want to control the car and the way that
we do that is just vehicle dot apply
underscore control and then it's Carla
dot vehicle control and we're gonna set
throttle equal to full throttle and
steer for now we'll just say zero point
zero we're just gonna make this car go
straight now every time we spawn
anything we want to actor underscore
list dot append and we want to append
that vehicle now we're going to say
blueprint is going to be equal to well
hmm I think what I'll do
now let's just make sure that runs
actually okay that's probably a good
idea so we need a couple other things
one we need the camera and then - we
need the collision sensor but for now
let me just run this let's just see
where we are right now where are we
Karla is it this yeah okay what do we
call it - Toro view yeah okay so PI -
three seven tutorial video - dot PI and
what I'll do is I'll just put this over
here I'll pull up the environment you
can't see the whole thing I know that I
just know the cars gonna spawn where you
like can't see it it's in your your view
I'm really just missing the bottom
though I'm gonna laughs if that's where
it spawns let's just respond see what
happens
oh wow okay it cleaned up immediately
because we didn't do it like we just
spawned the vehicle got rid of the
vehicle and everything happens so fast
we didn't even see it so let's sleep for
five seconds
try try that again okay let's go over
here oh oh darn it
what an amateur ok import time and pour
time save that come back over here rerun
it pop over here I actually did not see
the car I believe it was there um let's
see here let's try again oh there it is
I saw it it was over in this area
wow it's really hard to see it we're
like so far zoomed up maybe we should do
more than 5 seconds let's see if I can
spot it quickly enough this time here it
is it's gonna crash into the fountain
yep
okay so hopefully you can see yours uh
sorry if not but you could run for more
than 5 seconds or something like that
but basically we wanted to make sure
things run
so now what I want to do is attach a
camera sensor to the car I think we'll
do that and we'll deal with the
collision probably in another video or
something because that's good the
collision sensor really only has to do
with when we actually go to do
reinforcement learning okay so going to
sensors we can see hopefully didn't they
have a oh no we don't want collision we
want that RGB camera right and so in
this code basically we could pretty much
copy and paste this code the thing is I
don't actually want to do this with the
I'm not trying to say I don't know why
they're saving the image to disk I don't
understand when that would be actually
useful you would really want it that
that image in memory at best I'm not
sure why you'd want to deal with io2
disk but whatever so we can take this
code but I think we'll just I also want
to change a few things about this code
anyway whatever I'll just use the code
from minuttes because there's a couple
of different things that we want to do
from there so actor list a pen vehicle
will keep time not sleep at the very end
and now what we want to do is we're
gonna have another a blueprint let's
call it cam VP or something like that
and this price should have been vehicle
B P R I don't know whatever doesn't
really matter and we're gonna say that
is blueprint library dot find and we
want to find the sensor that is a camera
that is an RGB camera and RGB and don't
be fooled
it's actually an RGB a camera so Carla
people get on it because it also returns
a flattened array for some reason that
needs to be reshaped and if you reshape
it to whatever your dimensions are by
three you are gonna be a sad panda so
anyway continuing on
b.p dot sets underscore attributes i
actually don't know what the defaults
for this camera are but you definitely
if you're going to be feeding it through
a neural network you want to be very
specific so image underscore size
underscore X and we're gonna set that to
be we're gonna make it an F string
actually oh whoa we don't even have to
okay definitely make an app an F string
instead we're just gonna say M
underscore well it has to be a string I
guess I will use an F string I don't
know cuz it needs to be in string form
so M width I might be run there anyway
copy pastes and width and then M height
whoops make this all capitals XY okay
now we need to set in width and M height
so we'll just go up to the tippity-top
here and we'll say m underscore width
equals will do 640 m high it will make
that 480 fabulous and the next thing
that we want to do is we can also set
which is kind of cool field of view so
you can make kind of like a fisheye lens
so cam underscore BP dot set underscore
attribute field of view whoops needs to
being quotes field of view and then we
could set it to be like 110 I'm sure the
default is like I don't know 70 90 I
don't know anyways I don't know what any
of the defaults are you could query the
actual blue the default blueprint to
find out all of those things I don't
really care at the moment if you want to
know more things that you can do you can
always just do Durer over the blueprint
if you wanted but you would I guess that
would only give you set attribute I
guess you'd have to print out the string
form to see all the attributes I'm sure
it's also somewhere in the documentation
I don't know where exactly you'd have to
go to find all that really I just read
through the documentation saw some of
the exam
of them tweaking certain things and went
from there I didn't actually read
anything beyond that so okay so we set
the field of view okay so we've got the
blueprint set up now what we have to do
is one we have to yes spawn that
blueprint but where do we spawn that
blueprint and on there's kind of two
things one is you would spawn the what
you can say is you can spawn you have to
spawn you we want you quite literally we
want to attach the sensor to the vehicle
and there is a little flag when we go to
spawn the actor to attach to something
so it will always follow but we don't
not only do we want to attach it to the
car but we're on the car so what I'm
gonna say here is spawn underscore point
is equal to Karla daunt transform and
that's gonna be Karla dot location and
this is a relative location to the car
so I've just kind of trial a trial and
error to find this I want to say X is
forward I'm probably totally run X is
forward I think Y is left and right and
then Z is up and down
so I'm because I think the camera wants
to start like at the center point of the
car so I've moved the camera forward and
then up a little bit so anyway but
depending on what car you use these this
look this relative location is going to
vary like if you're in a truck and it's
it's up higher or something like that
you're gonna want to change that spawn
location I wish that and they might
totally have this and I'm just not aware
of it I wish they had just like a hood
camera or like Center windshields spot
or something like that just by default
so because part of the self driving will
be from different perspectives so you
know again we're seriously simplifying
this problem because we're just going to
do it with a Tesla Model 3 car to start
but there's other cars that we might
drive and we want
this model to drive in all kinds of
different conditions and different cars
different heights of cars and so on so
anyway that's going to be that for now
what we're gonna say now is our sensor
or camera is world dot spawn underscore
actor and what we want to do is spawn
the cam blueprint we want to spawn it to
that spawn point which is relative when
we attach it to our vehicle so now it
will follow around that vehicle and now
we've spawned an actor what should we do
well we need two actor list dot append
that sensor otherwise we're going to be
sad okay so we've got the sensor how do
we get the data so as you could see if I
pulled up what was it I think it's not
listen yeah you can see here they're
using these like little lambda functions
to interact okay copy that so basically
it's lambda and then the image
information so we can do anything with
that image that we want to do and
apparently image dot frame well that's
the frame number so image is apparently
an object that has a method dot save to
disk okay I don't totally know what this
like Carla image it is I honestly just
attempted an array conversion and it
worked so it's some sort of object that
allows itself to be converted to an
array so it must have I guess an array
method I have no idea I've absolutely no
idea how you would do that because the
numpy array is something else entirely
but anyway they have that built in so
awesome so um hello I don't want to talk
to you anyway okay so let's move this
over so so in order to actually get
information from the sensor we're going
to say sensor dots listen and then we're
gonna listen
using lambda and then you're getting
data it's not actually it's like some
sort of image object from what I can
tell and then what we're gonna say is
process image that data okay so we want
to do that now what we're gonna do oh is
that function just doesn't exist yet
that's why it's getting mad so now we're
going to do is just come up to the top
here and after our constants let's just
add define process image it gets passed
image or probably better called data but
whatever and from here what we want to
do is I'm going to say I for image is
equal to numpy array of image dot
actually reading the nose now raw data
so the raw data is a like just a
flattened array so if you tried to
display that you would find that it's
it's just a law it doesn't actually have
the dimensions of an image so also I
need to import numpy as in P I totally
forgot it had raw data method so yeah
it's some sort of like image object that
just so happens and again what I did was
I pass that to the function and I just
did a print dur image and read out my
options so printer image and we can even
do that image behind umpire yeah so I
can actually just save that and then
come over to our script here let's just
run that real quick image size one oh
it's a capital y we need to lower case
that Y and save and run again cool so I
almost don't want to break it so delete
yeah cool cleaned up so yeah so as you
can see that has these these methods
that are built-in so all these like
dunder methods you can just ignore but
there's a convert a field of view I
guess maybe we could I have no idea what
what field of view would do because this
is after the fact that we took the image
we have frame number height width and
then raw data I figured hey I'll look
into raw data that's probably what I
want save to disk which again I didn't
really want that and that was one thing
that was kind of hard in their Docs
so Karla people if you're listening
honestly I I would think most people
their goal is actually just to access
that data live not save it to an image
and then access that image that doesn't
make any sense to me why that would be
the default that you showed otherwise I
like all the other stuff that you guys
did great job so yeah but anyways raw
data is what you really want I don't
know what transform is with timestamp
maybe we could maybe transform would
help us transform the data I don't know
actually transform probably has to do
with the actual Karla transform I don't
know that doesn't matter not important
so so anyway that's how I found out that
raw data existed so once we have the raw
data and now it's converted to an umpire
ray we want to reshape it because again
print
ìdonít shape you can probably just take
my word for it but yeah it's just like
one big flat array not the shape of a
photo so we need to get this into the
shape of a photo so what we're gonna say
is i2 equals I dot
reshape and then the tuple that we're
gonna use is M height don't forget that
one M width so again an image is like
width by height but in array terms we
first have the height of the array then
we have a width of an array because it's
like a length of an array and then
you've got like the contents within so
anyway it'll that'll probably trip you
up lots of times if you do work with
imagery in programming anyway and then
it's by four because again it's RGB a
for alpha information so yeah cool I
don't mean I guess the Alpha is nice to
have I just wish they didn't call the
sensor RGB I was having a hard time with
that one because it's called an RGB but
it's returning RGB alpha
so anyway cool I actually really truly
want just RGB I don't care about the
Alpha values I mean maybe later we could
grab alpha values and care about them
but right now I don't
so I'm gonna say I three now is equal to
whatever I to was and then this is
useful numpy stuff to just get the first
three elements we're gonna use this :
syntax so : so everything comma
everything comma up to the three so for
the entire height
[Music]
this phone is killing me the entire
height the entire width and then in
South Africa the height in the width
what do we have well we've got our RGB a
values I just want the first three so
the first three are RGB values so kind
of a nifty way to quickly just grab RGB
it's quite it's very fast there is an
open CV method to do this exact thing
which reminds me we actually need open
CV so I'm going to import CV to and for
CV - if you don't have numpy or open CV
bt-dubs just do a pip install you can't
see that pip install OpenCV - python and
I'm buddy cool easy now
see and then you just import CB - so
coming back down here the next thing
that we want to do is anyway what I
started to say it I'm not sure if I
finished there is a functionality in
open CV that converts what's your
problem
okay converts RGB alpha to just
straight-up RGB interestingly enough it
is slower than the numpy method you
would have expected it to be identical
like they you would have expected it to
literally just do this but it does not I
don't know what it does and why it's
slower but it is fast moving riveting
information the more you know now we're
just going to show it so we're gonna
give the title nothing just empty and
we're just gonna show i three CV - wait
key camel
tastehh don't mess that up one
millisecond and then we're gonna return
I 3/2 55 so here we'll actually get a
representation of what the camera sees
and then returning it we're just gonna
we're just gonna normalize this data
before I forget to do it later on
anytime we're working with imagery data
passing it through a neural network
generally neural networks prefer
information to be between 0 & 1 or
negative 1 and positive 1 or something
like that so values of 0 to 255 are kind
of silly to a neural network so we can
normalize them or scale them basically
to be between 0 & 1 ok ooh I think we're
right and I think we're ready to see a
camera popping out from our car are we
still printing shape we don't want to do
that anymore let's go slow things down
let's run this bad boy so I should be
able to just go up sure can
tutorial to good let's write immediately
we do get the front camera the car drops
down and it goes forward awesome let's
just do that one more time make sure
everything's looking good drops down and
drives forward does not stop awesome
okay long tutorial but we got actually
made a lot of progress here we almost
have we basically have all the
information we need to either control
the car via other logic like if you
don't want to do reinforcement learning
with us that's totally fine if you want
to go more a rule-based approach you can
totally do that again there are also
lidar sensors I don't know if I'll find
one really quickly well it exists you've
got those you've got other us all kinds
of sensors that you can work with so if
you want to go that route have at it
we're gonna go off of pretty much a
camera based model to start and see how
far we can get with reinforcement
learning there we also need to add a
collision sensor and then convert
everything to more of like a
reinforcement learning environment
setting so that's what we're gonna be
doing the next tutorial if you've got
questions comments concerns feel free
given below you can also join us in
discord discord GG / centex and finally
a shout out to Rison
new and long-term channel members we've
got over one here Kevin see Alexander
Sarah Bay Robert diamond mojo war of
gooped I think that was Gupta I can't
remem I'd have trimmed your name if I
did I'm sorry cyber carp cigar Jarvis
and Suzanne Williams thank you guys so
much for one or more years we've got
people up to I think 14 months now which
is crazy six months Oh Chris Olin
Frederick fail
Joey Santana and prod geez 1605 and four
brand new members vide shaw otto cub
Koch pests kopetski
Jeffrey Miguel ski you guys are killing
me this next name I wouldn't know how to
pronounce it if you know how to
pronounce that name or that's your name
tell me in the comments and I'll do my
best to actually pronounce it next time
Vaughn show games michael schwartz and
or fur patrick Carol Sreenivas Aida
jetan nandwani minnow al see Harve de
man which I recognize your name did Yuri
sub what's going on
Kevin hey Gaines lots of people to think
but seriously guys I love what I do I'm
really enjoying focusing much more on
doing YouTube than I have really ever so
yeah awesome it's thanks to you guys who
are supporting the channel directly so
really really bottom my heart thank you
so much for becoming a member and
sponsoring basically so okay I will see
everybody in the next video