what's going on everybody and welcome to
a video slash tutorial on I'm just gonna
be poking around the karla library thing
if you're not familiar with Carl I don't
imagine many of you guys are it's an
open source simulator for autonomous
driving research okay so basically what
it does is creates a simulated
environment specifically dedicated
toward people who want to Train
self-driving cars so historically I've
always used Grand Theft Auto for my
shenanigans but this has certain
benefits over that because it's just
basically built from the ground up to
have an API for you to do all kinds of
automated trial-and-error kind of stuff
and so it just kind of makes sense the
last time I looked into it was like two
years ago and I don't think I even got
off the ground with it I forget exactly
the issue I think I could run the
simulation but not connect to it or
something it was some sort of strange
thing going on there so I thought I
would take another look into this kind
of package here and then share with you
guys the steps that I take whenever I'm
visiting some new library I pretty much
like I understand what Karla is but I've
really never worked with it because like
I said I couldn't even get the basic
thing to like run so I'm gonna try that
again if it works you'll see this video
if it doesn't work you won't know about
this videos overseeing it hey it worked
okay so I'll and on Karla okay the first
thing I want to do is just go to get
started I already kind of know what
Karla does it might be useful to read
this though like here you can see that
you can have like a server with multiple
clients that connect so you can run
multiple agents at any given time the
other thing that I noticed was basically
there's like remote connection now which
is pretty cool you can disable rendering
of the simulation which I guess both for
client and server would be quicker so
you could run things quicker you don't
necessarily have to run in real-time so
okay pretty cool I don't know anything
about I've never used the robot
operating system but but that's pretty
cool
they they have some sort of bridge there
like that's kind of interesting that you
know you can even test that kind of
stuff so if you wanted to use that with
self-driving cars that's pretty cool too
so they also apparently have some
autonomous driving baselines that's also
interesting I might want to check those
out just to see how they do a couple of
things because you want you definitely
want things to be as efficient and clean
as possible so I'll be looking into that
obviously for me I'd like to do
self-driving so I kind of want to poke
around and play with all the stuff that
they have here and then they also have
like cameras and different sensors like
lidar and stuff like that so those are
pretty cool too so anyways let's go
ahead and get started so I click on get
started just gonna try to make sure this
fits to my page here and at least here
they're saying hey make sure you're
using nine or greater okay got it
chickity-check grab a release so just as
a caveat this does use Unreal Engine 4
I'm just gonna be straight with you guys
I don't know anything about Unreal
Engine 4 other than it's a game engine
ok and I might even be wrong about that
so I know I've done some stuff in the
past to run this so I don't know if if
they can include all the files from
Unreal Engine 4 and like that's gonna
make the game run or something so like I
always try to think like do I have
things pre-installed that would make
things different for me as opposed to
people watching this so I'm actually
doing this on my main computer so I
might have some things already here that
you don't have so just keep that in mind
so I'm gonna go ahead and click on the
latest released I don't really think I
want the nightly build so from here the
latest stable version is o8 and then all
these that sounds like a long time ago I
wonder why that's the most stable
version yeah it's been over like 6
months before since a stable version
it's kind of weird and at least it looks
like here engine new agent class
reformed client-side it sounds like
these are some pretty important updates
so I don't think we want the latest
stable version
let's just go with the latest versions
Pete what he gave me like one second to
load this page are you kidding
Thanks idiot yeah okay so um that
couldn't possibly have been a proper air
anyway uh windows please no windows I'm
pretty sure they have windows support
I'm trying to use windows I know some
people relying on easily max because I
can record in Windows better whenever
you see me using Linux
I'm usually recording I'm using Windows
still to record because I do my
recording and editing on Windows which
is easier if I'm on Windows okay but I
use Linux all the time don't worry I'm
okay I'm gonna survive so I usually use
a virtual machine but in this case I
don't want to use a virtual machines the
VMS are kind of a pain in the butt
whenever you have full screen
applications so what I really want
Windows let's see if the cartlett I'm
getting a little sick and tired of your
shenanigans okay so it does have Windows
here it's experimental awesome
how about this one someone better yes
okay cool new town environment with
narrow roads open drive parser and
waypoints API cool okay so I would just
going down with this one if you're on
Linux you can download these other
things and I guess I'm just hoping
there's the windows-based towns I'd have
to imagine I'd have to imagine that the
towns are probably cross-platform like I
can't imagine why the town would be
operating system specific honestly but
okay so this is gonna take apparently a
minute while we wait on that I mean we
could just kind of check out what we're
what we're dealing with here so looks
like we want to have pygame and numpy
installed I already have both of those
so make sure you do too
then to run it it looks like we just run
Exe file and then on Linux you would
just run the shell this launches if you
over the city this is the spectator view
so it sounds like like yeah they have
more of like a client and server so then
you so you got the server I'm guessing
this is more like your server view and
then later any other scripts you're
gonna have will be your client and I
don't know if they're gonna show a
window or if they're just gonna simply
like connect in and then you grab stuff
from the sensors I don't really know
we'll find out okay Karla requires these
two TCP ports make sure if you're having
issues you might need to open those cool
got it
also Karla and then the second port is n
plus one so you could change it I guess
okay so to add life to the city we are
gonna run spawn and PC so again it just
sounds like there's a lot of this like
server client that's kind of weird
though because I wonder if you can just
like start the simulation with NPCs but
okay so it sounds like it'll just be the
city and then we can change the weather
so definitely with the Python
self-driving car like if you're using a
camera as your primary sensor you
definitely want to have tiny weekly
weather because otherwise like slight
changes are gonna just totally throw you
off if you're doing like lidar I don't
know I don't think it would really
impact it to be honest with you guys and
then finally they run this which is
probably your client and I guess they're
using PI game this is PI game so it
looks like it's gonna open another
window so that would be like your client
window so you'll have at least two
windows open if you're using the same
machine but what's kind of cool is yeah
you can connect to a server so Python
manual control and then with the host
argument you can pass in the host IP
which sounds like in the future I could
probably host a server for people to
bring in their self-driving cars and I
wonder I wonder what kind of load like
what kind of GPU load you have on the
server versus on the client side
especially like what if all you're
trying to pull is maybe lidar data or
something like that or maybe the imagery
at a low resolution I wonder
I wonder how that would go okay so it
looks like our thing has yelled let me
go ahead and start the extraction
process hopefully I didn't take too long
and I think the first thing I'd want to
do so like generally I'm always like I
mean the api's are useful if they have
good documentation but usually places
don't so let's go ahead and click on the
Python API reference yeah so that's
basically you know the index of a book
pretty not useful to an amateur newbie
like me
overview useless Python API tutorial
okay so maybe this has a little bit of
interesting information for us what is
that oh that was dinging at me because
of the let me just meet this real quick
okay so we've extracted everything so
now what we can do I'll probably check
out Python API but first let me just
finish reading what I was reading here
so in this tutorial blah blah blah yes
yes copy that
okay so so it looks like they're kind of
separating out core concepts I'm gonna
guess these are on drugs
you've got actor blueprint in world
actor is anything that plays a role in
the simulation so vehicles pedestrians
in sensors a sensor is an actor okay a
blueprint before spying an actor you
need to specify the attributes we
provide a blueprint library with the
definitions of all actors available I
don't know if that means like they have
defaults
I hope world world represents the
currently loaded map so you can like
probably reference the world although
I'm just not really sure why you would
do that other than maybe initially for
some reason but to change weather okay
so the world is more of probably server
type tasks as my guess because I'm
thinking probably your actors what you
would reference for like sensor values
right because that's basically at the
end of the day we want to move the
vehicle based on the the vehicle actor
based on the sensor actor data okay so
okay this is starting to look more like
the raw hood so first thing I want to do
is test out the those initial commands
that we had okay so the first thing we
need to do is run lovey we do have the
exe fantastic so like for example ue4
like do binaries and content and then
engine binaries so maybe you don't have
to install country legend for I don't
know I'm sorry I'm an idiot okay so far
so good
oh okay okay
my GPUs starting to make some noise let
me move this over as much as I can
anyway you can't really see the full
screen but that should be good enough
for demo purposes maybe at some point
I'll fix that okay looks a little lonely
in this city some sort of evacuation has
taken place okay we can use our WASD
keys to move around we are flying okay
yeah there's definitely nobody here but
I think that was what was to be expected
because apparently we have to load in
the NPC's so let's go here spawn NPC 80
pythons spawn NPC up I I am an expert
Python programmer and I know that's not
gonna work here I'm gonna guess maybe
they're in Python API Carlo maybe not
seeing agents no probably not no no no
no back it up bro code oh let's go no
examples over to manual control steering
wheel hold on sorry I'm getting
sidetracked but I got to see it I got to
see it
it's I'm still looking for a steering
wheel what does that mean what I must
have scrolled by it anyway I'm curious
to know can i plug in a steering wheel
somebody answer that for me no rendering
mode example that's pretty cool I need
to stop getting distracted where was
that going spawn in PC okay so here is
spawn in PC cool so let's go ahead and
run that real quick python spawn npc de
pie and then it was n80 no module named
carla so if I go to Carla then I click
here and then go to dist for example
yeah so it is here
oh okay so the problem is here it's
looking for Python threat like this is
for Python three seven and for me Python
corresponds to three six so so let me
just try that real quick so let's just
say PI - three seven spawn NPC to PI n
80 let's see if that works 80 vehicles
cool okay so that does work and I guess
they're probably using a directory
change let me just check that real quick
spawn NPC open with sublime yeah right
yep right here cool okay so then yeah
just it looks for the specific version
of Python that we're using that's why I
couldn't find it so that's pretty
impressive
they are already updated to three seven
cool very good and and we have cars very
cool I believe at this point I'm already
further than I have ever been with Carla
cuz I don't I couldn't even I couldn't
get this far cool that's awesome
okay I want more cars what happens here
ctrl C to exit oh dear
the whole thing is crashed because I
left my NPCs okay fine fine I can deal
with that let's just run it again that's
funny get up there get up there
cool there goes my GP you can hear it
it's crying a little bit which is funny
because this does not appear to be a GPU
intensive thing but it is that's why
those triple-a titles get paid the big
bucks okay so and just for the record I
could not do any better I would do way
worse I would write the most inefficient
games possible okay so 80 was too few
I'm gonna do oh you can't see it I'm
gonna do 250 NPCs I think that's enough
oh but could only find 200 spawn points
so I wonder I wonder if I run it again
though that's a lot of cars though it
would be hard to drive in this as it is
but I'm curious
oops Python epi examples this really is
my methodology it's fine and PC and
let's just it I'm so used to 80 let's
just do another let's see if we can get
away with 80 I guess see what happens
can we do spawned yeah cuz collision at
the spawn location but otherwise we got
some more ah yes
beautiful that's pretty cool
I'm having way too much fun spawning
NPCs okay uh what were the other
commands okay so we've got NPCs
okay dynamic weather that's probably a
good idea in the self-driving car a
series with Grand Theft Auto like
especially if you're relying primarily
on a camera sensor you definitely want
to use dynamic weather because the
camera let's just run this real quick
the you know if the Sun Goes Down like
obviously the day and night difference
but also yeah I like clouds rain wind it
affects things like shadow I'm not sure
what wind is doing I'm not sure how
crazy of a physics simulation we're
dealing with here I can't imagine wind
has a huge impact here but I don't know
that would be pretty amazing if wind
actually has an impact here didn't know
this was truly a driving sim okay what
was the other thing
Oh manual control absolutely let's try
that one out
pi - 3/7 space manual control oh we were
so close everybody okay hmm um index
error oh you can't see the error let me
look at this here although maybe you got
the same error list index out of range
and it's out of range because of this
line here and the only thing that's
indexing list is here so fonts would
therefore have to be an empty list so
now we need to find out why is fonts an
empty list manual control let's check it
out
let's check it out together everybody
okay so what we're looking for
we just search for mono fonts
okay so fonts was our empty list so X
for X in PI game got find I get fonts so
either PI game font get fonts is empty
or the more likely Occam's razor
if mano in X I'm gonna wager
so the heck is this it's only looking
for mono fonts so I'm guessing we don't
have mono what if I do are real that's
fine we can leave let me just fix it
real quick so let's just do it
I'll find and replace so mono and then I
pull this up here will replace all cases
of mono with Arial which is probably
gonna it could mess up some other stuff
actually well we'll see what happens and
then what did I yeah but boom - hey they
said experimental they did they warned
us okay but we made it so far I'm gonna
wager a fonts issue I'm gonna let that
slide let's just run this again PI three
seven manual control dot PI come on here
we go we got up window let me fix where
it is here beautiful on this horribly
rainy day okay
again WASD controls I'm not sure okay
yeah we are dang cool it oh we collided
with them how do I reverse how do I
reverse didn't they show yeah okay here
we go so here we have our commands
throttle break toggle reverses cue so
we're actually changing a gear fancy
space that's pretty good though to
separate out that was one thing I didn't
really like so much about grand theft
auto is you know what s was both reverse
if you held it long enough but also a
brake didn't really make
since p.m. toggle autopilot with P
interesting toggle manual transmission
awesome I can't imagine why would you
want to do a self-driving car with a
manual transformation that makes no
sense but okay change vehicle with a
backspace heck yes record image to disk
cool sounds good probably screwed it up
by highlighting okay so let's continue
on our path so cue to reverse oq2 Unruh
verse and we're off let's just thread
the are you kidding
there was definitely space y'all gotta
fix your hit boxes let's go okay okay so
one thing I'm noticing here is yeah
right at the bottom you can see it's
noting when you clunk I crossed a broken
line so that's like the dashed lines
like here can you see the math yes
so now let me go let's brought cross
that and then cross that book you know
oh now it tells you okay so we crossed
the solid line then the movement back
it's probably gonna alert us again let's
cross over this median
oh we just collided oh that's cool too
it shows the collision there I'm getting
good at shifting gears now guys okay
let's cross over this solid sidewalk
you're calling that a sidewalk I don't
think so nobody should be walking on
that Sidewalk okay so at least works
that's pretty good this car has some
serious steering issues what does that
change the car a backspace Lou's
monstrosity
Carla Cola nice ooh
that's not a car the bicycle I'm in gear
- this is crazy
okay so there's lots of cars apparently
that we bicycle again lots of cars and
bicycles apparently move a Ford Mustang
this has some get up and go as well
which just had noise like sound but I
don't think it does
oh gosh that didn't go well okay so
enough playing around you guys can drive
if you want it looks okay I wish the
driving physics were a little better
that was one somewhat nice thing about
obviously grand theft auto driving
physics aren't spectacular but I would
say they're better than this but this is
not terrible it's not terrible and
really what you're trying to work on is
that vision task oh I think I missed it
nice little smart car though it's cute
it's adorable I like that it knows when
it's crossed all those things so like my
hope and objective is one we can detect
a collision so I for a long time wanted
to do reinforcement learning so any time
I'm not gonna care if we cross a line
but if we collide with something that's
a bad thing
so that's pretty cool we can use that as
a penalty event we can know how fast
we're going I still I don't really
actually want to make a self-driving car
that someone would actually use I like
us really fast self-driving cars so
that's always been my goal so maybe I
can do that with this but that's pretty
cool so closing out of that let me see
so we've made it this far I might just
cut it here in the next tutorial we
could start going with you know how to
actually control the car and all of that
like I'd like to load in and actually
control the car and maybe in the next I
don't even know I have no plans what
what I'm going to do I'm just kind of
poking around at this stage but looking
here this is like a pine tutorial it
looks like here we connect to the client
cool blueprint library so I'm guessing
blueprint random choice but okay so it's
for the vehicle if it has the attribute
color change it cool
spawn actor is the blueprint okay so
that's how we make our vehicle easy
enough set autopilot to true a depth
camera attached to the vehicle
okay doesn't like the location we gave
to the vehicle I don't know what that's
about
uh-huh and then here we're adding NPCs
to the city okay cool so let me just run
that tutorial example MD PI - 3 7
what was that tutorial about PI so it
created all those vehicles and ourselves
and for some reason we don't actually
see a rent boats don't Oh
destroying actors and then done yeah so
ok so it creates it for 5 seconds I
guess and then we're done ok so we'd
probably want like some sort of loop
okay so probably what I'm gonna do is
just kind of poke around further
probably what I'm going to do is poke
around further with like the Python API
tutorial and I think probably what I
would like to do is just simply create a
vehicle and just drive that v8 the
vehicle and then just drive it forward
and then just to see if we can do it and
then we want to get the sensor data and
probably just have like a hood camera
there are other sensors like lidar and
stuff like that and maybe later we can
use those but for now I just really want
that hood camera so I think what I'm
gonna do is I'm just gonna start with
tutorial hi
kind of dumb it down so for example I
would just get rid of the creation of
all the other vehicles and just have our
one car that way cuz apparently that
didn't that only spawned our city which
is this but I think this city is dead
now like I don't think anything's
running it's also kind of blurry I'm not
sure why that's happening anyway I think
nothing's actually happening there so so
I just want to spawn the one car and
just looks
have it drive it forward and make sure
we can control the car and then from
there what I'd like to do is try to do
some sort of reinforcement learning
situation so that's what we're doing
that next video if you've got questions
comments concerns whenever you know the
deal feel free to leave them below
shouts out to my most recent or actually
not my most recent these are long-term
Channel members four five and six months
we've got Eric Gomez Robo Ajay Kumar
payments.i Adi all in for six months
Chris Allen Frederick fail
Joey Santana and piranha ji for five
months and coming in at four months a
drug addict Liz
Ivan buck neck youtube series cheeky
buns with the best name ever Franklin Jr
ami crog's Miguel Hurtado lateral parada
Brandon Merrithew yan guo trevor hughes
and a VJ thank you all very much for
your support you guys are awesome and I
will see everybody in another video