today in a special edition of drive labs
we're taking you on an autonomous Drive
and we're going to show you the pieces
of software we're building running
together in the car enabling the vehicle
to drive itself our pilot is Dennis I'm
your co-pilot let's go we are now on the
road and we'll be engaging autonomy once
we get on the highway but before we do
that I want to show you our perception
functionality already in action in the
car and perception is basically what
enables the car to see we take in raw
sensor data and translate it into
semantic understanding of the world of
the scene that we're in so take a look
at that happening on our front camera we
have drive net detecting obstacles the
bounding boxes around the cars we have
weighted detecting the intersection the
yellow box around everything weight then
also detecting traffic lights and
traffic signs and light that is
classifying traffic light staid
correctly is red we also have sign
classification going on using sign in at
the same time driving that is detecting
pedestrians in the cyan bounding boxes
on the far side of the intersection we
also have open road net tracing out the
free space around obstacles on the scene
and on top of that we have our object
tracking from frame to frame you see the
track IDs on the top of each bounding
box we also have our camera base DNN
distance estimation running so you see
the distance and metres displayed at the
bottom of each box clear sight net is
also running in the background assessing
whether and how well the cameras can see
in our four cameras around perception
set up on our embedded
EDX platform and all of this rich
perception functionality is what our
planning and control software is going
to use to execute the autonomous driving
maneuvers that you're about to see
[Music]
okay we're now getting onto the highway
on-ramp and entering the coverage area
of the high-definition map that we're
going to use today for the car to create
a route plan that we're going to follow
and basically the car will localize
itself onto the map and create a lane
plan that tells us when it needs to stay
in the lane when it needs to take a lane
change to stay on the route and when it
needs to take a highway interchange the
second thing that's about to happen is
that we're going to transition out of
human driven mode driven by Dennis into
autonomous machine driving mode where
the car is going to drive us so taking a
look at the top right of our screen we
see automatic cruise control ACC and
lane keep LK when they're both off
dennis is driving when they come on the
car will be driving us
so here we go taking a look at the
screen lane keep is now on ACC is now on
we're driving fully autonomously
Dennis's hands are off the wheel but
staying closed for safety reasons and we
are officially starting our autonomous
drive okay we are now in full autonomy
the car is keeping us in the lane let's
take a look at how that is happening
that thick green Center path that you
see that is the path perception ensemble
drive labs episode 1 and it is computing
not just the center path in the edges of
our lane but also the center path and
edges of the left adjacent Lane and the
right adjacent lanes and we visualize
that with different colors so green is
our ego lane left adjacent is red and
right adjacent is blue
[Music]
next we need to determine which of the
obstacles belong in which of these
different lanes the way that we do that
we have the bounding box detections from
drive net we have free space boundary
detections from open road nets where
those two meet is what is called the
object fence and that fences off where
that object is in space we combine this
object fence information with Lane
geometry information from path
perception ensemble and this now enables
us to do obstacle to lane assignment the
car fence takes on the color of its
assigned Lane we are now approaching our
first maneuver first autonomous maneuver
the car is letting us know that based on
our route plan and when you take a lane
change to the right here we go the car
is performing a surround radar in camera
lane change safety check and we are now
moving from Lane keep mode into speed
adaptation in order to figure out the
speed profile to get into the next Lane
and into lane change mode moving from
the center path of the current Lane into
the center path of the target Lane and
we have now completed that lane change
okay we're now getting ready for our
second set of autonomous driving
maneuvers going straight into the
highway interchange onto 280 now
although we know this is coming up based
on localization to the HD map we will
not be using any clues from the map to
actually navigate this maneuver we are
handling this using path perception
ensemble only plane handling mode on the
screen is split because this is a lane
split interchange and now the challenge
is going to be for path perception
ensemble to maintain confidence
throughout this interchange because it
has both high curvature and high grade
but take a look at path perception
ensemble it's still green and meaning it
has high confidence but it's navigating
this difficult curved graded highway
interchange correctly
[Music]
we are now coming up on our next set of
autonomous driving maneuvers to get onto
highway 87 the first thing that we're
going to need to do is another lane
change to the right to get into the
correct exit lane and then handle
another lane split highway interchange
followed by another lane change under
time pressure so here we go first lane
change you see Lane handling mode go
into speed adaptation finding the
lateral path into the next Lane Ensemble
going from red to green as it's landing
in the target Lane finding confidence
that it's found the lane
we have just handled another lane merge
and we are going to have a little bit of
grade profile changes in the road coming
up
right there this is why it's important
to have calibration continuously running
in the car we see the lane handling mode
move into split mode the car needs to
correctly take that lane split to the
right to not unintentionally exit the
route pass perception ensemble is now
navigating another high curvature
interchange we see the Center path
staying green and we are now moving
right into that third maneuver this is a
lane change under time pressure we don't
have a lot of time here to move from the
right lane into the next adjacent left
lane in order to not incorrectly exit
from our planned route so here we go
with switching from keep mode into speed
adaptation into change mode and landing
in the center of the target Lane to
complete that set of maneuvers and we
are now going to complete the rest of
our autonomous route and head back to
the garage
[Music]
and we're back we hope you enjoyed our
autonomous drive today and enjoyed
seeing how our software is enabling car
to drive itself for any questions reach
out to us with a comment section check
out our other Drive Labs videos and
we'll see you next time
you