Having a Waymo Driver that can respond to things in the
real world as quickly as possible is really important for
safety critical situations. Hey, my name is Jonathan. I'm
a Product Manager at Waymo on the Perception team. O
work on machine learning models that allow the Waymo
Driver to see, hear, and understand the world around it.
Today, I'm going to be taking you through a case where the
Waymo Driver is driving through the Phoenix area and a
cat darts out from underneath the vehicle on the right.
Let's back up and see what went into it. The Perception
team's responsible for taking in raw data from things
like camera images, lidar point clouds, and radar
returns and turns them into logical concepts. Like there's
a person here at the crosswalk or there's a vehicle in
this lane traveling at 22 miles per hour. What we can see
on this camera view is the Waymo Driver cruising through
suburban Phoenix and on the right, all of a sudden, a cat
jumps out from underneath the vehicle. There's actually a
few challenging pieces about this case. One is that the
object is very small. Another is that it's moving at quite a
high speed and a third is that it's quite low to the ground
The key thing I want to highlight here is actually how
little time the Waymo Driver needs to react. In this case,
that would mean the time between the cat appearing up from
underneath the vehicle to the time at which we started
applying brakes for it. It's especially challenging
because we have an occlusion. There are lots of challenges
that come with building a low latency perception system
like this. One way that we try to do this is by
reporting objects as quickly as possible. Even when our
detectors aren't quite ready to say what it is yet. By
doing this, we allow downstream components like planner to
know that there's an object there earlier instead of
waiting for a more detailed object confirmation. In this
example, we're able to react in less than half a second
which thankfully was fast enough to allow the cat to
safely cross. Part of building the safest driver means that
we want to build a reliable, consistent, and responsive
driving system. How do I feel about cats? I'm slightly
allergic to cats, actually.