everyone norm from tested here I'm in Las Vegas Nevada location of the AWS
deep racer League championship Cup Blaine
tell me about this competition who the competitors are and what they've been
working toward this past year well Norman we're very excited about this
because this is the final round of 16 after a full year of competition that
actually started in Santa Clara back in March we have had thousands of
competitors hundreds of miles of track where people have raced to get the best
times to qualify in each one of the cities we've had 29 cities that
qualified in addition to a virtual race where people from anywhere in the world
if they can get the best time in the virtual track they're able to compete as
well we took the best yesterday there are 64 competitors are qualified we are
down to the round of 16 these are the 16 fastest developers in the world right
now and we're giving people a sense out there of what the race actually entails
of course we've covered the deep racer RC vehicle but that's the physical part
of it it runs on the machine learning reinforcement learning models that these
developers have made that's right in a nutshell the developers teach the car to
teach itself how to drive which is different than any kind of an RC
competition they're slowly autonomous fully autonomous the only thing the
drivers have control over is the percentage of battery that gets applied
to the motor kind of like equivalent to a throttle it's almost a throttle but we
actually don't want to call it a throttle because there's no direct
relationship to 80 percent gives you 5 m/s 90 percent gives you 6 meters per
second or whatever it might break down to it's just a raw battery percentage
now the skilled developers right now are on the track testing the batteries as
they are today and testing the cars and loading their models and how they behave
but once the model gets in the car and it's on the track it's kind of like
they've trained a dog and it's a dog show and you hope the dog does
everything you trained it to do but these are real intelligences in the car
and sometimes they don't behave as expected right so even with the best the
best you'll see them racing around the track in the car will miss a turn and
plows straight into a wall because the car at
that moment thought that was a good idea yeah and this is the physical
environment and as much training as you can do as much simulations as you can
run nothing will prepare you for unless you actually tune your model for the
actual physical car on a physical track that's been designed which is the
amazing thing about this virtual simulator they've been using some of the
competitors have been so neurotic about this virtual environment equating to
real but they've done things like include bounce in the real world the
car's got shocks me we've got monster shocks on these vehicles that's the car
jiggles around virtual world there's no jiggle and so some of the
competitors were smart enough to realize if they simply add camera jitter like we
get in video games nowadays if they add that to their virtual camera when they
translate sim to real the physical model understands what to do when the cameras
bouncing around right right and even in a moment of the race that sees time
trials themselves competitors are adjusting on the fly so they're
practicing the responsiveness that connectivity absolutely all these kind
of variables give me a sense of some of the best times and some of the fun
things have been happening sure so yesterday was the very first day any of
the competitors globally got to put wheels on this brand-new track this is
the 2019 reinvent track there are no straightaways there's a lot of real
wicked s curves in this in fact already these brand new walls have a lot of skid
marks where cars are plowed into them that meant last night our field of 16
I've talked to a number of them most of them were in the virtual lab retraining
their models for areas they found were going bad on the physical track Wow
which means today times are faster than yesterday the world record holder Salah
from Japan team DMP she holds the record on the old track of 744 yesterday she
clocked the world record for this track at nine point zero five overnight the
work she did she came in this morning post an unofficial time of 8.1 to almost
a full second faster which for humans may not sound like much but at these
speeds that's monstrous and having some hands-on time with the deep racer
adjusting its easy it's not just press and go only the
fact that she's getting under 10 seconds it's really impressive
so this is where teams really help out so laws part of team DMP there are
actually three competitors in this field of 16 that are from her organization
it's a printing company in Tokyo that is working to reinvent itself through
machine learning and their company has actually decided to use deep racer as
the engine to train their developers what reinforcement learning does well
this team has gotten really good at log analytics and heat maps are trying to
find out where things work or they don't work and tweaking Waypoint models and
the analytics they do are phenomenal and last night they do a little more and it
pays off other competitors are working as teams we have a team from a
university in Taiwan that it came with a number of competitors and two of them
qualified for the round of 16 and there's a lot of work that they've been
doing over this past year who gets that point a lot of collaboration among their
teams and all that work is being put on display in competition starting in just
a few minutes and even collaboration where you don't even work together there
was an underground group of developers that just got together and we're buddies
because of deep racer it's ballooned into a group of 1200 developers globally
all sharing log analysis and models and ideas building a whole canoe from
scratch AWS didn't build it the community built it what started as a
learning tool to get developers introduced to machine learning and
reinforcement learning now has become this obsession it's fading doors
everywhere yeah super cool thank you so much Blaine norm thank you for having us
here to meet you we're gonna chance to take a look Elise races okay
so this is Darren we just watched you compete your bracket and you won your
bracket yes this group see ya as you're practicing and moving on for your next
round of competition I noticed that in your bracket as you are doing your trial
you switch models tell me about your approach to designing them off and then
the one just there yeah so yesterday we won is great to say with nine point
three just morning sorry running the same model same car same track just
wasn't finding the lines at all 14 sections was the best I could do and
when you can only control the spades you see how all he can do is change the
model you have two options model changed babe if the speed change isn't working
so I just gotta keep trying different models so I didn't have a good first run
at all and then I got the other car and I just asked you to just check the
calibration and they just went through one calibration set that made all the
difference just started flying rhymes over and over again and once I got a lot
to qualify I switched to a more risky model that I was making last night and
that's what's that the 0.9 so you're developing and tuning that model even as
late as last night there's joy at the moment oh and they're trading right now
you need to like it still isn't enough to be salar de 8.4 yes no i won't face
them in the bracket if i chase those times it should make me better as well
so I'm not trying to go too crazy but small adjustments in the sacred space I
mean in layman's terms you're giving you're teaching the the deeper to how to
learn or kind of developing a lesson plan and and kind of the carrots to
reward for 400 and you're basically changing these lesson plans up on the
fly and then saying go go through the simulation go through the training
update you're right and take it to the center real world
into the real world where there a lot of variability easily what's the lighting
yeah it's even if you watch this one this is the same model that's my
come--it civically juice it's just not quite exactly as fine-tuned those over
there could be the track could be the car it's hard to say it's the driver
because the driver is so level input yeah but I am if you want to make it
here I think have a couple of mindsets are different from each other
I'm starting to be too consistent with one - so if that model didn't work known
of its Browse search over so I have to be sure car fly to dive headfirst into
into one particular are you noticing that even during a run the way the
deeper under your auto behaved you're noticing that is a direct result of song
change that's the way it took that corner the how cautious was 1% yeah if
you want to keep it to the left there's parameters to set keep left and you'll
see it doesn't care and say you care about the reward so it's a bit of back
and forth you're part of the delivery mutual competitors so tell me about that
environment is your along with other developers in the IT department kind of
just working on this spare time and yeah and spare time and loads of support so
Liberty Mutual Liberty i-team and Belfast which is a technology are mutual
and then my unit a team of individuals made up
all of them together have come to say we're gonna support the slave we're
gonna support the teams run on it we're gonna get you the resources of trainer
models we're gonna set loads of guidelines and rules so everything's
fair between all the teams and then once we want that league it was just a bit
more support what do you need to be competitive in Vegas yeah so whether
it's with rent in the track down from Amazon Dublin so Fernando he's walking
about here he said he was a big help as well
just going through the whole a process coming here today a whole team being
paid to come to Vegas reinvent comes to support us it would be a different story
that was just me by myself even if it's the same model
I do believe helping people with you support me makes just to give you a
courage the faster things on my own don't put your head time there's a lot
of team spirit I like company pride it sounds like company's trusting you guys
compete here and take advantage of all the learnings you guys are going through
with AWS definitely like Lelouch was 100% behind
reinforcement learning machine learning as a way to explore that in a more
competitive environment so we as a car but we're learning height so you program
the Python hyper kilometers prior to me get data if I'd avoid bad data if you
mean about a data center maybe change the tracks
I'm tuning to do thank you thank you so while Darren is over there practicing
for his next competition we want to give again a better sense of how the models
are developing this is Matt here you're also part of the Liberty Mutual team
you're Darren's teammate essentially a lot of you guys out here and it sounds
like you guys have a lot of support back at home to work on your models and and
train them mm-hmm absolutely if we started off this league with the
expectation that maybe 10 or 15 teams might be interested by the time we'd
finish signups we had 64 Liberty teams competing which was just phenomenal so
we had a similar bracket system to what's here and then our our team from
Li T Liberty international they just
absolutely blew everyone out of the water on the final competition we had a
good time that we thought was eight seconds and then they set almost nearly
a world-record pace so it's all about how your strategy in the deep racer can
affect how quickly you can go around the turns how smooth it looks and what
happens when it gets itself into a corner so tell me about the different
strategies what are the variables that you and the other developers are tuning
not only in the reward function but also in how capable how many actions you can
actually program and allow deep brace or move absolutely if you go into the AWS
console it gives you a lot of things you can change
first is the action space so what can the car do you can either turn left or
right it can go fast or slow and you can set as many variations in between that
as you want the problem is is if you have the wider variations the larger
numbers take a much longer time the Train so our strategy was to have a
relatively low number of actions we just had five one that's very fast rate and
then the side angles got slower and slower so the car wouldn't start to
drift too much and that worked out well for us because we were able to train a
lot of models very quickly right but the other IT team strategy which wound up
being obviously a world-record pace was to have one model that they could train
on multiple different tracks so it was an agnostic race model you
could take it to a track it's never seen before and it's probably gonna be able
to get along just fine and if
any issues you can go in and make minor corrections in the model so you could
you know maybe get a time like what we saw today
it's really constraints you have are not only the intuition you put into the the
model but how much time you have to iterate and if you're working on four
different models something riskier something more precise and you're tuning
that and you're trading that for hours at a time that's gonna be a lot more
time overall than someone who's just doing generalize efficient model or
something that maybe had like a smaller action space less granularity in terms
of the movements but again you can iterate faster and that's the key to it
is iterating faster the MIT team tried so many different things I know our team
tried like five or six different strategies until we finally settled on
what we have today and we're getting to the point where it's so close between
the models that everything you do every minor tweak makes a huge difference so
training it at those faster action space speeds can really make the defeats
between a competitive model and one that just doesn't make the cut and a lot of
these top competitors are now that sub ten-second time which is a huge
improvement for when you guys all started it was in much longer than that
yeah I mean last year if you look at what we had for deep racer we have teams
and and you know our Liberty teams included who were just really excited to
get times of 15 seconds and now 10 seconds isn't acceptable right you have
to get that eight or nine seconds to just stay in at this point
are you looking to next season they announced the deepest or EVO it's gonna
be head-to-head direct competition what are your immediate thoughts on that I'm
super excited for the Evo I can't wait to get my hands I want to try it out
very excited to see the object avoidance I don't know what their plans are there
may be throw up in an appeal on the track but between that and the
head-to-head racing where you'd be able to like get in front of someone to
prevent them from passing or try to overtake someone that's a huge machine
learning problem I have no idea how to solve yet but very look much looking
forward to figuring out what we can do to try out and iterate again all these
system models kind of go out the window you have so many more things to contend
with it'll be a whole new world I think I'll start by listening whatever there
it's great to meet you Matt great to meet you
thanks so much of course