- I'm Carolina.
- I'm Jesper.
- I'm Kaspar Meijjer.
- Hi guys. Norbert. My code is only 30 lines big.
(upbeat music)
- Some pretty big stakes here, no doubt about it
and a chance to compete for the title
of DeepRacer League 2019 Champion.
- Maybe we should tune the hyperparameters.
Try only the speed.
(car motors running)
(laughing)
- It seems amazing to you if machine sub-intelligence.
- We will fight and see what algorithm is better.
(buzzer)
- We are down to just the last two and a half hours.
- Time. 42.50.
- Ben to six.
- 20.91. Oh.
- That's still good, that's still good.
- Two laps pretty close.
- There it is basically.
- Let's see what happen at the end of the day.
- [Blaine Sundrud] Welcome, to the world's first
autonomous racing league.
Developers of every skill level, from all corners
of the globe, compete for the ultimate prize
in machine learning.
Welcome to DeepRacer TV.
(steady bass music)
- The Dutch city of Amsterdam is known
for its canals, electronic music, and rusty bicycles.
But it was also recently named the best
tech city in Europe.
And there's tons of excitement here, as developers
are lining up outside the RAI to get ready
for the first global autonomous racing league.
To register an official time on the leader board,
the clock starts now.
- Our first competitor will be taking
to the track here very soon.
- [Blaine Sundrud] You can get two wheels in the green.
That's still good, that's still good!
It's only if all four wheels are in that
disqualifies you.
I'm Blaine Sundrud with AWS Training and Certification,
and I'm a subject matter expert
for AWS DeepRacer.
- Get your opportunity on the track with our
fantastic DeepRacer cars.
These are really cool tech.
- AWS DeepRacer is a 1/18th scale
fully autonomous racing vehicle.
It's controlled entirely by reinforcement learning.
(dinging bells)
(chatter)
(buzzer)
- (male) The car knows how to drive itself?
- This is the point of the reinforcement learning, right?
- [Blaine Sundrud] Each developer will have four minutes
on the track to place their best time.
They need to complete one full circuit around the track
with less than three faults in order for their
time to qualify.
- The top ten will get to bring home one
of these DeepRacer cars.
The top three will have some special gear
to bring home as well.
- The best time today
will win the AWS Amsterdam Cup
and an expenses-paid trip to compete in AWS re:Invent
against all the other Summit winners.
(groovy music)
- [Ryan Myrehn] Carolina is on the track right now.
- I'm Carolina. I'm from Argentina.
I have a daughter, Abril.
She was born last year.
And I have a dog.
I don't know how to drive.
I don't have license.
I'm working at Booking.com.
In my company, we use a lot of machine learning.
I know the theory behind it, but I never
worked close to this.
I arrive early because I want to be the winner (laughing).
- Oh, yeah, yeah. (beep) Oh! 27.53.
- [Ryan Myrehn] That would be good enough
to move to second.
Carolina is using Model #1
- [Blaine Sundrud] For competitors who
are just getting started with DeepRacer,
they have the option to pick three pre-trained models.
- Ready to go?
- Yeah. Can I have a Model #1?
- [Blaine Sundrud] Model #1 is designed to keep the car
along the center line.
It values being right in the middle of the track
over any other parameter.
- I have no experience at all with machine learning,
so I just
- [Racer] I went with #2.
- Control over speed.
- [Blaine Sundrud] Model #2 is about reducing
the amount of steering the car does.
It penalizes over-steering, instead asking the car
to try to drive as straight as possible,
even if that means veering off the center line.
- [Racer 2] I want to try #3 today.
- I have the Model #3 and it's blazing fast.
- [Blaine Sundrud] Model #3: speed.
This one is built to prioritize the ability
to stay on the track fast over any other parameter.
- Go, go, go go.
- Oh, 23 eight to five.
- That's a new record.
- [Ryan Myrehn] Emo just jumped to
the top of the time charts
with the fastest lap we've seen so far.
- Yeah, my own solutions, our points easy.
You just click and get them running.
- [Ryan Myrehn] You might just end up at the top
of the charts at the end of the day.
- I'm confident that someone else will break my record.
Hopefully the human race.
- (upbeat music)
- [Ryan Myrehn] Competition starting to heat up,
it's getting tighter at the top of the leader board.
If you think you can go faster, show us.
- [Blaine Sundrud] As the day wore on, people's models
became more sophisticated, and we started seeing
some experimentation.
Norbert, tell me who you work for.
- I work for Bright Cubes.
It's a very small company in the Netherlands
and I am their AWS consultant.
Hi guys. Norbert.
There's my hobby room and this is my 3D printer
that I built by myself.
I'm building my own car.
Here's where I live, train, run, walk, enjoy my life.
Before the Summit, I saw some YouTube movies
about standard code that AWS already provided
for the DeepRacer.
And I thought that can be a little bit smarter.
- Okay.
So I tuned a little bit of the code.
- Wonderful.
Norbert thought, "I can build better code."
- My code is only 30 lines big.
- I didn't actually see the code.
He's keeping that to himself, and I respect that.
But he did show me the parameters.
He built a rewards system to say, "I'm going to
start penalizing you."
He's taking ten percent off the total
if the two wheels gets off the track.
- [Norbert] If all wheels on track, he gets a reward
multiplied by one.
And if he's almost off track, he get a discount.
- It's just absolutely some carrots in the middle
of the track, but you get too far off and yeah,
the stick comes out and in his case,
it's a pretty big stick.
- Maybe we should tune the hyperparameters.
- Yeah. Learning rates...
- [Kasper Meijjer] Try only the speed.
- Jesper and Kasper.
- Yes!
- So twins?
- No, no. Unrelated.
- I'm Jesper.
I'm from Amsterdam. I really like speed and trying
to push it.
From a mountain biking experience, I'm quite competitive.
I was the Dutch national champion under 23 years old.
My partner is Kaspar.
We work closely together on developing the model.
- I'm Kaspar Meijjer, and I'm a data scientist
in the Netherlands.
I'm not an aggressive driver, but yeah,
I sometimes exceed the speed limit.
(bell)
I'm pretty competitive.
I have a sail regatta, sail with some colleagues,
and we won the whole competition.
I do a lot with machine learning.
I never did something with the reinforcement learning.
- At the moment we're in the simulation.
We just saw the track actually and just shown
the simulation is actually applicable to real life.
- Some cars were very slow but stayed very precise
in the middle of the track.
- So we decided to discard the position of
the car on the track, and just focus on speed.
- If the car is not aware of its position on the track
relative to the center, it might be zigzagging
back and forth, and that's going to bleed speed
all night long.
How's the testing?
- This version is not so good.
- Okay, all right.
- Yeah.
- But we have multiple versions, because
it's quite risky to just look at speed.
It might happen that we accidentally just
destroy the car by driving too fast into walls.
- Optimize for speed, see what happens.
- (marimba music)
- [Blaine Sundrud] As you talk about the car,
it's important to separate
the physical car from the virtual world.
In the virtual world, there's all sorts of things
the car is aware of.
It's aware of its compass position
and where it is in relation to the track.
In the physical world, there is one input
and that is the camera.
It has to be able to find the geometric pieces
in that image, and pass that into the model.
Now the model is being processed by
an Intel Atom Processor, which was chosen
by the DeepRacer engineers as the
perfect balance between processor speed
and battery consumption.
- (heavy bass music)
- [Announcer] Just over two and a half hours
of racing to go here today.
- If you think about a race car, it doesn't always
go in the center.
You take the corners.
So I'll try to replicate some of the logic
that a drone uses to make its way to waypoints.
- Bart's been focusing on waypoints.
That's how you're actually getting the car
to not be on the center line, but start
to curve inside the track as you move through.
This is the next level we're looking for.
How is your test times going on the virtual track?
- Latest I ran was 16.8.
- That's close!
I believe Bart has a shot if the models had
enough time to train.
We really won't know until he gets on the track.
(applause)
(coin machine clanging)
- Start. You can increase its in a bit.
Oh, it drained out, sorry. Oops. Nope, nope. (laughing)
It likes left.
It likes going to the left.
No, whoa.
(car motor)
Really?
Come on.
- [Female Referee] Time is up.
- Okay. Ah, I'm sorry, the time is up. Yeah, yeah.
- This really was a case where the model
did not have enough time to train.
- I wanted more time to train it and it comes off...
- It takes time to reach convergence.
There's a certain element in machine learning
that's a combination of exploration and exploitation.
Think about training a dog.
A dog is going to, for the first while,
not really know what you're asking the dog to do,
so she'll try jumping up and laying down and doing things
to get the treat.
That's an exploration phase.
In DeepRacer, that's the car driving into the grass,
or going really fast in a straight line.
It's trying to experiment to see what it is,
and then see, "Did I get rewards from it?"
Then it moves into an exploitation phase
where it knows it's got a good path.
That convergence from "What happens if I do something?"
To "I really like doing this thing," that's convergence.
But that takes time.
- Can't wait for the Virtual League
to start as well, to compete there.
- The Virtual League allows anybody from anywhere
in the world to create, train, and race
their own models virtually and submit their times
to a Global Leader Board.
The top times there also win a trip to Las Vegas.
- Our next developer on the track has developed
a model of his own creation.
- Go.
(car engine)
- Ahh! Ooh. No, no, no, no, no, no.
- [Ryan Myrehn] This car's taken a bit of a beating here.
- In DeepRacer, we provide a basic model that'll
work good, but if you really want to get
the Championship Cup, you got to not be afraid
to try to break some things.
- Watching it crashing is actually more fun
than seeing it going straight.
(laughing)
- I think we're trying it again.
- (laughing)
- [Blaine Sundrud] It's getting tighter at the top
of the leaderboard. Still some time on the clock.
- I was watching the podium, going down,
going down, going down.
Then, when I reached the 23rd position, I said,
"Okay, I need to play again!" (laughing)
- [Ryan Myrehn] Carolina is back.
This is her second time.
- I will choose the third model, velocity,
because we need to speed up.
I don't know how, but we need to speed up. (laughing)
- (crashing machinery)
(cheering)
- It's off to a really good start.
Carolina's going with a slightly more aggressive model.
Still on track.
One that values speed and cutting the corners
more than just staying on the track.
One more corner to go.
This could be a good time.
- I ended the race without it going outside.
- 15.09
- [Announcer] 15.09!
- It was a really good time.
- [Ryan Myrehn] Carolina goes to the top of the charts.
Big smiles all around, well done.
- Carolina right now is our leader.
She's a full second faster than the second place time.
I've seen people with pre-train models get below
15 seconds, but it takes skill.
- I'm on the podium again, in the first position.
I want this trophy on my desk! (laughing)
- [Announcer] Under an hour to go on Amsterdam's speedway.
- Norbert, how's the testing?
- Of course, limited in time.
The model took 60 minutes for it.
- All right. That is staying right on the track.
And that's a competition run right there and it's still
trying to explore.
If it plays as well on the track as it does
in the virtual simulator, we could be looking
at today's winner.
- Let's go. (laughing)
- Good luck.
(car motors)
- I think you should just increase the traction.
And let it out on freedom.
Wow, your motor is really good.
- [Ryan Myrehn] 13.5!
(cheering)
Norbert, with our fastest time so far.
Round of applause for him. Yes.
- Bringing high-end technology to a play toy inspires me.
You can have a lot of fun with it.
And you can get immediate results.
And it was great to see your own codes working
in that model.
And it's great of Amazon that they're bringing
complex things on an easy way.
And I had a lot of fun today. (laughing)
(quicker, upbeat music)
- Typically, on race day, the last 60 minutes
are where we see the best times,
because our competitors will have been training
their models throughout the day,
and this is when they break them out,
and we get to see who's really ready to take
the Cup home.
- [Announcer] Still 40 minutes to go in competition here.
- Can't wait to see how this plays.
I like the aggressive stance.
- Yeah, yeah.
- I haven't seen anyone else be that aggressive.
- It just rewards when you go forward.
- As long as I'm on the track and I'm moving forward,
give me treats.
- Yes. Yes.
- And hopefully you'll stay on the track.
- Yes. We'll see how it plays out.
- Good luck.
- Thank you.
- [Announcer] Our final contestant on the track now.
Time running down.
- I do have some experience with racing under high pressure
when the stakes are high, so I'm just going to put
it full throttle and see what happens.
- Hopefully it will stay on track.
- (upbeat music)
- [Ryan Myrehn] It's off to a really good start.
(car motor running)
(cheering)
- That looked like a fast lap.
(applause)
- What was the time?
- We were really nervous, and we were standing next
to the track, and keep an eye on the scoring board.
(cheering)
- [Ryan Myrehn] 13.03!
Right on cue, a new fastest lap!
How about that?
13.03 seconds.
The fastest time we've seen.
Under the time set by Norbert a few moments ago.
That bumps Carolina outside of the podium.
- Then I was not able to shine, to play again.
- [Ryan Myrehn] That concludes the final run here
at Amsterdam speedway.
Finishing in third, Robert Vandenburger, a 14.15 second lap.
Robert, congratulations.
Why don't you take your spot there in the third place stand.
Well done.
Our second place finisher here today with a lap time
of 13.50 seconds, please welcome Norbert Overbeak.
Congratulations, Norbert, well done.
Now you developed your own model as well.
What did you know about machine learning
before you showed up here and tried your hand
at this DeepRacer thing?
- Now to be honest, nothing.
- Nothing?
- I just learned it from YouTube movies
and the AWS site.
I'm happy about this.
And I was happy that he finished first. (laughing)
- To finish first, first he must finish,
the old racing adage.
Well, congratulations, why don't you take your
spot up there on the podium, well done.
(applause)
- But there can only be one champion,
our champion, in addition to the excellent jacket,
also the trophy, and the DeepRacer car will
also receive an expenses paid trip to re:Invent 2019
in Las Vegas with a lap time of a blistering 13.03 seconds
from PAN DataLab, please welcome Jesper Slick!
(applause)
- Well done, Jesper.
You turned in your time relatively late in the day.
Were you in the workshop?
Were you working on this before you showed up
and took to the track?
- From the workshop we learned how to train
and then we just tried to train for as long as possible
before we could set a time.
- Well, well done today.
Congratulations, we hope to see you
in Las Vegas in December.
- Definitely, thank you.
- With this experience, I want to try to
use reinforcement learning inside my daily job.
- Automated marketing for example is lots and lots
of decisions that you can improve in businesses
with algorithms such as these.
- [Ryan Myrehn] There they are, your podium finishers
here today in Amsterdam!
(cheering)
- Second place.
I'm happy about it.
It's going to be on my desk, I think, at the office
for a week, and then it's going to my home.
- First position, Amsterdam.
AWS Summit in Amsterdam was perfect
(in foreign language)
(upbeat music)
- Jesper and Kaspar, today's winners, will move on
to compete at the AWS re:Invent Champions Cup
in Las Vegas.
Our next stop on DeepRacer TV: Sydney, Australia.
We invite you to participate by coming
to any of our upcoming Summits or
participate in the Virtual League.
Go to DeepRacerLeague.com.