This Mario Kart here is not driven by a human: It’s driven by a machine learning model.
And that machine learning model is not driving it an emulator, but on a real, physical Nintendo 64!
And it’s actually quite good at driving! in this video, I’ll show you how it works,
how I trained it, and also what you need to reproduce it!.
As many others, I’ve got a casual interest for machine learning:
I’ve dabbled around with tensorflow, and created a couple of simple classifiers.
However recently I wondered how difficult it is to use something like TensorFlow to
control something in real-time - and what better target to use
than one of the first console racing games I ever played: Mario Kart 64.
My plan was simple: Get the video-output from the N64 to my computer,
do the calculations there, and then send back the required controller inputs to the N64.
Now getting the video output was pretty simple: I connected the N64 to an AV to
HDMI adapter I had laying around and hooked that HDMI output up to an Elgato HDMI capture card.
For getting the controller signal from the computer to the N64 I had to get a
bit more creative: I couldn’t find anything ready to use, so I built my own adapter,
using - you guessed it - a raspberry pi pico and the cable of an old broken third
party controller I had laying around. The Programmable I/O features of the
Pico came in really handy here, you can find the source linked in the description!
With everything hooked up I had this setup: My N64, the AV Adapter, the capture card,
the laptop and the pico. I also hooked up a small monitor to the capture-card
passthrough to be able to see what’s going on without having to render it on the computer.
Time to look at the software side of things! When I started this project
I started looking for others who tried the same - and pretty much immediately
I found an awesome project by Kevin Hughes called TensorKart:
TensorKart is a machine learning model and toolset for controlling
Mario Kart 64 on an emulator - it uses a specially modified version of mupen64plus,
so that it can basically single-step frame for frame, and send joystick inputs for each frame.
While I did not want to use an emulator - mainly because I was interested in the
challenges that come with doing all with real hardware - I decided to use
TensorKart - without the emulator part - as a base, The code of TensorKart beautifully
simple and comes with a training pipeline that is easily adjustable for what I wanted to do.
What’s also really nice is that tensorkart is based on a
network from an nvidia paper that is actually able to control real cars,
and so I was super excited to see whether I could get it working on a physical Nintendo 64!
With all this in place, lets talk about training data:
My goal was to create a supervised model:
This means that to train the model I want to give the model what’s called labelled training data:
In this case, the input to the model is a frame of the game,
and the expected output of the model is a steering input aka the label for the training frame.
Now to successfully train a good model,
I need thousands of frames of the game AND the correlated steering input!
To generate this type of data I first wrote a small Python script that takes
the input of a connected game pad - in this case a
ps4 controller - and sends it via my Pico-based adapter to the N64.
Combined with the HDMI capture card and a modified script from TensorKart, I am able
to capture frames from the N64 on my computer, and capture my steering inputs for that frame.
Now as you can see here, the whole signal chain has a ton of lag:
From my controller, through my computer, through the USB adapter, to the N64,
and then back through the AV converter, into the capture card and to the screen I’ve got like a
second or so of lag. I optimized the Pico code a bit more to make it a bit better, but in the
end I just hoped that the lag is not too bad and that it’s good enough to train my model.
With this setup, I started recording training data. Now a big issue is that
when you play Mario Kart 64, your inputs normally are not particularly smooth:
You tend to have very harsh steering inputs: This does not train well on the machine learning model,
and will create quite a bad model, especially without reinforcement learning.
To make it easier to have smooth steering inputs I reduced the maximum steering angle
in my controller script, and I decided to also start training with basically cruise control:
I set the gas button to be pressed for 500ms and then released for 500ms automatically,
so that the kart always drives at more or less the same speed.
Using this, I drove 7 full time-trial runs of Luigi’s Raceway,
trying to steer as smoothly as possible, and ended up with over 10000 frames of data.
And with that training set, it was time to train the model! Well.. Almost. In reality I
spent a ton of time trying different training approaches, models, fixing bugs and so on,
and so what looks easy in this video actually took a lot of time and experimentation!
I also modified the output shape of the model: Instead of outputting gas,
break, and joystick x and y I wanted the model to only output a steering signal.
And with all those things setup, lets train the network!
I started out with 200 epochs, and as mentioned we are training on a very
slow driven dataset. The training is really fast on a decent GPU,
taking less than a second per epoch. On a CPU, this will take much much longer.
With the training done we can look at our loss, which describes The difference between
the output of the network, and the labeled training data. As you can see, it looks like
our training performed really well: Both loss - and validiation loss - which is the loss of
the network tested on data that it wasn’t trained with - is going down a lot, and more importantly,
the validation los sstays low: this indicates that we are not overfitting: Overfitting is when
the network learns to recognize the training data, instead of actually learning what we want it to.
So lets give this model a try on the actual hardware!
I modified the script that normally controls the emulator to capture frames from the capture card,
run them through the neural network, and then send the joystick input via USB to the N64.
For the first tries, I used the cruise-control again,
making the testing annoyingly slow - but you have to walk before you can run.
As you can see it already tries to do something,
but it’s not really driving well yet - it’s steering in the right direction, but
the steering inputs are too harsh and it always lands in the sand. But definitely a great base!
After a lot more debugging and changing different model parameters, I eventually
got it to a point where it drove - in slow mode 3 full rounds of Lugis Raceway - without bonking
against any wall and without driving into sand! One of the most interesting things I noticed was
that the network seemed to learn from the HUD - if I had the map enabled,
it seemed like it would primarily learn from the map! Disabling the map, and cropping
out the HUD parts such as the timing display drastically improved the models performance!
With that working, it was time to create some additional training data:
This time driven at full-speed! I also added two rounds of Kalimari Desert to the training data,
in the hope that it would help the network to learn to generalize better.
And with that, it was time to train again. And what was really
cool was seeing how with each increase in epochs the model got better and better.
After 10 epochs, the model would start steering in the right direction,
but always bonk against the sidewall. At 20 epochs, the entry to the curve
was quite good, but then it would steer outwards, bonk against the out wall, and land in the sand!
At 50 epochs, the first curve was no problem,
but it wasn’t yet able to keep things in control in the tunnel!
At 100 epochs, it still bonked in the tunnel,
but not too badly: It was able to keep driving and finish the round!
At 150 epochs hower, it was pretty perfect: It drives the corners well,
doesn’t hit any wall, and finishes three rounds without any problems.
In fact it’s so consistent that rounds two and three were only one hundreths of a second apart!
It was finally working! A machine learning model driving Mario Kart on a physical N64! Awesome!
It even - with just two rounds of training data - manages to drive Kalimari Desert!
It leaves the road here and there, but is able to finish a full round!
On other tracks, it doesn’t perform so well yet, I guess more training
on a wider variety of tracks is needed - but for luigis raceway it’s pretty good!
As always you can find all the source-code and so on linked in the description,
and if you liked this video please subscribe.
Also, if you have ideas for some other fun machine learning things let me know in the comments!
Thank you for watching and see you soon!