So, today’s video is going to be something a bit different.
Instead of talking about the engineering behind an existing theme park attraction, I thought
it might be interesting to do a deep dive on a personal project that I’ve been working
on which combines roller coaster design with artificial intelligence.
AI is an exciting branch of computer science that I have been really interested in lately,
and I wanted to try an experiment to see if I could teach an AI program how to generate
track layouts. Designing a fun and thrilling layout is something
that is well-suited for the human brain because it involves a high level of creativity and
reasoning, and these are both cognitive abilities that cannot be replicated by a computer.
In contrast to the human brain, artificial intelligence tends to work in a more structured
and predictable manner using algorithms and programming, however it can be trained to
complete a given task in an optimized way through the process of machine learning.
My initial thinking was that if it is possible to teach an AI how to design a roller coaster,
then in theory, it should be possible for it to come up with the most optimal layout
within a specified area. It turns out that this task is much more difficult
in practice, though, since designing a track layout is a multi-objective optimization problem
with several conflicting objectives, and it’s nearly impossible to define what an optimal
roller coaster looks like compared a bad one. Unfortunately, there’s no mathematical equation
that can calculate how fun a roller coaster is based on its physical attributes like speed
and g-force, and such an equation would be useless anyway because fun is something that
is entirely subjective. Despite this, I was able to work around the
problem to build an artificial intelligence program that I’ve called CoasterAI, which
can design simple 2D roller coasters using a genetic machine learning algorithm.
The designs that it comes with are not necessarily the most optimal by definition, but they are
completely generated by AI, and it has even invented some brand-new roller coaster elements
that have never been built before. Today I would like to show you exactly how
this program works, and I’m going to explain how you can use machine learning to teach
a computer how to design a roller coaster. To start, I think it’s helpful to differentiate
between what is artificial intelligence and what is machine learning, since these terms
are similar but not exactly the same. Basically, machine learning is a subset of
artificial intelligence that is responsible for actually achieving a state of AI, whereas
AI itself describes the end product of machine learning where a computer or machine exhibits
intelligent behavior. To create artificial intelligence, you first
need some type of machine, program, or bot that is capable of performing a specific task,
albeit quite randomly in the beginning, and a machine learning algorithm is then used
to teach the bot how to perform the task in an optimized way.
So, to make an AI program that can design roller coasters, you need to start by creating
a bot that is able to generate random track splines, and you need to train it so that
it understands which spline shapes make up a good roller coaster design, and which spline
shapes make up a bad one. To accomplish this, the program needs to consist
of four key components which I’ve broken down as follows:
A method for generating splines using a bot that is comprised of a neural network;
A basic physics engine for computing things like velocity, acceleration, and g-force;
A rating system that provides a quantitative score for each roller coaster design;
And a machine learning algorithm that trains the program and optimizes its behaviour.
I’m going to run through each of these, focusing primarily on the AI and machine learning
components, starting with spline generation using a neural network.
A neural network is essentially the brain of the AI, and it is inspired by the arrangement
of neurons in a human brain with several layers of interconnected nodes.
Every neural network has one input later and one output layer, and there can be any number
of intermediate layers in the middle called hidden layers; In this case, two.
The number of nodes in the input and output layers depends on the given problem being
solved, while the number of nodes in each hidden layer can be adjusted to alter the
behavior of the network. Every node can hold a single numerical value,
and each line connecting two nodes represents a mathematical computation to get from one
to the next. To see how this works, let’s consider the
first node of hidden layer 1, which is connected to all four nodes in the previous input layer.
In order to compute the value of this node, we start by multiplying each input by a numerical
weight, and then we sum all of these together and add a bias value.
This is basically just a sum of linear equations in the form y = m*x + b, where each weight
represents the slope of a line, and the bias represents the total translation.
We can repeat this for the second node of hidden layer 1 using a different set of weights
and biases, and then again for every other node until we have completed the entire layer.
The process is exactly the same to get from hidden layer 1 to hidden layer 2, and then
from hidden layer 2 to the output layer, with the only difference being the number of nodes
and computations. Now, this is obviously quite a bit of algebra
just to get from 4 input values to 1 output value, but fortunately we can simplify it
down to just a few lines using vectors and matrices.
If we represent each layer of the network as a vector, then we can get from one layer
to the next simply by multiplying the previous vector by a matrix of weights and adding a
vector of biases. This makes the computations a lot easier to
handle, especially when coding a neural network in a computer program.
It is also desirable to normalize all the values in order to help with convergence when
running a machine learning algorithm, and so we can apply the hyperbolic tangent function
here to compress the value of each node between -1 and +1.
This will allow us to optimize the network faster later on because we only have to deal
with values that are contained within a small, fixed range as opposed any possible number.
At this point, you may be wondering what any of this has to do with designing a roller
coaster, so now let’s see how we can use this neural network to generate a track spline.
To begin, we need to pick a starting point where the coaster has a known velocity, height,
energy, and track angle, and we need to choose a node spacing that will discretize the track
into a finite number of segments. We then take the initial parameters and normalize
them between -1 and +1, and we feed them into the neural network as the input layer.
The number that comes out the other end as an output is converted to an angle in radians,
and we use this as the track angle for the next segment.
The angle is measured relative to the previous segment rather than from a fixed reference
frame, and I’ve found that it’s helpful to limit it to a specified range such as +/-
30 degrees in order to avoid sudden changes in direction.
Using the angle and segment length, we can now use a bit of geometry and physics to find
the velocity, height, and energy at the subsequent node, which we can then normalize again and
send them back through the same neural network to get a new angle for the next segment.
The same process is repeated over and over until the track spline reaches a certain length,
or until the roller coaster runs out of energy. In order to automate this, the spline generation
sequence is coded into an algorithm that is executed by a bot, where the neural network
essentially acts as the bot’s brain. Its behavior is determined by the weight and
bias values contained within the neural network, and it will make different decisions that
result in different spline shapes depending on how these values are changed.
From here, you might be able to see that it is possible to train the bot by attempting
to optimize the weights and biases, but first we need a physics engine and a rating system
to evaluate the quality of the track splines. I’m not going to spend too much time talking
about roller coaster physics here since I have already covered this in other videos,
but I do want to give a brief overview of how the calculations work.
The physics engine itself is basically just algorithm that traverses along a given spline,
and it uses the law of conservation of energy to compute the roller coaster’s velocity
at every node. In order to account for losses, it calculates
the amount work done by rolling friction and aerodynamic drag along each track segment,
and it subtracts these values from the total energy.
In addition, the algorithm also needs to find the radius of curvature, which is required
to compute centripetal acceleration and g-force. There are a few different methods that can
be used to do this, but I found that the most reliable way is to use Menger curvature, which
works by fitting a perfect circle to every set of three points.
This provides an approximate solution, as opposed to an exact solution that could be
found using derivatives, however it is still quite accurate as long as the node spacing
is relatively small. Once the radius of curvature and velocity
are known, the centripetal acceleration can then be calculated at every node, along with
the total g-force that the passengers would experience on the ride.
All of this information gets stored in a series of vectors as the algorithm traverses along
the track, and after it reaches the end, the compiled physics data is then passed on to
the rating system. This part of the program is responsible for
giving each track design a numerical score, similar to the rating system in the roller
coaster tycoon computer game, and it’s one of the most challenging components to get
right in order for the machine learning process to work properly.
Like I mentioned at the beginning of the video, there is no single equation that can define
how fun a roller coaster is, and there are countless ways that you could go about evaluating
a roller coaster since it tends to be very subjective.
For CoasterAI, I wanted the scoring system to be as objective as possible while still
providing a level of customization, and although the one that I came up with is far from perfect,
it does give satisfactory results in most cases with only 5 basic criteria.
These are: Speed;
G-force; Steepness, or slope;
Boundary constraints, such as maximum or minimum height;
And inversions. To generate a score for a given track spline,
the speed, g-force, and slope values are first normalized at every node, and each one is
multiplied by a weight that is assigned to the corresponding parameter.
The scoring algorithm then traverses along the spline and it adds each weighted value
to the total score, but only if that value does not violate a constraint.
If the speed or g-force falls outside of an allowable range, then the non-compliant value
will not contribute to the score, and instead, the algorithm will apply a deduction.
A deduction will also be applied for any node that violates a boundary constraint, as well
as for any node that violates the law of conservation of energy.
Once all of the numbers are added together, the final result is a single score that represents
the overall quality of the design. The process itself is fairly straight forward
here, but the tricky part is fine-tuning the weights and deductions so that the rating
system is well balanced. If one of the parameters is weighted too high,
then the AI will learn to maximize that parameter only while ignoring all the other constraints,
and if a parameter is weighted to low, then the AI will simply ignore it.
In order to get the best results, the initial weights need to be set up so that each parameter
has a nearly equal contribution to the total score, and small adjustments can then be made
to control how the AI learns. For example, if we want it to design a roller
coaster with more g-forces, then we can assign a slightly higher weight to the g-force parameter,
or if we want it to design steeper hills, then we can assign a slightly higher weight
to the steepness parameter. Now, teaching the AI to make inversions is
a little more difficult, but I’ve found that it is doable by switching between two
different rating systems where one has an added incentive for creating a closed loop.
Unfortunately, this isn’t an ideal solution because the incentive needs to be so high
that it throws off the balance of the other parameters, however it does work well enough
most of the time until I can come up with a better method.
So, to briefly recap up to this point, we currently have a bot than can generate a random
track spline using a neural network, and we also have a physics engine to analyze that
spline, as well as a rating system to quantify how well it meets our criteria for a good
roller coaster. The last thing that we need to complete the
program is now a machine learning algorithm, which we will use to train the bot so it can
learn how to make the best possible design. This is basically just an optimization problem,
and our ultimate goal here is to find some combination of weights and biases within the
neural network that will allow it to produce a track spline with the highest possible score.
One way to train the network is to simply change the weights and biases at random until
it eventually gives a good result, however this is extremely inefficient, and it would
take far too long for the values to converge at an optimal solution.
Instead, it is much more effective to use a genetic machine learning algorithm, which
is based on the biological theory of evolution. With a genetic algorithm, we start by creating
an entire population of bots, and we then exploit the process of natural selection by
passing down the best traits from one generation to the next.
In other words, we are not really teaching a bot how to design a roller coaster, but
rather we are incentivizing a group of bots to evolve so that they become better at designing
roller coasters naturally. If everything works correctly, then after
a certain number of generations, we should end up with at least one top-performing bot
that we can say is artificially intelligent. To start the machine learning process, we
first select the number of bots that we want to have in our population, and for each one,
we generate a neural network with completely random weights and biases.
We then allow each bot to create its own track spline using some given input parameters and
constraints, and these are passed through the physics engine and rating system so that
each design receives a score. This part can be quite computationally intensive
depending on how many bots we have, but the greater the population size, the less generations
we will need to find an optimal solution. From here, the top-performing bots that rank
in the top 10% are carried forward to the next generation, while all the others are
left behind and discarded. The top-performers are then replicated multiple
times in order to fill the top half of generation 2, however each copy undergoes a series of
mutations so that it becomes a slight variation of the original.
This is done by taking a probabilistic approach, where we go through the neural network of
each bot and make small random changes to the weights and biases.
Each one is assigned a given mutation probability, and the magnitude of each mutation is limited
to some percentage of the initial value. Two random variables are then generated for
every weight and bias within the network, where the first one determines whether a mutation
will occur, and the second one determines the magnitude.
By adjusting the mutation probability and limit, we can control how fast or slow we
want the bots to evolve between generations, and to what extent.
Now, moving to the bottom half of generation 2, we populate this section by generating
a new set of bots with brand new neural networks, which introduces a bit of extra randomness.
This basically just helps to prevent us from getting stuck in local maxima of our optimization
problem, so we don’t end up trying to optimize a single spline shape that is actually a sub-par
solution. With our second generation complete, we can
use the new population to generate a new set of track splines, and from here, the whole
cycle can repeat over and over until we reach a specified number of generations, or until
we find a roller coaster design that we are satisfied with.
Even though the ultimate goal of a genetic algorithm is optimization, in this case, it
can also be a useful tool for finding new types of elements that are simply unique or
interesting. To see the machine learning process in action,
we can take a look at the CoasterAI program that I’ve built in MATLAB which combines
everything from this video together. Down the left are the input parameters and
constraints for generating splines; At the top right are the genetic algorithm
and mutation parameters for machine learning; And at the bottom right are the weight modifiers
for the scoring criteria. Once these are set, we can go ahead and hit
run to start the program, and the top figure will begin plotting the best results from
each generation. Here, the blue line represents the spline
that currently has the highest score, while the orange lines represent all the splines
in the top 10% along with their respective mutations.
This allows us to visually see how the AI is learning as it tries to find the most optimal
spline shape with the given parameters. At the same time, the bottom window displays
the top 5 corresponding scores from each generation, and when the program is terminated, it also
gives us some stats for the final roller coaster design and lets us know if it violates any
constraints. If we are satisfied with the design, then
the spline can be exported to other roller coaster modelling software such as NoLimits,
or we can discard it and try running the program again.
I’m going to put a download link for CoasterAI in the description for anyone that wants to
try it out, and I’m also going to include a simplified version that can run directly
in MATLAB, as well as a printout of the script for anyone that just wants to see what the
code looks like. It can be a little tedious to get the settings
dialed in, and there’s a couple of bugs that I’m trying to work out with the rating
system, but it’s still possible to get some pretty cool results with a bit of patience
and fine-tuning. I think one of the most important accomplishments
so far is actually getting the program to design vertical loops, since this is something
that typically requires complex physics and math formulas.
By using a genetic algorithm, CoasterAI can learn how to make optimal loops that are almost
identical to the modern teardrop shapes that are commonly used today, which just goes to
show that it is possible to design roller coasters with artificial intelligence.
From here, the next step is to expand the program so that it can string multiple elements
together, and eventually I would like to try building a more advanced AI that can create
a full track layout in 3D. Designing roller coasters has turned out to
be a very interesting application of AI with some unique challenges, and I really have
to thank my Patrons for allowing me to put so much time into it so far, as well as today’s
sponsor, NordVPN. Using a VPN is one of the best ways to protect
your personal information online, because it allows you to browse the internet anonymously
by routing your connection through an encrypted tunnel.
This helps to prevent your ISP and third parties from tracking the websites you visit, and
NordVPN doesn’t keep any data logs. In addition to security, a VPN can also be
useful for accessing geo-restricted content that may not be available in your region,
such as certain videos on YouTube and online streaming services like Netflix and Disney
Plus. All you have to do is click on a country where
the content is available, and in seconds, you can browse the internet and watch movies
as if your device was actually located there. NordVPN has servers all over the world with
no data caps, so you’re free to enjoy unlimited content from anywhere.
I highly recommend that you try them out by going to NordVPN.com/ArtofEngineering, where
they are currently offering a huge discount on a 2-year plan.
It’s completely risk-free with their 30-day money-back guarantee, and if you sign up now,
you can also get 1 extra month for free. NordVPN is a service that I personally use
myself, and they always help me out by sponsoring these larger projects, so please go check
them out at NordVPN.com/ArtofEngineering using the link in the description.
You can also support this channel and everything I do over on Patreon, or simply by leaving
a like and hitting subscribe below. I hope that you found this video educational
and inspiring, and as always, thanks for watching!