hi all and welcome to this new tutorial
series to create a neural network in
javascript which we will hopefully use
to automate the game asteroids now we
made this game asteroids in another
tutorial series so feel free to check
that out you can download the code from
the links below there should be a sounds
folder and an html file
so let's open the game up
so there's a triangular ship that can be
moved around the screen
and it can shoot
the asteroids
i'll just mute that
so the goal of this tutorial series is a
very simple one and intentionally so
because new neural networks can be quite
challenging
the goal is to rotate towards the
closest asteroid
and shoot
rotate to the next closest asteroid and
shoot
it's not going to be an algorithm we're
going to train a neural network to do
this
so what is a neural network
well here it is
basically they're made up of a number of
layers so an input layer a hidden layer
and an output layer you can have
multiple hidden layers but we'll focus
on just having a single hidden layer to
keep things simple
now the input layer it will take our
game information so we could pass things
such as the angle to the nearest
asteroid or the current angle of the
ship
the hidden layer is more like a
processing layer i guess and the output
well we'll probably have a single output
whether to turn left or turn right so
that'll be a single boolean so zero will
mean left one will mean right we'll keep
things nice and simple because these
things can be quite complicated
in between each of these layers there's
a whole bunch of black arrows now each
of these black arrows represent a weight
a weight between -1 and 1 i believe
so for each neuron in the previous layer
it sends out information to each of the
neurons in the next layer
so ultimately we'll be sending in some
training data for example the asteroid
position and the ship's current angle or
something similar and we'll feed that
through with an expected output so
either to turn left or turn right
now if it gets it wrong we'll send that
information back
to the input i guess
and use that to correct these weights so
ultimately we'll be sending in thousands
of points of information
correcting these weights until finally
we have a unique set of weights that can
take any of our game information and
give us an accurate and reliable output
so here's the position of an asteroid
what do i do
turn left here's another position of a
different asteroid what do i do turn
right
let's just close that down
we'll have to create a new javascript
file
just a plain text document called say
neural network
neural
network and give that the js extension
yes and open both of those in your
favorite editor i'm using visual studio
code here
so the first thing we should do is set
up a flag to let us know that automation
is on
so currently we have a whole bunch of
flags in here how about we pull them out
and we'll put them in a different
section just to make it a little bit
tidier how about developer flags
paste them in there
sorry just doing some housekeeping
so let's create a new constant called
automation
automation on
we'll set that to true initially
so when the automation is on we'll want
to set up our neural network so just
after we set up the game parameters here
so after new game here
set
up the
neural network
so if automation is on
let's leave ourselves a to-do here
because we can't do it at the moment to
do
neural
network
good now also when the automation is on
we don't want the player to be able to
to control the ship do we so down in our
key press uh functions so key key down
yep here we go key down and key up so if
the ship is dead or
automation is on
return don't do anything so they can
press the keys as much as they like
nothing will happen
similarly in the key up method
if ship is dead or automation is on
return we'll let the computer control
the ship
speaking of computers controlling ships
let's head down to the update function
that's where all the
objects drawn and moved and so on so
just the start of the update function
just up near here somewhere how about we
say use the
neural network
to
control to rotate the ship
and shoot
so again we'll just leave ourselves
so if automation is on
we'll leave ourselves a to-do here so do
uh
control ship
okay good
let's just make sure that we haven't
broken anything so head up the top uh
first of all let's try it with the
automation flag on
okay so i'm moving the arrow keys i'm
pushing the spacebar nothing's happened
and just to make sure that we haven't
broken anything let's set it to false
yep i can control the ship in that
situation so when it's turned on
when it's true
i've lost control of the ship awesome
now head over to our new file neural
network dot js at the top of that file
let's use a flag use strict
now what use strict does it basically
enforces good coding practice things
like you must declare variables and so
on we're also going to focus on es6
javascript now es6 is like the latest
major revision to javascript syntax
things like class and constructors and
so on so we're going to use that we'll
try to use that throughout this tutorial
series
now neural networks use a lot of maths
specifically matrix maths
so how about we create a
heading here called matrix functions
a matrix is essentially just a
two-dimensional array
so
it has a number of rows and a number of
columns
so i'm just making this stand out a
little bit
matrix functions there we go so how can
we represent a matrix well how about we
represent it as a class so class matrix
give it a constructor
the constructor will take a number of
rows and a number of columns
and how about optionally we'll have data
here as well so if we put an equal sign
here it's a default value so by default
it will be an empty
array
so inside the constructor
let's set the variables now usually
private variables private instance
variables are prefixed with an
underscore so we'll follow that
convention
do the same sort of thing for the
columns
and the data
calls and data now to access those
variables we should put some getters so
just get
rows
and all that does is return
this
dot underscore rows
and do a similar thing for each of the
other variables
so coles
data
this
calls and this
data
so to use that when we declare an
instance of this matrix we'd go matrix
dot rows and that would return
this dot rose
so inside the constructor let's
initialize
initialize with zeros initialize with
zeros
if no data
provided
so if data
equals null
somebody could pass null couldn't they
or data dot
length
equals zero so in the default case
that's true
if either of those things occur then we
want to populate this data this
data will equal a new empty array
you'll have to loop through each of the
rows so four let i equals zero so let's
is one of these new es6 words basically
it means that i is only visible within
the for loop
let i equal 0 i is less than this
rows
i plus plus
lowercase i plus plus
we'll want to initialize the row so this
data
i
will equal an empty array
and we'll loop through each of the
columns so let j equals zero
j is less than this
coles
j plus plus
this
data
will this data i
j
so we'll set each of the elements
to equal
zero
good
if data is passed we'll probably want to
check its integrity for example somebody
could put three rows three columns but
provide data with only two rows so
that's not good is it so let's put an
else here
else
will check
data integrity
so if the data's length
doesn't equal the row number doesn't
equal rows
or
data
maybe data zero dot length so that's the
column length isn't it
doesn't equal coles
then let's throw a new error so throw
new
error now the error takes a message
you can just put something like
incorrect
uh data dimensions
that should be enough
so let's test this let's go back to our
main file
we'll have to import that
file so we can do that up in the
style script
create script tags so script
and we'll need a type type will equal
text
javascript
and the source will equal
dot sorry dot slash that means the
current directory
uh neural
network whatever we named our file
neural network dot js
close the script tags
so head down to where we will initialize
or set up our neural network just in
here
let's try to set up some matrices so let
m0 equal new
matrix
uh rows say two rows three columns
and let's output that to make sure that
it's working
we could probably use the table command
here because it outputs table data
tabular data
so m0.data
that's the data of our matrix
let's take a look at that just need to
open the console
here we go we have a 2 by 3 matrix full
of zeros
that's good
let's try to pass some data
so we can do that by setting up an array
here
so
an array of arrays i should say so 2 1
just adding some values here
and the next row four
three
uh four three zero say
okay let's give that a go
there we go two one minus one four three
zero
two one minus one four three zero now
what if we declared the incorrect
dimension here just so we said a three
by three matrix instead of a two by
three let's check our area checking
good error incorrect data dimensions
put that back
to
awesome
head to our other file to create another
function
so let's create a function that
randomizes the weights or creates a
random set of weights remember between
our neurons there's going to be some
weights
between -1 and 1 so let's create a
function for that so apply
random
weights
between
minus one and one
so we'll call this random
weights
so this is inside our class
for
let i equals zero
i is less than this dot rows
i
plus plus
for the columns for let j equals zero
j is less than this dot columns
j plus plus
this dot data
i
j
will equal
math
random
that's a number between 0 and 1
times
2 times two minus one
so a random number between zero and two
minus one is minus one between minus one
and positive one
good let's test it in our other file
so on m0 we can just go m0 dot random
weights
and hopefully that should work
so currently that's what it looks like
let's run
it ah there we go yep that looks good we
got some random weights negatives
positives and everything in between
great
head to our other file and let's create
our next method
add to
matrices
so we'll make this a static method that
just means we don't have to instantiate
the class we can just call it directly
we can go matrix dot
add
it will require two matrices m0 and m1
we probably want to check that them to
make sure that they have the same
dimension so we can go we'll create a
new method a new static method matrix
dot
check
dimensions
it will require m0 and m1
let's create that now actually
so check
matrices have the same
dimension
same dimensions
so
static
check dimensions m0 and m1
if m
if
m0.rose
doesn't equal m1 dot rose
or
m
zero dot coles
doesn't equal
m one dot coles
then let's just throw a new error throw
new
error
the message we could
how about matrices are
of different
dimensions
okay
so we check that
now we'll need to create a in our add
method we'll need to create a new
return matrix so let m equal new matrix
uh the rows will just be the same as
what's passed so m0 dot rows
m0 dot columns
and we'll need to loop through each of
those so let i equal zero
i is less than m
dot
rows
i plus plus
and another loop for let
j equals zero
j is less than
m dot coles
j plus plus
what will we do here so m dot data
i
j
will equal i'll just copy that
m
0
m 0 dot data i j
plus
m 1 dot data i j so we're just adding
the each individual cells together and
assigning it to the new matrix
and we'll have to return that matrix
return m
okay let's test that in our other file
first we need a couple of more matrices
to test with
so let's create two more m one which
will be two by three and just change up
some of these numbers here
that will do
and m two
will be a different sized matrix say two
by two
so let's just delete
that okay good
uh let's try it we don't need to use
this random weights anymore let's try to
add m1 and m2 it shouldn't work because
there are being invalid sorry they're of
different dimensions so matrix dot add
m1
m2
let's try that
error matrices are of different
dimensions and that's in the
that's on line 81
yep that's correct
how about we try to add
m0
well m m0 and m1 they should add we'll
probably want to print out the data
before we add them
m0 data m1 data
and we'll print out this starter
there we go so we'll add that and show
the data of the
result matrix
okay so the first matrix two one f one
four three zero next matrix zero one
blah blah so adding each of these so two
plus zero equals zero
sorry
two plus zero equals two
four plus two equals six that's right
one plus one equals two three plus minus
three is zero minus one plus minus one
is minus two and zero plus zero
zero plus zero equals zero good adding
works head back to our other file
we also need to be able to subtract
matrices so it'll be very similar to
adding so just copy all of that
paste it down here
subtract
two matrices subtract
almost identical except that we just
need to change this plus to a minus so
it'll be m0 data minus m1 data so the
order is important here
let's test that
right so instead of adding m0 and m1 how
about we subtract
okay so two minus zero is two four minus
two is two one minus one is zero three
minus minus three is six
minus one minus
one
is zero and zero minus zero zero good it
looks like it's working just make sure
the error checking is working okay so m
two minus m one should not work
there we go error matrices are of
different
dimensions great
we also need to be able to multiply two
matrices together
so in a similar way that we've
subtracted them
let's set up and
multiply two matrices multiply two
matrices multiply
if you know anything about matrices this
is not the dot product so let's make a
note not
the dot
product
that's a separate thing we'll need to do
that too but it's a different operation
it's just multiplying each cell by each
cell so all we have to do is change that
minus there
to at times
uh head back and test it so matrix
multiply
mol
multiply
uh m2 and m1 shouldn't work let's just
make sure
yep it doesn't work
and m zero and m one should work
good two times zero is zero four times
two is eight one times one is one three
times minus three is minus nine minus
one times minus one is 1 0 times 0 0
looking good
next let's calculate the dot product
let's create a function to calculate the
dot product of two matrices so dot
product
of two
matrices
so static
dot
m0 and m1
now for two matrices to be dot
compatible as such
the
first one's columns must be the same as
the second one's rows for example a four
by two matrix and a two by four matrix
they're dot compatible the inside
numbers are the same so for example
three by two and two by one they're
compatible
so let's test for that if
uh m0.cols
doesn't equal m1 dot rows
then we have to throw an error here so
throw
oops throw new
error
and the error message can be just be
matrices are not
uh dot
dot
compatible
okie dokie
now the size of the resulting matrix so
let's create a new matrix here let m
equal new matrix
the size of the resulting matrix is
equal to the outside numbers for example
four by two and two by four the
resulting matrix will be a four by four
three by two
times a two by one would be a three by
one so that'll be the m zero dot rows
uh and the number of columns will be the
m1 dot
coles
right
next we have to do the similar sort of
loops that we've been doing in all of
these
so that
so for let i equals zero i is less than
the m dot rows i plus plus and for let j
equals zero j is less than the m coles j
plus plus
so inside here is where we need to do
our tricky stuff basically we need to
add so we need to multiply each cell of
the first
matrices row by each cell of the second
matrices column and then add up those
products
i'll post a link below how to do this by
hand if you're interested so we'll need
to create a variable called sum let sum
equal zero
and another for loop for
let k equal zero
k is less than the inner number which
will be the m0.cols
k plus plus
and then we'll have some plus equals so
we'll continually add this together sum
plus equals
the
m0
data
i
the ith row and then the kth column
multiplied by the m1's
data
and it will be the k
row
remember the inside numbers are what's
matching them by the jth
sorry the jth column
i think that's right sum plus equals
that and then finally we need to set the
i j so m
dot data
i
j
will equal the sum
okie dokie and then finally we'll need
to return that matrix
return
m
so let's test that out in our other file
so matrix dot
dot
m0 m1 now m0 is a 2x3 matrix m one's a
two by three as well so they're not dot
compatible let's just prove that
yep error matrices are not dot
compatible
so which of these are dot compatible
well how about m 2
m 2 times m 1
so a 2 by 2 times a 2 by 3 is dot
compatible because the inner numbers are
the same
let's just update this here
give that a go
right so the first row entries times the
first so times the second
first column entries so
1
times 0 is 0
plus minus 1 times two is minus two
yeah
the second one so one times one is one
plus minus one times minus three is
positive three is positive four in total
yep looks like it's working fairly well
next function that we'd like to do is to
create some sort of conversion function
to convert a regular array to a matrix
because when we pass in our input data
from our game into the neural network
we'll probably just pass it as a regular
one-dimensional array
so
convert
array to
a one-road
matrix
so static
uh convert array we'll call it convert
from
array
we'll pass an array
and all we'll do is return a new matrix
a new matrix with one
row
the columns will equal the array.length
and the data will just be the array
itself
okay let's try that
first of all let's create an array so
let array
equal
just make an array of any length with
some random numbers inside
we'll want to print that out to the
console so console log array
and we'll also want to print out the
converted array so using table
so matrix dot
convert
from
array
pass our array
and we want to see the data of that
okay oh we don't want to see the rest of
this stuff so let's just comment that
out
let's give that a go
incorrect data dimensions okay
something's gone wrong here
25 in our matrix
uh function
where are we so up the top of here page
f line number 25
okay so we failed this data integrity
check i can see why because it's
checking the data.length now the length
of a single dimensioned array is the
total length so it would be four in our
case but we don't want that we want that
to equal the rows which is one
so to get around that all we have to do
is create a
two dimensional array in our convert
array so convert from array
here we just need to put square brackets
around that so essentially we're putting
an array inside of an array a 2d array
okay let's give that a go
that's better so our regular array four
five six seven and our converted array
to a matrix four five six seven so it's
a four columned single row array
great
next we'll need to create a function
that can apply a function to each cell
of the given matrix so we'll call that
map
so just down here
apply a function to each cell
of the
given matrix
the reason we need to do that i do this
is because in neural networks we'll be
using a function called the sigmoid
function
and we'll need to apply it to every cell
so let's create a static method static
map
which takes
m 0 a matrix and a function we'll just
call that m function
and we'll need to create a new uh
matrix so let m equal new matrix it'll
be of the same size as the past so
m0.rows
m0
dot coles
and we'll have to loop over in similar
fashion to what we've done up here
so let i equal zero i is less than m
zero dot well m rows is okay too
m rows i plus plus
j
yep
and inside here all we need to do is go
m
data
i
j m data i j will equal the function
of
that data so the function of that of
itself effectively m data
i
j
and then all we have to do is return
that
matrix
hang on we don't want to assign the m
data because that's empty we want to
assign the m0 data so the one that's
been passed so that data we want to
apply the function to and then we'll
assign that result to our empty
array sorry our empty matrix let's give
that a go
we don't want to see that array anymore
so console table matrix dot map
we want to map let's just choose one of
these arrays so we'll choose m2
and let's apply a function we can use
what they call an arrow function so
that's just x equals greater than
it looks like an arrow pointing
rightwards and then we can just what do
we want to do to x so x times two so
that means we'll times every cell by two
and also let's copy this just above
let's show the original table
okay let's give that a go
so the original table one minus one
three zero
one times two is two three times two is
six and so on that's working well and
one more function so x divided by two
say
so point five three divided by one point
five zero divided yet minus one divided
by two is minus zero point five great
that's our map function handled
and the last function for today is to
find the transpose of a matrix now the
transpose of a matrix is simply when we
switch the columns with the rows and
vice versa
so find the transpose
of the given matrix
so a 4 1
matrix will become a 1 4 matrix
so static
transpose
it'll just take a single
matrix as its parameter
and basically we'll be doing something
very similar to this
just need to modify a few things so let
m equal new matrix we need to switch the
columns for the rows
and we need to make sure that we loop
over the
the original the past matrix
so m zero rows m zero columns
now m data i j we want m data j i
m data j i will equal
[Music]
m zero data i
j let's test that out
okay so matrix dot transpose
let's transpose say matrix 1 so m1
and we'll just put the m1 data before it
okay so the original
the original matrix is 0 1 minus 1 and
then the column down here is 0 1 minus 1
2 minus 3 0 2 minus 3 0 so a 2 3 matrix
has become a 3 2 matrix
excellent
last time we created these matrix
functions that like for example to add
matrices and multiply them and so on so
today we're focusing entirely on
creating the neural network so first of
all let's create the class neural
network so class
neural network
create the constructor now the
constructor needs to take the number of
inputs the number of hidden and the
number of outputs so num
inputs
num
hidden
and num
outputs
let's set the private
variables the properties as such so
we'll just call give them the same name
just prefix it with an underscore num
inputs equals num inputs
num hidden
num hidden equals num hidden and num
outputs
equals num
outputs now going back to our diagram
remember we have all these weights that
are in between the input and the hidden
layer let's call that weights zero and
we'll call these weights here between
the hidden and the output weights one
so this
underscore weights
0 will equal a new matrix
the number of rows will be the number of
inputs so this is the number this num
inputs and the number of columns will be
the number of hidden so this
num
hidden
similarly weights 1
will equal
a new matrix except that its
number of rows will be the number of
hidden
and the number of columns will be the
number of outputs
okay
for the weights 0 and weights one we'll
need to set that we'll need to update
them during our training process
so let's create getters and setters for
them so get
weights
zero
that's simply return
this
underscore weights zero
and we'll also want to set that so set
weights zero
uh weight
weights
set weights zero weights so we can just
go this
underscore
weights zero equals weights
right
and we'll do a similar thing for weights
one
weights one
this weights one weights set weights one
this weights one equals weights
great
so
we've done this one here
next we need to randomize the initial
weights so let's just
use that as our next comment
so randomize the initial weights
we've already created a function in our
matrix functions actually where we can
call randomized weights on any
on any uh matrix so weight zero dot
random weights
and do a similar thing for weights one
good that's that step done we should
probably test this so go to our other
file the html file
and let's go down to where we set up
here we are to do neural network so
we'll have to create a variable here so
variable nn we'll call it
and here we can go nnn equals new
neural network
now we need to pass the number of inputs
hidden in outputs here i think we should
set them as constants so up where we're
setting our constants let's create a
section called
neural network
parameters
const
num of
inputs i'm not entirely sure how many
inputs we'll need but let's just say two
to begin with
const num
hidden
um again there's lots of theory behind
what number this should be
the higher usually means the more
complicated processing it can do so
we'll just start with five we can always
adjust it later
and const num
outputs now this one i am confident
about we just want one boolean flag
zero turn left one turn right so one
output
so head back down to where we're setting
up our neural network
put in these as their the arguments and
num inputs
num hidden
and num
outputs
so we don't need these matrices anymore
so we can get rid of them
to test that our neural network's
constructor is working all right how
about we output the weights so nnn
weights
zero
dot data
and we'll do a similar thing
for weights one
we don't need this transpose anymore
so open that up in our web browser
okay so we have one
for the first matrices of weights is two
by
three four yep two by five that's right
and our output one our weights one is a
five by one
with random weights inside
awesome
to head over to our neural network class
let's go down and see what we need to do
next
feed forward now all that means is that
we're going to input some information
into our network it's going to feed
through it and some output is going to
be created
so let's create a new method called feed
forward in our class
it will take an input array as an input
as a parameter
let's paste these comments in here
so the first step is to convert input
array to a matrix that's easy enough so
let's declare a new
array let inputs equal matrix
dot
convert
convert from array we made this earlier
so we can just put the input array in
there
and that will convert it to a single
road matrix
that's fine
next one find the hidden values and
apply the activation function okay to
find the hidden values we can look at
this diagram here you can see that each
hidden
neuron is made up of these
weights one coming from each of the
previous inputs
so to calculate that it's just the first
input multiplied by the first weight
plus the second input times the second
weight times the third input sorry plus
the third input times the third weight
and that can be handled using a matrix
dot product
we have a function for that as well so
let
hidden
equal
matrix dot
dot
which takes two matrices the first one
being the inputs and the second one will
be the weights which we all have already
determined it's been randomly determined
so weights zero
okay
now the activation function
is simply well with lots of neural
networks the activation function is
something called a sigmoid function
let's take a look at that
now a sigmoid function is simply this 1
divided by 1 plus e to the minus t now
what that does we can input any value
into this system and it will return a
value between zero and one no matter how
big that value is so if we put one
trillion into this we'll get a it'll
output a value close to one
okay which suits our needs very well
so let's go ahead and create a function
down here just outside our class
function we'll just call it sigmoid
sigmoid
it'll take some value we'll call that x
and it will return
1 divided by
1 plus the
exponent so that's e
a numeric expression representing the
power of e yep so math.exp to the minus
x
that's right
now in order to apply that function to
each of the cells of the matrix we made
a function for that too so we just go
hidden will equal
matrix
dot
map
okay so it takes the
the matrix so which is hidden
and the function to apply to it well we
can use an arrow function here so x
equals greater than
x
sigmoid x so sigmoid x so that should
apply the sigmoid function to each of
the values within our matrix
good
similarly
so for the second part find the output
values and apply the activation function
it's the same sort of thing
so we can just go let
outputs
equal matrix dot
hidden will be the inputs
so this will be from here from the
hidden layer to the output layer so
hidden will be the inputs and output
will be the outputs
so matrix dot hidden this dot sorry this
dot weights one that's the second set of
weights
outputs
matrix map
so itself
outputs and
apply the sigmoid function to each value
and finally we want to return
return those outputs to the sender
the next step apply bias i don't want to
do that right now and the reason is
because not applying bias can cause
issues and i'd like to see what those
issues are
so let's first test this feed forward
function so let's do some console output
console log
inputs
and we'll just output the uh the data in
the array so we can use table to that
console.table
inputs
dot data
and we'll do something similar each step
of the way here so when we work out our
hidden
neurons we can just do that
and we'll also want to see what happens
after we apply the sigmoid function
similarly for our outputs
and
after the outputs
have had the sigmoid function applied to
them so i'll just call that output sig
that will be hidden sig
okay head over to our other file
let's call that function so all we have
to do is go nn dot
feed
forward
it requires an input array so something
like zero comma one now remembering that
the number of inputs that we declared
for our neural network is two so we need
to pass to
an array of length two here now if we
didn't let's just try that if we didn't
it should break
let's have to open our file
open up the console
yeah so inputs got through a single
length array but there's an error
matrices are not dot compatible so to
fix that all we have to do is
put the second element
put the second uh
input in there let's give that a go
yep that worked fine so we've got two
inputs zero and one
the hidden values are calculated i'll
just believe that
the sigmoid is applied the output and
the sigmoid is applied let's just keep
resetting that until we get an output
that is negative
here we go so here's a negative output
can you see that the sigmoid function
has made that positive
so that seems to be working okay
so head over to our other file we can
probably get rid of some of these
console logs so let's just delete a few
of them
we'll keep one set we'll probably need
it for our next function
and that is train train using input and
target arrays
right so currently our system doesn't do
very much we give it some inputs and it
outputs some randomly generated outputs
not very useful so in order to make it
do something how about we create a
function that allows us to put in some
training data so we'll call that train
so up inside our class
train
it will take an input array
and the target array
so for example if we
had the inputs 0 and 1 and we expect 1
to be the answer then that's what we
would put in here
let's paste those comments in there
so the first step feed the input data
through the network well we already have
a function for that called feed forward
so let's say let outputs
equal
feed
forward
this dot feed forward
the input array
okay that's that done
next calculate the output errors
target minus output okay in order to
subtract matrices we'll have to convert
the target array first so that
let's targets
equal matrix dot convert
convert from away from array
target
array okay good
and
let's output errors
so we're calculating the output errors
here let output errors equal matrix
dot subtract
and we'll subtract the first which is
the target so the targets
minus the second which is the output
the outputs
let's test that so we'll grab this
console data that we kept up here
uh outputs
targets and output errors
so outputs that'll be fine
targets
targets and output errors
output errors let's print out every step
okay head back over to our other file uh
nn.train
we need an input array 0 1 and an output
array remembering it has to match the
number of outputs if it doesn't we're
going to get an error we'll just
give that a go we'll put a 2
column one there
okay so error matrices are of different
dimensions so when it tries to subtract
them it failed so let's put that to a
single one
right so we have our outputs which is
0.639
the in the expected the target is one so
the difference is point three six that
sounds correct
so on to our next step
calculate the deltas
which are equal to the errors times the
derivative of the output
so the derivative of the output is
actually the derivative of the sigmoid
function so let's update our sigmoid
function to handle derivatives
how about we just pass a flag derive
equals false we'll set we'll give it a
default value so you don't have to
provide it and if you don't it'll just
return what we always have
if derived
then we'll return the derivative now the
derivative of this is x times 1 minus x
that's not the true derivative of the
sigmoid function
but
we've already applied the sigmoid
function so we don't want to return the
true so i'll put a note here where
x equals sigmoid
x
right so the true derivative is the
sigmoid x times 1 minus the sigmoid x
but we've already applied it so this
will be a good representation x times 1
minus x
so let's create a variable let output
derives
equal we can use our map function to
apply this matrix.map
the outputs
and the function will be we can use an
arrow function again
will be the sigmoid
x but this time we'll set the derived
flag to true so it'll apply the
derivative to each of these outputs and
assign it to that
and finally
output
errors no calculate the delta output
deltas is what we're trying to find here
output deltas will equal
matrix
dot so error is times derivative so
multiply we can use our multiply
function
the first one is the errors output
errors
and the second one will be the output
derivatives output derivatives
and we'll just copy
this console log here so we can test
this
output just the deltas we'll do output
deltas
let's test that
let's run it again
so we have outputs targets output errors
and output deltas
who am i to refute that it looks okay
okay on to the next step
calculate hidden layer errors so it's
the deltas dot the transpose of weights
one
right so let's calculate the transpose
first so we'll call that weights
one
capital t for transpose will equal we
have a function for this matrix dot
transpose
it just takes a single matrix so that'll
be the this dot weights
one
okay and then we'll need to calculate
the hidden layer errors we'll call it
hidden errors hidden errors
will equal delta dot so matrix
dot
uh
what was the first one the deltas dot
the transpose okay so that'll be the
output deltas output deltas
dot what we just worked out the
transpose their weights one t
again let's output some just to make
sure it's not
throwing an error basically we don't
really know exactly what we're after
here
uh the hidden areas is what we'll want
to output here hidden errors
let's give it a go
right so we get some output deltas and
then we get the hidden
errors across ways that looks to be okay
so the next step calculate the hidden
deltas errors times derivative of hidden
sounds very similar to what we've done
up here so let's just copy and paste
that
let hidden derives
equal matrix map
hidden
i just had a thought we don't have
actually have visibility to hidden here
hidden because hidden wasn't was used
within this feed forward function
here we go let hidden equals that so how
about instead of creating a local
variable here we use a property on our
neural network we'll say this dot hidden
this dot hidden so anywhere we use
hidden inside here we'll go this dot
hidden
this dot hidden this dot hidden
and we'll have to go create a property
on the neural network so we can do that
easily enough just at the top of it this
dot underscore hidden will just equal an
empty array
and we'll have to set up some getters
and setters for it so that we can access
it
get
hidden
we'll just return this
underscore
hidden
and set
hidden
just pass a
variable called hidden this
hidden
will equal hidden
that should access yep that's the same
as that
right so that should all work in our
feed forward now
and back down where we have
back in the train function
where we have hidden here so hidden
derivative with a matrix map this dot
hidden so we're accessing the actual
hidden values at this stat at this point
in time
x
uh sigmoid x is true yep we're applying
the derivative of it
calculate the hidden deltas so the
hidden deltas
matrix multiply the hidden errors
times the hidden derives
right so let's output those hidden
deltas
let's give it a go
so we have some hidden errors with some
hidden
deltas
good
now for the final step in our little
neural network update the weights so we
have to add the transpose of layers dot
deltas so we add that to the weights
okay so let's first calculate the
transpose so let hidden
transpose equal matrix
dot transpose
uh this dot hidden
sorry
um
next we have to add the transpose to the
weight so this weights the associated
weights is one it's the second set of
weights will equal matrix
add
add itself we have to add to itself so
this
weights
one
the
layers the transverse layers dot deltas
so matrix
dot
the transpose hidden t
hidden
t
and the deltas which will be the output
deltas of the associate are associated
with the hidden layer so output deltas
now we have to do a similar thing for
the first weight so weights zero
so we need to calculate the transpose of
the inputs so inputs t well matrix
transpose
in do we have
inputs
do we have visibility on inputs
no we don't because again within the
feed forward
we're converting it there
and keeping it as a local variable
so
there's a couple things we could do we
could just convert it again within our
train array but we have to remember that
this train function will probably be
called like thousands of times if not
hundreds of thousands of times so it's
probably better just to assign it to
what how we're doing with the hidden
here so let's do that
this dot inputs
this input so we'll have to go up and
create a property
same as what we've done for hidden
so this
inputs will just equal an empty array
and we have to create a couple of with
create getter and a setter
get inputs
this
return this inputs
set inputs
inputs equals inputs
this inputs will equal inputs that's
looking good so our feed forward
function should work now we'll just
assign instead of creating a local
variable we're just assigning as a
property now down in our train
uh function
this inputs should work now that's fine
and we're updating weights zero at this
point weight zero weight zero dot
the inputs t
inputs t
and the deltas that are associated with
that of the hidden deltas
the next
layer deltas let's run that make sure
it's not broken
okay it's not breaking so that's good
enough test for me i think we should
actually try this training and see if it
actually works or not how about we get
rid of all these comments first because
they're just not
we don't want them printing out every
iteration when we've got like a thousand
or ten thousand iterations
remove all them remove all them
this the weights are updating great go
back to our other file
right so we'll be calling train but we
want to loop over the number of training
samples that we have so we can go for
i'll title this train
the network
for let i
equal zero
i is less than say let's set up a flag
for this so num
samples
i plus plus
in there we'll do our training for
something
a number of samples let's go up and
create that now
num samples
i'm not exactly sure how many we'll need
but let's start at say one thousand i
know in other examples there's ten
thousand and a hundred thousand and so
on we'll just we'll start with one
thousand num samples is 1000.
and how about we use the classic example
to test neural networks is the zor sorry
the xor gate the exclusive or gate
so x we'll call this a test test
x or
gate logic
let's take a look at that xor gate here
it is just this table down here so when
you have two inputs 0 0
output 0 0 1 or one zero output one and
one one outputs zero so zero zero zero
okay let's write that in here i think
zero zero
equals
zero
zero one equals one
one zero equals one and one
one equals zero so only when there's one
of them
flagged will it be one otherwise when
they're both the same it'll be zero
so let's create some random input we'll
create one
zero one there and one zero one there so
we'll input one input two or input zero
and input one input zero will equal
well
it'll be a random number and we want to
round it so math round just rounds to
the nearest integer
and we want to round a random number
math random which is a number between
zero and one so that should produce a
zero or one
similarly for input 1
so we can pass them into our
training array here so input 0 and input
1
and we also need an output so let output
let output equal well what's the logic
behind it if input 1 input 0 and input 1
are the same then the output is 0 right
otherwise the output is one so we just
do a conditional here
input zero
equals
input one if that's true the output is
zero
else the output is one that's the
zorgate logic
and put our output into this array here
out
put
so basically we've created a whole bunch
of random
training data with the correct outputs
fed it into our system in the hope that
it's going to create an accurate uh
prediction
so let's test it test output
we can just use the console for this i
think
console.log
we'll put in each of the
combinations of inputs so 0 0
will equal
let's feed it into our system together
to get a prediction
feed
forward
we need the array 0 comma the input
array 0 0
and we want to output to the screen the
data of that so do that for each of the
possible
input combinations so zero one one zero
and one
one
zero zero zero one
one zero and one
one
let's give that a go
okay zero zeros we're expecting a zero
here and a zero here and a one here and
one here it could be that we haven't got
enough samples so let's try by bumping
up the number of samples to say ten
thousand
ah that's looking pretty good that's
close to zero that's close to one that's
close to one that's close to zero let's
just keep trying make sure there's no
discrepancies here
now i have read that not having bias can
cause an issue in these situations
so let's see if it comes up with a
strange
reading on occasion
so here for example the prediction for 1
1
is 0.5 so that could be because of the
bias not being there so i think for
safety we should put the bias in
so first of all what is bias
well here's a diagram basically we have
our input neurons
we just add an extra neuron for bias it
always takes a value of positive one
and we add it and all it does is output
its own weights in effect it doesn't
take an input of any kind it just
outputs its own weight we'll have one
bias node on the input layer and one
bias node on the hidden layer the reason
we do this is because the input
neurons could take all zeros for example
in our zorgate logic if we have zero
zero as our inputs can you see that that
could
skew the results because 0 times any of
these weights will be 0 so it's not
going to behave correctly so that's why
we add this bias node
head into our other file
go right up the top
in the same way that we've created
weights here let's create a bias zero
and a bias for one
so
bias zero will equal a new matrix of
just a single row so one row
the number of columns will be the number
of hidden
and bias one same sort of thing one row
but the number of columns will be the
number of outputs
okay and we'll have to randomize in the
same way that we've randomized the other
weights we'll randomize the bias so bias
zero random weights
do that for bias one as well
okay that looks good
head down to our feed forward function
we've left the node here apply bias
to apply the bias all we have to do is
before we apply the sigmoid function to
the hidden layer we can just apply we
can add the bias so this hidden will
equal matrix
add
this hidden so adding to itself
this
bias
zero so that is applying the bias apply
bias
and in the same way we'll do that with
the output so before we apply the
sigmoid function
instead we'll be doing it to the output
so outputs will equal matrix add
outputs
this
bias
one
now down in our train function we've
left a note update biases
now to do all we have to do there is go
in the same way that we've updated our
weights here we'll do the same thing
this dot
bias
1
will equal matrix
add
this
dot bias
1
and we're adding the
the deltas the output deltas for this
one
same sort of thing for the bias 0 except
that we'll be adding the hidden deltas
so this by zero
hidden deltas
so let's test that
oh we have an error m1 is undefined
okay
uh m bias
uh
maybe no i didn't create the uh getters
and setters for these private
so let's do that now that was a bit
silly so we're just as we do just as we
get and set the weights let's get and
set the bias so bias
zero
this
bias
zero
set
bias zero
and this
bias
zero just pass the bias
oops
bias
okay so but get biased you return this
by zero set by zero bias this bias zero
equals bias do that for bias one as well
just update that to one
one
and one
okay that should be okay now let's give
that a go
great so if we run that a few times my
hunch is that it will be fine we'll get
a value close to zero a value close to
one and finally a value close to zero
we will actually be feeding in game data
into our neural network so that we can
rotate this ship i'm doing this manually
of course so we can rotate the ship and
shoot at the closest asteroid
okay so let's head into our code i'll
just turn automation back on
okay and head to our neural network file
the first thing i'd like to do today is
to set up some error logging in our
neural network class the reason for this
is that i'd like to be able to keep
track of how well our
training function is doing as we're
training it so let's set up a flag up
here just a constant let's call it log
on
set it to true
and that's just whether or not to show
error
logging
we'll need to keep track of how often we
show this message like show a message we
don't want to show it every iteration
because we could in theory be calling
this
uh train function like thousands of
hundreds of thousands of times if not
millions of times so let's set a log
frequency
uh initially we'll set it to a thousand
say
and that's just how often
to show
error logging
error logs
and that's in iterations
in
iterations
okay we'll have to set up a private uh
member variable here
just down here so error
logging
this underscore we'll keep track of how
many calls we've had or how many
iterations we'll call it log count
and it'll start the day at log frequency
and we'll also because it's private
we'll want to set up some getters and
setters
it's not technically private i'm kind of
making it private i should say
so get log count
i don't get log count
return this
log count
set log count
count
this log count
equals count
head down to our train function
now what's the most appropriate thing to
output here i'm thinking the output
errors because the output errors are
simply the target minus the output so if
the target's 1 and the output's 0.7 then
the error is 0.7 it's easy to understand
so let's do our error logging here error
logging
if
log
on
and if the this dot
log
count yep if this.log count equals the
log frequency
well let's do a console output so
console log
uh output errors
output errors
now it's a matrix so we'll need to grab
its data array and we there's only going
to be one error because we've only got
one piece of output so we'll just do the
first
column
sorry the first row first cell
okay
probably prefix that with something so
we know that what we're outputting error
equals that
then we'll have to decrement the log
count so this dot log count minus minus
because we don't want it occurring every
iteration
and when that reaches zero if this dot
log count equals zero
what will we do well we'll just reset it
this dot log count
equals
log frequency
let's test that out
so currently we we'll just need to turn
our console on
reset it
okay here we go so we've got errors of
0.32 0.29 0.08 blah blah blah it's going
down slowly isn't it uh how about so
this is our zorgate logic that we used
last time
how about we increase the number of
samples so go back to our other file our
html file we'll increase the number of
samples to say 100 000
and we'll increase the number of hidden
nodes to 10
but because we're doing a hundred
thousand samples now how about we change
our log frequency to say 10 000 we don't
want it showing so often
let's give that a go
point seven four point zero one as you
can see very quickly the error is
reduced with the zorgate logic so that's
good our logging is working great
that's all we really need to do for our
neural network file so let's head over
to our other file
now we have to think about how many
inputs are there going to be in our
system
i've always said that the goal was to
turn our ship towards the closest
asteroid and shoot it so knowing that we
need to know the asteroids position
don't we so there's going to be an x
position and a y position so that's two
inputs
and we also need to know the ship's
angle because if we're facing this way
we need to know that we'll have to turn
left to get to the asteroid or turn
right or whatever so that's the asteroid
x the asteroid y and the ship's angle so
that'll be a minimum of three inputs
uh how about just for testing purposes
we'll just turn the roid number down to
one there'll be a single asteroid
right head down to where we set up our
neural network where we train the
network we don't need to do any of this
all gate logic anymore that was just for
testing our system
and we don't need this test output
anymore
so let's create some variables well i
need the asteroids x position asteroids
y position the ship's
angle and just for completeness let's
include the ship's position as well so
ship x and ship y
so the first thing we'd like to do the
first thing we'd like to include in our
training data is a random asteroid
location random asteroid location but we
have to remember that asteroids can
float off the left and right of screen
as well as top and bottom up to their
radius so we'll have to include
off screen
data here as well
so a x will equal a random point random
times by the width of the screen so
canvas dot width
but because
it can go up to its radius off the left
and its radius off the right well that
will mean we'll have to add the total
with the size of the roid here roid
size
which is the diameter in effect just
bracket that
and then fine so that'll generate a
number a number between zero and say the
width is 500 so 500 plus 100
is 600 so 0 and 600 but we want it to be
able to be negative so we'll have to
subtract the radius so it'll be the roid
size
minus the roid size divided by 2 which
is the radius
do a similar thing for a
y
except that it will be the height
good and now the ship's
uh angle
and position
now the ship's angle
it'll just be random we'll just randomly
generate an angle anywhere between 0 and
360 degrees so in radians let's do a
math random first math random
times 360 degrees in
in radians which is just pi times two
good now the x location and the y
location i'm not planning to move the
ship at all that'll be far too
complicated and i think it's a pretty
good strategy just to sit in the middle
turn to the nearest asteroid and shoot
so we'll keep it fixed so sx will just
equal the ships
starting sorry the ship's starting x
and s y
will just equal the ship's starting y
next we'll need to work out which
direction to turn
based on the angle of our ship and the
angle of the asteroid
so calculate
the angle to the asteroid
how about we write a function for this
because it's possible that we'll use it
again
just down here
function we'll give it a generic name
such as angle
to
point
angle to point we'll have to use some
trigonometry here so we need to know our
current position so x and y
we'll need to know our facing or bearing
we'll call that bearing
uh we'll need to know the targets x and
y location target x and target y
and i think that's all we need to know
so let's angle
to
target
so let angle to target equal well it'll
have to be the inverse tan because we
have x and y coordinates so tan is
opposite over adjacent so a10 there's
two types of atans here there's a10
which is our regular mathematics
tangent
but that doesn't take into consideration
coordinate space you know how we have
four quadrants in coordinate space
but a10 2 does take into consideration
that so we'll use that now it requires a
y and an x so we'll have to find the
differences between our
target and our self
so we'll put
so this will be the y differences so
it'll be target
y
minus
y
right and the x's will be the symbol a
similar thing target x minus x
however because y's on screen like pixel
wise are in the opposite direction so as
we go up the screen we're actually going
in a negative y direction so we'll have
to apply a negative to that and a
negative to that so it'll be minus
target y plus y
target x minus x y x yep that's good
we'll have to find the differences
between our angle to the target and our
bearing so we can just go let say diff
equal
our bearing
minus
the angle to target
angle to target
now that could result in a whole range
of numbers because this a 10 2 function
it returns a number between effectively
180 degrees and minus 180 degrees so it
could be negative
our bearing could be anywhere between 0
and 360. so just that was 0 and we had
180 here we'll get a negative number
won't we
if the bearing is say
360 and our angle is minus we'll get a
number
over 360. so we'll need to do some
normalizing as such here so how about we
have or we can just return this i think
return
the
diff
remembering we can add 360 degrees to
any angle and it's still the same
effective angle if i'm facing 40 degrees
and i add 360 degrees so 400 degrees i'm
still facing at the same angle aren't i
so
we will add
the
we'll add 360 degrees math pi times 2
but that would that could get us above
360 couldn't it'll get us above zero
which is what we want but it could also
get us way above 360. so we'll find the
modulus of that the remainder so modulus
math
pi
times 2 so that will
guarantee us a result between 0 and 360
degrees in effect in radians right so
back up here we can just go let
angle say angle equal call that function
angle
angle to point
we have to pass the correct parameters
so the x is our ship location ship x
ship y
the bearing is the ship's angle
the target x is the asteroid x and the
target y is the asteroid y so that'll
give us our
absolute angle as such
and then we need to determine
the direction to turn
right so that's probably let's say let
direction
equal
[Music]
well if that angle is greater than 180
degrees so angle
is greater than math.pi that's 180
degrees
then we'd like to turn
left so remember we have to um put in
zero one here how about we set these as
constants i think that would be the best
move
so back up here in our neural network
parameters let's have constant
output
left
will equal zero so that will be the
expected
output expected neural output
for turning left
similarly we'll have the output right
which will equal one
and that will be the expected neural
output for turning right
okay
so back down here if the angle is
greater than 180 math pi we'll
output left
else will output
right so finally we just need to train
the network
so we can call
neuralnetwork.train to do that it takes
an input array
so the input array is just our
position of our asteroid ax and ay
and the
angle of the ship
and the output array the expected array
is just an
array with direction in it it's just a
single
length sorry a one length array
that's our training so we'll be training
that every iteration
now if i foresee one problem
where this a x a y and s a they take on
strange values so ax could be a number
like 600 a y could be minus 23.
sa could be
uh
3.8 or whatever right
they're not normalized are they but
let's see what they do it might not be
an issue
okay
you can see that these errors don't seem
to be getting any better let's pump up
the
number of
pump up the number of samples
just while we're here we'll that's the
number
of training
samples
so we'll pump that up to 1 million let's
see what that does
to me that looks like it's not getting
anywhere
because these errors are still
incredibly large
they're not getting any closer to zero
so i don't like that so i think we'll
need to normalize our data
we'll put that back to oops we'll put
that back to a hundred thousand i don't
think we need a million samples but a
hundred thousand should do us head back
down to our training here
we'll need to normalize this data here
before we input it into the training
array so how about we create a method a
new function called say normalize input
or similar so just before shoot laser
function
normalize
input
we'll need to pass the roids x location
the roids y location and our ships angle
they're our three neural inputs now what
we mean by normalize is normalize
the values
to between
zero and one
so we'll divide them by something in
order to get them to be between zero and
one
so let's create a new
uh array an empty array
called input let input equal empty array
and input
zero
that will be
our roid x
value
we'll need to now normally you could
just go right x divided by the canvas
width that would give give us our screen
position our relative screen position
but because we've allowed it to be off
screen up to its radius then we'll need
to modify this so roid
x
so that could be negative 50 could be
negative it could be the negative radius
it could be negative 50. so we'll have
to add the radius back to that to get it
back to zero so roid
size
divided by two
and we'll have to divide all of that by
the new canvas width including the
radius on the left and right so that'll
be canvas
width
plus
the diameter which is just the roid size
let's just work through that logically
so roid x could be minus 50. so minus 50
plus
50
equals zero zero divided by anything is
going to be zero that's fine
now the maximum value this could take
would be
just say we'll choose an arbitrary width
of 500 so 500 plus 50 is 550
plus 50 is 600 600 divided by 500 plus
100 is 600 so that'll be one that's good
and halfway between say 250 250 plus the
radius of 50 is 300 300 divided by 500
plus 100 600 300 divided by 600 equals
0.5 that sounds good
we'll do the similar sort of thing for
the x sorry for the y
position
except that we'll use the height instead
now the next one ships
angle this is a bit of an easier
exercise all we have to do is
divide the
past angle so that'll be a number
between 0 and 360 in effect in radians
we just have to divide that by 360. so
math
pi
times two so an angle of zero divided by
that would be zero an angle of 360
divided by that would equal one and
everything in between that's great let's
return that input
array so let's go up and implement that
in our training section
train the network so all we have to do
is normalize this data beforehand
normalize input before we put it through
our
chain function
let's give that a go
it's looking better
at least it's getting closer to zero
there's some discrepancies here maybe we
just need to modify some of the
parameters here so how about we bump up
a number of hidden to say 20
and we'll set our number of samples to
say
500 000.
just while we're here we might as well
change the number the amount of logging
let's change that to say every 20 000
instead
okay
let's give that a go
so it's a bit slower because we've got
more hidden uh neurons
but it does look like it's getting
better the the errors aren't too bad are
they
look at that one down the bottom very
close to zero so i think we've done
quite well with that if we put that
if we put the hidden nodes back to 10
it'll speed it up a bit will that impact
the quality let's have a look
that's pretty good too actually
either way well let's leave it at 20 and
500 000 number of training samples
just while we're here i'm thinking that
we'll need an output threshold because
none of these predictions are going to
be exactly zero or exactly one are they
but they might be close
so how about we create a threshold
for example 0.25 just an arbitrary value
to begin with
and this is how close the prediction
must be so how close
the prediction
must be
to
commit
to a turn
so at 0.25 for example if the prediction
is 0.75 or greater we would commit to a
right turn if the prediction was less
than 0.25 we'd commit to a left turn
however if the prediction was say 0.5 we
won't do anything we'll just sit in the
middle so how about we start
implementing some of this so let's go
down to well first of all let's see what
happens when we push the left and right
arrow keys so in the key down method
so when we've got manual control of this
all we're doing is setting the ship's
rotation to this
how about we create a function that can
handle because when we turn right it's
the same thing except we're going the
opposite direction so let's pull those
out and create a function for that
because we'll need to call this function
what will we call it rotate
ship
it was pass a boolean we'll say that
false equals left
right false equals left and here
we'll rotate ship
true so that'll be right
and let's what do we do in the key up
method the key up just sets the ship's
rotation to zero so that's fine we don't
need to do anything with that so let's
go down make that rotate ship
just after our normalize input function
rotate
ship
it'll take a boolean which is equal to
right
so we'll be doing that
now if it's right we want to put that as
negative so how about we make a variable
called say let sine
equal now if it's right
if it's right the sign will be -1 else
it will be positive one
and we'll just multiply this by
the sign
that should be okay just to make sure we
haven't broken anything how about we
turn it back into manual mode
so automation on is false
and make sure that we can control this
ship
yep i can turn left and right no problem
that's all good
put that back to true
and now let's go down to our update
function where is that down here
to do control the ship
so we'll need to make a prediction make
a prediction
based on current data
so let's make some variables so we need
to know the asteroids x which will equal
roids
um
we'll just use because there's only one
asteroid to begin with we'll just use
roid 0 for the time being roid0.x
similarly a y will equal roids
sorry roids0.y
and we also need to know the ship's
angle don't we so that's just the ship
dot a
ship dot a
now to make a prediction how we go let
predict
equal we just need to run our feed
forward function
feed
forward
it will take the input array so input
array will be ax
but we have to normalize this don't we
so hang on do the normalize first
normalize input it takes the voids x so
a x
a y
and the ship's angle
normalize input
remember that returns a matrix an output
matrix so we'll have to take its data
and there's only going to be one output
so it'll be the first column and the
first so the first row first column 0 0
that should be our prediction
now we need to decide which way to turn
so make a turn
now we're going to get a value it's not
going to be exactly zero it's not going
to be exactly one so how about we find
the difference between it between the
prediction and our
left and right turns so let
we'll call it delta left or difference
left whatever you're going to call it
delta left will equal the prediction
make this the absolute value math
absolute value
it'll be the prediction
minus the
uh output
uh constant output left
right and delta right
will equal the prediction minus the
output right
so let's just
try to analyze that so the prediction
just say the prediction is 0.2
right
so 0.2 minus output left is 0.2 that
means there's only it's 0.2 units close
to the left
0.2 minus
1 is 0.8 the absolute value will be 0.8
so that's not going to be what we want
to do so we'll have to compare it we'll
have to compare these to our thresholds
okay
so if
delta left
is less than our output threshold so if
it's less than 0.25 or whatever value we
choose to set it at
then we wish to rotate left rotate ship
now left is just false
okay else if
delta right
is less than the output threshold
then
we will rotate the ship right
which is true
now if it's neither so for example 0.5
0.5 minus the output left is 0.5 0.5
minus output right it's not going to be
less than the threshold in either of
these situations so how about we just
put an else here saying that
um
[Music]
stop rotating
so if it's not confident it'll stay
still so ship dot rotate
equals zero so that's equivalent to us
releasing the arrow keys
i think we should just log this
prediction here so console
log
predict
just so
we can see if there's any problems with
it
okie dokie let's try it
so we're just training it to begin with
with 500 000 samples
the errors look pretty good
that one wasn't very good
okey-dokey what's it doing
it seems to be stuck i know
now it is turning
it's kind of slowly turning
towards the asteroid will it turn when
it goes past it yes
it's okay it's maybe not the best
now i wonder why
we'll just watch it a bit longer it's
kind of getting confused a bit isn't it
okay i've been watching this for a while
now it looks like something's going
wrong because it eventually the ship no
matter how many times you run it the
ship eventually starts spinning around
in its inner circle without
and it's got these really
very confident predictions so see this
prediction here it's very very low it's
close to zero
so
i'm guessing something's going wrong
with our angle so let's go into our
code maybe we should log the angle here
ship
dot
angle
let's see what that does
okay it's loading the training data
again
okay so the angle remember it'll be a
number between 0 and 2 pi which is about
6 or so
now that's not good see the angle is
going negative there we want the angle
to be zero between zero and two pi so
there's something
i think i might know what's wrong
if we head down to where we rotate our
ship in the update function
i suspect that we're just letting it add
on endlessly
we are so for example the ship out the
ship angle can just keep going up and up
and up and up so it'll go beyond 720 and
so on it could go below zero no problems
which is fine it was working fine in the
game but for our prediction we want it
to be between 0
and 360. so we want to keep the angle
between
0 and 360 which is 2 pi remember 2 pi
so if the ship's
angle is less than 0
we'll need to add 360 to it remembering
if we add 360 it'll still be the same
angle so plus equals
math
pi do we need to bracket this i'm not
sure
i'm not sure what the order of
operations are for these
these assignment operators we'll just
bracket it won't hurt math times two
we'll add 360 to it
else
if
a ship's angle
is greater than
or equal to
360.
360. math times i'll bracket this one as
well math dot pi
times 2 if it's greater than that then
we'll want to subtract we want to do the
same thing except that we'll subtract
360. we don't want exactly 360 because
our randomly generated uh position will
never be 360 i don't think
minus equals that
and
that should do it
let's give that a go
let's load it up
loading the training data
so the training data looks pretty good
it should be working quite well
okay
i think that's acting a lot better
where's the asteroid there's the
asteroid yeah i think it's tracking it
quite well
i think that was our problem
yep so it's choosing the first asteroid
which is whatever's in the in the array
first in the array and following it
good okay how about we add some shooting
to this little ship
so let's head back all the way up to the
top to set some
parameters
so we'll probably need to give the ship
a
rate of fire so just in under our neural
network parameters here let's create one
called rate of fire
and we'll set that to say five shots per
second so shots per second
we'll also need to keep track of how
many frames we count down to the next
shot so
where we set up our neural network here
let's just put in a new something called
say a i
shoot time
and we'll set that to zero to begin with
where will we update that well we'll
update that in
our water where we do our prediction
so just after we make it turn now what's
the strategy for firing when when will
we shoot well we could shoot just when
it's not rotating
but that's not really what i do i keep
turning and just shooting all the time
because eventually you're going to get
lucky too so how about we shoot all the
time it'll shoot as often as it can so
shoot the laser
if a i shoot time
equals zero which will it'll start at
zero then we want to reset that a issue
time not this
a i shoot
time
will equal the frame rate divided by the
rate of fire
i think we should find the ceiling of
that
math.seal
so if we had a frame rate of 30 divided
by the rate of fire of 5 that'll every 6
frames we'll shoot if it was 10
so 30 divided by 10 would have we'd be
shooting every third frame so quite
quickly
uh we'll need to shoot the laser shoot
laser
just
checking that function
because part of that condition is that
the ship can shoot and when we shoot a
laser the ship can't shoot
that's because the player must push the
space key then release it so we'll have
to set that up to be able to handle that
or we can we can just do it here ship
dot can shoot equals true so whenever it
can fire we'll let it fire
else so if the ai shoot time doesn't
equal zero we'll want to count down a i
shoot time minus minus so this is just
counting down the frames left
so let's give this a go
so loading in our training data
so hopefully this little ship will
follow the first asteroid and just keep
shooting at it until it's destroyed and
then it'll follow the next asteroid
yeah that's what it's doing
bang bang bang bang yeah just keep
shooting it's pretty effective actually
even when we're not targeting the
closest asteroid
so how about we program it to target the
closest one so head back into our
update function
so compute
the closest
asteroid
uh we'll need to keep track of the index
so how about we just make a variable
called say let c equal zero so this will
be the closest index in the array
uh we'll need to calculate the distance
we'll need to keep track of the closest
distance so we'll call that say distance
zero
will equal we have a method for
calculating distance distance between
points
uh the x1 y1 so the ship dot x
the ship dot y
uh the x2 will be the asteroid so roids
zero
dot x
roids
zero
dot y
and then we'll have to loop through each
of the asteroids so four let i equal
zero
actually let it equal one because we've
already calculated the first one
distance zero there so let i equal one
i is less than the asteroid's length
roid's dot length
i plus plus
and then we need to
get the distance of the current
asteroids we'll call that distance one
distance 1 will equal distance it's
going to be the same thing really
basically
except that we'll be using i instead of
0 here
and then all we have to do is do a check
to see if that one's that distance is
smaller less than the uh
current distance so if distance
1 is less than distance 0
that means it's smaller doesn't it so
we'll replace distance zero distance
zero will equal
distance one and we'll want to update
our
uh index closest index so c will equal i
so all we have to do is put that c
into where we've got zero down here so
ax will equal right c
a y will equal roid c dot y
and let's give that a go
loading our training data again
so hopefully it'll turn towards the
closest asteroid and shoot at it
there's only one asteroid here
that reminds me we should probably up
the number of starting asteroids back to
its original so roid numb will equal
three
save that
load it one more time
okay is it turning towards the closest
one
certainly it's not shooting the furthest
away ones anyway
so it's doing fairly well
it's not perfect
i'm thinking we could probably make this
a little bit better i think we'll do one
more one more update to our little
function here
how about we pass some more information
because we're only passing kind of the
bare minimum we're passing the asteroid
location
and the ship's angle i think we can do
better than that so head back into the
code
to head down to where we set up our
training data
what else could we pass
well we are calculating the angle to the
asteroid
how about we stick that in as one of our
inputs so we'll put it in after our
asteroid position
we'll have to bump up the number of
inputs to four
so down here our normalize input we'll
have to add an extra
here we'll pull that roid
angle right angle it'll be our third one
it's going to be very similar to how we
do the ship angle we'll just put right
angle there roid angle
ship angle update its index to three
uh wherever we call this what that fix
it up which is just down in our update
function
so we'll have to pass an angle in there
and we'll have to calculate that angle
so what do we need to know for that we
need to know the ship's x and y
coordinates so s x will equal ship dot x
you didn't you don't really have to do
this but this is to make it clear
ships y
ship dot y
and let angle equal
angle
angle to point now what does it take x
and y which is the ship x and the ship y
it takes the bearing which is the ship's
angle
the target x which is the asteroids x
and the asteroids y
and then we'll pass that in through our
feed forward function to get a
prediction i don't think we need to log
this anymore with this prediction
uh let's take a look at that
okay import um
doing our training
wow those errors are very small so
they're e to the minus 15 there's a
couple of zeros in there so they're very
precise
so looking at our ship's performance
yeah he's doing pretty well
he should be doing very well the only
issue i can see is that see that jitter
there that's because we're only making a
prediction every frame if we increased
our frame rate we could probably make
that better
and also if we reduced our threshold to
decide whether we turn or not so let's
do that now
so head up to the top
we'll reduce our output threshold to say
0.05
that means in order to make a right turn
we'd have to have 0.95 confidence
let's increase the rate of fire for fun
to say to 15 shots per second
and we want to increase our frame rate
to say 60.
let's give that a go eh
okay just loading our
training data
the errors are very very good
and here he goes
wow
we've created a monster
so with his increased
with his increased rate of fire
and the faster frame rate
he can make more accurate more subtle
turns
and really pepper those asteroids with
laser fire
cool
so anyway that's the end of our tutorial
series so we've gone from using matrix
maths
to creating a neural network only a
fairly simple neural network but a
neural network all the same i hope
you've learned a lot from this i
certainly have because i had to do a lot
of research in figuring all this stuff
out
but in the end we managed to use our
network to control a simple game such as
asteroids and this is the result
i wonder how far he could go how far
could this uh what level could this
ship reach
well anyway feel free to download the
code and try it out for yourself update
the parameters have a play and learn
something new
until next time
talk to you then bye
[Applause]