JASON MAYES: All right, it's time to follow along.
This time, you'll start from a blank canvas
and build everything up from scratch.
Head to the TensorFlow.js boilerplate project page
and fork a new copy that you'll use
to implement this project with.
Once you've got a copy, open up the index.html
and replace the code contained within the body tag
with what's shown on the next slides.
OK, so let's go through the changes
to the code contained within the body tag.
First, you'll set some descriptive text
with a header and paragraph.
Next, you'll define a section with class of box.
This will contain a heading, some more descriptive text,
and a canvas element whose width and height
are set to 28 pixels to which you'll later
render images of the hand-drawn numbers
from the MNIST data set.
You'll then add a second section element,
also with class of box, and again, a header and paragraph
are added along with a final paragraph
with ID of prediction, which is where
you'll render the model's prediction to once it's
trained and used.
Finally, you can import the TensorFlow.js library
and the page's script.js just like before.
And make sure you add a type attribute
to set the module for script.js so that you can import other JS
modules.
Great.
Now you'll add some CSS to give that HTML some style.
Open up style.css and replace the contents
with what's shown on the next slide.
As before, CSS was a prerequisite for the course
so I won't be going into the details of this code.
Essentially, though, these three columns
of CSS position and size the HTML elements
correctly to create two boxes that are side by side
and change the colors of correct and wrong predictions
so they're either green or red respectively.
OK, so now onto the JavaScript.
Time to import the MNIST training data.
Open up script.js and replace the contents
with what's shown on the next slides.
First, import for example training data
I've provided at the indicated URL.
It should be noted that this training data is already
in normalized form.
As you know that grayscale images like these
can only contain numbers between 0 and 255,
this is not an issue should you want
to normalize a new grayscale image later
that you draw yourself.
You can just divide all the values
by 255 to get a normalized value between 0 and 1.
Next, create some convenient references
to the inputs and outputs respectively as shown.
And then you need to shuffle the inputs and outputs
while maintaining their relative orders as the MNIST data you
just imported contains 10,000 examples,
but they're all in order.
What that means is that roughly the first 1,000 or so examples
are all images of 0s, the next 1,000 are all 1s, and so on.
Now, to avoid issues, you must ensure you shuffle the arrays
before training occurs.
Otherwise, you'll find that your validation results
will be pretty terrible.
But why is that?
Well, if you set aside 20% of a data,
and it all comes from the last 20% of the training data,
you will essentially drop all the 8s and 9s,
which will be very, very bad.
Thankfully, TensorFlow.js has a handy utility function
to shuffle arrays while maintaining relative ordering
of those arrays called tf.util.shuffleCombo,
as you can see here, to which you can pass the two
arrays you want to shuffle.
What this means is that if element 0 in array 1
moved to position 100, then element 0 of array 2
will also move to position 100 so the inputs will still
match the outputs--
very useful.
Finally here, you can convert the newly shuffled arrays
to tensors.
In this case, your inputs tensor will be 2D,
as grayscale image data is just a list of numbers in an array.
And you have many of those, so those are then
contained in array themselves.
Next, your output's tensor will be a one-dimensional tensor.
However, note here how you convert
the values to be one-hot encoded using the tf.oneHot function,
to which you pass the newly created one-dimensional tensor,
which must be of type int32.
The second parameter is how many possible classes there are--
in this case, 10, as there are 10 possible digits to predict.
Great stuff.
Now it's time to define a new model architecture.
So just like all the other examples so far,
you start with a tf.sequential model.
Next, you can add the input layer.
As you know you're dealing with images of size 28 by 28 pixels,
that's a total of 784 inputs, so the input shape is 784.
In this first input layer, you'll
set the number of neurons to be 32 with a activation of relu.
But why 32?
Well, it seemed to work well.
I first had 100, but noticed it converged to a minimum loss
very fast and didn't get much better.
So I figured I could probably safely drop it
down to a smaller number.
After some experimentation, 32 was a good compromise.
Remember, it's up to you as a machine
learning engineer to figure out the best values
to deploy to production that give you the best
results for your needs for your training data
while keeping things fast.
All right, back to the code.
And you'll now add a hidden layer with 16 neurons
also with a relu activation.
Again, this value I chose after trying what works well
through some experimentation.
Next, you can add your output layer,
which must have 10 neurons that represent each
of the 10 possible digits that the network can possibly
predict.
Notice here the activation is softmax
that you learned about in the prior section that
will ensure all the outputs add up to 1,
essentially giving you percentage confidences for what
each output is and what it thinks it saw.
Finally, you can print the model summary to the console,
and then go ahead and call the train function
that you'll define next.
At this point, if you check the output you just logged,
you see here that this model uses almost
26,000 trainable parameters.
You may want to come back to the code
later and see if you can find a smaller set of neurons
and connections that still performs well or close enough.
I've purposely left some room for improvement
here so you can try that out.
OK, so moving on to the code to train,
you're going to try a couple of new things out this time,
but you're a pro at this now, so you've got this.
In your train function, let's focus on model.compile as
highlighted.
This time, you're going to try a new optimizer called
adam instead of the stochastic gradient descent
algorithm you used before.
It should be noted that you could still
use gradient descent if you wanted,
but adam is actually pretty cool as it automatically
changes the learning rate over time, allowing your model
to converge to a better solution pretty quickly,
and is well-suited for image problems like this.
As with most things in this industry,
trying out different options and seeing
how they affect the training is all part of the fun,
so feel free to change this later
if you want to see how the performance differs
with this training data.
Next, you define the loss function.
As previously mentioned in the last section,
mean squared error loss is no longer
suitable for this classification problem,
and instead, you must use categoricalCrossentropy
instead.
With TensorFlow.js, changing that is as simple as changing
the string shown here--
easy.
No need to worry about actually implementing it yourself.
Finally, you can specify a metrics property
that contains an array of extra metrics you want to log.
As this is a classification problem,
you can also log accuracy, which is
a measure of how many of the images in the training
data are predicted correctly.
You want to see that the validation accuracy is going up
over time--
the higher the better-- showing that the model
can generalize well enough for unseen examples in the future.
OK, on to the actual training using model.fit.
Just like before, we pass your inputs tensor
and your output's sensor that you're trying to learn from.
First, set shuffle to be true so data is
shuffled each time as you go through it.
Next, as you have a lot of data, you can set aside 20% of it
for validation testing.
So set validationSplit to 0.2 to achieve this as shown.
Here, I've set the batch size to 512.
This seemed like a good number given
the amount of training data while still achieving
great accuracy on the training.
Experiment with playing with this number if you like.
Next, set the epochs to 50.
I actually started with 100, but realized
it converges pretty fast so reduced to 50 epochs instead.
If you copied all the other hyperparameter configurations
above, then this should work well
with all of those settings.
Finally, in this section, you can
specify your onEpochEnd callback and ask
it to call a function called logProgress
at the end of each epoch with the loss and accuracy scores
that you'll use later.
Great.
By this point in the code, training
will now be complete, so you can safely
dispose of the inputs and output's tensors
and then call the evaluate function
to test the model with some images.
So as you just saw, the train function
is pretty much the same as what you've been doing before.
The key changes here were the optimizer you used,
the loss function, and the extra metrics you're now able to log.
OK, so let's go to the code and add the log function
and evaluate stuff.
First, grab a reference to the element in your HTML
where you'll render predictions to.
You gave this element an ID of prediction in your HTML,
so you can use document.getElementById to grab
a reference to that and store the result in a constant called
prediction_element.
Now you can start to fill out the evaluate function.
First, grab a random image from your inputs array.
To do this, you can generate a random number
using Math.random, which will return
a number between 0 and 1.
You can then multiply this number by the inputs array
length, and finally, Math.floor that result
to round down the number to the nearest whole number.
You now have a random number you can
use that's going to select one of the elements
in the inputs array, so now let's use it.
First, wrap your next chunk of code in a tf.tidy,
as you'll be generating output tensors when
you make a prediction.
Next, you'll actually fetch a new input to use.
To do this, use your calculated offset number
from above to select an example image from the inputs array
and turn that into a tensor1d.
Remember, a single grayscale image
is just a list of numbers in a 1d array.
Now you can actually perform a prediction using model.predict,
but remember that it expects a batch of images as an input.
Here, you can call expandDims on the tensor1d created
to turn it into a tensor2d with just one value
to avoid any errors.
The results of the model prediction
will be stored in a variable called output.
And you can now print the output to the console for reference,
of course.
And if you run the code at this point,
you can see an answer like this on your console.
Can you guess what it predicted?
Note the 1e minus 7 value at the start.
This just represents a really tiny number,
so it uses mathematical notation to indicate
this is the value 1 divided by 10 7 times, which
just saves some space on the screen printout.
Hopefully you remember from the prior section
that the model's prediction of what number is represented
by the input image is simply the output with the highest value.
The highest number in this example
was in position 1 in the return tensor as highlighted.
Remember, arrays start at index 0,
so the model was telling you that it
thinks it saw a number 1.
As in this case, it just so happens that the array index
position directly correlates to the digits that they represent.
So now you've got a prediction for each of the digits,
you can return the index of the element in the output tensor
that has the highest value using a handy TensorFlow.js
function called argMax.
Note that you need to call squeeze on the first output
tensor in this case, as the output would be a 2D
tensor as it was technically a batch result.
The TensorFlow.js squeeze function
will essentially do the opposite of expandDims.
This will then allow you to take the index of the highest
number in 1D tensor instead.
Now, if you did not use the squeeze here,
you would need to explicitly set the axes of a tensor
to use argMax on.
If you prefer to do that, you can pass the axes
as a parameter if needed when calling argMax.
Now that you have an answer that contains
the position of the highest value,
you need to actually read this value.
As this is a tensor right now, you need to use the .array
function, which will give you the value for tensor contains
returned as an array.
Note that this call is asynchronous,
so you can use the then keyword to call
a function with the result when it's ready.
Once the data comes back, you can
set the inner text of the HTML elements for the prediction
with the position that was found from the argMax,
as this will be what the model thinks it's seen.
Note that the position in this case
directly relates to the number it represents, so you can just
print this returned index rather than looking up what position x
actually represents, which you may need
to do with other data sets.
So an argMax value of 0 represents it saw a 0.
A value of 7 represents a 7, and so on.
If you are not classifying numbers,
and instead you are classifying animals,
you need to have a lookup somewhere that tells you
that a dog is 0, and a cat is 1, and a chicken is 2 and so on,
so that you could print something
understandable to the human that's using the system.
OK, so now you can also set the class attribute of the HTML
element to be either class correct or class
wrong depending if the index value that was returned
is equal to the correct answer that
is stored in the outputs array at the same offset position.
This allows you to check the ground truth of this example
image as you're just using the images from the training
data in this case.
Finally for this section, dispose of your answers tensor
now that you've used it, and go ahead
and call a new function that will draw the chosen input
image that was just classified to the canvas in your HTML.
Here, you'll call a function called drawImage
that will be defined on the next slide.
All right, so the only thing you have to do now
is to draw the input image to your document
so you can view and see what you just classified.
First, grab a reference to your HTML canvas using
document.getElementById, and pass
it the ID of canvas, which should
be stored in a constant called uppercase CANVAS.
You can then also grab the 2D context for this canvas
and store that in a constant called uppercase
CTX, which will allow you to draw to the canvas later on.
Moving into the drawImage function,
you can first get the current canvas data
using CTX.getImageData.
You can pass to this function the starting position,
which here is the top left of the canvas,
so it's position 0, 0.
You then specify the width and height of the canvas
you want the sample.
Here, you want the full 28 by 28-pixel square,
which is the same size as your input images.
This data will be stored as a variable called imageData.
Next, you can create a loop that will iterate
through every pixel value in your image
that you want to render, which is
called digit in this function.
Remember, this is just a 1D array of numbers.
You can now set the image data for the current canvas image
pixel from the input image.
However, note that canvas works with red, green, blue, and
alpha color channels, and your MNIST data
was grayscale, so each pixel has four number values.
In this case, you can calculate the correct
offset in the image data array for the current pixel value
in your digit image that represents each color channel
respectively.
Here, you set the red, green, and blue channels
to simply be the input image's value multiplied by 255.
Remember, your input images were in normalized form
for training the model, so to get them
into regular color values, you need
to multiply back out by 255.
Finally, for the alpha transparency channel,
you can set this to be 255 for all pixels, which basically
represents not transparent.
Now that you've updated the image data array with new
values, you can paint them back to the canvas by calling
CTX.putImageData and passing to that function the image data
array you just updated along with the starting position
to draw the image to--
in this case 0, 0, which is the top left of the canvas.
Your page will now update with the new image data,
and you'll be able to see the image you just
classified in your document.
The last step here waits 2,000 milliseconds,
which is the same as 2 seconds, and then calls
the evaluate function to start the process off
all over again with a new random input image.
Now every 2 seconds, it will test a random image
from the input data that you already
used in your trained model, and you can see how it performs.
And with that, congratulations.
If you copied and ran the code, correctly, you
should now have a working model that
can classify random handwritten digits every 2 seconds as shown
on this slide.
Consider changing the code to allow
you to draw your own digit on a canvas
and pass that through your trained model instead
and see how that works.
If you do decide to try this, take
note of the following tips.
Remember to convert the canvas data to grayscale.
You should have 784 input values only for a single input image.
By default, canvas will give you values for RGB and A,
so you'll find you'll have four times as much data than you
need.
Secondly, don't forget to normalize those 784 values
by dividing them all by 255 before you send them
through your trained model.
And thirdly, ensure you have the same color conventions.
Canvas background should be black,
and any drawn digit should be white,
as this is the data your model was trained on,
and you must do the same thing for new examples too.
Next, if you head to the console after training
and check the validation accuracy,
you can see in my case, it was getting
0.93, which represents 93% of the time,
it was correct on completely unseen data.
Now, that's pretty amazing, and if you check the examples
it gets wrong, even you as a human
might struggle to identify some of those.
I highly recommend experimenting with different model
hyperparameters to see if you can beat
the accuracy I'm getting here.
Can you get above 95% for your validation accuracy?
Share your results in the forum.
Finally, if for some reason you're stuck,
check out my completed working code
using the link shown on this slide
to see if you missed anything.
All right, in the next section, you'll
go one step further to learn about a more advanced technique
known as Convolutional Neural Networks, which
are even better than multilayer perceptrons for image
classification, especially when dealing
with more complex imagery than hand-drawn digits.
See you there.
[MUSIC PLAYING]