[MUSIC PLAYING]
JASON MAYES: OK.
It's time to put your knowledge into practice
and make your very first convolutional neural network.
Let's go.
For this project, you'll be training
a CNN to classify data from the fashion MNIST data set.
This is just the original MNIST data
set but for fashion items instead
of numbers, some of which are shown on the slide.
Your starting point will be a completed version
of the fashion MNIST demo that I made
that currently uses a regular multilayer perceptron
to classify the data, just like you learned about before.
Head to the project page shown here
and fork a new copy that you'll use to implement this project.
Let's quickly take a look at some
of the updated code in the starting template
versus the MNIST number classification example
you previously made.
First, it uses a new training data
set of fashion images instead of numbers.
But they're still 28 by 28 pixels in size and grayscale,
just like before.
There are 10 classes of fashion items,
so no changes to the output layer size either.
Next, the normalization function you created before is back,
as the training data for the fashion this data set
is not normalized by default unlike the MNIST numbers, which
were.
Also, notice that I've updated the normalization
function to be a bit easier to use with image data.
Here, you can just pass the min/max values directly
as numbers, as you know the images will always
be in the range of 0 to 255 when not normalized,
instead of having to use tensors when you call the function,
so the code is a little cleaner to read.
Next, after the model is trained and you
perform a classification, you'll now
need to convert the position of the highest output number
to a human-readable name of what the index position actually
represents.
Here, you can just use an array of strings
as shown that corresponds to the order of the one hot encodings
used.
So if the model had the highest number output position
at position zero on the outputs layer,
then you can just look up the element zero in this array
to find the human-readable name of "T-shirt" in this case.
Finally, I added some code to set
an interval between classifications
once the model has trained.
In the HTML shown at the top here,
a div is added containing a paragraph and an input range
slider that you can use to change
how fast you want to classify random images from the data
set.
All this code does is add an event listener for the change
in value to the input.
And when a change is detected, it
updates a variable called "interval"
that will be used by the set time-out in your classification
loop.
By default, the interval is two seconds,
but you can drag the slider to go much faster.
OK.
So now you're up to date with the small changes to the demo
code, it's time to actually edit this demo code
to create a CNN instead of using the multilayer perceptron
that it currently uses.
In this section, you'll modify parts of index.html,
so go ahead and open that.
First, update the header and paragraph tag text.
And then, replace the second section tag
with the content shown, so the wording is up to date
and relevant for this demo.
These are just textual updates and do not
affect any of the functionality of the demo.
Great.
From here, you'll be editing the existing
code found in script.js.
The first thing to do is to comment out
the call to the train function right
after the model definition code.
But why?
Well, training a CNN in the browser
is quite a resource-intensive task.
Even though the final model may run very fast in the browser,
the math required for the changing
of the weights and the filter values
is very high for the training phase.
In fact, typically, you do the training
for more advanced models like this
on the server-side via Node.js.
But for the purpose of learning, you
should try it in the browser first.
In this case, you don't want to start the training process
until everything is defined correctly,
as Glitch will refresh and try to run your code automatically
if you've got the preview window open.
OK.
Next, remove the current model definitions
to make room for your CNN code.
First, you'll call model.add with a tf.layers.conv2d layer.
This is the layer that will do the convolution step
that you just learned about.
Let's walk through some of the hyper parameters that
have been set.
As this is the first layer in the model,
you must define the input shape just
like you've done so before.
However, for a 2D convolutional layer like this,
expect the shape to be in the form of width, height,
and number of color channels.
Here, the input image is 28 by 28 pixels and is grayscale,
so has only one color channel.
If you're using an RGB image, this value would be set to 3.
Now you can set how many convolutional filters you want
to generate with this layer.
Just like the diagram at the end of the last section,
you'll choose 16 filters for this part.
Now, you need to define the size of each of those 16 filters.
Remember that ML folk like to call filters "kernels."
So here, you specify a property called kernelSize,
which for this demo is set to 3, meaning
you'll generate 16 three by three square filters
to pass over the input image.
If you wanted to use a rectangular filter,
you could do so by specifying an array containing the dimensions
for the width and height as needed.
Next, stride is set to 1, meaning
you're going to move and calculate
a filter value for every pixel in the input image.
And related to that, you can set a padding method
to deal with the pixels around the edges of the image.
Here, you use a property called "same"
that will ensure the missing values
are filled with zeros so the filter can still
work on those edge pixels.
Alternatively, you can specify a number to use directly instead.
Finally, just like you had in your multilayer perceptron,
you can also specify an activation function.
Here, you use the popular relu activation
to ensure non-linear relations can be
learned from the training data.
Great.
You can now add a tf.layers.maxPooling2d layer.
Here, you'll set its pool size to 2 and stride of 2,
just like you learned in the prior section.
This means your 28 by 28 pixel inputs
will be transformed into an output that's
14 by 14 pixels in size after best maxPooling layer.
All you need to do now is replicate this code
one more time, as shown.
Let's dive into the differences.
In this conv2d layer, you have double the number of filters--
in this case, 32 instead of 16.
But the other properties are actually exactly the same.
You then have a second maxPooling layer
after this second convolutional layer.
Also, as a side note, you may see in some CNN architectures
that they have multiple convolutional layers
before adding a maxPooling layer.
So do note that you can chain these layers together
in a very flexible manner.
But it can use a lot more processing power
to train the model if you end up doing that.
Finally, you can add code to run the outputs
for a regular multilayer perceptron.
Let's walk through it.
First, you must flatten the outputs
from the maxPooling layer in the previous step.
Remember, at this stage, the maxPool
has 32 seven-by-seven outputs.
And as you're aware, a multi-layer perceptron
needs a big, long list of numbers
to sample with its starting layer.
You can therefore use tf.layers.flatter
to do exactly that and convert those outputs
into a big, long list of numbers that
can be fed into your classification network.
Next, you can add a dense layer with 128 neurons
with relu activation, as you've seen
before, and then finally another dense layer of 10 neurons.
As this is the output layer, this
means you need to use a softmax activation
in the case of classification tasks like this.
Now, if you take a look at the console at this point,
after a short pause, you'll see the model summary
printed similar to this.
Notice how the total trainable parameters is over 200,000.
If you check your original multilayer perceptron example
for image classification that you just
copied at the start of his tutorial,
it had about 25,000 parameters.
So this is almost 10 times the size,
and it would take quite some time to train.
But thankfully, when you actually use the trained model,
it will still be really fast.
Next, let's head to the train function and adjust
a few parameters.
First, and most importantly, we need
to reshape the current training data inputs to be in a form
that the CNN input layer can digest.
Remember that the training data is just
an array of arrays containing a list of 784 numbers.
But your CNN wants inputs to be a batch of 28 by 28
by 1 in size.
To do this, you call .reshape on a tensor and then pass an array
of numbers specifying the new shape you want to resize it to.
The first value is the batch size,
so inputs.length is used here.
The second and third values represent the height and width.
And then, finally, the fourth value is for channel count.
Next, you can set the validation split
to be 0.15 to set aside 15% of the training data
to see how well the model does on unseen data.
And after experimenting with different values,
30 epochs turned out to be a good trade off between time
spent training and obtaining decent accuracy.
However, feel free to experiment with this value yourself.
Next, you can set the batch size to be 256.
Again, this seemed to work well for me,
but feel free to experiment with it.
OK.
So finally, let's head over and update the evaluate function.
First, you'll need to normalize the input you're about to use.
Here, you can call the normalize function
from the existing code with a tensor1d containing the input
image and pass it the known min/max values of zero and 255,
respectively.
As you're dealing with image data,
you know all the values in the training data
will be set in this range of zero to 255.
Next, you can call model.predict with this normalized input,
but remember to reshape it to be the form that the Sienna needs,
else you'll get an error.
And just like before, you can call
reshape on this tensor with a batch size of 1,
and then specify a 28-by-28 pixel image
with one color channel.
The rest of the code in this function
has not changed from what you've worked with before,
so it should look familiar.
OK.
So the only thing left to do now is
to uncomment the course of the train function
after the model definition code.
Then, open your console window in the browser,
and wait for it to finish training.
Now, this is going to take a really long time, much longer
than you're used to.
On my reasonably modern laptop, it takes around five minutes
to check that the epochs are incrementing
in the console output to ensure there's no errors
and nothing has crashed.
The screen recording shown here is recorded in real time
on my machine to give you a sense
of how long each epoch takes, which
for me is around 10 seconds.
So do be patient.
Also worth noting is that when you're training in the web
browser you must keep your web browser
window open and visible.
Otherwise it will not get the full system resources
and will take an extremely long time to do anything.
This is just how browsers work and has nothing
to do with TensorFlow.js.
Basically, any background tab or window gets less resources,
so make sure you keep it visible at all times while training.
OK.
It's time to wait for your model to finish training.
So hit the pause button on this video,
and come back once it's complete.
So if all went well, after about five minutes,
you should see it reach the final epoch with accuracy
similar to the one shown here.
For me, I got a test accuracy of 92%
and a validation accuracy of 87%--
not too bad, even though it's managed
to fit the training data as shown by the higher test
accuracy.
But given this image data is a lot more complicated
than the hand-drawn digits, this is still a great start.
Now, on the subject of overfitting,
there are special layers called dropout layers
that you can also look into on the TensorFlow.js API
that you can add to your model architecture
which can help reduce overfitting.
Essentially, these layers deactivate some fraction
of the outputs of the prior layer
randomly at training time by setting very selective values
to zero.
It should be noted this only happens during training.
When using the trained model, these layers
will just not do anything.
Now, this added noise, if you will, at training time,
can help, as only the features that
really matter all of the time will keep showing up
in the training data.
That means the network will learn
those key patterns instead of ones that
only show up occasionally.
Common values for dropout are 0.5 or 0.25,
representing 50% and 25% dropout respectively.
So finally here, you can see a screen capture of the completed
result as I change the speed trying
random images from the data set using the slider.
This is certainly much better than something
that most folk could have created
without using a machine learning model
and runs super fast once training completes.
Not to mention, you can reuse it for other image data classes,
too, without writing any more code.
Just change the training data that you feed into it.
So congratulations on creating your first CNN
with TensorFlow.js.
If you're having issues getting any of the code to work,
check out my completed example at the link shown here
to see if you copied anything incorrectly.
Now, there's certainly room for improvement.
State of the art models can get well over 95% accuracy,
with some at 99.5%, which is truly incredible.
But to do that, you'd probably want
to train the model with more parameters within Node.js
on the server with a decent graphics card
to leverage the special Cuda acceleration that's
available on the server-side to make training go much faster.
Now, thankfully, the code for Node
is essentially the same as what you just
learned, the only differences being small things--
like instead of using JS module imports for the training data,
as you've been doing so far, you probably
would load it from a local file system on the server, which
would be faster, especially for larger training data sets.
And on the subject of server-side execution
in Node.js for training, it should
be noted that there are two builds of TensorFlow.js
Node that you can install on your Linux server.
The default one uses the CPU known as tfjs-node.
And there's also one that can leverage
for GPU with Cuda acceleration known as tfjs-node-gpu.
However, to use the latter, you need
to install the correct drivers for your graphics
card along with Nvidia's Cuda library.
It should be noted that in order to use that,
you must be using an Nvidia graphics card on that server.
This is a somewhat involved process,
but there are plenty of tutorials online that
can show you how to do this.
In fact, check the official TensorFlow.js Node
documentation on the link provided for more information,
if you want to try it out.
Now, in the next chapter, you'll learn
all about transfer learning, which
can greatly improve the time needed to retrain a model.
You saw in this lesson how slow things can get for training.
But if you train the model for something similar,
you can just add some layers at the very end
and only train those instead of the whole model itself,
saving a lot of time while still getting great results.
In fact, this is what Teachable Machine does
that you saw at the beginning of the course, which also uses
CNNs to classify custom images, but as you saw, trains a lot
faster.
See you in the next session.
[MUSIC PLAYING]