LAURENCE MORONEY: Hi, and welcome back
to Machine Learning Foundations for Google developers.
I'm Laurence Moroney, your host.
And today, we're going to look at how
to use convolutional neural networks to classify
complex features.
In the last video, you took what you had learned about CNNS.
And you saw how to improve the fashion classifier that you
had created much earlier on.
You had an exercise to apply that learning to handwriting.
So before we get started, let's look at the answer
to that exercise.
So the last exercise had you looking at using mnist
and trying to build a convolutional neural network
that could classify mnist.
For bonus points, you could use the callback
that you learned about earlier so
that once it reached a certain accuracy,
you would cancel training.
And here's the solution.
It's very similar to the fashion mnist.
And don't forget you have to reshape your training
images and your test images to add this extra dimension so
that your input shape can be 28 by 28 by 1.
For fun I've just done a single convolutional layer.
And hopefully, it'll run a little bit faster.
Let's say if I train it for 100 epochs, what would happen?
Now important thing to note when you're
doing this with convolutional neural networks
is to make sure that you're runtime type is actually
GPU before you start running.
So I'm going to do that.
And now I'm going to run the code.
It will take a minute to allocate the GPU.
But there it goes.
It's connected.
And now the code is running.
It's going to start by downloading the data.
And let's see how many epochs we actually need.
I've set it to train for 100.
But our callback is going to stop training
once it reaches 99.8 accuracy.
And here we can see after nine epochs,
it hit 99.81 percent accuracy.
So the callback was hit.
And the training was canceled.
So here's the source code.
You can take a look at it for yourself.
I've given the URL in the slides.
And have fun playing with it.
That wasn't so bad, was it?
You were able to improve the performance of the computer
vision simply by using convolutions
to try to spot features in the image
and then match those features to labels,
instead of just doing raw pixels and hoping for the best.
That technique should, in theory,
work for images that are far more complex than the fashion
ones.
And in the fashion ones, there was just one subject.
And it was centered and zoomed in.
So in this video, we'll see if we can apply that
to other images like these.
This is a data set of horses and humans
with a number of pictures of different horses
in different positions, as well as different, very diverse
humans also in different poses.
So for example, consider the horse in the top middle.
You can only see three of its legs.
In the one at the bottom, you can only see two.
The men and women are also in different poses.
And some have parts of their body obscured.
Like the woman with the red dress is actually
cut off at the knees.
As you can hopefully see, this is a far more challenging
problem than we had with fashion mnist and handwriting digits,
where every item was posed similarly.
The first thing that we need to do
before we start coding our network
is to have an easy way to label these images,
where we can tell the computer which ones are horses
and which ones are humans.
One way to do this that TensorFlow
supports to make your life easier
is to use sub directories.
So if I have a master directory of images,
and I then subdivide that into training and validation images,
and the training directory contains
sub directories called horses and humans--
each containing the appropriate images
with horses in the horses directory
and humans in the humans directory--
and similarly, if I have a validation directory containing
horses and humans in sub folders,
I now have a fully labeled set of images
for training and testing.
With TensorFlow, I can pass these directories
to something called a generator.
And it will auto label the images
based on the directory name.
This labeling is achieved using an image data generator
in the Keras libraries.
You import it like this.
You can then use this to do some transforms
on the image, such as normalizing them.
Now this is a very simple normalization where I just
divide each channel by 255.
There's better ways of doing that.
But I'll keep it simple for now.
You then create a generator for the images
by flowing them out of the directory by calling
the flow from directory method.
You specify the directory that contains the label sub
directories.
So for example, in this case, we're training.
So this will be the training directory
that contains the horses and humans sub directories.
It's a common area to use one of those instead.
Make sure to use the parent directory.
You need to specify the size of the images
that the generator will provide to the model a little later.
Remember when, we did the fashion images,
they were all 28 by 28.
When using real world images, you
aren't guaranteed to have them all the same size.
So thus, when flowing them from the directory
as well as rescaling, it's a good idea to resize them, too.
You can specify the batch size for training.
So for example, in this case, they'll
be taken from the directory 128 at a time in order
to be fed into the neural network.
And finally, there's the class mode.
Keep an eye on this, as it's an easy source of bugs.
If you only have two classes like we do here,
keep this as binary.
If you have more classes, it should be categorical.
For your validation data set, you
do exactly the same, except that you
create a validation generator and point it
at a validation directory.
These two generators now provide the images
that your model can use for training and validation.
So now it's time to define your model architecture that
can use these.
And later, we'll see how you can use them
when you're fitting your images to your labels.
Here's the code for simple CNN that can classify the horses
and humans images.
Now this should look a little familiar by now.
First is a few stacked convolutional layers,
where every convolutional layer is
followed by a max pooling one, as we saw on the last video.
The number of convolutions in each layer is purely arbitrary.
You can experiment to get the best results.
I've done it here by increasing the number of filters
as the image size decreases.
Remember that the pooling layer is
quarter the size of your image.
So at the smaller images, I'm trying more filters.
But you're free to experiment.
Remember the input shape, though.
That's super important.
Here, we're telling the initial layer
to expect to be fed data in a 300 by 300 by 3 format.
So each image is 300 by 300 pixels.
And there are three bytes per pixel.
This needs to match the size that you
specified in the generator a little earlier.
Finally is your output layer.
The number of neurons should match the number of classes
that you have.
There is one exception.
With a binary classifier like this one,
you can get away with only one neuron and a sigmoid activation
function.
And this pushes the value towards 0 for one class
and towards 1 for the other class.
If we look at our model architecture,
we'll see something like this.
And the journey of the image through the layers is apparent.
It starts 300 by 300, loses a pixel border
to become to 298 by 298, gets halved in each dimension
by the pooling to become 149 by 149,
and it loses another pixel border, et cetera, et cetera.
Overall, this network will need to learn
about 40 million parameters.
So training might take a little while.
When you compile your model, you specify a loss function
and an optimizer.
In this case, we're going to use a loss function called
binary cross entropy.
And this is a common one when there's
a binary classification.
We'll also use an optimizer called RMS prop, which
is able to accept parameters for something called the learning
rate.
Start with 0.001 like this.
And you can tweak it as you go.
And we'll capture the accuracy matrix while we're training.
The learning rate parameter defines
how the mathematical functions in the transformer
can learn using something called gradient descent.
It's a little bit beyond this video
to go into that in detail.
But if you want to learn more, there's
a great video from Andrew Ng at this URL.
Now it's time to do the training.
And if you're familiar with model.fit,
which we saw previously, where we specified
the data and the labels when using a generator,
all you have to do is specify the generator.
And it will infer the labels from the provided sub
directories.
And you can see that here.
My training data.
And that has the images and the labels
that are provided by the training generator.
I'll just train it for a short time, 15 epochs.
And you'll see how well this model can perform.
The validation data will come from the validation generator.
And in the same way as the training,
it will get both the images and the labels.
The verbose parameter just tells TensorFlow how much
detail to report per epoch.
If you remember back when we specified the training
generator and we defined batch sizes
to flow the images from the directory.
You can also specify the batches for training.
And you'll find that training will be drastically faster.
It takes a bit of experimentation
to get the right combination.
But when I didn't use any, using Colab, even with a GPU,
it was taking many minutes per epoch.
However, if I specify the number of steps to use when training,
it massively increases performance
so that each epoch was taking less than 10 seconds.
So watch out for this while you're training.
Here's the code with the step set.
The rule of thumb here is to consider
the amount of items in the data set
and then divide it by the batch size.
If you remember earlier, we had 128 batch size.
And there's a little over 1,024 items in the data set.
So I chose a step size of eight.
After 15 epochs, this model should reach close to 100%
accuracy on the training set and about 85%
on the validation set.
This is called overfitting.
And it's a common error in neural networks
that can lead you to a false sense of security.
What has happened is that the network
got to be really, really good, almost perfect at classifying
data that it has already seen.
And that's the training data.
But it's not quite as good at understanding data
that it hadn't previously seen, such as the stuff
in the validation data set.
It's a little bit like if the only shoes you had ever seen
in your life were hiking boots.
Then, you may not recognize a high heel as a shoe.
You've overfit yourself into thinking that all shoes look
like hiking boots.
There are techniques to avoid this in neural networks.
And we'll look at some of them soon.
But before that, let's just explore
how to use the network that you've just
created to classify images.
Here's the code to upload an image to Colab
and have it use the model to predict
the contents of the image.
This works in Colab only.
And you import the files library to use it.
And to get a list of uploaded files
by calling files to upload like this.
Then, for each file that's uploaded,
you have to convert it to be 300 by 300, turn it into an array,
and call expand_dims to ensure that it's
a three-dimensional array with the third element
being the color depth that we saw when
we were defining the model.
Once you have your image or images in a list,
you can then call model.predict to get the results back.
If the value is greater than 0.5, it's a human.
Otherwise, it's a horse.
Now that we've gone through everything,
let's take a look at a hands-on lab where you
can build this for yourself.
I'll run through it first.
And then I'll provide a URL where you can try it out
for yourself.
Let's start looking at using convolutions
with complex images.
And in this case, we're going to use
the horses or humans data set that we spoke about
during the lessons.
This download will give you the horses or humans training set.
And this one will be the validation set.
They're separate zips.
So you can download and have a play with them.
This following code will then use Python's operating system
libraries just to unzip that and put it
into our required directories.
Here, I'm going to specify what the training directory is
for horses and humans and what the validation directory is
for horses or humans, taking a look at slash TMP
horse or human horses.
We can actually look at the files in Colab.
It's quite nice.
You can look at the file system.
And we can see that we have a horses or humans directory
with horses and humans.
And we have a validation horse or human directory
with horses and humans.
If we list them, we can start seeing some
of the files that are in there.
So within the training horse directory,
we have files like horse48-7, horse21-1, et cetera.
And within the training human directory,
we have images like human12 or human15-18.
We can see the total numbers that we have.
So training horse images, this 500.
Training human images, this 527.
We can plot some of these to take
a look at what they look like.
And here's a bunch of eight horses and eight humans.
And I've designed this data set to have humans of various skin
colors, hair length, dress, pose, all that type of thing.
And you can begin to see some of the diversity
that there, though these two images are very similar.
And the similar thing with horses.
Different colors, different shades of horse,
different pose, different backgrounds,
that kind of thing.
The idea is to try and extract features
as much as possible from these.
And it's the feature that tells us what makes a horse a horse
and what makes a human a human.
Now I'm going to take a look at building my model.
I'm going to run this code and import TensorFlow.
I'll print out the TensorFlow version.
We see we're on 2.2 release candidate 3.
And if you're watching this later,
you'll see a later version.
I'm going to create my model.
I can do a model.summary so we can see our neural network.
And we can see the journey of the image through the network.
I'm going to use RMSprop root mean square propagation
as my optimizer and set the learning rate to be 0.001.
And now, I need to create my generators that
are going to flow the images from the directories
that we specified earlier on.
I can see that my training generator found 1,027 images
belonging to two classes.
My validation generator, 256 images
belonging to two classes.
So now, all we have to do is model fit.
I'm just going to run it for 15 epochs just
to see what it looks like.
So now, at the end of 15 epochs, we
can see that we actually reached a 100% accuracy on the training
set and about 78% accuracy on the validation set.
What's happening here is something called overfitting,
where the network has become too specialized to be able to spot
what's in the training set.
And it's not general enough to spot other things.
If you notice, on the validation set,
the loss is actually increasing per epoch, increasing
quite sharply, in fact.
And as a result, that's a great sign
that shows your network is overfitting.
In later lessons, we'll talk about techniques
to avoid overfitting like this.
But this can lead you to a false sense of security
that your network is actually 100% accurate where,
according to the validation set, it's more like 80% accurate.
But let's try it anyway and see what happens.
One of the things that's kind of interesting about this data set
is that if you notice when you're looking at the images,
they're all actually CGI.
But because they're CGI images of horses and humans,
they still have features that are horses and human features.
And as a result, we should be able to try them
against real images.
So if I run this piece of code, it's
going to ask me to choose some files.
So I'll pick files.
And I have some pictures on my desktop,
for example, this one here.
From the file name, you can tell it's a horse.
It will upload that.
And it will classify it.
It actually classifies it correctly as a horse.
Another thing that's pretty cool that you can do with this code
is you can actually upload multiple images
and have them all classified.
So I'm going to pick all of these.
The first three are horses with horse in the name.
The next three are humans.
Two of them are female.
And one of them is male.
Let's take a look at what we get.
So here's all the images being uploaded.
And now we can see.
So horse is a horse.
Horse is a horse.
Horse is a horse, which is great.
This image, beautiful JPEG, is a horse.
So it's miscategorized a human as a horse.
And these two are correctly categorizing human.
So we could take a look at what's
in that image, the beautiful 1274.
And I got these images from Pixabay.com.
But this image, for example, ended up
being classified as a horse.
Now you and I can see that it clearly isn't a horse.
It's a human.
But what about the features?
What is it about this image that made the network think
it was a horse?
This is an example of the overfitting
that was actually happening.
It may be her long hair.
It's one of those things that, as you're
working through your images, and as you're
working through convolutional neural networks,
that you'll see if you can fix.
But for now, it's done a pretty good job.
And give it a try for yourself.
Maybe change the model architecture.
And later on, you'll learn about something
called image augmentation that may be
able to help with fixing this.
Also, as you walk through the lab,
you'll see these intermediate representations
that you can have a play with.
So you can see what the image looked like as it was traveling
through the network.
I'll pick one at random.
And it will show you.
Like here's one of a man.
And we can see features beginning
to be extracted from that.
Have fun with it.
And I'd love to see what you come up with.
Now you can try it for yourself if you like.
Pause the video.
And then go to this URL.
When you're done, come back.
And then, you can try the exercise.
As always, I'll have the answer to the exercise
in the next video.
So don't forget to hit that Subscribe button.
And I'll see you soon.
[MUSIC PLAYING]