LAURENCE MORONEY: Hi, and welcome back
to Machine Learning Foundations for Google developers.
I'm Laurence Moroney, and I'll be your host and guide.
In the last episode, you took your first steps
into computer vision, and you built
a neural network that could recognize items of clothing.
You then did an exercise with a similarly sized data
set but with different content.
And that was handwriting recognition.
So before we get onto the new stuff,
let's take a look at the answer to the exercise that you did.
So let's take a look at the answer to exercise 2.
The code is here.
One of the things I've done is to implement a callback.
And I've done that by creating a class called myCallback.
And in this class, I defined on_epoch_end,
and this takes self, epoch, and logs.
From the logs I can get the accuracy.
And if the accuracy, for example, is greater than 0.99,
I can say print, we've reached 99% accuracy,
so the training will be canceled.
So now for mnist, I can just say mnist
is tf.keras.datasets.mnist.
I can load and normalize my data as before.
I can create an instance of my callback
that I'm just calling callbacks.
And then when I'm training, I can
add this extra parameter called callbacks and say
callbacks equals this instance of myCallback.
So if I were to run this, I'll get this warning.
But let's see it running.
And here you see after five epochs,
the accuracy was 0.9917, which is greater than 99%.
So it cancels training after only 5 of the 10 epochs.
And it was able to hit that 99% accuracy.
So that's the answer to exercise 2.
Hope you were able to get it.
OK.
Now that you've looked at both fashion and handwriting
examples for computer vision, you've
probably noticed a big limitation for computer vision
with these types of example.
And that was that you could only have one item in the picture,
and then it had to be centered, and it to be well-defined.
So shoes had to face left, for example.
A neural network type called a convolutional neural network
can help here.
It'll take us a little time to build up to it.
And in this video, we'll talk all about what convolutions are
and how they can be used in combination with something
called pooling to help a computer understand
the contents of an image.
It's going to take a little while to put it all together.
But let's start with just understanding
what a convolution is before you can start using one.
The idea behind the convolution is
very similar to image processing with filters.
If you use something like Photoshop before,
it has filters to do things like sharpening an image
or adding motion blur.
So while the effect is complex, the process behind it
is quite simple.
So let's take a look at it in action.
On the left here, I have an image
of a boot from the fashion MNIST dataset.
On the right, I have a representation of nine
of the pixels in the image.
I'm going to call the center one my current pixel,
and all of the others are its neighbors.
Here I define a filter, which is a set of values
in the same shape as my pixel and its neighbors.
So for example, if I had one neighbor
in each direction for a 3-by-3 grid of pixels,
I'd have a 3-by-3 grid in my filter, too.
Each value in the filter can be called a weight.
So to calculate a new value for my pixel, all I have to do
is multiply each neighbor by their weight, my current pixel
by its weight, and then add them all up.
And I'm going to do this process for every pixel in the image.
The result will be a transformed image.
So for example, if you look at the picture on the left
here, if I apply the filter in the middle to it,
I'll get the picture on the right.
This filter has led to a huge emphasis on vertical lines.
They really pop now.
And similarly, this filter leads to an emphasis
on horizontal lines.
So you might be wondering at this point, what
does this have to do with computer vision?
Ultimately, the goal of trying to understand what an item is
isn't just matching the raw pixels to labels
like we did with the fashion and handwriting examples.
But what if we could extract features from the image
instead, and then when an image had this set of features,
it was this class, or if it had that set of features
it was that class?
And that's the heart of what convolutional neural networks
do.
They process the images down into raw features,
and then they find sets of features
that will match the label.
And they do this using filters.
And just like neurons learned weights and biases
to add up to what we needed, convolutions
will learn the appropriate filters
through an initial randomization,
and then using the loss function and optimizer to tweak them
for better results.
So for example, take a look at these images.
And you can see how they've been transformed by filters.
And these filters learned how to isolate and abstract features
within them.
You can see, for example, that something
like our horizontal line detector
was able to detect a commonality in these images.
And they're all shoes.
And that line is the sole of the shoe.
But if you consider these images with the same filter,
they had different results.
So if a lot of pixels are lighting up
with the horizontal line filter, we
could kind of sort of describe that as a sole detector.
Now, you might hear that phrase a lot-- "detector."
But what it's referring to is ultimately
a filter that can extract a feature that can
be used to determine a class.
I first heard it in the cats versus dogs classifier, which
we'll look at in a later video, when somebody visualized
a floppy ear detector, which determined something
was a dog and not a cat.
One other thing that you might have noticed
in the previous slides was also that the resolution
of the images was decreasing.
This is achieved through something called a pooling.
The idea here is pretty simple.
If there is a way that we can extract the feature
while removing extraneous information,
we can actually learn much faster.
Now, let's take a look at how this works.
The concept is actually quite simple.
Imagine this is a chunk of our pixels.
To keep it simple, I'm just going
to have them as monochrome.
We then look at them in blocks of 2 by 2 like this,
and then we'll only keep the biggest value,
which in this case is 192.
We then repeat the process for the next 2-by-2,
and keep the biggest, which is 144;
and the next, which yields 255; and the next, which yields 168.
We then put these four results together
to yield a new 2-by-2 block.
And this contains what had been the largest
values in the 2-by-2 constituents
of the previous 4-by-4 block.
We've now reduced the size by 75%, and maybe-- just maybe--
we've kept the important information.
So let's see what this actually looks like.
Here is the image that we had filtered earlier
to detect vertical lines.
On the left is what it looks like before pooling.
And on the right is what it looks
like after pooling in the way that we just demonstrated.
Notice how the information hasn't just been maintained.
You could argue that's also been emphasized.
Pooling is an important way, also,
of reducing the amount of information
your model has to process.
If you think about it, say you want to learn 100 filters.
That means you would have to keep track
of 101 versions of your image--
the original plus the results of what it would look
like after the filters have been applied.
If you have thousands of images, you'll
soon start to eat memory.
And that's just one layer.
What if you had another layer beneath this
that also learned 100 filters?
That would mean each of your first
100 will also have 100 products from it,
giving you 10,000 images for every image you need to train.
Any technique that can reduce size
while keeping this information is obviously very valuable.
OK.
So now that we've seen what a convolution is,
and how it works, as well as how it
can go hand-in-hand with pooling,
let's take some time to see how to code for them.
Here's the URL of a Colab containing code
that can filter an image as well as do pooling.
You can play with the filter parameters
or try different effects, or just
use the ones that I've provided.
Pause the video now and give it a try.
So let's take a look at some convolutions in action.
And here's the lab that you'll be doing yourself
in a few moments.
You might remember some of the images from fashion MNIST.
But I want to talk about, for example,
if you were going to classify a shoe instead of the one
that you had in fashion MNIST.
One of the ways of doing this is with convolutions.
And in this lab, you'll just take a quick experiment
to see how the filters of convolutions work.
So we'll start by importing some of the required Python
libraries.
And one of the things that's built into these libraries
is this image called ascent.
So we can use the pyplot library to draw it
so we can see what it looks like.
And it's a person walking up some stairs.
I'm now going to just copy that into a numpy array
so that I can manipulate it.
And here I've defined a number of different filters.
Remember the 3-by-3 filters that we were showing?
I'm just implementing them as three arrays of three items.
So for example, we used this one filter here.
And one other thing that I've done
is added a weight to this so that one of the things
is that all the digits should add up to 1.
But if they had to add up to something more than that,
then you can multiply them out by a factor to normalize them.
So for example, if they added up to 10,
you could set the weight to 0.1 so the final result
would be normalized back to 1.
Now here's is just simply a loop going over the image
and multiplying out the relevant pixel
and its neighbors by the relevant item in the filter.
Once it's done that, we can plot it to see the result.
And here you can see the filter that I'm using
is really emphasizing the image's vertical lines.
For example, I could go and change
to a different filter like this one,
run this to set the filter, and then run the code again,
and now plot the image again.
And we'll see the one that really emphasizes
the horizontal lines.
So you can consider following filter values
and look at what their impact on the images
is, or you can experiment with your own.
As for pooling, we saw in the lesson
that pooling is taking groups of pixels
and just picking the biggest value.
So if we were to do a 2-by-2 pool, we'd take 2-by-2 blocks,
pick the biggest value, so a 4-by-4 block like this one
would become a 2-by-2 block.
We'd lose 3/4 of the information but hopefully
maintain what's in the image.
So for example, here I've implemented a simple pooling.
So if I'm to run this on the image that we've just seen,
it will get pooled to this.
If you look at the axis, the axis is now-- it's 256 by 256.
And if we look at our original image, it was 512 by 512.
This is the original image after the filter.
This is the pooled image after the filter.
We haven't really lost any information.
So again, have a play with it and try it for yourself.
Great.
So you've now had a look at how filters can convolve images,
providing the basis of convolutional layers,
as well as how pooling can reduce the image
size while maintaining the features of an image.
There's no assignment this time.
Yay.
So in the next video, you'll start implementing convolutions
and pooling in code, and you'll see
how these can improve the fashion MNIST classifier.
After that, you'll take on some more real-world and more
challenging images.
And you can learn how convolutions
can help you to classify them.
So don't forget to hit that Subscribe
button for more great content.
Thank you.
[MUSIC PLAYING]