Hi Everyone!
today we will see how to use machine learning databases
in particular, we will focus on MNIST
which consists of 70,000 images of handwritten digits
we will use a library called Pytorch to load MNIST
and then I will show you how to adjust its data
so we can feed it to a neural network
are you ready? let's do this!
we will begin by copying the imports from the description
and you can also find all the different installation commands there as well
so first thing's first - let's load our database!
we can do this by typing: datasets.MNIST()
and we will assign this expression to a variable called "training_data"
but that's not all!
we will, of course, need to add some arguments to these round brackets
the first one would be the root folder
where we'd like to store the images we're about to download
we will call this folder "data"
and next, we need to decide if we'd like to download the training data or the test data
now, the training data is very very big
it has 60,000 different images
while the test data is much smaller - it only has 10,000 images
now, judging by the name of this variable
we will of course load the training set
and we can do this by typing "train=True"
without the typo, of course
then we will need to download this dataset
and we will also set "download=True"
then lastly, we will add another argument called "transform"
and at first, we will set it to "None"
we will talk about it later on
and then right before we run this cell
I would like to actually print the length of this training data
now let's go ahead and run this cell with Shift+Enter
and we can see we have downloaded 60,000 images
great!
cool! now let's see how these images look like
we will first load one with training_data in the index of 8
and this will actually return 2 values
the first one would be the feature which is the image itself
and the second value would be the target or the label
now, in order to fetch the image only
we will focus on index 0
we can then assign it to "feature"
and we will do the exact same with our label
so we will copy this line of code
we will adjust index 0 to index 1
and we will adjust "feature" to "label"
now let's go ahead and print it!
so we will first print the label
we will then print the size of our feature
with feature.size
and lastly, we will print the feature itself by simply typing "feature"
let's go ahead and run it with Shift+Enter
and there you go!
we can see our label is a perfect match to our image
and we can also tell that each of our images would be 28 pixels by 28 pixels
if we then go ahead and change index 8 to index 432 for example
we re-run this cell
and we are fetching a brand new image with the exact same size
and we can load all kinds of different databases with Pytorch
and more specifically - Torchvision
For example, we could have used CIFAR-10
which is an RGB database of colourful images of animals and objects
CIFAR-10 only has 10 classes
while CIFAR-100 has 100 different classes
and you can find among them: frogs, birds, trucks, planes and what not
another cool database is Fashion MNIST
which consists of black and white images of clothing items
and we are also not limited to images only
we could have loaded a videos database
for example: HMDB51 that tracks human motion
we can then use this data to train our neural network about actions
so I'm actually going to include a link in the description of the video
with all the available databases that Pytorch has to offer
next, I wanted to talk about data transforms
now, do you guys remember how in that last tutorial -
we've said that slightly boosting brightness
or slightly rotating an image
will have a huge effect on its pixel values?
so that's exactly what we're tackling with transforms
we randomly zoom out, we randomly zoom in
we randomly crop or rotate the image
we are basically adding a lot of variation to the original image
to do this we will create a brand new variable above our training data
and we will call it "training_transforms"
we will assign it to transforms.Compose() with a capital "C"
and since we're looking for a few of these transformations
we will pass them in a list
now, the first transform in my case
would be transforms.RandomResizedCrop()
and then inside the round brackets
we specify the size of the image we would like to get in the end
now, in our case, we will leave it at 28 pixels wide and 28 pixels tall
because that's the original size
now, what this command will do is it will randomly zoom-out
randomly zoom-in and it will crop the image
to whichever size we specified in the round brackets
now the next transformation
would be transforms...
ugh... Google you are so annoying!!! holly smokes!!!
and this transformation would be Random Rotation
and then inside the round brackets instead of... ugh!!
instead of the size - we specify the angle
in my case, I would like to limit it
to a rotation of 45 degrees maximum
and then we can go ahead and copy
our "training_transforms" variable name
and instead of the "trasnforms=None"
we will specify "transforms=training_transforms"
we will then re-run this cell with Shift+Enter
and we will then re-run the next cell as well, also with Shift+Enter
and we get a very similar image -
but it's not the same!
because this time we have a slight rotation
and we are also chopping off a few pixels from the top
variation, at least when it comes to our training data -
is very important!
it's OK not to have transformation on the data you'll use for prediction
but the data you use for training
it has to be diverse!
cool! but we can't really feed this data into a neural network just yet!
and that's because neural networks, they don't deal directly with images
but rather with something called "Tensors"
that's why our very last transformation
would be "transforms.ToTensor"
we will then end this expression with an empty set of round brackets
and that's it! our transformations are complete!
however, if we were to deal with colourful RGB images
there are so many different things we could have added here
for example - we could have added random brightness
or if we were not dealing with digits -
we could have also randomly flipped our images horizontally
or randomly flip them vertically
there are so many available transformations
and I'm actually going to include a link in the description
so you guys can experiment a bit
I think it's really cool to try all kinds of different things
and see how they affect your images
now, when you do this, I highly recommend
to comment out this transformation into tensor
it will be much more convenient to view them as images
awesome! our data is now ready for training!
so in the next tutorial, I will show you how to load
a state of the art neural network
and how to adjust it for our needs
if you guys found this tutorial helpful,
please give it a like
maybe subscribe to my channel
turn on the notification bell
leave me a comment or share this tutorial
with many many people
thank you so much for watching and I will see you very soon
in a brand new tutorial