can i teach an ai model using tensorflow object detection api
how to identify the subject of a painting
the only way to find out is by trying and i think you may be surprised by the outcome of
this project but let's narrow down this problem first we want to identify if a painting is
about the nativity or not which is simple binary classification problem during the last
2000 years since jesus birth countless artists like michelangelo leonardo da vinci caravaggio
just to name a few have created paintings about the birth of jesus so there should be plenty of
training data for us to collect just in case you don't know what a nativity painting is let me
explain it a nativity painting is a painting where the subject is the birth of jesus christ when
trying to identify a painting about a nativity you almost always see baby jesus in the center
and almost always you'll see his mother mary and his father joseph around him sometimes
animals will also be surrounding him because he was born in a stable or you'll see him in
a cave often he'll be outdoors and surrounded by shepherds i hope you can agree with me there is no
way we can write any python code that can take into account all these different variations so
this should be a perfect project for a neural network without further ado let's get started
before i start i just wanted to let you know that i created this notebook with code from
this tutorial here on image classification with tensorflow this is not using binary classification
it's using categorical uh classification so it's slightly different but i use a lot of the code
here to also uh create my own notebook so i think that's important to mention so let's get started
so the first thing i'm going to do i'm going to import all the tensorflow and any libraries
that we're going to need today so you can find the notebook that i'm going to run today in the github
i'll put the link in the description also you'll find the test data the data is set that i created
as a csv file in here so three files you need nativity data set other data set and test data
set so download them as well because you're going to need those files let's do that now we first
upload the csv files with the urls for all the images that we need to download okay so now we can
go through the notebook first we import all the tensorflow libraries any other guys that we need
the ones i wanted to highlight we're going to use keras which is a library that makes tensorflow a
lot easier to work with and we're going to use pandas pandas is used for anything to do with uh
like parsing csvs creating tables and data frames and so on
and yeah that's those are the two uh libraries i wanted to highlight in order to train an image
classifier we need to have a training image data set a validation data set and a test data set
since we are training a binary image classifier we will have images of only two classes the
nativity or any other images that we have during the training of our model will use
the training data set to teach the model how to classify painting either nativity painting or not
at the end of each training cycle epoch we will use the validation data set to
score how well the model is doing by calculating the accuracy and the loss
the accuracy measures how many times our model got the correct answer and the higher is the better
and the loss measures the delta i.e the difference between the predicted value and the actual vowel
and the lower is the better it is important that the validation data set
is separate from the training data set because the ai model is very good at
cheating and if you don't separate the two the model will simply memorize the answers
instead of learning the increasing characteristics of what we are trying to teach you
at the end of the training we'll also use a separate test data set from the training
data set and the validation data set and to do an independent benchmark of the model performance and
you will notice that we are downloading uh three files the nativity data set which contains all the
nativity paintings only the other data set which contains any paintings except nativity paintings
and then we have a test data set which contains both and they are labeled if you look at these
files we have urls here a image url a web page url so the image url is what we actually care about
the webpage url is just telling us where this image was taken from this is quite important
so that you know you keep track of where your images come from later on they might be useful
to know so the same thing for the other data set it's just a bunch of urls and then you know we
have here the actual image on the left and then we have the webpage url where we took the image from
and then we have tested the set which is slightly different where i actually mix the two labels the
two labels that we have is nativity and others and you'll see here that there is the same structure
image url is the actual image then you have webpage url and then you have a label here
okay this is how it was done so we're going to be downloading all the images inside these csv files
but wait a moment did i not just say that the training data set should be separate from the
validation data set so why do we keep them in the same files it's true but we are also doing some
data exploration we're experimenting and it is a good thing to have some flexibility typically
you are advised to have eighty percent of training data and twenty percent of
validation data but this is not a hard and fast rule um we might want to change this uh
these percentages to see what gives us the best results as part of our experimentation
this is also known as hyperparameter tuning on the other hand the test data set should
be fixed so we can compare different models with different architectures in a consistent way here
i'm defining some utility methods that we're going to use to download the images in our
data sets so we have here get file name from url and the purpose of this method is just to extract
the file name from the url itself and actually remove some illegal characters if there's any it's
a very simple method i've only created the code that is needed for the type of filenames i have if
you have other more strange file names you might need to put more rules here but i kept it simple
for now then i have here this method download image this download image is a method that goes
through basically downloads the image url into a destination folder something worth mentioning here
is that we are sleeping we have a one second sleep time between downloads and the reason for having
this sweep time is to make sure that we don't hit the rate limit that for example uh wikimedia has
when you try to download images from them they are quite happy about you downloading images from them
because those images are free to use but they will they will control uh how fast you can download it
so that's why i have one second here so we have to be patient so i'm going to just run this cell so
now we can download images we need patience so i'm going to uh i have to wait and you'll probably see
uh this is going a lot faster than it actually was i'll see you in the future
so we've now downloaded the images that we needed and i have here another section of code and i
shall i'll just briefly explain why i'm doing this some of the images in our data set are over
80 megabytes in size and if i try to resize the image with python it will try to all the images
into memory and it will kind of blow up so not a great idea and that's why i'm actually
having a separate resize method here the reason i point this out only i'm only doing this here the
resizing because the images are too large because keras itself in the pre-processing pipeline allows
you to it will resize the images for you anyway first i'm going to install image make which is a
utility that allows me to resize images very easily and very fast and let's install so here
i have a resize images method i've created which does the resizing uh from one folder
to another so we're going to be you know our data set is here so we have here our data set
so that this is the training and validation data set and here is below you can see the test data
set so the the resize images is going to copy the images from this folder into another folder
which is going to be our destination folder and it will be resized so it's all this method and now
here we resize the images so all the images have been resized and all the images are here in this
folder the resize folder and the resize folder has the training and validation data set here
and this is the test data set and you can see the folders they have two labels here
nativity and others and the same for the test data set now because we are going to be using
uh keras uh this method called image data set from directory uh this will
this method actually loads the all the images from the directory into memory into a
basically a tf.data.dataset because we're using a binary cross entropy for our model we need to make
sure the labels are either a 0 or a 1. so we need to map nativity so this name here needs to become
a one and this name here needs to become a zero so that's all we need to do so i've just written
very simple code nothing special just i'm just renaming the folders as you can see here and here
so i'm just going to run that we should see that now the folders are renamed the next step is let's
just do a count of all the images that we have so we have 454 images that's for the resize data set
so let's take a look at some of the images that we have in in that data set you can see here
this is a image from the nativity data set and another one here clearly nativity and then some
other random paintings that we have in others dataset well this one looks almost like this
almost looks like a nativity painting but it's not um so as you can see it's a bit tricky sometimes
to figure out even for a human being imagine our ai model okay there's another one here this is a a
a straightforward uh image to classify for us okay so now we have here some parameters okay
so the batch size that we're going to use is uh 32 the image height and the image width i'm going to
go with 300 and i have to say that the the resolution of your image can affect the results
uh before i was trying with 150 pixels and i was getting worse results than what i get with
300 pixels so i'm going to start with the this optimal size uh maybe i can still go up further so
it's quite possible that the higher the resolution is the the better the results
are but it's not straightforward so anyway i have good enough results for 300 pixels by 300 pixels
so i'm going to uh run with that for now okay here here we have we're going to load
our images from the training data set into a an object well this is not exactly it's not going
to hold everything all at once it's just going to create a reference to these images and it's going
to also allow us to split so here this is uh where we actually decide to split the training data from
the validation data so here you'll notice this parameter called validation split so here we are
saying we want 0.2 uh 20 is going to be test data sorry 20 is going to be validation data
okay 80 is going to be training data and then we use a seed here this number is important because
we're going to have to call this method again for the validation data set so this is a trained data
set and then the validation data set niche uses the the same seed number so it gets the
reminder of the images if you have different seeds then you're going to be mixing training
images with validation images and that's not a good thing as we know our model is very good at
basically cheating really and notice here also this parameter label mode this is important that
you have it should be labeled most set to binary for binary classification so let's
run this you can see 452 files are belong to to the two classes that we have and it's using 362
images for training and when you we run this one it's going to use the reminder for
for the validation data set so we have now a third call here for test data directory the test data
set is completely independent so i don't need to have the same seed number so this is just going
to load all the test data that we have into this object and we have 37 images for our test data set
and let's just see what class names we have in our train data set we have zero and one one is
nativity and zero is others so the next step is just for us to visualize our images so we're
going to print nine images and using matplotlib we're going to bottom you can see here examples
of this is just a random painting and then we have nativity data set random painting it's random in a
way like know what we want to identify and then we have nativity here and activity here okay so this
is just a sample of what we have in our trained data set okay so here we're going to just take a
very quick look at our batch size so we've we've we've gone with 32 of batch size and remember we
use 300 pixels uh width and 300 pixels height and you can see here this is a tensor right
this is actually printing uh the value of the tensor so we have a tensor for the image batch
and you'll see the there's 32 items 32 images really and then we have 300 pixels by 300 pixels
and then we have three is basically rgb right so the colors they don't have three uh cells and then
for the labels it's just 32 labels and it's a single dimension so it's not a big deal okay so
we've run that okay so some of this code here is for um performance as you can see the idea is that
we are trying to load as many images in memory as possible at the same time so that it doesn't
go to disk all the time because whenever you doing the training it can be much slower if you you have
to go to to train uh to basically fetch the images from disk here we have a method that i've defined
to help us test the model against the test data set this will give us a very nice
visualization uh of all the 35 images that we have against the model to see if whether the model
gets gets it right or not this is an independent benchmark right from the other benchmark that
we're going to get which is normally done after training the model against the validation data
set so let's do this uh method okay so the next step this is uh called normalization as we know uh
rgb are in uh the zero to uh 255 range and this is not really great for a neural network because in
general you should keep your input values as small as possible um in fact the the neural networks
like values between zero and one and that's why we do this step here and you'll see it all the time
in all the different tensorflow tutorials you do you'll see uh this uh normalization so all we are
doing is dividing all the different colors by 255 so that they they fit between zero and one this is
good for the the gpu right okay keep the numbers like that so here it's an example of how you could
uh do the normalization uh in a different way so the first example here this is going to be part
of the model itself as one of the layers and you can see it uh in fact i can show it here