In this machine learning tutorial, we
will look into what is K nearest
neighbor classification. We'll go over
some theory then we'll write
python code, we'll build a model
and then you will have an exercise to
work on.
This you are doing a classification for
iris flowers.
Here I have a picture of versicolor
flower, which is one of the three types
and based on a sepal width and height
you can actually figure out
which of the three flowers category it
is in. So, we are classifying an irish
flower
into one of the three classes- Setosa,
Versicolor, Virginica;
based on
sample width and height. You can plot
sepal width and height in that this kind
of 2D
scatter plot and you can plot all your
data points, you know all the data points
for which you know
what is the class.
Now,
let's say
you have build a model and you have a
new data point. So yellow data point you
don't know what class this is,
and using KNN you want to figure out.
By looking at the graph itself, you
immediately get an intuition that this
has to be blue color which is virginica
because it is more near to this cluster
and KNN works just like that. In KNN,
you first figure out was it what is the
value of my K,
and you can figure out the value of K by
trial and error. There is just no
like specific rule to it. Usually people
use 5, but you know you can change it.
So, let's say here I use K is equal to 3
and this means I need to figure out the
most near by 3 data points.
So, the most nearby 3 data point using
euclidean distance, which is just a
simple distance between these two points
are these three.
And since these three are virginica my
yellow data point is also virginica.
K nearest algorithm
neighbor near.. K neighbor nearest
neighbor algorithm is super simple. You
figure out
the most nearby K data point and
whichever data point category that data
point belong, my new data point belongs
to that category.
Let's say my K is 10,
again you figure out the most near by 10
data points. So here I have 7 virginica 2
versicolor. You take the highest number
which is 7 which is virginica.
How about if K is 20 well then you have
a problem, because
your total number of data point
in virginia class are very less.
So now, it is
misclassifying this yellow data point
saying that this is versicolor.
Because the rule is you figure out the
most nearby K data point and you take
the maximum count.
So here K is 20. Some maximum data points
which are near
are versicolor which is 11 data points.
See, 11, 7, 2. 11 maximum number.
But,
this will be wrong because clearly by
looking at the graph you know that this
yellow data point is actually virginica.
So you have to carefully choose a value
of K such that is not very high or not
very low.
Again style and error depends on your
situation depends on your your your
training data set and so on.
Let's write python code now.
One thing I would like to clarify is
that here we had only two features
sample width and sample height but KNN
works equally if there are more than
two features. You know there could be n
number of features and KNN will just
work fine, I mean as a human we can't
visualize it but mathematically it
should just work okay.
We're going to use same irish data flower
data set for our coding.
If you go to youtube and search for
support vector machine python, I have
this svm
video
in which,
I used iris flower data set. So I'm going
to use some of this same code here
so if you go here and look at
the code and if you have followed this
video you would
already be familiar with this code.
So, I will
use the same code till the point that I
create a classifier.
uh, so same code is here. Basically you
are loading
irish data set from
Sklearn on data sets.
Then,
obviously the feature names are
sepal width and length, petal width
and length. Sepal and petal are two
different flower leaves like the
components of the flower,
and based on these four features we
determine
whether it's
setosa versicolor or virginica. So our
target classes are three.
I have loaded
iris.data in a data frame
and then
I'm adding a new column called target
in the same data frame.
So, you see now my data frame has five
columns, four are my features the fifth
one is my target. Basically,
zero means
setosa one means versicolor
two means virginica. This code my friends,
is very very easy.
If you have followed my SVM video you
would
you will have no time understanding this.
You need to have some knowledge of
pandas as well.
Now if I do df target equal to 1, it will
show me all the versicolor flowers and
you see this start from the index 50.
If I do two which is all virginica
flowers, they start from 100.
So basically,
I have 150
training samples out of which, first 50
are setosa,
50 to 100 is
versicolor, 100 to 150 is virginica and I am
going to
split this main data frame into three
data frame.
So when I do
nothing colon 50
df 0 will have all setosa flower df1
will have all
versicolor flower and df2 will have all
virginica flowers.
And I'm using matplotlib to do some
plotting. I have a scatter plot of
sepal width and length between
df0, which is setosa df1 which is
versicolor.
So these are the
two kind of clusters that we have
and looking at all these clusters
we get immediate intuition that we can
apply KNN classifier here.
Alright,
and I'm doing train test split usual
thing, if you have followed my videos you
would know what this is.
So we are dividing our samples into
training and test data set and training
set now has 120 samples,
train test has
uh sorry test
set has
30 data sample.
Alright now comes the most interesting
part, which is creating a K nearest
neighbor classifier.
Google is your friend you probably all
know that
in that specify sklearn KNN classifier, you
will get
scikit-learn documentation.
This is how you create
KNN classifier in scikit learn
and usually they give a nice little
example at the bottom. So I'm just going
to
use my
most important skill, which is
ctrl c ctrl v
and create
a KNN classifier. So, I would say KNN
classifier. Okay, what are the parameters
in the classifier?
Obvious one is neighbors. This is
basically the value of K.
So, I'm just going with
you know three
and there are other values,
you can read the documentation. I'm going
to keep everything else default, this is
a beginner tutorial
but metric
minkowski. What is that? Let's read.
So metric
minkowski means
it's
basically your
if p=2 and metric is minkowski
it is using euclidean distance.
Okay so where is here...okay
so
see read this.
It is going to use your standard
euclidean distance for computing the
nearby neighbors. You can use some other
distance as well
and this parameter can control that.
I'm just going to use whatever is
default and
simply
call
KNN dot fit method. So I created KNN
classifier.
Call KNN fit on x train and y train
and my classifier is trained already.
And first thing I do usually after
training my classifier is compute the
score.
So when you compute the score, you are
giving x test so using the mod train
model
it will find out the prediction,
and then it will compare that with y
test and it will tell you how accurate
is your model. My model is so much
accuracy
it got all prediction right.
So you can
maybe
change this value. I mean 3 is ideal but
I'm just changing the value just for fun
and see with different value of k you
get different score. So, whenever you are
using KNN
try to figure out best value of k for
your use case.
you can use grid search cv or
k fold cross validation to figure out
the optimal value of n neighbors.
For us the optimal value is 3, but I am
going to keep it 10 so that you know we
can see some variation in our confusion
matrix.
So in the
previous videos, we have already plotted
confusion matrix.
So confusion matrix tells you for which
classes it got the prediction right, for
which classes
it did not get the prediction right.
So to plot it you will import confusion
matrix from
sklearn
matrix library
and then what you do is
you use KNN for making prediction,
so you give x test
you find y predicted
and then in confusion matrix,
there are two parameters
first is truth
and second one is prediction. Okay what
is my prediction? Prediction is this.
What is my truth? Truth is y test,
and there you get the confusion matrix.
Now if you don't know
how to read it properly, basically this
is your confusion matrix but for better
visualization we can use a c bond
library. We have used it in previous
video, so I'm not going to go over this
code in detail; but it's the same
exact grid. You are just having a little
better visualization.
Anything on the diagonal,
are your correct prediction.
This means
this 0 is setosa, this 0 is setosa and
this is truth, this is prediction.
What this is saying is
11 times,
11 times
it was setosa and my model predicted it
to be setosa.
So this diagonal is all correct
prediction anything that is on not not
on this diagonal
is
wrong prediction.
So one time what happened was truth was
versicolor
and it said it is virginica.
You see 2 means virginica ones means
uh versicolor, and this one is how many
instances? So we totally did how many
predictions did we did uh did we do in
total; 30. So 12 and 1 13 13 and 6 19
19 and 11 30, because we have total
30 samples in our test data set
and out of those 30
only one time it got the prediction
wrong,
remaining 29 times it got it right and
hence the score is so high. You can also
use
confusion matrix. So...
not confusion matrix, the classification
report.
So, from sklearn matrix you can import
classification report,
and this classification report
takes
your
y test and y
predicted
and you know.. okay. The formatting is not
great so if you put print statement it's
going to show you
a little better result.
And this is showing you precision recall
reference score for each of the classes.
This one class 2 class and so on. So you
have to
get your fundamentals clear on precision
and recall and if you go to youtube
search for
codebasics
precision recall. You'll find my video
if you watch it you will get an idea on
all these parameters.
So that's all pretty much that I had for
this video.
KNN again is a very simple classifier
very very easy to understand
if I explain it to high school student
they will also understand it easily. The
code you saw was so simple- three lines.
It's like freaking
simple classifier.
So after you watch this video you need
to work on an exercise.
So if you go to my github page, I'm going
to provide all these links in the video
description below.
exercise dot md has
this exercise and this exercise friends
is the most important part of
my tutorials,
because
learning coding or machine learning is
like
learning swimming. Until you practice
you're not going to get much benefit. So
practice is extremely important.
I want you to load
digits data set from sklearn
and use KNN classifier to classify
digits from zero to nine.
You also need to plot confusion matrix
classification classification report
like how we did it in the current video.
Also for
for our main classifier, you need to find
an optimal value of K that gives you the
maximum score
either by
trial and error, manually trying it out
or by using research cv.
A good student would try
on their own and then only they will
click on solution link.
A bad student would directly click on
solution link and try to look at the
solution, but the downside is that
they will get a virus which I have
embedded
encrypted into this github
in your
computer will start sneezing and your
computer will have a fever! So if you
don't want your computer to have a fever
please try the solution on your own and
then only click on solution link.
Alright!
Thanks very much for watching. If you
have any question post in a comment box
below.