I'm going to explain bias and variance
using a very simple language and you are
already wondering what's the meaning
behind showing all these three pictures.
Well look at the pictures carefully and
try to find out what I
want to show you here.
The first picture of course shows a
t-shirt that I'm wearing, which is
which is too much fit. Basically it's
over fit. So if my body increases or
decreases you know if I start eating
junk,
it will
not fit.
The second one is
underfit, which is too loose basically.
You look at my sad face here clearly, it
indicates what's the issue here. The
third t-shirt however, is a perfect fit
where
even I grow in size a little bit of or
if I you know
thin up a little bit, it's gonna be okay.
Same concepts applied machine learning
world, where your model can be an over
fit, underfit, or balance fit and we're
going to look at these scenarios using a
simple housing price prediction example,
where
we are using one feature which is the
square foot
area of a home
and based on this feature we are trying
to build a machine learning model that
can predict the home price.
Here I have a scatter plot of all my
training data set. Basically my data set
super for the supervised learning and
in this scatter plot when we
train our model, what we do is we split
this data set into training and test
samples.
Let's say all these blue dots are my
training samples, the orange dots are my
test samples,
and we can train a model that fits to
these blue dots. Let's say our model
is such that our training methodology is
such that,
we end up in an over fit model. An
overfit model
tries to fit exactly to the training
samples where your training error
becomes close to zero.
But the problem here is
the
test error
can become high,
because once you have this model train
which is this green line.
Now let's say you want to figure out an
error for this particular orange test
data point,
the predicted value will be somewhere
here,
and the error will be this gray dotted
line. And you can measure the error for
all your test data set and average it
out. Let's say you get this error as 100.
I'm just making this up. Let's say this
error is 100.
Now remember one thing
when you have a data set and when you
pick your training samples, you pick it
up at random you have trained a split
let's say 80 20 percent, you pick your
training samples
at random. There's a different data
scientist,
they might be
choosing different set of training
samples let's say this and their model
might look different. Here they're using
the same model, same methodology so your
training
data set error is still zero because you
both are trying to over over fit the
model. But the problem that happened here
is
your test data set error
is, let's say 27.
You can compare these two images and you
can easily see that
the test data set error here is much
higher than the second scenario. You see
all this, you just sum up all these gray
dotted lines here
and sum up the gray dotted line here.
This error is much less than
the previous error. So if you compare
these two scenarios side by side, in one
case your test error is 100, second case
test error is 27, which means the test
error varies greatly
based on your selection of training data
point.
And this is called high variance, because
there is a high variability in the test
error based on what kind of training
samples you are selecting. Now you are
selecting training samples at random, so
your test error varies randomly which
is not good and this is the common issue
with over fit model. They tend to have
high
variance. So high variance again just to
summarize
is high variance happens when your test
error varies by great deal,
based on your selection of training
data points.
So just to remember when you're talking
about variance, high variance you are
always talking about test
error.
Let's look at another scenario where I
have same data set I have split into
train and test samples and this time
instead of having a complex model that
over fits
my training samples, I'm trying to come
up with a very simple model. Let's say
linear equation,
which which is under fitting your
training sample. Because linear equation
is a straight equation, it cannot truly
capture the pattern in your training
samples,
okay.
The straight line cannot pass through
all the blue dots. So it's a simple model
where
your training
error is actually high. In previous case
the complex
model was complex your training error
was zero here it is, let's say 43 and
your test data set error is around 47.
Again 43 47 these numbers I have just
made up to explain you this concept.
When you select a different set of
training data points, see
in both the cases the line will be
different but your training and test
data set error is still
kind of similar. You know in the first
case it was 47, second case
it was 37. So this is called low variance
because
based on what data point you select as
your training samples your test error
doesn't vary that much.
47 37 that's still similar it's not like
100 and 21 you know. So,
your test error
doesn't vary too much hence
low variance.
But if you look at your train error in
the previous case your train error was
zero. Here you have some big train error
and that is called
high bias. High bias is a measurement of
how accurately a model can capture a
pattern in a training data set.
So when you're thinking about bias,
you're always thinking about train error.
When you're thinking about variance
you're always thinking about test error.
Bias is basically your model's ability
to capture,
the
pattern in your training data set in an
accurate way.
In the previous case, where we had over
fit the model the bias was low because
the train error was close to zero. So if
if the higher the train error, higher the
bias, okay.
Now let's look at the idle scenario
where
you come up with a model that
kind of accurately describes the pattern
that is available in your training data
set. Here your training error and test
error both are low,
even if you select different set of
training data points. See the training
data points here here are different, but
still your model selection is such that
your train and test error both are kind
of low,
and in this case it's called low
variance-
low bias model,
because your test error doesn't vary too
much based on what training samples you
have selected.
Also, your train error in general is
low.
So that's why it's called low bias.
So we already covered these concepts,
like whenever you have an overfit model
it's likely that you will get high
variance. When you have under fit model,
it is likely that you will get
high bias,
and when you have balance fit you will
get
low variance low bias and as a data
scientist
you want to
come up with a with a model that has a
balance fit you know just like our
t-shirt. You want to go to a store and
buy a t-shirt, you will always buy a
balance fit t-shirt. Now there is a
bull's eye diagram, where
the central circle
represents the truth. This is your truth
samples okay,
and
when your predicted values so these
white
diamonds are let's say your predicted
values.
When they're near to the inner circle,
which is your truth it is called low
bias.
Bias is all about
how close you are to the truth.
So if you are closer to the middle
circle you have low bias.
If you are far away from the middle
circle,
you have high bias. But in both the cases
these diamonds they are clustered
together you know they are like a close
family they are together. So whenever
they are together it is called
low variance.
In high variance they are scattered
apart.
You see they are scattered apart
but they're still near to the truth
which is the inner circle hence there is
a low bias.
Worst case scenario is higher bias high
variance where you are far away from the
truth, which is this middle circle.
That means you have high bias
and also these individual predict
predicted values they are separated out.
Hence
you have a high variance.
If you remember this image, I think
remembering bias versus variance
tradeoff is going to be
easier for you.
How do you get a balanced fit model? So
there are several techniques and the
first one is cross validation. I have had
a video on this before, so if you watch
it you'll get an idea. Another one is
regularization, l1 and l2 dimensionality
reduction and symbol techniques bagging
boosting. I'm gonna be making videos on
this in next 10 days maybe. So those
videos should be available. So these are
different techniques to get a
balanced model. I hope you like this
video if you did give it a thumbs up and
share it with your friends. Thank you! If
you have any question post in a comment
box below.