in this quick video we will learn what
is
mean absolute deviation and standard
deviation
let's say you have a history test scores
here i have six data points where i have
shown the scores out of
hundred and the average score here is
70.
and when you plot this on a chart you
will see that 70 is an average this
yellow line
and these are the individual data points
for example you have 75
here you have 65 here and so on
and you can see that all these data
points are near
the average data point so what we're
trying to learn
is how far apart the
individual data points are from the
average or how spread out the data
points
are with respect to average
you can have a different data set let's
say a score
in the mathematics test and those scores
might look like this
again the average is same but when you
plot them on a chart you will find that
data points are quite spread out
they are quite far away from average
data points
and in statistics and data science
it helps if you can figure out how far
apart individual data points are from
the average data point or how
spread out these data points are and
we need a metric or a number to
represent this concept here my data set
is simple
i can easily say that in the right hand
side the data points are far apart
but when you have millions of data
points and if you want to come up with
a single number or a metric which can
tell you this
then that will be super useful so pause
this video for a moment and think about
what that metric could be
well a a busy very basic approach common
sense approach would tell you that
why don't you take the difference of
individual score
with respect to average so let's do that
75 how far away it is from average well
my average is 70.
so this guy is five points
away similarly 72 is two points away 60
point is
68 is 2.7 and so on and if you take a
mean
of this number that will be 3.16
and for the second case you do the same
thing and when you take a mean
you get 23. so now this is interesting
look this number is telling you that
for the history test my mean is 3.16
but for mathematic days my mean is 23
which means here i have a higher number
and that implies that this particular
data set the mathematic test data set
is wide spread out you know with respect
to
the average number so this is
interesting
now this mean number is nothing but mean
absolute deviation you can also call it
mad
mad all right so m80 could be
quite a significant or a useful metric
that can represent how spreader spread
your data points
are but there could be a scenario
where using mean absolute deviation
might not be enough
let's take this particular example here
see here mean absolute deviation is 3.33
on left and right hand side but if you
look at the chart you can see that
on a right hand side although i have
four points
right at the average but i have one data
point which is far away see
this is far away on the left hand side
all my data points are between 65 to 75
range
but here my data points are
in 63 to 83
range so my range is widespread so here
the data distribution
is kind of spread out
and when you look at mad it tells you
it's the same number 3.33
so in this case mad is not very useful
so we need to come up with
something else again pause the video can
you come up with
some other concept which can represent
this
data distribution accurately
well if you give it a little zy you
might come up with this idea that
how about we again we keep the same data
set same absolute everything but we add
a new column
and we take a square of these individual
numbers so here see look at this column
this column is nothing but the square of
this second column so
5 square is 25 2 square is 4 2 square is
4
5 square is 25 and so on
and then you take an average of this
column okay i took the average of this
column which is 12.66
then i took the square root of that
average
why did i take square root well because
you squared the numbers here individually
therefore you want to do a square root
okay so here i come up with number 3.55
on the right hand side i do the same
mathematics
and come up with average and square root
of average but this time i came up with
6.02
so see here the number is 3.55
here the number is 6.02 so higher the
number
means the data points are spread out
more they are more widely spread out
this square root is called standard
deviation
standard deviation is nothing but
you take individual data points so here
this
this particular symbol is called sigma
and sigma is your standard deviation and
standard deviation is nothing but
you take individual data point which is
x i
minus mu
is average then you take
square of individual data points then
this
symbol means you sum it up and then you
take the average
and then you take the square root see
this is not complicated friends
we already did this math if you look at
this is exactly what we did
so we took the difference first okay so
that difference is x i minus u
so that is this particular column then
we took a square
of that okay so that is the square
then sum sum and then
divided by n n is number of data points
which is an average
so that is this average and then we took
the square root
which is this square root and hence you
got 3.55 versus 6.02
so standard deviation is something you
will use a lot in
data science machine learning statistics
these are these are the
very very basic concepts that you need
to know
when you're learning statistics or data
science
you might have heard about this term l1
and l2 norm if you're doing machine
learning
let's say linear regression you might
have done
a ridge regression and lasso regression
so those are l1 and
l2 l1 usually refers to mean absolute
division
and l2 refers to the standard
deviation so we'll be using this
standard deviation in
future videos as well my next video is
going to be
most likely on normal distribution so i
hope this video
gave you some basic understanding of
standard deviation and
mean absolute deviation if you like this
video please give it a thumbs up
share it with your friends and i will
see you in a next video thank you