I was home last night
barians of standard deviation so I
estimated them and its goal that quest
[Music]
hello I'm Josh stormer and welcome to
stat quest today we're gonna continue
our series on statistics fundamentals
this time we're gonna talk about
estimating the mean variance and
standard deviation note this stat quest
assumes you already know about
histograms statistical distributions and
specifically the normal distribution if
not check out the quests the links are
in the description below also this stat
quest assumes you already understand why
we want to estimate population
parameters if not check out the quest
the stat quest on population parameters
we counted the number of mRNA
transcripts from gene X in five
different liver cells alternatively if
mRNA transcripts and liver cells didn't
mean anything to you we counted the
number of green apples in five different
grocery stores or green t-shirts and
five different clothing stores or
whatever you want to measure in five
different units
this green dot represented a liver cell
that had three mRNA transcripts for gene
X and this green dot represented a liver
cell that had 13 mRNA transcripts 1924
and 29 now if we had a lot of time and
money on our hands we could count the
number of mRNA transcripts for gene X in
all 240 billion liver cells
now we can draw a histogram of the
measurements if we wanted to fit a
normal curve to this histogram like this
then we need to calculate the population
mean in the population variance or
population standard deviation
calculating the population mean is easy
we take the average of all 240 billion
measurements BP booty P due to baby boo
doo bee doo doo boo boo and we get 24
the population mean
and we Center the normal curve on the
population mean note because we
calculated the mean with all 240 billion
measurements in the population this is
not an estimate of the population mean
it is the population mean
however since we rarely if ever have
enough time and money to measure every
single thing in a population we almost
always estimate the population mean
using a relatively small sample in this
example we have the measurements from
only five of the 240 billion cells
estimating the population mean is super
easy we just calculate the average of
the measurements we collected doodoo
booty boot boot boot boot and in this
case the estimated population mean is
seventeen point six
oh no it's the dreaded terminology alert
statisticians often use the symbol x-bar
to refer to the estimated mean which is
also called the sample mean
and they use the Greek symbol mu to
refer to the population mean
the estimated mean x-bar is different
from the population mean mu but with
more and more data
x-bar should get closer and closer
going back to the full set of population
data
we will now determine how wide to make
the curve by calculating not estimating
the variance and standard deviation in
other words we want to calculate how the
data are spread around the population
mean
this is the formula we use to calculate
not estimate the population variance
note I'm making a big deal about
calculating versus estimating variance
because it makes a big difference that
we'll talk about later
this part X minus mu means we subtract
the population mean mu from each
measurement X boo-boo-boo-boo-boo the
square tells us to square each term
character Sigma tells us to add up all
the terms
lastly we want the average of the
squared differences so we divide by the
number of measurements n which in this
case is 240 billion
thus we're just calculating the average
of the squared differences between the
data and the population mean
note squaring each term ensures that
each difference is positive
otherwise the measurements on the left
side of the mean would give negative
differences
which would cancel out the positive
differences from the measurements on the
right side of the mean
note if you are wondering why we don't
take the absolute value of each term
great we'll talk about that in the
follow-up video that dives deep into
these details
anyway and now we just do the math and
we get 100 for the population variance
BAM okay we calculated the population
variance and we're all proud of
ourselves however there is one thing
that is annoying about it
because each term is squared the units
for the result 100 are mRNA transcripts
squared note if the data have been the
number of apples and grocery stores then
the variance would be 100 apples squared
either way we can't plot the variance on
the graph since the units on the x axis
are not squared
to solve this problem we can take the
square root of everything and that gives
us the population standard deviation
so the population standard deviation is
the square root of 100 the population
variance which is 10
and we can plot that on the graph
this shows the main 20 plus and minus
the standard deviation 10 mRNA
transcripts
um
note before we move on I want to
emphasize the point that we almost never
have the population data so we almost
never calculate the population mean and
population variance in standard
deviation
instead we estimate the population
variance and population standard
deviation from the relatively small
number of measurements that we have
remember the population variance and
standard deviation determines how much
the curve spreads out
and that means the estimated variance
and the estimated standard deviation
should reflect how the data are spread
around the population mean
however when we do an experiment we
don't see the curve or the population
mean we only see the data
so we have to use the estimated mean
x-bar instead
this is the formula we use to estimate
the population variance
because we almost always work with a
relatively small sample and not the
entire population this is the formula we
will use most of the time
the differences between this formula and
the one for the calculated population
variance are subtle but important first
since we don't know the population mean
mu we use the sample mean x-bar
second we are dividing by n minus 1
instead of n
dividing by n minus one compensates for
the fact that we are calculating the
differences from the sample mean instead
of the population mean otherwise we
would consistently underestimate the
variance around the population mean
this is because the differences between
the data and the sample mean tend to be
smaller than the differences between the
data and the population mean
thus the differences around the
population mean or result in a larger
average and the larger average is what
we are trying to estimate
note if you're like me and want to know
more details about why we need to
compensate for calculating differences
from the sample mean check out the
follow-up stat quest the link is in the
description below
now let's do the math
before we calculate the differences
between the mean and the data doopa
doopa doopa doopa - then we square each
term
add up each term
but now we divide by n minus 1
and the estimated population variance is
100 1.8
now we just take the square root of the
estimated variance to get the estimated
standard deviation and we get 10.1
we can draw the mean plus and minus the
standard deviation on the graph
double BAM
the estimated population parameters
correspond to this purple curve with
mean equals seventeen point six and
standard deviation equals ten point one
which isn't too far off from the true
distribution with mean equals 20 and
standard deviation equals 10
with more data the estimated parameters
would be more accurate and we would have
more confidence in them
however with just five measurements we
still did pretty well and that saved us
a ton of time and money hooray
in summary if we have all of the data
from a population we can calculate the
population mean the population mean
equals the sum of the measurements
divided by the number of measurements
and that equals the average measurement
mu
when we don't have the population data
we can estimate the population mean with
the same formula the estimated
population mean equals the sum of the
measurements divided by the number of
measurements which equals the average
measurement x-bar
when we have the population data we can
calculate the population variance and
standard deviation
the population variance is the average
of the squared differences between the
data and the population mean mu
in other words we square these
differences to prevent the ones on the
left from canceling the ones on the
right and then take the average
and the population standard deviation is
just the square root of the population
variance and since the standard
deviation is in the original units that
we measured we can draw it on the graph
however we almost never have the
population data so chances are you
should not use these formulas
said we almost always estimate the
variance and standard deviation
when we estimate the population variance
we divide by n minus one to compensate
for measuring distances from the sample
mean instead of the population mean
and the estimated standard deviation is
just the square root of the estimated
population variance and since the
standard deviation is in the same units
that we measured the data we can draw it
on the graph and one last shameless plug
for the follow up stat quest if you want
to know why dividing by n underestimates
the variance check out the quest the
link is in the description below
triple bell
[Music]
is in this stat quest I made a big deal
about how we rarely have the population
data and we almost always estimate the
population parameters one reason I did
this was because while many software
packages estimate the variance and
standard deviation by default Microsoft
Excel does not
instead it gives two choices one
function VAR p calculates the population
variance the other VAR s estimates it
since we almost always have a relatively
small sample rather than the population
data we should almost always use far s
hooray we've made it to the end of
another exciting stat quest if you like
this that quest and want to see more
please subscribe and if you want to
support stack quest well consider buying
an original song or a t-shirt or a
hoodie or just donating the links are
all in the description below alright
until next time quest on
you