step quest step quest stack quest
hello and welcome to stab quest this
time we're gonna talk about standard
errors and we're also gonna have a
bootstrapping bonus we'll start by
talking about error bars which are very
closely related to standard errors for
example you might collect measurements
from three samples labeled a B and C and
plot them on a scatter plot just like we
see here
you could then calculate the means for
the three data sets and we Illustrated
those here with three green horizontal
bars approximately halfway up the
clusters of data points after that we
could calculate the standard deviations
and add those to the graph and we've
shown those here with red error bars in
manuscripts and presentations people
often don't display the original data
but instead just show the mean in the
standard deviation and what's called a
dynamite plot because each column in the
plot looks like it's the igniter for a
stick of dynamite
there are three common types of error
bars the first type of standard
deviations which we just saw and I'm
sure you're all familiar with these tell
you how the data are distributed around
the mean big standard deviations tell
you that some of the data points were
pretty far from the mean in most cases
you want to use standard deviations in
your graphs since it tells us about your
data the data points that you collected
yourself the second type of error bar
comes from standard errors these tell
you how the mean is distributed not just
the data but the means which sounds
crazy but it'll become clear once I draw
some pictures the third common type of
error bar are confidence intervals and
these are related to standard errors
confidence intervals will be explained
more in a future stat quest since this
stat quest is all about standard errors
that's what we're going to talk about
let's start by considering a normal
distribution in this case we can imagine
that we weighed a lot of mice and
plotted the distribution of differences
from the mean the y-axis is the
proportion of the mice that we weighed
in the x-axis is the difference from the
mean most of the mice had weights close
to the average a few of the mice weighed
much less than the average Mouse and a
few other mice weighed much more than
the average Mouse usually you can't
afford to measure the weight of all the
mice so you just take a sample in this
example we'll just assume we took five
measurements from the population rather
than measuring all the mice since most
of the mice have weights close to the
average most of our samples are going to
be close to zero now just like we always
do we can calculate the mean and
standard deviation from our sample in
this case the mean of our sample is
minus point two and the standard
deviation is one point nine to three and
we can plot the mean and standard
deviation on our graph as the mean
plus or minus the standard deviation
around the mean and for all you stat
Questers out there here's a rule of
thumb remember that one standard
deviation on each side of the mean is
supposed to cover about 68% of the data
two standard deviations on each side of
the mean is supposed to cover about 95
percent of the data this will come in
handy later
the mean is now a lighter color because
we're going to take additional samples
and overlay additional means and
standard deviations on this same graph
here we've taken another five
measurements and from those five
measurements we've calculated the mean
and the standard deviation and here
we've plotted that mean plus or minus
one standard deviation on each side and
now we take another five measurements
this is the first sample or one of the
measurements is relatively extreme
however that one measurement doesn't
sway the mean that far from zero that is
to say the means are relatively close to
each other compared to the raw data this
is because for a mean to be far from the
middle most if not all of the raw data
points would have to be in a single
cluster that is far away from the middle
for example the sample of purple points
all form a cluster that are far from the
middle this could happen but very rarely
what's much more likely is to have a
sample where most of the points are
close to zero and only one or two are
far away so far we've shown that you can
calculate the standard deviations for
each sample but now that we have three
means we can also calculate the standard
deviation of those means because one
standard deviation will cover 68% of the
values and two will cover 95% of the
values the standard deviation of the
means won't be as wide as the standard
deviations of the data here we've
plotted the mean of the means plus or
minus one standard deviation of the
means notice that this standard
deviation is much smaller than the
standard deviations we got from the
individual samples the standard
deviation of the mean is called the
standard error of the mean or more
simply the standard error the standard
error gives us a sense of how much
variation we can expect in our means if
we took a bunch of independent five
measurement samples so to review this is
how we calculate this
an error of the mean first you take a
bunch of samples each with the same
number of measurements or in in this
case in equals five the second step is
to calculate the mean for each sample
here we calculated the mean and standard
deviation for each sample but for the
standard error all we need to do is
calculate the mean once we've calculated
the means for each sample we can
calculate the standard deviation of the
means in this case the standard error
equals zero point eight six here we
notice that the standard error is much
less than the standard deviations
because the means aren't as widely
dispersed as the raw data we've shown
how to calculate the standard error of
the mean but there are other standard
errors for example we can also take the
standard deviation of the standard
deviations this is called the standard
error of the standard deviations which I
guess is to avoid a tongue-twister it
tells us how the standard deviations of
multiple samples are dispersed you can
calculate the standard deviation of any
statistic for example the median the
mode percentiles are anything anything
that you can calculate for multiple
samples you just calculate the standard
deviation and then you have the standard
error of that so if we calculated many
mediums we could calculate the standard
deviation of those mediums and we have
the standard error of those mediums to
summarize everything we've talked about
so far know that the standard error is
just the standard deviation of multiple
means taken from the same population so
if there's a population and we can take
a bunch of different samples from it all
we have to do to get the standard error
is to calculate the standard deviation
of the means of each sample well at this
point you might be wondering if we can
calculate standard errors without
spending a lot of time and money on
doing the same experiment a bunch of
times the good news is the answer is yes
in rare cases there's a formula you can
use to estimate
the standard error for the mean is 1 the
formula for that is very simple it's
just the standard deviation divided by
the square root of the sample size
however there aren't many other cases
the good news again is that we can use
something called bootstrapping for
everything else
every time we don't have a simple
formula we can bootstrap it the nice
thing about bootstrapping is it's very
simple conceptually and it's easy to
make a computer do this work here's a
bootstrapping example just like before
we have an experiment where we took 5
measurements as an aside usually for
bootstrapping it's good to have 10 or
more measurements in a single experiment
now we bootstrap our data with the
following steps first we pick a random
measurement from the sample that we just
took this random measurement isn't a new
measurement that we haven't taken before
it's not a new number that we haven't
seen it's part of the sample that we
already have now we just write that
value down in this case it's one point
four three in step three we just go back
to step one and pick a new random
measurement and write that value down
and we do that five times our second
measurement is minus one point three
eight the third measurement is minus
three point one one our fourth
measurement is one point four three
we've already picked that measurement
before but that's okay when you're
bootstrapping you just pick five
measurements from your sample and it
doesn't matter if you've picked the same
one before our last measurement is minus
zero point one zero step four in
bootstrapping is to calculate the mean
median mode or whatever the statistic it
is we're interested in understanding the
standard error of and we calculate that
with our sample in this case we're
interested in the standard error of the
mean so all we do is calculate the mean
from our new bootstrap sample the fifth
step is to go all the way back to the
beginning step one and repeat that until
you have a lot of means or medians or
whatever you're interested in
calculating the standard error of the
sixth and final step in the
bootstrapping procedure is to simply
calculate the standard deviation of all
the means that we generated in steps one
through five that's all there is to it
in this case we calculated the standard
error of the mean and we've plotted it
as a black line in the graph so if
there's no fancy formula to help us
calculate the standard error we can do
it ourselves from scratch we can just
use bootstrapping and get the job done
and that's it in this static quest we
learned that the standard error is a
measure of how we might expect the means
from many different samples to vary from
one sample to another we also learned
that if we don't have a fancy formula
for calculating the standard error we
can do it ourselves using bootstrapping
okay so tune in next time and we'll talk
about how to use bootstrapping to
calculate confidence intervals and
that's when things get really cool