this clouds outside but who cares it's
time for stat quest stat quest hello I'm
Josh starburns welcome to stat quest
today we're gonna talk about statistical
power and it's gonna be clearly
explained note this stat quest assumes
that you are already familiar with
p-values if not check out the quests
this stat quest also assumes that you
are already familiar with the normal
distribution if not check out the quest
let's start with two distributions the
one on the Left represents the weights
of mice on a special diet and the one on
the right represents the weights of mice
eating normal Mouse food
since there's only a little bit of
overlap it's pretty easy to see the
difference between these two diets most
of the mice on the special diet weigh
less than the mice on the normal diet
and if we collect a small set of
measurements from the special diet and
another small set of measurements from
the normal diet
and plot these points on a graph and
compare their means then in this case
we'll get a p-value equal to zero point
zero zero zero four and this small
p-value less than 0.05 would cause us to
correctly reject the null hypothesis
that both sets of data came from the
same distribution in other words if this
distribution which is somewhere between
the special and normal diets said all
data a tums
from me then the small p-value would say
in a loud and confident voice I reject
your hypothesis if we repeated this
experiment a bunch of times there's a
high probability that each statistical
test will correctly give us a small
p-value in other words there is a high
probability that the null hypothesis
that all of the data came from the same
distribution will be correctly rejected
but every once in a while we will get
something like this where the data
overlap and when this happens we will
get a large p-value greater than 0.05
and that means that even though we know
the data came from two different
distributions we cannot correctly reject
the null hypothesis that all of the data
comes from the same distribution so the
large p-value says in a very small and
meek voice dang I can't reject the null
hypothesis
that said because these distributions
are so far apart and there is so little
overlap the probability of correctly
rejecting the null hypothesis is high
power is the probability that we will
correctly reject the null hypothesis
alternatively you could say that power
is the probability that we will
correctly get a small p-value
in this example because we have a high
probability of correctly getting a small
p-value and rejecting the null
hypothesis we have a large amount of
power BAM
[Music]
note if there was no difference between
the special diet and the normal diet and
they both shared the same distribution
and we collected one set of measurements
for mice on the special diet and one set
of measurements for mice on the normal
diet then the null hypothesis that both
datasets came from the same distribution
would be true
in this case there is no such thing as
correctly rejecting the null hypothesis
so the concept of power the probability
that we will correctly reject the null
hypothesis doesn't apply in this
situation
contrast if this special diet wasn't
very good at helping mice lose weight
but it still made a difference then even
though there is a lot of overlap we have
two distinct distributions
and that means power the probability
that we correctly reject the null
hypothesis applause if we were to weigh
three mice on the special diet and three
mice on the normal diet and plot these
points on a graph and compare their
averages then in this case we will get a
p-value equal to 0.34 that means we will
fail to reject the null hypothesis that
both groups come from the same
distribution
this is a bummer because in this case we
know the data comes from two different
distributions
and when we repeat this many times most
of the time we will get a large p-value
and fail to reject the null hypothesis
however every once in a while we will
get something like this where the data
do not overlap and we will correctly get
a small p-value
when this happens even though the null
hypothesis says all day today comes from
me the small p-value will say in a loud
and confident voice I reject the null
hypothesis so out of all these tests
this was the only one that gave us a
p-value small enough that we correctly
rejected the null hypothesis and that
means when there is a lot of overlap
between the two distributions and we
have a small sample size we have a
relatively low power
medium BAM
the good news is that we can always
increase power by increasing the number
of measurements we collect and a power
analysis will tell us how many
measurements we need to collect to have
a good amount of power
shameless self-promotion we'll talk more
about how and why we can increase power
in the stat quest on power analyses BAM
in summary power is the probability that
we will correctly reject the null
hypothesis
when we have two distributions that have
very little overlap we will have a lot
of power because there is a high
probability that we will correctly
reject the null hypothesis however when
the two distributions overlap a lot and
if we have a small sample size we will
have a small amount of power
however if we want more power we can
increase the sample size lastly a power
analysis will tell us how many
measurements to collect to have a good
amount of power double BAM hooray we've
made it to the end of another exciting
stat quest if you like this stat quest
and want to see more please subscribe
and if you want to support stack quest
consider contributing to my patreon
campaign becoming a channel member
buying one or two of my original songs
or a t-shirt or a hoodie or just donate
the links are in the description below
alright until next time quest on