Step three in the five steps of hypothesis testing is to select a level of
significance. The level of significance sets a fence that we use to determine if
our test statistic is significant, i.e. statistically significantly different. We
are doing hypothesis testing. The hypothesis that we test is the null
hypothesis, and the null hypothesis states that there is no difference
between the sample mean and the population mean. So we are going to
establish two fences - one on the right and the other on the left. Values over
the fence in the purple shaded area (which is called the critical region) are
so extreme that it is unlikely we would randomly select a sample with this mean
if the null hypothesis is true. The fence is called our critical value. The
specific location will vary depending upon our sample, but we will use the same
standard to set the critical value every time. That standard is an alpha level of
.05. We set the fences so that ninety-five percent of the scores under
the curve are inside the fence, and five percent of the scores under the curve
are outside the fence. Here, because we have two tails, we will put 2.5% in
either tail for a total of five percent. If we get a sample mean that is outside
of the right fence, we will reject the null hypothesis. If we get a sample mean
that is outside of the left fence, we will reject the null hypothesis. Only if
we have a sample mean that is inside the fence will we fail to reject the null
hypothesis. Another word for fail to reject is to accept the null hypothesis,
but there are some philosophical problems with that word accept, and I'll
explain more later. The level of significance is the Alpha value. The
Alpha value is the same as the critical value,
both of which mean the same as the cutoff score. What to call it depends on
how you're measuring it. If you're talking in terms of how many standard
deviations it is from the mean, you might call it a critical value. If you're
talking about an alpha level then you might say alpha equals .05. If
you're looking at X values on the number line it will be your cutoff score.
Regardless of what you call it, the level of significance is the fence that sets
the critical region. The critical region, also called the region of rejection, is
the purple shaded area of the curve. The critical region was set by the fence, and
the fence is the level of significance. We want to know whether our sample mean
is outside of the fence or inside the fence. If the cow is the population mean,
and we are the sample mean, are we on a different side of the fence? Sample and
population are different? Or are we on the same side of the fence? Inside the
fence? Sample and population are the same? Remember that we are testing a null
hypothesis, that the sample mean and the population mean are the same. You should
reject the null hypothesis if the sample mean is outside of the fence. The null
hypothesis says that there is no difference between the sample mean and
the population mean. If you reject the idea that there is no difference what
are you saying? You're saying that there is a difference. Now that is the logic of
hypothesis testing. That is what you must get your mind wrapped around. The null
hypothesis says there is no difference. If you reject the idea that there is no
difference, you are saying that there is a difference. The most common level of
significance is .05, especially in scientific
and consumer research. .05 is five percent. Another way to think about
this is to imagine that a dollar is 1.00, or one hundred percent
so five percent, .05, would be like a nickel. In medical research, where the
quality matters more, and the consequences for mistakes are higher, the
Alpha level is often set to .01. Following our analogy .01
would be one percent, or a penny. Political polling often uses a .10
level, because their results are not as dire if they're wrong. This .10
level is analogous to ten percent, or a dime. For most tests, assume alpha equals
.05 and a two-tailed test. We will be using this term "significant" very often,
so it is important that you know how to interpret when results are significant.
The word "significant" is an abbreviated way of saying statistically
significantly different, so significant means outside of the fence. If your
sample mean is outside of the left fence, the results are significant; the sample
mean is different than the population mean. If your sample mean is outside of
the right fence, the results are significant; the sample mean is different
than the population mean. But if the results are inside the fence, not in the
region of rejection, then the results are not significant. The sample mean is not
significantly different than the population mean. For our example, using
baby weight, the Alpha level is .05. We are using a two-tailed test, so if we
had a large enough sample, our critical value would be positive or negative 1.96.
Now remember that number - 1.96. It is the critical value or cut off
score that we will learn about with a z-test.