>>Recalling the continuing theme
that we will never know the truth,
what can we really conclude when we conduct
a hypothesis test?
Imagine for a moment that we actually do know the truth.
This table gives us all the possible outcomes
of a decision we might make
concerning a particular hypothesis.
Two are correct decisions and two are errors.
If the null hypothesis is actually true,
and our test says we cannot reject it,
then we have made the correct decision.
The status quo stands.
If, on the other hand, the null hypothesis is false,
and we conclude we cannot accept the null hypothesis,
then again, we are correct.
We have rejected a false null.
The other two possible decisions,
however, are a different matter.
In both cases, we have made an error.
Each type of error is given a name.
If we reject a good null, think put an innocent man
in prison, then we call it a type I error.
If, however, we cannot reject a false null,
that is, we accept the false null,
then we have made a type II error.
Here we have let a criminal go free.
The null was innocent and the evidence
was not sufficient to reject the null
when, in fact, he was a criminal.
Each error has a probability of occurrence.
The probability of a type I error is called alpha
and yes, this is the same alpha
we met doing confidence intervals.
Before, alpha determined the level of confidence
and it will do so here also.
The probability of making a Type II error is called beta.
We can control only one of these two probabilities
and we choose to control alpha,
the probability of putting an innocent man in jail.
That is, the probability of rejecting a good null.
Remembering that the null represents the status quo,
this amounts to controlling the probability of overturning
the current way of understanding the world.
The system is designed to control how fast we decide
our current way of thinking is wrong.
The statistical test is biased in favor of retaining
the null hypothesis, the current world view.
That is to say, if we are to make an error, it is in favor
of the null hypothesis.
In the example of the criminal justice system,
we put the burden of proof on the prosecutor.
Thus, we will set the alpha level; the probability
of rejecting a good null, innocent man goes to jail,
at a very small level.
This of course means that we have a very high confidence
that we did not make a Type I error,
send an innocent man to prison.
The easiest way to see this relationship
between the alpha error and the level of confidence
is with the following figure.
In the center of this graph is a normally distributed
probability distribution mark, H-naught.
This is a sampling distribution of X-bars
and by the central limit theorem,
it is normally distributed.
The distribution in the center is marked H-naught
and represents the distribution of the null hypothesis.
H-naught is Mu equals 100.
This is the value that is being tested.
The formal statements of the null
and alternative hypotheses are listed below the graph.
The distribution on either side of the H-naught distribution
represents distributions that would be true
if H-naught is false under the alternative hypothesis
listed as H-a.
We do not know which is true and we'll never know.
There are, in fact, an infinite number of distributions
from which the data could have been drawn if H-a is true
but only two of them are on the graph representing
all the others.
To test the hypothesis, we take a sample from the population
and determine if it could have come from the hypothesized
distribution with an acceptable level of confidence.
This level of confidence is the alpha error
and is marked on the graph as a shaded area
in each tail of the H-naught distribution.
Each area is actually alpha over two
because the distribution is symmetrical
and the alternative hypothesis allows for the
probability of the value to be either more than or less than
the hypothesized value, called a two tailed test.
If the sample mean, marked X-bar one is in the tail
of the distribution of H-naught,
we conclude that the probability it could have come
from H-naught is less than alpha.
We consequently state the null hypothesis cannot be accepted
with one minus alpha level of confidence.
The truth may be that this X-bar did come from
the H-naught distribution but the sample
was from way out in the tail.
If this is so, then we have falsely rejected
a true null hypothesis and we have made a Type I error.
What statistics has done is provide an estimate
with a known probability of being wrong, called alpha,
and this probability of being wrong
is under our control.
We can also see in the graph that the sample mean
could actually be from one of the H-a distributions
within the boundaries set by the alpha level.
Such a case is marked as X-bar two.
There is a probability that X-bar two actually
came from H-a but shows up in the range of H-naught
between the two tails.
This probability is the beta error.
The probability of accepting a false null.
Our problem is that we can only set the alpha error
because there are an infinite number of alternate
distributions from which the mean could have come
that are not equal to H-naught.
As a result, the statistician places the burden of proof
on the alternative hypothesis.
That is, we will not reject a null hypothesis
unless there is greater than 90 or 95 or 99 percent
probability that the null is false.
This system of how we decide what we believe
and how we move from one world view to another
is deeply embedded in the scientific method
which is the guiding principle on such matters.
There is a lot at stake here.
We should be as aware of these issues
as we are of the math formulas we will use to do
the specific tests of hypothesis.
In a real sense, these formulas are the mechanics
while these issues are the philosophy
which has a powerful impact on policy.