>>We’ve talked about generating confidence intervals and performing hypothesis tests
about the difference in two population means. So far, we’ve assumed that those two populations
are independent of each other. That is, we collect a sample from population one and we
collect a sample from population two, and those two samples, as well as those two populations,
don’t affect each other. For example, if we collect samples of maple and samples of
oak lumber, and we perform strength tests on each, the strength measurements of maple
don’t affect the strength measurements of oak, and vice versa. The strengths of maple
lumber are independent of the strengths of oak lumber. And we make our comparison of
these strengths of these two populations with the assumption of independence.
But there are certain situations where the independence assumption fails. This is best
illustrated with a few examples. We might want to determine if a training course improves
the productivity of operators that work on a factory floor. Now we could collect a sample
of operators who have had the training and measure their productivity. And we could collect
a sample of operators who haven’t had the training and measure their productivity. Then
we could compare their means of productivity with the techniques that we’ve already talked
about. But that’s not the most effective way to address the specific effect of the
training course. The best way would be to measure the productivity of a sample of operators
who haven’t had the course. Then let them sit through the course. And then measure their
productivity afterward. We could compare their productivity before and after the course,
considering those to be two populations: the “before” population and the “after”
population. And we could answer hypotheses about how different those two populations
are.
But those two populations contain the same individuals. The operators are in both the
“before training” and “after training” populations. The populations are not independent
of each other.
As another example, we might ask the question: did change in tire brand impact fuel economy
of taxis? We could collect a sample of taxis with one brand of tire, and a sample with
another brand of tire, calculate the fuel economy of each, then make a comparison. And
if we only have data describing those two populations separately, then we have to assume
independence and use techniques that require the independence assumption.
However, if we are designing this experiment from scratch, we could measure the fuel economy
of a sample of taxis whose tires are all of a specified brand. Then we could change the
tires on that sample, and measure the resulting fuel economy after the change. That would
get directly at the effect of the tire brand on fuel economy.
With both of these example, the two populations that we’re interested in are “before”
and “after.” Before some change we take a sample and measure something about that
sample, then we make the change and measure on the same sample. In such a case, we have
to intentionally design an experiment such that the independence assumption fails. We
have to recognize in advance that the best way to answer our hypotheses is a before-and-after
kind of experiment where the sample is in both populations. We call this situation a
paired sample or a matched sample.
As we said, each observation belongs to both populations. How we analyze the data for each
observation is by taking the difference in outcomes for each observation. Let’s look
at an example: say we have a number of observations n, and each of those n observations belong
to both Population one and Population two. For each observation, we take the Population
one measure and subtract from it the Population two measure. In our example, for observation
one, the measure taken from Population one is fourteen and from Population two is seven.
So the difference is seven. For the second observation, the difference between Population
one and Population two is thirteen minus nine, or four. We find this difference, d-sub-i,
for all n observations.
Then we take the sample mean and sample standard deviation of the differences. That is, for
the mean d-bar, just like we’ve always done, we sum up the differences and divide by n.
For the standard deviation, we take the difference between each difference and the mean of differences,
d-bar, square it, sum it over all n observations, then divided by n minus one.
And so, d-bar and s of d are our point estimates for the mean paired difference and standard
deviation of paired differences for the populations. The actual mean paired difference is denoted
as mu-sub-d and the standard deviation of paired differences is sigma-sub-d.
We might be interested in estimating mu-sub-d with d-bar. Of course, we know that we can
do better than a point estimate. Let’s look to formulate a confidence interval as the
point estimate, d-bar, plus and minus a margin of error. And that margin of error, as we
well know, is a function of variability, sample size, and confidence.
Our one minus alpha times one hundred percent confidence interval is pretty straightforward.
It’s d-bar plus and minus t for our particular degrees of freedom with alpha over two in
each tail multiplied by s of d over the square root of n. We’re one minus alpha times one
hundred percent confident that mu-sub-d, the true mean paired difference, is inside the
bounds somewhere. The degrees of freedom is calculated with n minus 1.
Let’s take a look at an example. Say that we’re assembling components into a larger
radar assembly. We have a standard approach to assembling these radars, and we want to
experiment on a new assembly approach that might eliminate some unnecessary movements
and ultimately reduce assembly time. We collect a sample of twelve operators, and we measure
how long assembly takes using the standard approach. And then we measure how long it
takes for the new approach. When we treat this as a paired sample, we account for individual
performance using the two approaches. For example, some operators may be a little quicker
than others naturally: we account for that with the paired sample.
With operator one, the assembly time with the current approach sixty-three point three
seconds, and with the new approach, it was fifty-seven point eight seconds, for a difference
of five point five seconds. We find the difference calculation for all twelve operators. The
sample mean paired difference, d-bar, is four point eight three seconds. And the standard
deviation is one point seven six seconds.
And so we want a ninety-five percent confidence interval for the true mean paired difference.
Our measure of confidence, the t value with point oh two five in each tail is calculated
for n minus one or eleven degrees of freedom. If we go down to the row in the t table for
eleven degrees and over to the column associated with point oh two five in the tail, we get
a t value of two point two zero one. So, our confidence interval is calculated as four
point eight three plus and minus two point two zero one times one point seven six divided
by the square root of twelve. The bounds on the confidence interval are three point seven
one seconds and five point nine five seconds. That suggests that we’re ninety-five percent
confident that the true mean paired difference between the current standard assembly approach
and the new approach is between three point seven one and five point nine five seconds.
Note what it means to have two positive bounds: it says that the current approach minus the
new approach leads to a positive number. That means that the current approach must be larger
– it must take more time. So we’re ninety-five percent confident that the current approach
takes longer than the new approach by somewhere between three point seven one and five point
nine five seconds.
We can also form hypotheses about the mean paired difference. For example, we might want
to know if the actual mean paired difference, mu-sub-d, is D-naught or something other than
D-naught. In such a case, we have mu-sub-d equal to D-naught in the null hypothesis,
as equality is always in the null. And the contradictory statement, mu-sub-d not equal
to D-naught, in the alternative hypothesis. That is, is the mean paired difference equal
to some value, or is it not? And as always, we have two other options for framing our
hypotheses: greater than or equal to in the null with less than in the alternative, or
less than or equal to in the null and greater than in the alternative.
The test statistic has a similar look to what you’d expect. It’s a t test statistic
that measures the difference between the sample evidence, d-bar, and the hypothesized value,
D-naught, in standard deviation units, s of d over root n. And we could use the critical
value or p-value approaches the same way that we’ve always used them.
Let’s look back to our assembly time example. Say we want to know whether the mean difference,
current standard minus new approach, is more than four seconds. More than four seconds
implies strictly greater than. Since that’s not an equality statement, greater than must
be in the alternative hypothesis. Therefore the contradictory statement that includes
equality must be in the null hypothesis, mu-sub-d less than or equal to four seconds. Recall
that d-bar for our example is four point eight three seconds and the standard deviation of
sample paired differences is one point seven six seconds.
Implementing the test statistic, we get t-naught of one point six three. If we check out the
t table, we find that for eleven degrees of freedom, we find a value of one point three
six three on the t axis associated with point one in the tail and a value of one point seven
nine six on the axis associated with point oh five in the tail. Our test statistic of
one point six three is found between these two values. That suggests that the p-value
is between point oh five and point one. Depending on how much confidence we want to have in
the result, that might be compelling enough for us to reject the null hypothesis and conclude
that the mean paired difference is indeed greater than four seconds. If we use the tdist
function in Excel, we find the p-value to more exactly be point zero six six.
Note that those without the course in statistics might say “well heck, the sample mean of
four point eight three is obviously more than 4 seconds so reject the null.” But we know
better than that. We know that we have to be compelled by the sample evidence when we
account for variability and sample size, and we want to have some amount of confidence
in the result. We can’t just take a point estimate and draw a conclusion – we need
to use the methods of inferential statistics. And in this case, the decision is fairly close.
A p-value of point zero six six is only somewhat compelling.
And so this concludes our discussion of paired or matched samples, where the two populations
whose means we’re studying the differences of are not independent. We see the need for
a paired sample test arise often in before and after studies, though any situation where
the same observations make up both samples would require a paired sample approach.