The test that we will learn about now is called a paired samples t-test.
The paired samples t-test allows us to compare means between two paired or related samples.
There are two ways to create paired samples:
repeated measures and matched samples.
The paired samples t-test uses repeated measures.
Repeated measures are exactly what they say on the label. Each
Individual is measured repeatedly.
This is typical of a before and after design. The same subjects are used in all treatment conditions.
The measures of those subjects are repeated, so for example, a
repeated measures design would be if the same group of men is weighed before and after an exercise program.
They are weighed once before the program, and then again after the program. This design is sometimes called a
dependent measure because the score on the second measure
depends upon the score from the first measure. If you lost 100 pounds,
the meaning of that 100 pounds depends on whether your starting weight was
200 pounds or 400 pounds, so in that sense each subject is his or her own control group.
Your score on the pretest is your starting point and the comparison for your score on the post-test.
This same amount of change means something different depending on where you started.
This makes repeated measures more powerful and more likely to find
statistical significance.
Another way to use a paired samples t-test is with matched pairs. With matched pairs or matched samples,
pairs of subjects are either naturally occurring or
deliberately matched with control subjects who share the same characteristics,
So, for example,
naturally occurring pairs would a twin study
because identical twins share so much of their genetics in common,
we cannot assume that their scores are independent of each other. In fact, their scores are very related, and
they allow us to control for the effects of genetics.
So instead we take the twin pairs,
and we group them based upon twin who exercises more versus twin who exercises less
So each twin will be in a different group.
We will have two groups, but the groups are not independent of each other.
Another way to use matched pairs is to deliberately match subjects with control subjects who share the same characteristics.
We are studying patients with a heart condition who are in the hospital
For our control group, we cannot simply use healthy individuals who have not been hospitalized.
We need to find people who are very similar
except for not having the heart condition.
So we deliberately match each heart patient with a similar hip replacement patient who is the same age,
gender, etc as the heart patient We deliberately match patients of similar weight,
smoking history, and other characteristics to try to find someone who is as similar as possible
to our heart patients, but who does not have the heart condition.
Matched pairs of studies are often used when the experiment is testing rare
variables, such as a rare form of cancer or
comparing psychopaths to other incarcerated individuals who are not psychopathic.
Because we are deliberately matching people in each group the groups are not independent of one another.
At the heart of the paired samples t-test is the difference score.
A difference score,
Abbreviated as capital D,
measures the amount of change between measurements.
D equals the final score minus the beginning score.
A difference score is computed
for each individual in the sample.
The null hypothesis is that the average difference in the entire population
will be 0 or μD = 0.
The paired samples t-test is a parametric procedure. It tests whether the average difference between
two measurements is statistically significant from zero. It is often used in
before-and-after designs where the same individuals are measured both before and after a treatment to see if change occurred over time.
For example, a television show
starts with a group of individuals who are very overweight.
Over time through diet and exercise they lose weight.
Here the same sample, which I have called Sample A, is measured before and after.
Their before scores have a mean and standard deviation and
there after scores have a mean and standard deviation.
Is their ending weight significantly
different from their beginning weight?
Now if not, the average difference between the before scores and the after scores will be 0.
Some people will go up a little, some people will go down a little, but overall when you average it all out
the difference is 0.
The paired samples t-test is also called a matched-sample test, related-sample test,
repeated-measures t-test. Now, why so many names?
Statistics developed in many places around the world: Germany, Russia, America, Britain.
Statisticians in each country gave the mathematical formulas different names.
Sometimes those names were translated into other languages.
In the end, we have been handed down many names from many traditions for exactly the same thing.
The important thing to remember is that the assumption for an independent measures design is that the groups are independent.
If your groups are not independent, then you will use this repeated measures design.
Those are your only two options. Regardless of what names they go by.
So let's review the research design for a paired-samples t-test.
You will remember that our research question for each of these examples is the same:
Is recall better under hypnosis or without. We test whether people will recall events better under hypnosis.
As always, we start with an existing memory test that has been standardized on people who are not under hypnosis.
We randomly select participants,
and then we randomly assign people to recall a list of words without hypnosis and the same people are tested again
under hypnosis.
The independent variable is conditioned with vs without hypnosis.
Here, however, we may become concerned about practice effects.
If we test everyone without hypnosis first and then again under hypnosis,
some of the reason for their improved performance might be that they had a chance to practice on the first trial.
In order to control for practice effects we would randomly assign people to participate in either
under hypnosis or not in a random order. So roughly half of our sample was hypnotized during the first trial
while the remainder were hypnotized during the second trial, and that should cancel out any practice effects.
The dependent variable is memory, the number of words recalled.
We may find that the mean for one condition was higher than the mean for the other condition.
If the means of those two groups differed we would be able to tell whether hypnosis effects memory.
Do participants remember more words when they're hypnotized or when they are not hypnotized?
All inferential statistics are built on certain assumptions.
If your assumptions are wrong, then your conclusions will likely also be wrong, so it is important that you check your assumptions.
There are some assumptions for the paired-samples t-test
that you can get away with violating. Others you absolutely cannot. One that absolutely must be true
is that a paired-samples t-test
can only be used to compare two repeated or matched samples.
You must have a before and after design or matched pairs, like twins.
And you can only have two groups. If you have more than two repeated measures,
use a repeated measures ANOVA.
The next non-negotiable assumption for the paired samples t-test is that the dependent variable must be
quantitative and at the scale level, meaning interval or ratio
data like likert survey scale data with at least five answer options.
You will select the scale level dependent variable measure when you establish your research methodology.
You should choose a data collection instrument or survey or measurement that will give you scale level data.
You should also check your dependent variable for errors and outliers when you do your data cleaning, and
data cleaning is the time to make absolutely certain that your variables are scored correctly.
For instance, if you are using a survey with items that are reverse scored.
For example, you need to change high values to low values.
Then make sure that items in your scale have been coded in the same direction.
Now, on the other hand, there are some assumptions for the paired samples t-test in which you have some more flexibility or wiggle room.
The test assumes that you do not have extreme outliers.
If you have outliers you can fix that by removing them, or better yet, winsorizing them to the next highest
reasonable value.
The sampling distribution of the dependent variable should be normally distributed.
So the dependent variable should be drawn from a population
whose sample means are normally distributed.
Assuming that the other assumptions are met, this is not an issue
Especially if your sample size is greater than 30.
See the video on the central limit theorem for more details,
Now, unlike the independent samples t-test
there is no assumption for homogeneity of variance with a paired-samples t-test.
Remember, that in an independent samples t-test
some of the variability in each group is affected not just by random differences
but also perhaps affected by
pre-existing differences between the groups.
We need a Levene's test to evaluate the assumption of the homogeneity between the two groups.
However, in a paired samples t-test
there's no need to do a Levene's test because the same group is being measured twice.
That group is drawn from a single population.
There is no variability with another population. We do not need to evaluate homogeneity of variance between two groups
because we really have one group drawn from one population.
The random variability in that group is the same before and after. Any
change in variability is therefore due to the treatment.
The upshot of all of this is that the paired samples t-test is
robust as long as there is a minimum of 30 subjects in the sample group.
Robust means that the type I error rate does not increase if the assumptions are violated.
Here are the settings for null hypothesis significance testing.
The null hypothesis is that the mean difference between the groups is 0,
which implies that the means of the two groups are equal, or said more accurately,
because the sample is drawn from a single population .
If there is no treatment effect,
then the difference between the before and
after measurements will average out to 0.
So here is how you would write your null hypothesis:
H0: μD = 0, which is the same as
μ1 = μ2.
The alternative hypothesis would be:
H1: μD ≠ 0, which is the same as μ1 ≠ μ2.
The alpha level is typically set to alpha = .05,
but it could be set to .01 or any other level chosen by the researcher.
The degrees of freedom for paired samples is N -1, where N equals the number of pairs.
The critical value is determined based on degrees of freedom and
alpha, and you will look up the critical value in Student's t-Distribution Table, which is also available on the last page of your notes.
So you can assume that the t-test is alpha = .05
two-tailed test unless you are told otherwise.