We are now going to conduct a paired-samples t-test
using SPSS.
We will be using the Puppy Training data set, which is the same data set that we used for the independent samples t-test.
Here is the setup for our research.
There are two popular ways to train dogs.
One involves using a clicker which creates a sharp click that reinforces when the dog has done something right.
The other method, of course, is food training or treats in which the dog gets a small morsel when he responds correctly.
Because this is a paired-samples t-test the researcher is going to use matched pairs of dogs in each group.
Mickey is a dog trainer studying reinforcement effectiveness.
He gets 16 puppies from 8 litters with 2 puppies from each litter.
He matches each puppy in clicker training with its litter mate in food reward training.
After training the puppies, he measures the number of correct responses by each puppy in 10 exercises.
Was there a difference between the two training techniques?
Let's walk through the five steps of hypothesis testing.
Step number one: select the appropriate statistic
We will compare two dependent samples.
One sample is the puppies who were trained using the clicker training, and the second sample was their litter mate, the twin
sblings, if you will, who were trained with food reward.
Because they are matched pairs, their scores are not independent to compare these two samples.
We will use a paired-samples t-test.
Step two: state the null and alternative hypothesis
We want to know if the means of the two samples of puppies are different.
We're not saying which group will be better,
so this is a two-tailed test. If we said that one group would be better, it would be a one-tailed or directional test.
Because this is a two-tailed test, our null hypothesis will be either
μ1 = μ2 or
μD = 0, the mean difference equals zero.
That means that our alternative. hypothesis would be:
H1: μ1 ≠ μ2
or μD ≠ 0.
Step 3: select a level of significance
We have already established that this is a two-tailed test.
Our alpha level is going to be .05.
The degrees of freedom for the paired samples t-test is N-1, where N
is the number of pairs of samples.
We have 8 pairs of puppies divided into two groups, so N-1 = 7
From student's t-distribution table, the critical value for a two-tailed test,
alpha = .05,
degrees of freedom =7, is a critical value
of +/- 2.365.
So any t-values exceeding 2.365
indicate that the sample of puppies being trained with clicker training
was statistically significantly different than the puppies being trained with food reward training.
Alternatively, you can download the RStats effect size calculator for t-tests and use it to determine your critical value.
Here you can see the six things that you can do with this effect size calculator spreadsheet.
Click on the green tab for critical values.
In the upper left we have a typical setup for alpha = .05, two-tailed tests.
Simply enter your degrees of freedom: 7
Click or hit return, and your critical value is 2.365.
But wait.
There's more. if you are not doing the typical two-tailed alpha of .05 test enter your degrees of freedom here.
Click and you get critical values for both one and two tailed tests at a variety of alpha levels.
You would use this box if you were doing a t-test by hand.
Step 4: calculate the statistics
So now we need to turn to SPSS
The first of the two variables that we will use is called Clicker. It has 0 decimals and is at the scale level.
The second variable is Food which also has 0 decimals and is also at the scale level.
We have two scale level variables that have been matched for each pair of dogs.
Now click over to Data View.
Let me remind you that I created this data. set specifically for this example with both an
independent and paired test in the same data set, and this is an extremely stupid way to set up data.
Don't ever do this.
The only reason that I did this was to show you the difference between two types of t-tests with exactly the same data, and
so that you would have one data set for both tests.
In the real world, always keep your independent and paired designs separate.
Now we are ready to do an analysis
Click on "Analyze"
"Compare Means"
"Paired Samples T-Test"
We need to place the variables according to their pairs. If you are seeing variable labels,
you can right click on any variable name and select "Display variable names"
If this was a before and after design, we would enter the Before first and the After score second.
In our matched pair design we will enter the Clicker group as Variable1 and the Paired food group scores as
Variable2.
When you enter these variables you will notice that SPSS
automatically adds a drop zone for a second pair of variables.
If you had a larger data set, you could add more variables here.
SPSS would give you a third pair and so on.
We only need this first pair.
Now the options button does not have any interesting options. The only thing worth pointing out are these arrows.
If you had multiple comparisons going on, this arrow would move the pairs of variables up one level.
This arrow moves the pairs of variables down one level, and this one
is the order of the pairs of variables.
Now click "OK"
In the output window we see the paired sample statistics.
We can see the means and standard deviations for each group along with sample sizes and the standard error of the mean
We will use these for the APA write-up.
We also get a box for paired sample correlations.
Now this is a correlation between the two samples based upon their paired data.
This box tells us two things.
First, this is a fascinating glimpse into the reality that all of the tests that we do test that might seem very
different, like t-tests and correlations, are all actually very similar.
They are all applications of the general linear model.
Later we will learn a form of correlation called a point-biserial correlation.
That is actually the same math as an independent measures t-test in reverse.
The second thing we can learn from this box is that this is not the t-test results box.
Do not use this box for interpreting the t-test.
This correlation significance value can be significant while the t-test is not significant, or in this case, the other way around.
This next box gives us the results for the paired samples t-test.
We do not need to do a Levene's test for equality of variances for the paired samples t-test
because this test does not have an assumption of homogeneity of variance.
Although we are treating them like two groups, any
variance in the measurements between the two groups would be due to random error.
All of these findings are under the heading "Paired differences"
So this mean is the mean of the paired differences.
This is the standard deviation of the paired differences, the standard error of the paired differences, and so on.
If you subtract the food mean from the clicker mean, the difference between these two means would be
1.25
So how can we interpret these findings?
First the t-test value is 2.758, which exceeds the critical value of 2.365.
Degrees of Freedom is 7.
We can see that the significance value for the t-test is
.028, which is less than .05.
We also see the 95% confidence interval for the difference between paired sample means
shows a lower bound of +1.78 and an upper bound of 2.322.
Because this confidence interval does not cross 0, it does not include the null hypothesis value of
0 difference. All three of these findings tell us the same thing.
There is a statistically significant difference between these two means.
Now let's go back to our five steps of hypothesis testing.
Step four: calculate the statistics
The t-test value was 2.758, which we will round to 2.76.
The critical value is 2.356.
The obtained t-value exceeds the critical value. That means that the t-test is
significant, and the two groups are statistically significantly different.
Finally, step five: make the decision.
Clicker training was more effective than traditional food reward training.
t with 7 degrees of freedom = 2.76, p = .028
To calculate Cohen's d effect size we will use the RStats effect size calculator for t-tests.
We are calculating a Cohen's d for
paired-samples t-tests, so click on the tan colored tab for effect size within.
Here we have three options. The first option would be for if we already had output from the t test.
Enter the sample size for the number of pairs: 8
And the t value: 2.758
This will give us Cohen's d and Hedge's g, which would be most useful if we were conducting a meta-analysis.
But this Cohen's d is actually a little high.
Using Cohen's formula overestimates the effect size because the paired variables are correlated.
Therefore. Dunlap, et al. have argued convincingly for adjusting the effect size
by the strength of the correlation between the variables.
Here I am calling this Dunlap's d-sub c, for corrected d.
To get this value we need to enter the correlation from the paired-samples correlations box, .683.
And we get the adjusted effect size.
The effect size is calculated using Dunlap's correction was 0.776.
According to Cohen's conventions, this is approaching a large effect size.
The reference for Cohen's effect size and its interpretation,
Hedge's g, and the formulas that I used are below the table.
The second option, or maybe sub option, in blue is for when you have not yet run a t-test,
but you have the group descriptive statistics. You can enter the mean and standard deviation for each group.
Group 1 Mean: 6.38
Standard deviation: 1.408
Group 2 Mean: 5.13
Standard Deviation: 1.727
Plus we still have the N for the number of pairs.
That will give you Cohen's d and Hedge's g but not the Dunlap correction, which of course, requires a
correlation coefficient.
This third option uses the output from SPSS
to get the measures of effect size. Using this option you should go to the SPSS output window.
Double-click on the paired samples test box activate it.
Here we are using the paired differences output.
These are the mean of the paired differences and the standard deviation of the paired differences.
Highlight
Copy
Then flip back to the RStats effect size calculator for t-tests, and paste the output here.
Now go back to SPSS, and this time from the paired-samples
correlation box. Grab the sample size and correlation.
And while we are here, notice that t = 2.758.
Back to the spreadsheet.
Paste the values into the spreadsheet, and
finally enter the t-value that we memorized. 2.758 and click.
You can see that the Cohen's d value here, 0.9, is much larger than the Dunlap corrected value.
When we analyzed this same data with an independent samples t-test Cohen's d was
.79, so Dunlop's correction for paired-samples,
.78, is much closer.
Although I'm going to report Cohen's D in this lesson, I would definitely report the corrected effect size by Dunlap,
.78, for a publication.
Here is how we would write up these results in APA style.
The groups differed significantly.
The mean for the clicker training group was statistically significantly higher than the food reward training group.
These findings support the idea that clicker training is more effective than traditional food reward training.
You can see that
I included the means and standard deviations for each group, as well as Cohen's d effect size and the confidence intervals.
That last statement about the meaning of the findings would go in your discussion section if you were writing this up for a paper.
Finally, if you look at the data for these two tests you will notice something remarkable. The data are exactly the same.
The group means are exactly the same in both tests, and yet the independent samples t-test was not significant
But the paired samples t-test with exactly the same numbers was significant.
This illustrates that a paired-samples t-test, and
repeated measures in general, is more powerful than an independent samples t-test.
That means that they are more likely to find a statistically significant difference between groups.
It doesn't mean there's anything wrong,
but only that you should consider the power of your test when you do your research design.