So let's figure out how to do a paired-samples t-test in JASP.
We will be using the PuppyTrainingPaired.CSV data set, which is the same data that we
used for the independent samples t-test. In fact, if you have not already seen it,
I would encourage you to watch the independent samples t-test video first,
before this one, because these two approaches to the t-test
are designed to be taught side-by-side. So here is the setup for our research:
There are two popular ways to train dogs. One involves using a clicker, which
creates a sharp click that reinforces when the dog has done something right.
The other method, of course, is food training, or treats, in which the dog gets
a small morsel when he responds correctly. Because this is a paired
samples t-test, the researcher is going to use matched pairs of dogs in each group.
Mickey is a dog trainer studying reinforcement effectiveness. He gets 16
puppies, from 8 litters, with 2 puppies from each litter. He matches each puppy
in clicker training with its litter mate in food reward training. After training
the puppies, he measures the number of correct responses by each puppy in 10
exercises. Was there a difference between the two training techniques?
Let's walk through the five steps of hypothesis testing. Step 1: select the appropriate
statistic. We will compare two dependent samples. One sample is the puppies who
were trained using clicker training, and the second sample was their litter mates -
- the twin siblings, if you will - who were trained with food reward. Because they
are matched pairs, their scores are not independent. To compare these two samples,
we will use a paired samples t-test. Step 2: state to the null and alternative
hypothesis. We want to know if the means of the two
samples of puppies are different. We are not saying which group will be better,
so this is a two-tailed test. If we said that one group was better, this would be
a one-tailed - or directional - test. Because this is a two-tailed test, our null
hypothesis will be either H0: mu1 = mu2, or mu- D = 0
The mean difference equals zero. That means that our alternative
hypothesis will be H1: mu1 does not equal mu2, or mu - D does not equal 0.
Step 3: select a level of significance. We have already established
that this is a two-tailed test. Our alpha level is going to be .05
The degrees of freedom for the paired samples t-test is n - 1, where
n equals number of pairs of samples. We have 8 pairs of puppies divided into the
two groups, so n - 1 = 7. From students T-table, the critical value for
a two-tailed test, alpha .05 degrees of freedom = 7, is positive or negative 2.365.
So any t-values exceeding 2.365
indicate that the sample of puppies being trained with clicker training was
statistically significantly different than the puppies being trained with food
reward training. Step 4: calculate the statistics. So now we need to turn to JASP
Begin by placing the puppy training data on your computer desktop.
We will open our data set by going to our main menu icon -> Open -> Computer -> Desktop
PuppyTrainingPaired.CSV and then clicking open. Let's begin by looking at
our data. The first of the two variables that we will use is Clicker, which is the
number of tasks correct out of ten, for the puppies trained with clickers.
The second variable is Food, which is the number of tasks correct out of ten for
the puppies trained with food rewards. Both of these are scale variables, but
they were imported as ordinal. On the left edge of each box, we can click here
to change column type, and set it to Scale. Do not click on the variable name,
or you will be changing variable labels. Click on the left edge of the box.
And now we are ready to do the analysis. Currently, I have six basic analysis
options available. I could add more modules with the plus sign, but I don't
need to do that now. Basic modules are exactly what I need. Select t-tests ->
paired samples t-test. We need to place the variables according to their pairs.
If this was a before and after design, we would enter the before first and then
the after score. In our matched pair design, we will move the clicker group
into the variables pairs box first, and then their paired food groups scores
second. If we had a larger data set with more variables, we could add more pairs.
Reversing the order of the pairs gives us exactly the same output, so we will
not need to do that. In the Results window, we see the paired samples t-test
statistics. We can already see that the p is less than .05, so this
test is statistically significant. This is a good start, but I want to know much
more. All I need to do is select the options that I prefer. We have an option
for the nonparametric alternative Wilcoxon signed-rank test, but we will
not need to do that now. Notice that we do not have a Welch's t-test option,
because, again, we do not have to account for pre-existing variance. Because we are
testing paired groups - or sometimes the same group - any variability between the
groups is coming from the main effects, and not pre-existing variability;
therefore, we do not need to check for homogeneity of variance with the
Levine's test, like we did with an independent samples t-test. This is not
an issue with this research design. There is no homogeneity of variance among the
assumptions tests, but we will check the assumption of normality. Here we notice
that we may have a problem. The assumptions check says that these
results suggest a deviation from normality. The probability is less than .05
What should we do? Well, we have the option of transforming the
data set by squaring the values or by using an inverse, but these
transformations can make interpreting the findings much more difficult, so
first we need to consider how badly violated the assumptions can be before
we start to worry. While we tend to interpret t- tests and other hypothesis
tests at the p = .05 level, for assumptions tests we usually
use the p = .01 level and with larger data sets it is very
common not to transform data or drop outliers unless they exceed p < .001.
So we are not going to worry about this p = .037
deviation. Our results will still be valid. You recall from Step 2 that our
alternative hypothesis is that the means of the two groups are not equal? We could
also choose one tailed test options here, if we wanted to. Under "Additional
Statistics," I am going to get a Location parameter, with 95% confidence intervals.
I want a Cohen's d effect size, descriptive statistics, and a descriptives
plot. In the output window, we see the group statistics. I want to begin by
scrolling to the bottom. I always like to see a picture of the data before I begin
interpreting. I see that the food reward group mean was lower than the clicker
training group. There is a fair amount of overlap in those error bars, but not as
much as when we examined the plot for the independent samples t-test. The top
of this food group error bar is still lower than the mean of the clicker group.
I would probably not include this plot in my write-up or in my paper, but it is
a good place to begin. I can also see the descriptive statistics for each group.
The food group was more than a full point lower than clicker training. These
will be important to report later. With these thoughts in mind, let's scroll back
to the top. We see the t-test results, the probability, and the mean difference.
If you subtract the food mean from the clicker mean, the difference between
those two means would be 1.25. We have our confidence intervals, and out here to
the right, we see the effect size. The Cohen's d is a .975
We'll round that up to a .98. This is a much larger effect size
than the .80 we found with the independent samples t-test.
One reason that this Cohen's d is larger is because these two variables - Clicker and
Food - are correlated, and that inter- correlation between the variables is
inflating the Cohen's d. There is a way to correct for this using a formula, but
I will cover that in another video. So how can we interpret these findings?
First, the t-test value is 2.758, which exceeds the
critical value of positive or negative 2.365. Degrees of
freedom is 7. We can see that the significance value for our t-test is
.028, which is less than .05, and we also see
that the 95% confidence interval of mean differences between paired sample
means, shows a lower bound of positive 0.178 and an upper bound of positive 2.322
Because this confidence interval does not cross zero, it does not include
the null hypothesis value of 0 difference, so the groups ARE different.
All three of these findings tell us the same thing: there is a statistically
significant difference between these two means. In addition, we see Cohen's d is a
.98, which is a very large effect. Let's go back to our five steps
of hypothesis testing. Step 4: calculate the statistics. The t-test
value is a 2.758, which we will round to a 2.76.
The p-value is a .028. Using the analogy of dollar, this
p-value is2.8 cents, which is less than a nickel or .05.
That means that the t-test is significant and the two groups are
statistically significantly different. Finally, Step 5: make the decision.
Clicker training was more effective than traditional food reward training. Notice
that I list the means and standard deviations for each group and the full
write-up for the t-test. Here is an example of how we might write up these
results in APA style: first, I would describe my research design and how I
conducted the experiment, then I would describe my assumptions checks. We will
interpret the Shapiro Wilks test at the .01 level, which is common for
assumptions checks, and we will not transform these data to correct for that
non-normality. They do not deviate from normality badly enough to warrant that.
Then I would report the findings from my t-test
including the confidence interval and the effect size. I would also include the
means and standard deviations for each group, and those come from the
descriptive statistics. Notice the conclusion: these findings support the
idea that clicker training is more effective than traditional food reward
training. This last statement about the meaning of the findings would go in your
discussion section, if you are writing this up for an academic paper.
And finally, look at the data for these two tests: you will notice something
remarkable. The data are exactly the same! The group means are exactly the same in
both tests, and yet the independent samples t-test was NOT significant, but
the paired samples t-test - with exactly the same numbers - was significant.
This illustrates that a paired samples t-test and repeated measures in general is more
POWERFUL than an independent samples t-test. That means that we are more
likely to find a statistically significant difference between the
groups. That does not mean that there's anything wrong with using an independent
samples t-test, only that you should consider the power of your test when you
do your research design.