Not all experiments are created equal, and EXPERIMENTAL DESIGN is an important aspect of
planning and evaluating experiments. Today we’re going to look at three common experimental designs
used in psychological experiments. ***
When we talk about experimental design, we’re not talking about logos and funky graphics on
the researcher’s lab coat. We’re specifically referring to a set of procedures that are designed
into the structure of the experiment in order to control the influence of confounding
and extraneous variables. Good experimental design produces results with high internal
validity – we can trust that the study is studying what it says it’s studying.
When experiments have been designed well, they look obvious,
but it actually takes a lot of thought to design a great experiment, as we shall see.
The three experimental designs we’re going to look at today are: Repeated Measures,
Independent Groups, and Matched Pairs. Experiments using a repeated measures
design have the same participants complete all sections of the experiment. If the research
question is what is the effect of orange smarties on a student’s score in a memory test,
then every participant will eat orange smarties followed by a memory
test … and every participant will eat all the other colour smarties followed by a memory test.
Repeated measures designs are perfect for studies with access to only a small
pool of participants – maybe it’s a study on parenting techniques for children with a
rare developmental disorder. The potential to recruit hundreds of participants isn’t there,
so participants may have to take part in each experimental condition.
Because it’s the same participants repeating the study in each condition, the researchers
are taking repeated measurements from the same people … hence the name, repeated measures.
So, it’s a great design for small participant samples, but what are the limitations of repeated
measures designs? Well, let’s say I asked you to eat orange smarties and then complete a spelling
test … and then I gave you other-coloured smarties and the same spelling test again
… you’ve already had a chance to practise the spelling test, so you may perform better
the second time due to this practice effect. But what if instead I was studying the effect of
colour on physical strength. I put you in an orange room and ask you to complete as
many press ups as you can before your arms start jittering like jelly and collapse.
Then I take you next door to a green room and get you to do the same thing. This time,
you may perform worse on the second attempt, not because of the colour of the walls, but because
you’re already tired. We call this Fatigue Effect. Practice Effect and Fatigue Effect are both
examples of Order Effects, which can influence the results of a repeated measures design.
So, to protect against order effects influencing their results, researchers
implement a counterbalanced design within repeated measures studies. You might know this as AB and
BA … or ABBA. All participants do all conditions, but some do A before B, and others do B before A,
theoretically helping the researchers eliminate order effects from their results.
Of course, the participants are still repeating the experiment multiple times,
which might mean they guess the real aims of the study, which may also change the way
they behave in later conditions. So how do we control for THIS?
We could use an Independent Groups design instead. The benefit of an independent groups design
is that each individual participant will only take part in one condition,
so the chance of them guessing the true aims of the study is much smaller.
Of course, it does also mean you need many more participants.
If your experiment has two conditions (and orange gym and a green gym from press ups), then you’ll
need double the number of participants. But let’s say you’ve been commissioned by a gym company to
investigate the impact of seven different colours on the results of exercise – now
you need SEVEN TIMES as many participants. Once you’ve got your participants signed
up for your independent groups experiment, you’ll need to divide them into separate groups. However,
unlike repeated measures – where MY results in the orange room are compared with MY results
in the green room – this time you could have me in the orange room and the Rock in the green room.
Obviously, I’ll give it my best shot, but something makes me thing your results aren’t going
to differ just because of the colour on the walls. Participant variables like age, gender,
IQ, physicality and so on could mess up your results, so researchers using independent
groups tend to randomly assign participants to their conditions in the belief that each group
will average itself out and extreme results will have less of an impact on the overall results.
But what can be done if a researcher wants to control participant variable
more tightly than relying on laws of averages? That’s were our final experimental design,
Matched Pairs, comes into play. This experimental design is the most
labour intensive as, for every “me” in the orange room group, the researchers have to find as close
to a perfect match as possible for the green room group. In a physical study including exercises,
the researchers might try to match us on biological sex, or height, or weight,
or perhaps on behavioural measures such as whether we smoke or regularly attend a gym.
Other studies might try to match participants on IQ scores, political persuasion, sexuality, age,
educational attainment, socio-economic status and so on… The more factors you try to match for,
the harder it gets – ultimately, you’re looking for a human clone.
So, when designing or evaluating experiments that have used a matched pairs design, be aware
of potential experimenter effects, because the experimenter has made a choice to match for hair
colour, but not eye colour… why? And what impact might this have on the validity of their results?
There’s no perfect experimental design, but there are designs that are better-suited to
specific studies. Why not redesign your favourite experiment and see if it would work as well using
a different experimental design… Do you agree with the original researcher’s design decisions?