Whether you’re DESIGNING an experiment or EVALUATING one, you need to understand whether
the results are VALID. What even is VALIDITY? And what sorts of validity should you be aware of?
*** The purpose of scientific experimentation
is to be able to say: we KNOW, with a high degree of confidence, that A causes B. We KNOW
that being bitten by a werewolf turns the victim into a werewolf. How do we know this? Because we
ran 4000 trials and every time the werewolf bit the participant that’s what happened.
Let’s break down the concept of validity and use a classic psychology experiment as our example.
In 1963, Stanley Milgram shocked his fellow psychologists by demonstrating that perfectly
ordinary people would murder a stranger if instructed to by an authority figure. This
landmark obedience to authority study is a great one to pick apart when thinking about validity.
By and large there are two umbrella categories within the overarching concept of validity:
internal validity and external validity. When considered,
these factors help us determine whether a study’s results are indeed valid... or true.
Internal validity looks at what is happening WITHIN the study. Does the study test what
it says it’s testing? If all of Milgram’s participants completed the experiment at midnight
after fasting all day, would the results definitely be a consequence
of the authority figure’s instructions, or was tiredness a factor? … or hunger?
To control for these potential confounding variables, the researchers tested participants
at different times of day so that they were able to say with increased confidence that
their independent variable – the authority figure, with his instructions – was indeed
causing the resultant participant behaviours – electrocuting the victim… our dependent variable.
Controlling confounding variables helps researchers improve the internal validity
of their research. Milgram’s study, however,
lacks a different type of internal validity: mundane realism. Psychological studies want to
answer the question of how people behave in the real world, and therefore it’s critical
for studies to be designed with mundane realism. They need to mirror real life as much as possible.
If the situation in the laboratory is too unusual, we might not be learning about how
people behave in the real world… just how they behave in a weird, unnatural, experimental lab.
When evaluating Milgram’s study, Perry (2012) found that many of Milgram’s
participants had been sceptical about whether the electric shocks were real.
Because they didn’t necessarily believe in the realness of the study,
participants may have acted differently from how they would in day-to-day life.
So if these are INTERNAL things, what’s left for EXTERNAL validity?
External validity is all about the GENERALISABILITY of the results.
A study’s external validity is intrinsically connected with its internal validity (of course),
and study’s that are weak in one are often weak in the other.
Let’s look at three examples of external validity. ECOLOGICAL validity refers to the environment
in which the research was conducted. Are the results from a laboratory study generalisable
to indigenous tribes in the rainforest? Because the environmental factors are so different,
possibly not. This connects with mundane realism; researchers want their studies to
reflect the real world as much as possible. As an aside, the "real world” looks very different
depending on where you are and what you’re doing, so a study of Amazonian tribes has a
very different looking “real world” from a study of retail behaviours in an American shopping mall.
POPULATION validity is another type of external validity, this time focusing on the participants.
If all of the participants of a study are male (as was the case in Milgram’s study),
we have to ask if this is really generalisable to the wider population, or just to men. If all
of the participants are in their twenties (as is often the case in research that uses university
students as participants), can we generalise the results to people in their fifties or sixties?
To ensure their studies have population validity, psychologists have to make a concerted effort when
recruiting participants to make sure that they have not succumbed to a sampling bias.
A third example of external validity is HISTORICAL validity. This considers a study’s
generalisability across time and eras in history. Continuing with the example of Milgram’s work,
this emerged out of the horrors of World War II and the holocaust.
There was a certain societal zeitgeist in the 1950s and 60s that simply doesn’t match today’s
world. Can Milgram’s experiments with a lab-coated authority figure really provide an insight into
the way we are influenced by the bikini-sporting tik-tok’ers of today? Possibly not … though
if an influencer ever asks you to electrocute someone, it’s probably best to unfollow them!