Hi. Today, we're going to be looking at randomized controlled trials,
some of their advantages, some of their disadvantages,
and how they relate to observational studies.
In later lectures, we'll give you some specific examples
of these trials applied to development.
Today is going to be an overview.
Let's get going.
So, what is a randomized controlled trial?
The basic ideas are pretty simple.
We begin with a population of interest or one of availability;
I'll talk a little bit more about that latter point somewhat later.
We're then going to pick somebody from this population.
Let's suppose this guy.
We then flip a coin.
If the coin comes up heads,
we're going to put that person in what we're going to call the treatment group.
If the coin comes up tails,
we put the person in the control group.
Let's suppose our first person ends up in the treatment group.
We then do the same thing again.
Perhaps, on the second time, the person ends up in the control group.
We keep going like this, flipping a coin every time,
until we have a sufficiently large treatment and control group.
Now, what do I mean by sufficiently large?
What do I mean is, large enough
so that we are confident, to within a small probability,
that these groups are statistically identical,
that these groups are statistical doppelgangers.
They are similar in all respects,
both in observed characteristics and in unobserved characteristics.
The mathematics of statistics tell us
how large these groups have to be
in order to get our confidence up to whatever rate we want.
So, these groups; we make them large enough so that they are statistically identical.
That's important
because, now, what we do for the treatment group;
we give them some treatment. For example, perhaps, we give them bednets.
Because randomization has made these groups identical,
the only difference is that one group has the treatment
and one group does not.
So, now we follow these groups over time
and we observe the difference that the treatment makes.
Perhaps, for example, we measure mortality rates in the two groups.
By the way, when this experiment has been done in India and in Africa,
with bednets,
it's found that the mortality rate among the group given bednets
is indeed significantly lower than the group without the bednets,
particularly for children under the age of five.
That's one example of a randomized control trial from development.
Okay.
Why would we want to do this? Why do we want to go through all this trouble?
That's what a randomized control trial is.
What are the advantages? Why are we doing this?
Let's take a look.
Randomized controlled trials have significant advantages,
but they can also be quite expensive,
which is one reason why the tradition in most of social science
has not been to use randomized controlled trials,
but has been to use multivariate regression in an observational study.
Now, the idea of an observational study is to take data which has been collected;
usually for some other reason,
say, by the government in the census;
and to try and use that data and statistical techniques
to answer your question of interest.
So, something like this equation has probably appeared in
millions of papers in the social sciences and elsewhere.
It's a multivariate regression.
Suppose, for example, we're interested in studying
the effect of a job training program on income.
So, Y would be our outcome of interest, in this case income.
T would be a variable denoting whether the person was treated or not;
0 or 1,
1 if they entered the job training program.
Then, all these X's are control variables.
So, these X's are all of the other things which might affect income
other than the treatment program itself.
So, for example, you might include the workers age,
the workers experience level,
the geographic location of the worker.
All kinds of other things as well.
Now, if you've included all of the major control variables,
then you can use statistics to deduce what's the effect of the treatment.
However, if you fail to include some important control variables,
which end up buried here in the error term.
And if the control variables, which you have failed to include;
if they're correlated with whether the worker has been treated or not;
then your estimate of the effect of the treatment on income will be biased,
will be mismeasured.
Now, this problem was understood at a theoretical level for a very long time.
However, more recently,
we've managed to compare randomized controlled trials
with the results of observational studies
and the differences have been quite shocking.
Let me tell you a story about this.
The story I want to tell you about is a true story about a former British colony.
The healers in this former colony had what seemed like a strange belief.
They believed that the urine of pregnant mares had great healing properties.
Specifically, when distilled down to a powder,
they believed that pregnant mares urine could make older women more youthful,
more feminine,
and that the powder would ward off disease.
The healers in this former colony were very highly respected,
and many women followed their advice and took the powders.
After many years, however, a new group of healers
decided to test this theory.
They did so using a randomized controlled trial.
And they discovered that pregnant mares' urine didn't reduce disease,
instead it caused heart attacks, dementia, and breast cancer.
Now, the most shocking thing about this story,
is that the former British colony that I am talking about is the United States.
And the results from the randomized controlled trials
were announced in just 2002.
The drug I'm talking about, was called Premarin,
pregnant mares urine,
and it was an estrogen treatment
which was prescribed as a treatment of hormone replacement therapy.
Many women took this hormone replacement therapy
and the arguments for it were not crazy.
For example, it had long been known that women had much lower rates of heart disease
than men until they hit menopause.
And then, during menopause, their rates of heart disease
quickly caught up to those of men.
Now, during menopause, estrogen declines tremendously.
So, the argument was that, with this hormone replacement therapy,
you could actually ward off heart disease.
Moreover, this wasn't just a theory.
In observational studies,
it was discovered that women on hormone replacement therapy,
in fact, did have lower rates of heart disease.
They also had lower rates of incontinence, lower rates of depression,
lower rates of Alzheimer's.
One negative was that they did have higher rates of breast cancer.
But, when calculating the total mortality rate,
it was suggested that hormone replacement therapy would actually reduce mortality
which is why many women took hormone replacement therapy
and, in fact, it became a standard prescription
for women entering menopause.
Not everyone was convinced however,
and it was decided; since so many people,
so many women, were on hormone replacement therapy;
that it would be a good idea to do a randomized controlled trial.
This actually was controversial, because some people said, "Look.
"The evidence is so strong in favour of HRT
that to do a trial would be unethical."
But the trial went ahead with 16,000 women.
This is the Women's Health Initiative.
Women were randomly assigned to either get the treatment,
the hormone replacement therapy, or to get placebo.
And what was discovered, over time, was shocking.
The women who got the hormone replacement therapy
had higher rates of heart attack,
higher rates of breast cancer,
higher rates of dementia.
This really shocked the medical profession
and people began to ask,
"How could the results of the randomized control trial
be so different from the observational studies?"
Now, even though the observational studies controlled for all kinds of factors,
you know, age and race and so forth, lots and lots of things,
it was discovered that the women who took the hormone replacement therapy;
they had more access to the health system,
they tended to be wealthier,
they tended to be better educated,
they tended to be more interested in prevention.
And, so, they were probably doing a lot of other things
which, in combination, reduced their heart attack rates.
But, it wasn't the hormone replacement therapy,
which was reducing the heart attack rates,
it was all the other things that women who got the hormone replacement therapy,
who chose the hormone replacement therapy.
It was all the other things that these women were doing which reduced
the heart attack rate.
When you looked, however, at randomly assigned groups,
it was clear that HRT was actually a very bad idea.
In fact, looking retrospectively,
it's now estimated that because so many additional women
were taking the hormone replacement therapy,
that there may have been as many as 200,000 additional cases of breast cancer
than there otherwise would have been.
This really indicates the importance of randomized controlled trials.