hi today we're going to be looking at
randomized controlled trials some of
their advantages some of their
disadvantages and how they relate to
observational studies in later lectures
will give you some specific examples of
these trials applied to development
today is going to be an overview let's
get going so what is a randomized
controlled trial the basic ideas are
pretty simple we begin with a population
of interest are one of availability I'll
talk a little bit more about that latter
point somewhat later we're then going to
pick somebody from this population let's
suppose this guy we then flip a coin if
the coin comes up heads we're going to
put that person in what we're going to
call the treatment group if the coin
comes up tails we put the person in the
control group let's suppose our first
person ends up in the treatment group we
then do the same thing again perhaps on
the second time the person ends up in
the control group we keep going like
this flipping a coin every time until we
have a sufficiently large treatment and
control group now what do I mean by
sufficiently large what do I mean is
large enough so that we are confident to
within a small probability that these
groups are statistically identical that
these groups are statistical
doppelgangers they are similar in all
respects both in observed
characteristics and in unobserved
characteristics the mathematics of
statistics tell us how large these
groups have to be in order to get our
confidence up to whatever rate we want
so these groups we make them large
enough so that they are statistically
identical that's important because now
what we do for the treatment group we
give them some treatment for example
perhaps we give them bed nets because
randomization has made these groups
identical the only difference is that
one group has the treatment and one
group does not so now we follow these
groups over time and we observe the
difference that the treatment makes
perhaps for example we measure mortality
rates in the
two groups by the way when this
experiment has been done in India and
Africa with bed nets it's found that the
mortality rate among the group given bed
nets is indeed significantly lower than
the group without the bed nets
particularly for children under the age
of five that's one example of a
randomized controlled trial from
development okay well why would we want
to do this why do we want to go to all
this trouble that's what a randomized
control trial is what are the advantages
why are we doing this let's take a look
randomized controlled trials have
significant advantages but they can also
be quite expensive which is one reason
why the tradition in most of social
science has not been to use randomized
control trials but has been to use
multivariate regression in an
observational study now the idea of an
observational study is to take data
which has been collected usually for
some other reason say by the government
in the census and to try and use that
data and statistical techniques to
answer your question of interest so
something like this equation has
probably appeared in millions of papers
in the social sciences and elsewhere
it's a multivariate regression suppose
for example we're interested in studying
the effect of a job training program on
income so why would be our outcome of
interest in this case I income T would
be a variable denoting whether the
person was treated or not 0 or 11 if the
enter the job training program then all
these X's are control variables so these
X's are all of the other things which
might affect income other than the
treatment program itself so for example
you might include the workers aged the
workers experience level the geographic
location of the worker all kinds of
other things as well now if you've
included all of the major control
variables then you can use statistics to
deduce what's the effect of the
treatment however if you fail to include
some important control variables which
end up buried here in the error term and
if the control variable
which you have failed to include if
they're correlated with whether the
worker has been treated or not then your
estimate of the effect of the treatment
on income will be bias will be
mismeasured now this problem was
understood at a theoretical level for a
very long time however more recently
we've managed to compare randomized
controlled trials with the result of
observational studies and the
differences have been quite shocking let
me tell you a story about this the story
I want to tell you about is a true story
about a former British colony the healer
is in this former colony had what seemed
like a strange belief they believed that
the urine of pregnant mares had great
healing properties specifically when
distilled down to a powder they believed
that pregnant mares urine could make
older women more youthful more feminine
and that the powder would ward off
disease the healers in this former
colony were very highly respected and
many women followed their advice and
took the powders after many years
however a new group of healers decided
to test this theory they did so using a
randomized controlled trial and he
discovered that pregnant mares urine
didn't reduce disease instead it caused
heart attacks dementia and breast cancer
now the most shocking thing about this
story is that the former British colony
that I am talking about is the United
States and the results from the
randomized controlled trials were
announced in just 2002 the drug I'm
talking about was called premarin
pregnant mares urine and it was an
estrogen treatment which was prescribed
as a treatment of hormone replacement
therapy many many women took this
hormone replacement therapy and the
arguments for it were not crazy for
example it had long been known that
women had much lower
rates of heart disease than men until
they hit menopause and then during
menopause their rates of heart disease
quickly caught up to those of men now
during menopause estrogen declines
tremendously so the argument was was
that with this hormone replacement
therapy you could actually ward off
heart disease moreover this wasn't just
a theory in observational studies it was
discovered that women on hormone
replacement therapy in fact did have
lower rates of heart disease they also
had lower rates of incontinence lower
rates of depression lower rates of
Alzheimer's one negative was that they
did have higher rates of breast cancer
but when calculating the total mortality
rate it was suggested that hormone
replacement therapy would actually
reduce mortality which is why many many
women took hormone replacement therapy
and in fact it became a standard
prescription for women entering
menopause not everyone was convinced
however and it was decided since so many
people so many women were on hormone
replacement therapy that it would be a
good idea to do a randomized control
trial this actually was controversial
because some people said look the
evidence is so strong in favor of HRT
that to do a trial would be unethical
but the trial went ahead with 16,000
women this is the women's health
initiative women were randomly assigned
to either get the treatment to hormone
replacement therapy or to get placebo
and what was discovered over time was
shocking the women who got the hormone
replacement therapy had higher rates of
heart attack higher rates of breast
cancer higher rates of dementia this
really shocked the medical profession
and people getting began to ask you how
could the results of the randomized
control trial be so different from the
observational studies now even though
the observational studies controlled for
all kinds of factors you know age and
race and so forth lots and lots of
things it was discovered that the women
who took the hormone replacement therapy
they had more access to the health
system
they tended to be wealthier they tended
to be better educated they tended to be
more interested in prevention and so
they were probably doing a lot of other
things which in combination reduced
their heart attack rates but it wasn't
the hormone replacement therapy which
was reducing the heart attack rates it
was all the other things that women who
got the hormone replacement therapy who
chose the hormone replacement therapy it
was all the other things that these
women were doing which reduced the heart
attack rate when you looked however and
randomly assigned groups it was clear
that HRT was actually a very bad idea in
fact looking retrospectively it's now
estimated that because so many
additional women were taking the hormone
replacement therapy that there may have
been as many as 200,000 additional cases
of breast cancer than her otherwise
would have been this really indicates
the importance of randomized controlled
trials