[MUSIC] Randomized experiments are designed and implemented by the researchers from the
very beginning.
Now there are going to be two similar situations that we're going to look at next.
The first is called a true natural experiment.
This is where treatments are actually randomly assigned, and the second is called an As-If
Natural Experiment and this is where treatments are assigned by a procedure which is arguably
random for all practical purposes.
Both of these cases are going to use observational data.
So we get treatment assignment by some process out there in the world and we as the researchers
come across it after the fact, after the data has already been generated.
So that’s going to be the main difference between natural experiments and randomized
experiments - is that a natural experiment is going to use observational data - but the
analysis will look very similar to the analysis of randomized experiments.
So let's talk about true natural experiments.
These are the most like an actual randomized experiment.
Because we know that the treatment was randomly assigned, and the only difference here is
that it was randomly assigned by somebody other than the researcher.
So usually in real life this happens actually because there wasn't enough money to treat
everyone, so they hold a lottery to decide who gets treated.
So a very common example of these is analyzing the effects of charter schools.
So charter schools are public schools that have more flexibility and autonomy than traditional
public schools.
So they typically draw students from the entire district instead of a specific area, they
often have many more students who apply for entering into the charter school than they
have capacity for, and so what they do is they randomly pick who gets in.
And so here, for example, I have a picture of a public lottery ceremony held at the Harlem
Success Academy in New York City.
So what many of these schools do is they actually hold lotteries in public where they pick the
student's names from this pot of who is going to get in and all the parents show up to see
if their kids got into the school.
And there is actually documentary made about this and charter schools called “The Lottery.”
So in some cases we going to see data like some students get randomly chosen to be in
the charter schools, some students get randomly assigned not to be in the charter school,
and that's a natural experiment.
We're going to be able to use that data to analyze the effect of being in a charter school
on outcomes like how much you make, etc, etc.
An As-If Natural Experiment is where the treatment with not literally randomly assigned, like
it was for example with the chartered school lottery, but the researcher can convincingly
argue - or they attempt to argue at least - that it was as if treatment was randomly
assigned.
And what that means is that the treatment assignment is not related to any variable
that might affect the outcomes.
So for example, you may have some social program that is implemented by the government and
the way it is implemented is it's slowly rolled out across different localities in a very
nonsystematic way.
So they don't go from North to South, or South to North or whatever.
Or they don't decide that really this county is going to be the one that gets the program
first, and then this one gets it first because they is better in some way or another.
They just for whatever bureaucratic reason decide that some region gets the program first
and some regions don't.
So the idea is that eventually all the regions are going to be treated, they are all going
to get the program, but the regions that get treated first are essentially chosen at random.
And this is called an As-If Natural Experiment.
[MUSIC].