[MUSIC] Analyzing natural experiments are basically the same thing as analyzing randomized
experiments - once you have the As-If Randomization Assumption.
So we're going to go through two examples to talk about where these assumptions come
from and how you check them.
The first one we'll look at is the effect of having property rights on your behavior.
More specifically, suppose we took squatters who are living on land that they don't own,
and then we give then rights to the land, what's going to happen?
Well, some economists looked at this in a paper from 2004, and they argue that under
the security provided by proper legal land titles saying that they actually own the land,
the people on the land are going to do things like invest in improvements in water distribution.
They're going to do invest in sewage control, treatment of fecal matter, garbage disposal,
safe garbage disposal, safety, how good the heating system is, air ventilation, quality
of the soil, etc., etc., etc.
Basically the idea is they own the land so they have a reason and incentive to invest
in improving it.
And those investments might affect the quality of life for their kids, and so it may affect
how healthy their kids are.
And so therefore, there may be this pathway from having property rights to having healthy
children.
So that's what these guys are going to look at in this paper.
And they're going to use an As-If Natural Experiment to do it.
So here's what they found.
Well, they came across this very unique situation that happened in Argentina in 1981.
So in 1981, about 1800 families were living in about 2 square kilometers of waste land
outside of Buenos Aires in Argentina.
Now these occupants were basically landless citizens who were organized by the Catholic
Church, and they purposely wanted to avoid creating a shanty town, so they partitioned
the land into small urban shaped parcels.
And at the beginning of this occupation they thought - the church and all the people who
live there - thought that the land was owned by the state.
So that's where they were living, and I'm paraphrasing for the paper here by the way.
So what actually happened?
Well, three years later they discovered that the land they were living on actually wasn't
owned by the government, it was actually owned by private citizens.
So three years later, the government said we're going to just take that land from the
private citizens and give it to the squatters.
Now some of the previous citizens who owned this land said “okay, fine, that's fine,”
they accepted compensation, some money in exchange for their land, but some of the other
people who used to own this land, they said “no, I don't think so, you're not taking
my land,” and they fought the government in court.
So basically what happened was 1800 families were split into two groups, the first group
was able to obtain property rights to the land they lived on because they lived on land
where the owner didn't fight the government.
The other families did not get property rights, because they lived on land where the owners
chose to fight the government.
So if we believe that this is an As-If random split of families into these two groups, then
all we have to do is compare outcomes of the groups - the people who got property rights
with the outcomes of the people who didn't get property rights - in order to learn about
the causal effect of having property rights to your land on things like your children's
health.
So, now how do we justify this as-if randomization assumption?
Well, they are basically going to do two things.
First, they are going to do balance checks.
They're going to first look at the pretreatment variables of the squatters, the people who
lived on the land.
They going to at their age, sex, citizenship, and education, and they're going to argue
that these variables are balanced across treatment and control.
So it's not like the treatment group - the people got property rights - are much much
older than the people who didn't get property rights, for example.
They're also going to look at the observed pretreatment variables of the parcels, the
actual pieces of land that they lived on.
These are the distance from the parcel to a polluted creek, the size of the parcel,
the distance from this parcel to a non-squatter area in Buenos Aires, and they're going to
say, well, this variables are also balanced across the treatment and the control group.
So it’s not like the groups that had, that didn't get property rights for example, were
very far from the city, or very close to polluted creeks, while the land that did get property
rights was close to the city and far from polluted creeks.
That's not true - it's actually, they're roughly equal distance on average.
So this is exactly what we would see if we had true randomization.
Now the problem is, we can't check balance for variables we don't observe.
If we had true randomization, then we would be guaranteed that all variables, whether
they're observed or unobserved, are balanced, but with as-if randomization, we can never
really know if the unobserved variables are balanced or not.
But it's still a good check to make sure that we do see balance on what we do observe.
So that's the first piece of evidence they give.
The second piece is they give an enormous amount of qualitative evidence.
So first they argue that the squatters didn't even know that the land was privately owned
when they were settling, so it's not like they could have picked and choosed which land
they wanted to live on: they didn't notice some land would have ended up with property
rights and some land wouldn't.
Suppose they did, why would that be a problem?
Well, then maybe the slightly more educated, the higher ability, more persuasive squatters
could have managed to get their way onto the land that was going to get property rights,
whereas the sort of less persuasive, less influential squatters were stuck with the
land that wasn't going to get property rights.
That would create an unbalance across these two treatment groups that would introduce
to possible confounders, and so comparing the outcomes between these two groups could
be chocked up to this difference, and persuasiveness, or influentialness of the squatters and not
to the actual treatment effect, the fact that some people actually had property rights and
others didn’t.
So would they say that's not the case, because nobody knew that this land was not owned by
the government when it was settled.
So second, they also do a ton of interviews and other qualitative fieldwork.
And summarizing all of these, they'd interview people on the land, interview owners, why
did some owners decide to fight the government, why didn't they?
They basically argue that just the reasons that some owners decided to fight the government
others didn't was just idiosyncratic that is unrelated to, say, how valuable the land
was.
It would be really, really bad for the authors if the owners who fought the government were
also the owners who had the most valuable land for some reason.
That would be bad because then that would be another confounder, another thing that
is not balanced across the treatment and control group.
And we wouldn't know if we compared outcomes of these two groups, we wouldn't know if differences
in outcomes is due to the property rights or different or because this land is more
valuable on average in the treatment group than in the control.
But they argue that's not the case, that it's idiosyncratic - just basically random.
So that's what they do to justify their assumption, and then they are going to move on in the
next few models, we'll look at what effects they actually found when they compare outcomes
in the treatment and control group.
And one lesson to take away from this paper is that it's really important to get a good
qualitative understanding of the context, in order to justify your as-if randomization
experiment assumptions.
[MUSIC].