[MUSIC] So let's look at some data.
We're going to try to figure out what the effect of having property rights is on things
like child health using this As-If natural experiment from Argentina.
Now, just like in regular randomized experiments, there are many problems that arise in as-if
natural experiments, one of them being noncompliance.
So in this case, some people lived on land that the government took, the owners said
“fine, have the land,” but the squatters didn't actually get property rights to the
land - they didn't actually get a title in their name.
And that could be for many reasons, but maybe they just didn't fill up the paperwork, for
example, you've to actually fill out the paper work and apply to get a title.
So they had the option of getting property rights, but they didn't actually receive it.
So they did not comply with the treatment: they were assigned to get property rights
but they didn't.
So what we're going to start out with is an Intention-to-Treat Analysis, where we just
completely sort of ignore this problem, and we forget about the fact that we want to learn
about the actual causal effect of having property rights on your child's health, and instead
we're just going to look at the effect of having an offer, an option of getting property
rights on child health.
So we're going to have, in this column, effects for people who had the option of getting property
rights, and in this column, effects for people who did not have the option, and our outcome
variable here is called Weight For Height Z score.
Now this is a short term measure of children's health status, and larger numbers are better,
and that's all I'm going to do to explain exactly what this number is.
And if you really want to know more, this is a really important part of research - trying
to define where your outcome variable are etc. - and it will talk about a lot more in
the paper, in this literature, on how to measure health, but for now, larger is better, and
it's a measure of child health.
So we have kids from ages 0 to 11, and this first row they are looking at all kids, in
the second row they are looking at just kids 0 to 4, at the third row they are looking
at kids 5 to 11.
So let's just look at the all kids effect.
So over here the people who had the option of getting property rights had an average
Weight-for-Height Z score of 0.279, the people who didn't have the option of getting property
rights it was 0.065, the difference, the average treatment effect is 0.214.
This is the average treatment effect of having an option to get property rights.
Now, is this a true difference or not?
Well, in order to know that, we have to unfortunately do some statistical inference, some confidence
intervals, hypothesis tests, and stuff like that.
The reason is our sample sizes are not enormous.
In brackets here they've got the sample sizes.
So in the treatment group, there are 239 kids; in the control group, people don't get property
rights, there are 132 kids, so it's not a huge number of kids.
So what are we going to do?
Well, what we're going to do is take this difference, which is our estimated effect
size, and then this number parenthesis here, 0.109.
That's called the standard error, and it tells us something about the variance, the variation
in effect sizes.
So we're going to take these two numbers and construct a confidence interval.
So I've got that over here, to just remind you from our earlier modules what's going
on, the confidence interval is going to be the effect size, 0.214, minus 1.96 times the
standard error, 0.109, that's going to be the lower bound, and here's it's -0.004.
And then the upper bound in our interval is going to be effect size, 0.214, plus 1.96
times the standard error, which was 0.109.
And I've done math for you: 0.432.
So how I've constructed this isn't super important, what matters is you get to this interval here,
and now we got to interpret it.
So remember, the first thing we do is zero in the interval or not?
Yes, zero is in the interval: -0.0004 to 0.432.
Now what we do is we claim if zero is in the interval, this is not a true effect: we're
not sure whether there's actually causal effect, or whether this big difference we found - and
it was quite big, it was five times larger outcome variable in the treatment group than
it was in the control group- but just because it wasn't a big enough sample size, we can't
tell.
But you may look at this and say, “well, that’s -0.004, that's almost zero, zero’s
almost not in that interval.”
So what you can do in this case is say maybe the criteria for deciding whether we have
big enough sample or not is a little too stringent.
Maybe we're being a little harsh, so what you can do is you can use something that I'll
call a less stringent confidence interval.
The way you do that is replace this two numbers here with 1.64, and then you do the same thing
and if you do that you're going to get this interval I've drawn here.
It goes from 0.035 up to 0.397.
Zero is no longer in this interval, and so we say, “hey, zero is not in the interval.
It looks like this actually is a true effect we actually have found, that having the option
of getting property rights has a positive causal effect on the child's health.”
So this first thing I drew is something called a 95% Confidence Interval.
The second thing I drew is called 90% Confidence Interval.
So 95%, 90% here is telling us a measure of how stringent we want to be.
So going from 95 down to 90, we say we don't want to be as stringent really as this first
one, we want to allow for a little bit more statistical noise, basically.
What that did is shrunk the interval, we went from a bigger interval that contains zero
down to smaller one.
So sometimes that's what done in practice, is you look at a few different confidence
intervals in order to see maybe there doesn’t look like an effect for one, that but if you
if you relax your criteria a little bit, you get an effect.
Ideally, every researcher out there would want to get, to have their interval not contain
zero for like 99% - you get even stronger, no matter how stringent you are, you still
find effects, that's ideal.
But you know research is hard to do, and sometimes you don't always get clear cut answers like
that, so in cases like this there's some subjective judgement about is there an effect here or
not.
There is if you are not too stringent, but there isn't if you are very stringent, and
whether you believe this one or this one that's really just up to your own judgement [MUSIC].