[MUSIC] Last time, we talked about controlled experiments: we take two units that are identical
in every variable, except you treat one and you don't treat the other.
Today we're going to talk about randomized experiments.
This is where you take a lot of units, and you randomly put some of them in a treatment
group, some of them in a control group.
And in the treatment group you treat everybody, and in the control group you don't treat everybody.
So why would randomization help us learn about causality?
Well, in a controlled experiment, you have your two units that are identical in everything,
except one is treated and the other is not.
In a randomized experiment, think about the time right after you've randomize everybody,
but before you treat people.
Well in that case, the two groups are identical, the control group and the treatment group.
Because by the definition of randomization, you've arbitrarily split everybody up into
these two groups.
So there's no difference between the two, except that you're going to treat one and
you're not going to treat the other.
Well, what do I mean by there being no difference between the two groups?
Well, suppose we picked one person from treatment group and another person from the control
group.
It's very unlikely that these two people are going to be exactly identical in every single
variable.
But, when we compare the average value of a variable in the treatment group with the
average value of a variable in the control group, we're going to get the same number.
And that's just because the distinction between these two groups is arbitrary.
That's the definition of randomization.
There's no systematic difference between the groups.
Now, suppose that in practice we have our treatment group and our control group, and
we looked and we saw there's a lot more women in the control group, for example.
What that would then mean is that women were a lot more likely to be assigned to the control
group than men.
That's not possible with true random assignment.
A different way of saying this, is that randomization produces balance in the variables across the
two groups.
Mathematically, you would say that the assignment of treatment is statistically independent
of all the other variables, like gender.
So, gender and whether they're assigned treatment or not, is uncorrelated, or independent.
Now, it's helpful to remember that there are three kinds of variables that we're looking
at.
Our outcome variable, our treatment or policy variable, and our pre-treatment variables,
which are all variables that are happened before treatment.
Randomization produces balance in the pretreatment variables like gender, but it doesn't produce
balance in the treatment variable.
That's the whole point.
The treatment group gets treated, the control group doesn't.
So now we can see what the main result of randomized experiment is.
Because these two groups are identical in all of their pre treatment variables, the
only thing that's different across the two groups is treatment, whether they are treated
or not.
Then any difference in the average outcomes between the two groups can be completely attributed
to differences in the treatment status.
So if we see the average outcomes in the treatment group are higher than average outcomes in
the control group, we can say that's because treatment has a causal effect on outcomes.
Now this argument requires there to be a large number of people in each group.
Think about controlled experiments once again.
That required us to have just two people.
We could get causality with just two observations, and that's because we had two observations
which were identical in every way except for their treatment status.
But now in the randomized case, suppose we looked at a really extreme case, where we
just had two people and we randomly assigned one to treatment and the other to control.
Well I randomly assigning them, so can't I just look at the difference and learn causality?
Well, no, not really, and that's because if you take those two people they are not going
to be the same in all their variables, so you can't conclude that the only difference
across the two groups is because of treatment.
It's only once you start getting a very large number of people in each group that you can
reasonably say that the groups are identical.
And therefore the only difference between them is whether they are treated or not.
This is one of the major differences between randomized experiments and controlled experiments.
Now in practice, you're going to have to figure out just how many people do we need?
And there's a whole body of techniques for doing that in statistics.
We're not going to talk about that, you can go look it up.
But basically the causality idea says you need a lot of people.
And in statistical inference, under this statistical technique we'll quantify exactly how large
you need.
Now in practice, there’s a lot of benefits to doing randomization.
One is that it makes data analysis very simple.
Basically when you have randomized experiment, you're just going to have to compare the average
outcome in the treatment group with average outcome of the control group, and the difference
is going to be an average treatment effect.
So it's going to tell you something about causality.
And we are going to go through a lot of examples of how to analyze randomized experiments in
the next few modules.
[MUSIC].