last time we talked about controlled experiments, we take
two units that are identical in every variable, except you treat
one and you don't treat the other.
Today we're gonna talk about randomized experiments.
This is where you take a lot of units, and you randomly put Some
of them in a treatment group, some of them in a control group.
And in the treatment group you treat everybody and in the control
group you don't treat everybody.
So why would randomization help us learn about causality?
Well in a controlled experiment, you have your two units that are
identical in everything except one is treated and the other is not.
In a randomized experiment, think about the time right after you've
randomize everybody, but before you treat people.
Well in that case, the two groups are identical, the control group
and the treatment group.
Because by the definition of randomization, you've arbitrarily split
everybody up
into these two groups.
So there's no difference between the two, except that you're gonna
treat one and you're not gonna treat the others.
Well what I mean by there being no difference between the two groups?
Well suppose you picked one person from treatment group and another
person from the control group.
It's very unlikely that these two people are gonna be exactly identical,
in every single variable.
But, when we compare the average value of a variable in the control
treatment group, with the average value of a variable in the control
group, we're gonna get the same number.
And that's just because the distinction between these two groups
is arbitrary.
That's the definition of randomization.
There's no systematic difference between the groups.
Now, suppose that in practice, we have our treatment group and our
control group, and we looked and we saw there's a lot more women
in the control group, for example.
What that would then mean,
is that women were a lot more likely to be assigned to the control
group, than men.
That's not possible with true random assignment.
So a different way of saying this, is that randomization produces
balance in the variables across the two groups.
Mathematically, you would say that the assignment of treatment,
is statistically independent of all the other variables, like gender.
So, gender and whether they're assigned treatment or not is uncorrelated
or independent.
Now it's helpful to remember that there are three kinds of variables
that we're looking at.
Our outcome variable, our treatment or policy variable and our pre-treatment
variables, which are all variables that have happened before treatment.
Randomization produces balance in the pretreatment variables like
gender, but it doesn't produce balance in the treatment variable.
The treatment group gets treated, the control group doesn't.
So now we can see what the main result of randomized experiment
is.
Because these two groups are identical in all of their pre treatment
variables, the only thing that's different across the two groups
is treatment, whether they are treated or not.
Then any difference in the average outcomes between the two groups,
can be completely attributed to differences in the treatment status.
So if we see the average outcomes in the treatment group are higher
than average outcomes in the control group, we can say that's because
treatment has a causal effect on outcomes.
Now this argument requires there to be a large number of people
in each group.
Think about controlled experiments, once again.
That required us to have just two people.
We could get causality with just two observations and that's because
we had two observations which were identical on every way
except for their treatment status.
But now in the randomized case, suppose we looked at a really extreme
case, where we just had two people and we randomly assigned one
to treatment and the other to control.
While randomly assigning them, so can't I just look at the difference
and learn causality?
Well, no not really and that's because if you take those two people
they are not gonna be the same in all their variables, so you can't
conclude that the only difference across the two groups is because
of treatment.
It's only once you start getting a very large number of people in
each group that you can reasonably say that the group
are identical.
And therefore the only difference between them is whether they are
treated or not.
This is one of the major differences between randomized experiments
and controlled experiments.
Now in practice,
you're gonna figure out just how many people do we need? And there's
a whole
And there's a whole body of techniques for doing that in Statistics.
We're not gonna talk about that, you can go look it up.
But basically the causality idea says you need a lot of people.
And in statistical inference, under this statistical techniques
we'll quantify exactly how large you need.
Now in practice there is a lot of benefits to doing randomization.
One is that it makes data analysis very simple.
Basically when you have randomized experiment, you're just going
to have to compare the average outcome in the treatment group, with
average outcome of the control group and the difference is going
to be an average treatment effect.
So it's going to tell you something about causality.
And we are gonna go through a lot of examples of how to analyse
randomized experiments in the next few modules.
[MUSIC].