This class is all about blocking.
The distinction between covariates and disturbances is in the prior
class, was that covariates are measurable, and disturbances are not.
Neither of them can be controlled.
We're generally comfortable with the idea of control in our experiments.
Variables that we can control, should be controlled.
Remember that idea that you should always keep
everything fixed, and only change what you're experimenting with?
The idea of keeping things fixed is what we mean by the term "control".
Now we're going to make a subtle distinction.
There are variables you can control, and choose to actively vary.
We call them "factors".
You can control these variables, and you can measure them.
Remember the cell phone app example from the previous class?
Recall the variables from that example were Factor
A was the type of marketing promotion used.
Either a free in-app purchase, or a 30 day trial of all the features.
Factor B was the marketing message and factor C was the in-app purchase price.
All three of those were controlled and measurable where the term "measurable" is
used fairly loosely as a way to say that you can quantify the value of your factor.
Take a minute now, to think about the experiments,
and the factors you have varied, during this course.
Are your factors actually controllable?
Can you measure or quantify all your factors?
If you have any doubt, it's a good time to consult
with your colleagues and post a message in the course forums.
Now here's the subtle distinction.
What if you had a factor you can control, it varies during
the experiments, but the factor isn't
actually the main focus of your experiments?
These are called nuisance factors.
Lets take a look at the cell phone example again.
The end user could be on an Apple or Android operating system.
You could control this, because you can select
only Apple or Android users during your experiments.
In fact, this could have been another factor.
You might have called it Factor D, but this
really isn't expected to be a significant factor of interest.
It's not what the aim of your experiments are about.
We call this a "nuisance factor".
Now from the previous video, you learned that you must randomize your experiments.
If you go run all your Apple experiments first, and then all
your Android experiments after, you've actually
confounded your variables on this factor D.
Ultimately, you would want successful sales, no matter
whether you have Apple or Android users, but you
must intentionally plan your experiments ahead of time, to
avoid this nuisance factor from having a confounding effect.
We call this "blocking".
There're a number of instances in which you want to block for nuisance variables.
In our baking example, imagine a situation where you are going to run out of flour.
You can do half your experiments on one brand of
flour, and half the experiments with another brand of flour.
Another case might be experiments in a factory.
Half the experiments are done during the day shift,
and the other half is done during the night shift.
Or if you're testing gas mileage in a
car, you might have one driver and another driver.
60 00:03:39,818 --> 00:03:43,199 People sometimes find blocking tricky to understand.
And here's one way I deal with it.
I ask myself, is my system, or process, going to have to
successfully work with different values of this nuisance variable in the future?
If the answer is yes, then I design with blocking in mind.
If I don't design my experiments with blocking in mind, then that
nuisance variable might affect the outcome,
and I won't really understand what's happened.
If the answer to the question is "no", that means that I've
got good control over the system, and I can avoid the nuisance variable.
Now that we've discussed the need for blocking, let's
see how to plan experiments where there are 2 blocks.
And the general rule for this situation is to
add a new factor to your standard order table.
We already have 3 factors, A, B, and C.
So that's 8 experiments in a full factorial.
Now consider adding the new nuisance factor as a new variable, D, to the table:
Apple versus Android.
This factor has two levels; minus and plus.
How do we go about picking which experiments to assign to our
Apple phone users, and which experiments to assign to our Android users?
Here's a hint.
If we were to do a full set of experiments in four
factors, we would've required two to the power four, or 16, experiments.
Instead, we're doing eight experiments shown here.
Eight is half of sixteen, and so there's no
surprise that this is effectively generated using a half fraction.
Once you understand the principle of half
fractions from the prior class, you'll perhaps
intuitively see that you assign factor D using the table we showed last time.
That factor D is generated as the product of A times B times C.
Here's the interesting part:
once you've generated Factor D in this way, we set all the minus
sign experiments to Android users, and all the plus sign experiments to Apple users.
Let's visualize this on a cube plot.
The closed circles are the Android users, and the open circles are the Apple users.
Doesn't this plot look familiar to you?
Now let's go take a look at the reasons for assigning the experiments in this way.
Imagine that there really is an effect that Android users are more
likely to keep using your app, and that Apple users are less likely.
We can imagine that our outcome variable
for Android users is boosted by a consistent
amount "g", and I will put a small tilde over these numbers to indicate that.
Experiments involving Apple users are reduced by some
amount "h", where "h" is a negative number.
And I'll add a small circle above their outcomes.
Remember how we calculated the main effects?
High minus low.
And high minus low.
And then we average the answer.
Well for the main effect of A, we notice that there's
an equal number of additions with a tilde, as there are subtractions.
The same for the circles.
Two positive circles, and two negative circles.
From a practical point of view, this implies that any bias due
to Android users and bias due to Apple users will cancel out.
So our main effect of A will be well estimated and
not affected by any differences that exist between Apple versus Android users.
That's a desirable requirement.
In fact, every parameter in the model will be well
estimated without bias, except for the three factor interaction, ABC.
That parameter is badly estimated.
All the tilde's have minuses, all the
circles have pluses, so no cancellation occurs then.
In fact, the ABC three-factor interaction is
confounded with this D effect of the nuisance variable.
That was intentional, and it is usually the best course of action in most cases.
We often expect that our three factor interaction to be negligible.
So we have sacrificed our three factor interaction here, in order
to minimize the effect that the nuisance variable has on our system.
Though it's outside the scope of this
course, more complex blocking schemes are possible.
For example, if we were testing Apple, Android,
and Blackberry users, we would have three levels.
In our baking example, if we were using large quantities
of flour, we might have had 4 different flour suppliers.
Experiments in a factory might rely on
3 shifts, a morning, afternoon and evening shift.
There're cases where there's blocking with more than two groups.
And what happens is, we simply create extra blocking factors in the design.
There are tables in many of the statistics textbooks that
show how to generate these blocks in an optimal way.
I'll leave that for you to explore on your own.
Let's return to our two-level blocking designs.
The general rule that you can remember for this case with two blocks, is that you add
a new factor to the system, and generate the
design as if you were running a half fraction.