We will now consider an example that illustrates the
difference between the notion of independence of a
collection of events and the notion of pairwise
independence within that collection.
The model is simple.
We have a fair coin which we flip twice.
So at each flip, there is probability 1/2
of obtaining heads.
Furthermore, we assume that the two flips are independent
of each other.
Let H1 be the event that the first coin toss resulted in
heads, which corresponds to this event in this diagram.
Let H2 be the event that the second toss resulted in heads,
which is this event in the diagram--
the two ways that we can have the second toss being heads.
Now, we're assuming that the tosses are independent.
So the event heads-heads has a probability which is equal to
the probability that the first toss resulted in heads--
that's 1/2--
times the probability that the second toss resulted in heads,
which is 1/2.
So the product is 1/4.
We have probability 1/4 for this outcome.
Now, the total probability of event H1 is 1/2, which means
that the probability of what remains should be 1/4, so that
the sum of these two numbers is 1/2.
By the same argument, the probability of this outcome,
tails-heads , should be 1/4.
We have a total of 3/4.
So what's left is 1/4.
And that's going to be the probability of the outcome
tails-tails .
Let us now introduce a new event, namely the event that
the two tosses had the same result.
So this is the event that we obtain either heads heads or
tails-tails.
Schematically, event C corresponds to this blue
region in the diagram.
Is this event C independent from the events H1 and H2?
Let us first look for pairwise independence.
Let's look at the probability that H1 occurs
and C occurs as well.
So the first toss resulted in heads.
And the two tosses had the same result.
So this is the same as the probability of obtaining heads
followed by heads.
And this corresponds to just this outcome that has
probability 1/4.
How about the product of the probabilities of H1 and of C?
Is it the same?
Well, the probability of H1 is 1/2.
And the probability of C--
what is it?
Event C consists of two outcomes.
Each one of these outcomes has probability 1/4.
So the total is, again, 1/2.
And therefore, the product of these probabilities is 1/4.
So we notice that the probability of the two events
happening is the same as the product of their individual
probabilities, and therefore, H1 and C
are independent events.
By the same argument, H2 and C are going to be independent.
It's a symmetrical situation.
H1 and H2 are also independent from each other.
So we have all of the conditions for pairwise
independence.
Let us now check whether we have independence.
To check for independence, we need to also look into the
probability of all three events happening and see
whether it is equal to the product of the individual
probabilities.
So the probability of all three events happening--
this is the probability that H1 occurs and H2
occurs and C occurs.
What is this event?
Heads in the first toss, heads in the second toss, and the
two tosses are the same--
this happens if and only if the outcome is
heads followed by heads.
And this has probability 1/4.
On the other hand, if we calculate the probability of
H1 times the probability of H2 times the probability of C, we
get 1/2 times 1/2 times 1/2, which is 1/8.
These two numbers are different.
And therefore, one of the conditions that we had for
independence is violated.
So in this example, H1, H2, and C are pairwise
independent, but they're not independent in the sense of an
independent collection of events.
How are we to understand this intuitively?
If I tell you that event H1 occurred and I ask you for the
conditional probability of C given that H1
[occurred], what is this?
This is the probability that we obtain, given that the
first event is heads, the first result is heads, the
only way that you can have the two tosses having the same
result is going to be in the second toss also
resulting in heads.
And since H2 and H1 are independent, this is just the
probability that we have heads in the second toss.
And this number is 1/2.
And 1/2 is also the same as the probability of C. That's
another way of understanding the independence of H1 and C.
Given that the first toss resulted in heads, this does
not help you in any way in guessing whether the two
tosses will have the same result or not.
The first one was heads, but the second one could be either
heads or tails with equal probability.
So event H1 does not carry any useful information about the
occurrence or non-occurrence of event C. On the other hand,
if I were to tell you that both events, H1 and H2,
happened, what would the conditional
probability of C be?
If both H1 and H2 occurred, then the results of the two
coin tosses were identical, so you know that C also occurred.
So this probability is equal to 1.
And this number, 1, is different from the
unconditional probability of C, which is 1/2.
So we have here a situation where knowledge of H1 having
occurred does not help you in making a better guess on
whether C is going to occur.
H1 by itself does not carry any useful information.
But the two events together, H1 and H2, do carry useful
information about C.
Once you know that H1 and H2 occurred, then C
is certain to occur.
So your original probability for C, which was 1/2, now gets
revised to a value of 1.
So H1 and H2 carry information relevant to C. And therefore,
C is not independent from these two events collectively.
And we say that events H1.
H2, and C are not independent.