As an introduction to the main topic of this lecture
sequence, let us go through a simple example and on the way,
review what we have learned so far.
The example that we're going to consider involves three
tosses of a biased coin.
It's a coin that results in heads with probability p.
We're going to make this a little more precise.
And the coin is biased in the sense that this number p is
not necessarily the same as one half.
We represent this particular probabilistic experiment in
terms a tree that shows us the different stages of the
experiment.
Each particular branch corresponds to a sequence of
possible results in the different stages, and the
leaves of this tree correspond to the possible outcomes.
The branches of this tree are annotated by certain numbers,
and these numbers are to be interpreted appropriately as
probabilities or conditional probabilities.
So for example, this number here is interpreted as the
probability of heads in the first toss, an event that we
denote as H1.
This number here is to be interpreted as a conditional
probability.
It's the conditional probability of obtaining heads
in the second toss given that the first
toss resulted in heads.
And finally, this number here is to be interpreted as the
conditional probability of heads in the third toss, given
that the first toss resulted in heads and the second toss
also resulted in heads.
Let us now continue with some calculations.
First, we're going to practice the multiplication rule, which
allows us to calculate the probability
of a certain outcome.
In this case, the outcome of interest is tails followed by
heads followed by tails.
So we're talking about this particular outcome here.
According to the multiplication rule, to find
the probability of a particular final outcome, we
multiply probabilities and conditional probabilities
along the path that leads to this particular outcome.
So in this case, it's (1 minus p) times p times (1 minus p).
Let us now calculate the probability
of a certain event.
The event of interest is the event that we obtain exactly
one head in the three tosses.
This is an event that can happen in multiple ways.
Here is one possibility where we have a single head.
Here's another possibility.
And here's a third one.
These are the three possible ways that we can have exactly
one head, depending on where exactly
that single head appears.
Is it in the first toss, in the second, or in the third.
To find the total probability of this event, we need to add
the probability of the different outcomes that
correspond to this event.
The probability of this outcome is p times (1 minus p)
squared, the probability of this outcome is what we
calculated.
It's, again, p times (1 minus p) squared.
And the probability of the third one is also p times 1
minus p squared.
So the answer is 3p times (1 minus p) squared.
Notice that each one of the 3 different ways that this event
can happen have the same probability.
So these 3 outcomes are equally likely.
Finally, let us calculate a conditional probability.
And this is essentially the Bayes rule.
Suppose that we were told that there was exactly one head.
So in particular, the blue event has occurred.
And we're interested in the probability that the first
toss is heads, which corresponds
to this event here.
These are all the outcomes in which the first
toss is equal to heads.
So given that the blue event happened, what is the
probability that the green event happens?
You can guess the answer, that it should be 1/3.
Why is that?
Each one of these blue outcomes has the same
probability.
So when you condition on the blue outcome having happened,
the conditional probability of each one of
these should be 1/3.
So given that the blue outcome happened, there's probability
1/3 that this particular one has happened.
And this is the only one that makes the green event happen.
But let us see if we can derive this
answer in a formal manner.
Let's see if we're going to get 1/3.
We use the definition of conditional probabilities.
The conditional probability is the ratio, first, of the
probability that both events happen, divided by the
probability of the conditioning event, which is
the probability of 1 head.
Now, the probability of both events happening.
That we have exactly one head and the first toss is heads.
This is the intersection of the blue event and the green
event which can happen only in this particular outcome,
namely the sequence heads, tails, tails.
And has probability p times (1 minus p) squared.
The denominator is something that we have already
calculated.
It's 3p times (1 minus p) squared.
And so the final answer is 1/3 as we had guessed.
Let me now make a few comments about this particular example.
This particular example is pretty special in the
following respect.
We have that of the probability of H2, heads in
the second toss, given that the first one was heads, is
equal to p.
And the same is true for the conditional probability of
heads in the second toss given that the first one was tails.
In other words, our beliefs about what may happen in the
second toss remain the same.
There's a probability, p, of obtaining heads no matter what
happened in the first toss.
Telling you the result of the first toss doesn't change your
beliefs about what may happen, and with what probability, in
the second toss.
And if you were to calculate the unconditional probability
of heads in the second toss, what you would get using the
total probability theorem would be the following.
It's probability of heads in the first toss times the
probability of heads in the second, given heads in the
first, plus the probability of tails in the first toss times
the probability of heads in the second toss, given tails
in the first.
And if you do the algebra, this turns out to
be equal to p again.
So the unconditional probability of heads in the
second toss turns out to be the same as the conditional
probabilities.
Again, knowing what happened in the first toss doesn't
change your beliefs about the second toss, which were
associated with this particular probability, p.
So what we're going to do next is to generalize this special
situation by giving a definition of independence of
events, and then discuss various properties and
concepts associated with independence.