>>Think back to the multiplication rule that helped us calculate the probability of the
intersection of two events. The probability of A intersect B is the probability of A given
B times the probability of B. As the intersection of A and B is the same as the intersection
of B and A, probability of A intersect B is the probability of B intersect A. That means
that the probability of A intersect B is equal to the probability of A given B times the
probability of B and is also the probability of B given A times the probability of A. This
means that the probability of A given B times the probability of B is equal to the probability
of B given A times the probability of A. Dividing both sides by the probability of B, gives
us an alternative way to calculate the probability of A given B if we can quantify the probability
of B given A. This result is referred to as Bayes Theorem.
It’s named after the Reverend Thomas Bayes, an eighteenth century English Presbyterian
minister and amateur statistician. His inverse probability problem was published after his
death, and has become very popular in calculating a particular conditional probability that
can’t be directly viewed and is estimated from probabilities that can be viewed.
So when do we use Bayes Theorem? When the problem begins with a stated probability,
the probability of A. And more information is gathered (usually via imperfect testing
or forecasting), and we know how often this new information is right or wrong, the probability
of B given A. Ultimately, we want to know how this new information changes the probability
of the event, the probability of A given B.
Let’s illustrate this idea with an example.
Stephanie is getting married tomorrow, at an outdoor ceremony in the desert. In recent
years, it has rained only five days each year. Unfortunately, the weatherman has predicted
rain for tomorrow. When it actually rains, the weatherman correctly forecasts rain ninety
percent of the time. When it doesn’t rain, he incorrectly forecasts rain five percent
of the time. What we want to know is: what’s the probability that it will rain on the day
of Stephanie’s wedding?
Let’s first define the events and the probabilities that we know. Define event R as the event
that it rains on a particular day. The probability of R is five out of three sixty-five, as in
recent years it has rained five days per year on average. Let’s define another event F,
which is the event that the weatherman forecasts rain. Note that this is different than it
actually raining – this is a forecast of rain. How accurate is the weatherman? Given
that it actually rains, the weatherman correctly forecasts rain ninety percent of the time.
That is, the probability of a forecast of rain given that it does actually rain is point
nine. Similarly, how good is the weatherman at forecasting rain when it doesn’t rain?
The probability of a forecast of rain given that it doesn’t actually rain is point oh
five. Here “doesn’t rain” is the complement of rain, or rain complement.
So let’s lay out everything we know. We know the probability of rain in general is
five out of three sixty-five, therefore we know the probability of no rain, the complement
of the probability of R, which is three sixty out of three sixty-five. We know how good
the weatherman is when it rains, that is the probability of F given R, which is point nine.
Therefore, we can calculate how often he’s wrong when it rains, the complement of point
nine which is point one. We also know how often the weatherman makes a mistake when
it doesn’t actually rain, forecasting rain instead with probability of F given R-complement
equal to point oh five. And the complement of this is the probability that he correctly
forecasted no rain given that it didn’t rain, or the probability of F complement given
R--complement, which is one minus point oh five or point nine five. This generally falls in line with
a Bayes Theorem storyline: we know the probability of an event, the probability of A. We gather
some information, in this case a forecast, and we know how often that forecast is right
and wrong, with the probability of B given A, and we ultimately want to know the probability
of A given B, the probability of the event given that the forecast told us the event
would occur. Here, the probability of A given B, or in this case, the probability of R given
F, the probability that it will rain when we have a forecast of rain, isn’t observable.
We only know how often it rains in general and how good the weatherman is at forecasting
rain and no rain.
Let’s implement Bayes Theorem to solve this problem. Recall that the general formulation
is the probability of A given B is equal to the probability of B given A times the probability
of A divided by the probability of B. Using the events associated with our problem, the
probability of R given F is equal to the probability of F given R times the probability of R divided
by the probability of F. We have the probability of F given R. We have the probability of R.
But we don’t have the probability of F. However, we can find the probability of F
using some of our probability rules. Recall that the probability of F can be found as
the probability of F given R times the probability of R plus the probability of F given R-complement
times the probability of R-complement. Remember, R and R-complement are mutually exclusive
and collectively exhaustive, so this probability rule works out. So, we’re left with the
Bayes Theorem formulation you see on the screen, and we have all of the probabilities that
make up this calculation. In the numerator we have point nine times five divided by three
sixty-five. In the denominator, we have point nine times five divided by three sixty-five
plus point oh five times three sixty divided by three sixty-five. This result is the probability
of point two. That is, there’s a twenty percent chance that it will actually rain
on Stephanie’s wedding day given that the weatherman said it would.
Only a one out of five chance? That’s a little low. Why is that? Because the weatherman’s
forecast is not perfect and because it really doesn’t rain very often in the first place.
It’s important to ask the right question. In Stephanie’s case, the weatherman might
say “I rarely forecast rain when it doesn’t rain, and I’m good at forecasting rain when
it does rain” so your wedding is doomed. But that’s not what we’re asking. We want
to know given that the forecast of rain has occurred, what’s the likelihood that it
does indeed rain. F given R is very different from R given F.
Let’s illustrate that difference with a scenario: say you’re interested in a female
human, but she’s taking longer than usual to respond to your Facebook messages. One
reason for that is might be that you refer to her as a “female human.” But let’s
talk about this some more.
Your friend might tell you: “She’s not into you anymore brah, because if she doesn’t
like you, the chances are high that she won’t respond as quickly to your Facebook advances.”
That last part is true – if she indeed doesn’t like you, she probably isn’t going to respond
quickly to you. But you haven’t witnessed that. You’ve only witnessed that it’s
taking her a while to respond. There could be a lot of reasons for that: she might be
studying, or her phone battery might be dead, or she’s in a movie theater. There could
be many reasons that it’s taking a while to respond.
So, “The chances that she doesn’t like you if she takes longer to respond” are
probably much lower than “The chances that she takes longer to respond if she doesn’t
like you.” These are two entirely different conditional statements. The first one describes
our situation: we witness that it takes longer to respond, and we’re drawing the conclusion
that she doesn’t like us. The second statement is more likely – given that she doesn’t
like us, she’ll take longer to respond – but we didn’t witness the event “she doesn’t
like you,” we only witnessed the event “she takes longer to respond.”
The point: we need to be careful in interpreting conditional probability statements and make
sure we’re witnessing what we think we are.