Today we're going to talk about
something emotional, which is what
happens when you go to the doctor, have a
basic screening for a test and you get
that letter in the mail saying you have
tested positive for this disease. If you
get that letter, if you're anything like
me your stomach falls into your feet and
you start to worry; a question that
mathematics can help us answer is how
worried do you really need to be? Maybe
very worried, maybe not at all. We start
with the question about what disease
we're talking about? Is it common like
the flu or is it a rare disease, usually
the kinds were really scared of, some
kind of cancer or some unusual genetic
disease? Maybe something auto-immune,
affects a very small fraction of the
population. So if it's a rare disease and
you get a bad result well the second
thing you worry about after how rare is
the disease is how accurate is the test?
There are two possibilities for you:
either you're sick
or you're healthy. And there's two
possibilities for the test, either says
you're sick or it says you're healthy. So
you may be sick or you may be healthy;
and the test may say that you're sick or
it may say that you're healthy. So if
you're sick and the test says you're
sick - well for one thing it's getting it
right. I think right should maybe be- right
in green. This is a true positive. And
this is a bad state, because you're sick
but maybe at least you've got the
disease caught so now you can begin to
treat it. Another possibility is that the
test says that you're healthy and in
fact you are healthy - so this is a true
negative. - (Brady: Again that's good news)
That's the best news, right? But there are
two ways in which the test can make a
mistake; it can make a mistake
by telling you if you- that you're
healthy when you're sick - it's called a
false negative. And there's another case
that can happen which is you're healthy
and you're told you're sick, and that
thing is called a false positive. So what
we've got here is we've got the correct
diagnoses in green and we've got the
false diagnoses, the mistakes that the
test can make, we've
got them in red. So I want to talk about
when a letter comes in the mail after
you've had this screening for, maybe a
rare disease, something that's pretty
scary but very remote, and you get a
positive result in the mail. So you've
tested positive for x, I won't name it
too upsetting, and you wonder what to do;
you think well I should call the
undertaker, I've got to let my relatives
know, this is terrifying I'm now going to
have a dreadful treatment followed by an
untimely death. This is, I think, a very
normal reaction to such a letter. And the
question is, do you need to have your day
ruined? Could be that the test is correct
and you're sick and that's very very sad,
and it could be that the test made a
mistake. And so the question that you
might ask yourself is, if you were so
inclined mathematically inclined, you
might say what is the probability that
I'm sick given that I had a positive test?
Maybe that's very high, maybe 99%, and
then you really do have to worry.
Maybe 0.01% - then you don't need to be
worried. How big is that probability,
right? You've got the bad news, so that's
a given for you now, and the question is
what's the probability that you're sick
given this information? Yesterday when
you didn't have this information you
weren't worrying about the disease at
all, right? There's a remarkable answer to
that question which dates back to the
1700s. So- and more surprisingly than it's
so old is that it was brought to us by a
Presbyterian minister named Thomas Bayes;
maybe you've heard of Bayes' rule. It has
a way of interchanging what you're
trying to find the probability of and
the given information. So I'm going to
write Bayes' rule, for this case, and say
that the probability that you're sick
given that you tested positive is equal
to the probability that you have a
positive test given that you're sick -
well that's not quite right I just flipped
the things over - but if you take this
inverted probability and scale it
properly you do get an equality. So what
do we do here?
We say that's times the probability that
you're sick, divided by the probability
that you got a positive test.
And this is the famous Bayes' rule that- this
simple formula was brought to us, you may
have learned it in a basic statistics
course or a basic math course, computer
science, data science - it's a very
important part of statistics. Rather
simple to derive in formulas and I
think hard to absorb in terms of its
broad implications. We want to figure out
the probability that we're sick given
that we got this lousy letter, and now
we've said that that's equal to
something that's got three things we've
got to fill in.
So to start Brady with what you asked
about, the probability that you're sick,
let's suppose that it's a rare disease,
one in a thousand, so I'll say the
probability that you're sick is 0.001.
And let's suppose that the test is
really really accurate; that when you're
sick it catches it. So just to make this
totally simple I'm going to say, suppose
the positive test given that you're sick,
if you were sick, it would catch it
100% of the time. Now most
tests aren't quite that good but a lot
of them are really good, like 0.99 or
0.98, so my simplification here is not so
far off. And then what we've got to do is
stick something in the bottom here,
what's the probability that you're going
to get a positive test? Probability of a
positive test has got two parts to it.
One of them is the probability that
you've got a positive test given that
you're sick, and one of them is that
you've got a positive test given that
you're healthy, that's those false
positives. Now this isn't quite right the
way I've written it, because it- here
I've just added up the positive tests
given that you're sick and the positive
tests given that you're healthy. But I
need to scale those numbers by who they
apply to. Now we've said that the
probability of a positive test given
that you're sick is 1, but we know that
the probability that you're sick is very
small so that's going to make only the
tiniest tiniest contribution. So we have
a 1 here for the positive test that
you're sick, 0.001 for the
probability that you're sick. Then
there's the positive test given that
you're healthy times the probability
that you're healthy, that's a very big number,
0.99. Now what about this
positive test given that you're healthy?
We go over here, that's the red, that's
our false positives. And let's suppose
this is a pretty good test; that it's
going to tell nine healthy people that
they're healthy and one healthy person
that he's sick out of every ten. So it
might catch all of the disease but it
also might tell a few healthy people
that they're sick as well, so this goes
back over to this box - there's two ways
to make a mistake. It can still make
mistakes by telling healthy people that
they're sick. And I'm worried that in
this particular test that that happens
in one out of ten. So the probability of
a positive test; well this number because
of that rarity of the disease is
effectively zero, and this number over
here is effectively 0.1, so this is very
close to 0.1. So I'm going to put that
0.1 here and say that now I've
computed that the probability that
you're sick given a positive test is
effectively 1 times 1 in a thousand
divided by 1 over 10, and this is 1 in 100.
So how worried should you be when you
get that bad letter? Well, if the test is
virtually all the time right when you're
sick, if it makes a one-in-ten rate of
false positives and it's relatively rare,
affects only one in a thousand in the
overall population that's called the
base rate, then your chances of being
sick or one in a hundred. Now seems a
little premature to call the the funeral
home I would think.
[Extras] ...applications. So
we can do the exact same analysis in the
case of somebody who's trying to
identify criminals in lineups, right?
Criminals are a small part of our
population, at least we hope so, I think
they seem to be. - You have forced Monty's
hand. He knows where the one remaining
goat is, he knows it! And he has to show
it to you.