>> Hello. I'm Dr. John Brush.
This short video is a brief review of how experienced clinicians make a medical diagnosis.
In this video I hope to give you some background and quick pointers
on how likelihood rations can improve your ability to incorporate diagnostic testing
into the process of making a medical diagnosis.
Imagine that we are seeing a patient in the emergency department with chest pain.
How do we make the diagnosis?
First we have to think of the possibilities.
Then we have to test the possibilities to see
which one emerges as our most likely possibility.
The goal in the end is to reach a conclusion with a probability estimate
that is strong enough to act upon.
How do experts generate a list of possibilities?
With experience, experts are able to recognize possible diagnoses quickly using nonanalytical
or intuitive thinking.
They recognize possible diagnoses, just
like we might recognize common everyday objects, like a chair or a dog or a cat.
If they have seen it many times before they can immediately recognize it.
Studies have shown that experienced clinicians generate three to five possible diagnoses
within seconds to minutes of starting a diagnostic inquiry.
If the diagnosis is not immediately recognizable,
we use a method called abductive reasoning,
where we reason toward the most plausible hypothesis.
To start this process we have to think backwards to imagine what might have happened
to cause the patient's present illness.
Once we establish a list of plausible hypotheses, we go forward to test them.
It's a little like starting a game of chess in a midgame position.
The game is all ready started and we are trying to figure out how we got to this position.
Our patient's disease is all ready present and we are trying to figure
out which disease is causing our patient's present illness.
So there's an order of operations that works best for diagnostic testing.
We should think of a plausible hypothesis, then test that hypothesis.
That's better than getting the test result first and then making
up an explanation for the test result.
First we should take a minute to estimate the baseline probability of a possible diagnosis.
And second, we should proceed with testing and update our estimate of probability
of that diagnosis based on test results.
Cognitive psychologists tell us that decision makers do this intuitively using a rapid
implicit mental habit or heuristic called anchoring and adjusting.
So what is the probability or our degree of belief that this woman is having a heart attack?
We set an anchor for an initial probability estimate based on initial impression.
Then we adjust our probability estimate based on the incorporation of test results to come
up with an estimate of posttest probability.
To understand this we need to quickly review probability.
Probability is uncertainty quantified.
It gives us a way to understand the variability of past observations in order
to make predictions about the future.
When we look back on experience we often note that results pile up in the middle and scatter
to the edges, much like the pile of sand pictured on the left.
We can draw a probability density curve, shown in blue,
that shows how a test result might be distributed
over a range of possible test results.
The upper right panel shows a probability density curve
in blue showing how probability is distributed over a range of possibilities.
The red cumulative probability curve shows how the sum of the probabilities
of all the possibilities moving from left to right on the graph ads up to one.
The lower panel shows how probability is distributed
for discreet possibilities like diagnostic categories.
If there is only one diagnosis, in other words the possibilities are mutually exclusive.
And if we are considering all of the possibilities, then the sum of the probabilities
of all the possibilities will add up to one, again as shown in red.
We can use an understanding of probability distributions
to help us understand how to define and abnormal test.
There are two ways that we can define an abnormal test.
With one method shown on the left we can take a group
of presumably healthy individuals and measure some lab test.
The results would vary as shown in the distribution.
And the inner 95 percentile of the distribution could be used to define the normal range.
With another method, shown on the right, we can take a cohort of patients in a clinical setting
and define a group of people with no disease and a group
with disease using some gold standard test.
On the x-axis we can then plot the results of a new test in these two groups.
If the new test is useful, it would separate people with no disease and patients
with disease into two distributions.
And usually those distributions would have some overlap as shown.
We can then draw a line of demarcation that partitions the test results into those
who we define as normal on the left, shown in red, and those that we define
as abnormal, the right, shown in green.
Here I've taken the distributions of people with no disease and patients with disease
and I've shown them separately in the bottom panel to demonstrate that the line
of demarcation identifies patients with truly negative test results shown in red on the left,
but also creates false positive test results shown in green.
Likewise, on the right, the line of demarcation identifies patients
with truly positive test results shown in red on the left,
but also creates false positive test results shown in green.
Likewise, on the right, the line of demarcation identifies patients
with truly positive test results shown in green,
but also creates false negative test results shown in red.
Unfortunately, this is the reality of clinical medicine,
because no test in medicine is perfect.
In the lower panel of this slide I have drawn the read cumulative probability curves
in each panel.
The left panel shows that if we move the line of demarcation to the right,
we would increase the number of patients with no disease who have a negative test result depicted
by the red cumulative probability curve, which is the true negative rate or specificity.
The right panel shows that if we move the line of demarcation to the left,
we would increase the number of patients with disease who have a positive test result depicted
by the red cumulative probability curve, which is the true positive rate or sensitivity.
We teach students about sensitivity and specificity, but the definitions are difficult
to memorize because the terms have different units for the numerator and denominator.
The terms, "True negative rate" for specificity, and "true positive rate"
for sensitivity are better because these terms almost defying themselves.
But we can do even better.
We can use likelihood ratios, which combine sensitivity
and specificity into a single number.
Likelihood ratios can be used to make simple calculations,
but even better they can be used intuitively to give an estimate
of the weight of new information.
Likelihood rations are dimensionless numbers, so you don't have to keep track of what's
in the numerator and what's in the denominator.
A likelihood ratio was defined as the percentage of diseased patients with a given test result,
either positive or negative, divided by the percentage
of well people with that same test result.
Therefore, the positive likelihood ratio is defined as the true positive rate,
divided by the false positive rate, or sensitivity divided by one minus specificity.
A negative likelihood ratio is defined as the false negative rate divided
by the true negative rate, or one minus sensitivity divided by specificity.
We can calculate likelihood ratios for anything that has a reported sensitivity and specificity,
such as clinical tests and even physical exam findings.
This slide shows how we calculate likelihood ratios.
Calculations for positive likelihood ratios are shown on the left,
and negative likelihood ratios on the right.
The first example is a test with a sensitivity and a specificity of 80%.
As you can see, this gives a positive likelihood ratio of four
and a negative likelihood ratio of 0.25.
The positive likelihood ratio of four is a pretty solid number,
as is a negative likelihood ratio of 0.25.
The second example is a test with a sensitivity and specificity of 91%,
which would be an outstanding clinical test.
For this test, the positive likelihood ratio is 10 and the negative likelihood ratio is 0.1.
It's hard to get likelihood ratios much better than that.
So you could consider a positive likelihood ratio as a measure of the strength
of a positive test result on a scale of 1 to 10.
And a negative likelihood ratio as a measure of the strength of a negative test
where lower is better on a scale of one down to 0.1.
This slide schematically shows how you might think about the anchoring
and adjusting heuristic and likelihood ratios.
This is a nomogram that shows how you can anchor your pretest probability on the scale
on the left, in this case at 50%.
And adjust your probability based on the likelihood ratio scale in the middle
to give you a post-test probability estimate on the scale on the right.
It shows that a likelihood ratio of one would not change your estimate or probability shown
by the line going from left to right.
The colors going up from one show how you should warm up to a diagnosis,
the higher up you go on the likelihood ratio scale.
The colors going down from one show how you should cool off
to a diagnosis the lower you go on the likelihood ratio scale.
This slide shows how a likelihood ratio of four to five, would markedly adjust the anchor
from a pretest probability of 50% to a post test probability of about 80%.
If we change the anchor to a pretest probability of 20%,
this would bring the post test probability back down to 50%,
even with a positive test result with the same likelihood ratio.
Forgetting to properly set the anchor
for the pretest probability is a fallacy called base-rate neglect.
This slide shows the schematic representation of the affect
of likelihood ratios using the nomogram on the left.
On the right are actual calculations based on a pretest probability anchor of 50%.
In order to use likelihood ratios to calculate post-test probability,
we first have to convert probability odds, and lastly convert odds back to probability.
Just for your reference, the simple formulas for doing this are shown at the top.
A probability of 50% converts to a pretest odds of one, which is an easy number
for demonstration purposes and the calculations are shown for various likelihood ratios.
In the top row, for example, a pretest probability of 0.5 or 50% converts
to a pretest odds of one, which multiplied by a positive likelihood ratio
of ten, gives a post-test odds of ten.
This converts to a post-test probability of 0.91 or 91%.
I only show these calculations for background to show how likelihood rations work as multipliers.
Seeing how likelihood ratios work as multipliers can help one gain an intuitive sense
of a true meaning of the likelihood ratio as a number.
Here is another way of visualizing the anchoring and adjusting heuristic.
The diagram shows a range of possible prior probabilities on the x-axis and the range
of possible post test or posterior probabilities on the y-axis.
The graph shows how your probability estimate would shift upward or downward,
based on whether the test result is positive or negative.
The amount of shift or adjustment depends upon the magnitude
of the likelihood ratio of the test.
The diagram on the right shows an example
of how we might anchor our pretest probability of 0.25 or 25%.
A positive test result would then shift our thinking,
causing us to adjust our probability estimate upward
to approximately 0.6 or 60% as seen on the y-axis.
This slide shows the positive and negative likelihood ratios for three commonly used tests.
A serum troponin test to diagnose a myocardial infarction, congestion on a test x-ray
to diagnose congestive heart failure,
and a serum D-dimer test to diagnose a pulmonary embolus.
A shifts in probability estimates are shown in the corresponding diagrams.
A positive troponin has a positive likelihood ratio that on a scale
of one to ten is pretty good at 4.75.
The negative likelihood ratio is also particularly strong at 0.06.
The diagram shows how our probability estimates would shift based on these test results.
For a chest x-ray if it is positive, it is really an overwhelming piece of information
with a positive likelihood ratio of 13.5, off the charts when our scale of one to ten.
The negative likelihood ratio is only so-so at 0.48.
The diagram shows that the shift in your thinking is asymmetric.
This is because the chest x-ray for congestive heart failure has very high specificity
but poor sensitivity.
You might want to remember this by remembering the pneumonic spin,
meaning a highly specific test, if positive, is good for ruling in a diagnosis.
But you don't have to remember anything about the sensitivity and specificity
because the likelihood ratio makes it obvious
that a positive chest X-ray is a strong piece of information.
In the last example, the D-dimer, on the other hand, has a poor or positive likelihood ratio
of 1.7, but an excellent negative likelihood ratio of 0.09.
The diagram shows that the shift in your thinking is asymmetric the other way,
because a D-dimer has high sensitivity and poor specificity.
This can be remembered by the pneumonic snout, meaning a highly sensitive test, if negative,
is good for ruling out a diagnosis.
But again, the likelihood ratios make it obvious that a negative D-dimer is a strong piece
of evidence and a positive D-dimer is weak evidence.
You could make an overt calculation
of probability using base theorem and the formula is shown here.
But we never do that in clinical practice and no one memorizes this formula.
You can also do a simple calculation with the likelihood ratios, but first you need
to covert probability to odds, multiply by the likelihood ratio to give you posttest odds,
then convert the posttest odds back to probability.
As shown previously, it's easy to play around with likelihood ratios to get an idea
of how they work as multipliers if you imagine a patient with a pretest probability of 0.5
or 50%, because that patient would have a pretest odds
of one, which is an easy multiplier.
But we don't even do that simple calculation of posttest odds or probability.
We estimate it intuitively.
This is where knowledge of likelihood ratios can help.
Likelihood ratios can help you calibrate your intuitive use
of the anchoring and adjusting heuristic.
Likelihood ratios give you an idea of how much your thinking should shift based on a positive
or negative test result, and how much you should warm up or cool down to the probability
of a particular diagnosis based on the strength of a test result.
So likelihood ratios can help you calibrate your intuitive probability estimate,
which is your degree of belief that your patient has a certain diagnosis.
Think of a positive likelihood ratio as the strength of a positive test result on a scale
of roughly one to ten, and a negative likelihood ratio as the strength of a negative test result
where lower is better on a scale of roughly one down to 0.1.
How do we verify a possible diagnosis intuitively?
We use a heuristic called anchoring and adjusting
to estimate pretest and posttest probabilities.
We can calibrate our intuition using likelihood ratios
to improve the accuracy of our probability estimates.
It is important to make accurate probability estimates, because the goal is
to use the test results to reach a conclusion that is strong enough to act upon,
in order to provide effective and timely treatment.
I hope this brief discussion of the anchoring and adjusting heuristic, and likelihood ratios,
will improve your interpretation of diagnostic testing.
Thank you for your attention.