- Asalamu alaikom, welcome back to another biostats ATP video,
this time around we’ll be talking about the “receiving operating characteristic” curve,
also known as the ROC curve.
Introduction: This curve helps us understand
how well a test can distinguish between two groups. In other words,
how well can the test see if the patient does or does not have the disease. The Y axis is the True
Positive rate otherwise known as Sensitivity. The X axis is the FP rate, otherwise known as
the complement or “the opposite” of Specificity or 1-Specificity if you will. In other words,
x axis represents true positive rate (TPR) while Y-axis represents false positive rate (FPR)
ROC Curve: The curve is
defined by its specificity and sensitivity and associated cutoffs. In other words, how well
it can identify those with a disease compared to those without a disease based on different
cutoffs is what determines the shape of the curve. For instance, let’s take a group of people we
tested for condition X with a cutoff of 100 units. Suppose after all the calculations,
we ended up with a sensitivity (SN) of 80% and specificity (SP) of 85%. However,
what if we want to see what would happen to SN and SP if we raised or lowered the cutoff,
for example, when we raised the cutoff by 20 units, the SN dropped to 70% while SP became 90%.
What if we lowered the cutoff by 20 points? Then SN became 90%
and SP dropped to 70%. I’m just making up the numbers here to keep it simple but notice how
we’re sort of getting a nice curve here. If you rinse and repeat this with many different cutoffs,
you’ll get a curve that shows you what the SN and SP would look like based on the chosen cutoffs.
This makes it easier to understand what cutoffs would be most helpful to apply in each situation
without having to manually write down all the possible cutoffs and their associated SN and SP.
Notice how the lower left corner suggests a SN of 0% and SP of 100%, this would be like
raising the cutoff so high that you won’t get any FP. Hmm, still not really that practical, huh?
On the other end of the spectrum, the top right corner suggests a SN of 100%
and SP of 0%, this would be like having a very low cutoff, sure you wouldn’t get any FN,
but this isn’t all that practical either. The top left corner suggests a test with
100% SN and 100% SP. That is where we would ideally want every test to be.
Please note that sometimes, the curve is not perfectly symmetrical. This is just a testament
to how real life doesn’t always provide you with perfect curves, but beggars can’t be choosers, eh?
ROC Curve Applications: Moving on, how can we apply this concept?
Well, sometimes a test is needed to screen or confirm diseases. If you are using the test
as a screening tool, your job is to look for the test with the highest sensitivity,
in other words, the highest point on the curve within acceptable detriment to specificity.
That is to say, we can’t just take the highest point overall because this would
leave us with a SP that is very close to 0, and thus the test would have much less value.
A good example of a high point that doesn’t completely wipe SP would be like this one for
example. But, if it’s a confirmatory test, you’re looking for the left most point, so you would pick
this point without completely wiping out SN. Golden rule when using tests in medicine:
if you are SCREENING focus on sensitivity, but if you are DIAGNOSING focus on specificity
As you can see, the ROC curve is immensely useful in helping us visualize the different
cutoffs of a test and allows us to make decisions as to which test or which
cutoffs of a particular test would be better suited to being a more SN tool or SP tool.
When it comes to comparing two different tests, eyeballing the ROC curves becomes very handy.
Since the top left corner suggests an ideal test, we want curves to be closer to that corner.
Another way you could go around this is to check which curve has more area
under it. Every curve will start in the bottom left and end at the top right. So how
much it approaches the top left corner will also determine how much area it covers under it. So,
for example, when looking at these two curves, it’s clear that A is the better performing test.
With this paradigm in your mind, we’d like to add a few more concepts to help tie things together:
Accuracy is another important term and relates to the validity of the measurement. I like to
think of it as the “trueness” of a measurement to help remind me of the formula which is TP + TN/
TP + FP + TN + FN (pretty much everything goes in the denominator, while only the true ones go into
the numerator). A test is considered more accurate the closer it approaches the top left corner, and
less accurate as it approaches the diagonal line. We wanted to just mention a few points here to
help tie in a few more concepts in relation to the ROC curve.
Last but not least, since the ROC curve can reveal to us the SN and SP of a particular cutoff,
we can actually find out TP, TN, FP, and FN. However, we also need a bit more given,
specifically, we need the amount of people we are testing and how many of them have
the disease, as known as, the prevalence. Using this given, we can find out the TP,
TN, FP, and FN. Turns out ROC curves can do a lot of cool things, huh?
- In summary ROC curve is built using 2 axes: sensitivity and 1-specificity,
otherwise known as false positive rate - It allows you to choose the optimal
cutoff for a test by looking at the ROC curve of that test. That way you can have an idea
about the SN and SP of that cutoff - ROC also allows you to compare
between different tests, by calculating the AUC,
to determine which one is better - Lastly, remember this relationship:
SN – Screening for a disease SP – Diagnosing/conforming a diagnosis
With that, we hope you benefited from this video! Consider liking and subscribing,
and as always, thanks for watching!