In the beginning of this series, you learned why to do CEAs and how to compare net costs
In the beginning of this series, you learned why to do CEAs and how to compare net costs

to health outcomes. Then you learned about DALYs and QALYs, and how to present CEA results
to health outcomes. Then you learned about DALYs and QALYs, and how to present CEA results

the base case.
the base case.

In this segment, CEA Sensitivity Analyses, we show you how to quantify the importance
In this segment, CEA Sensitivity Analyses, we show you how to quantify the importance

of uncertainty in your analysis.
of uncertainty in your analysis.

CEAs have lots of uncertainty. There may be dozens of input variables affecting your estimates
CEAs have lots of uncertainty. There may be dozens of input variables affecting your estimates

of cost and DALYs, if the model is of typical complexity. Each input has uncertainty. This
of cost and DALYs, if the model is of typical complexity. Each input has uncertainty. This

may be formal confidence bounds, for example HIV incidence of 1.5 per hundred person-years,
may be formal confidence bounds, for example HIV incidence of 1.5 per hundred person-years,

95% confidence interval 1.1 to 2.0 or differences in estimates across studies or a lack of relevant
95% confidence interval 1.1 to 2.0 or differences in estimates across studies or a lack of relevant

formal studies, forcing reliance on expert opinion. This uncertainty in inputs can lead
formal studies, forcing reliance on expert opinion. This uncertainty in inputs can lead

to large uncertainty in CEA results.
to large uncertainty in CEA results.

Sensitivity analyses are CEAs version of 95% confidence intervals in empirical studies.
Sensitivity analyses are CEAs version of 95% confidence intervals in empirical studies.

Well-designed sensitivity analyses deal systematically with this uncertainty. Each input is considered
Well-designed sensitivity analyses deal systematically with this uncertainty. Each input is considered

in isolation, and in combination with other inputs. Sensitivity analyses can convince
in isolation, and in combination with other inputs. Sensitivity analyses can convince

audiences that results are robust that is, that the same basic conclusion holds despite
audiences that results are robust that is, that the same basic conclusion holds despite

uncertainty in the precise numerical findings. Or, in contrast, that results depend on the
uncertainty in the precise numerical findings. Or, in contrast, that results depend on the

value of certain inputs. Sensitivity analyses can show that high profile uncertainties such
value of certain inputs. Sensitivity analyses can show that high profile uncertainties such

as intervention effectiveness or the lifetime costs of treating a disease), which many readers
as intervention effectiveness or the lifetime costs of treating a disease), which many readers

worry about may not affect findings significantly. In sum, sensitivity analyses are mandatory.
worry about may not affect findings significantly. In sum, sensitivity analyses are mandatory.

But they're also interesting. And fun.
But they're also interesting. And fun.

Here"s an efficient and fun way to summarize one-way sensitivity analyses. It's called
Here"s an efficient and fun way to summarize one-way sensitivity analyses. It's called

a tornado diagram, because of the shape. Each horizontal bar indicates the uncertainty in
a tornado diagram, because of the shape. Each horizontal bar indicates the uncertainty in

the ICER associated with uncertainty in that input. A longer bar means more uncertainty.
the ICER associated with uncertainty in that input. A longer bar means more uncertainty.

Thus, for this analysis of gestational diabetes screening and treatment, the input value that
Thus, for this analysis of gestational diabetes screening and treatment, the input value that

most affects results is the cost of postpartum care (at the top, spanning $900 to $2,400
most affects results is the cost of postpartum care (at the top, spanning $900 to $2,400

per DALY averted) and the input value that least affects results is the cost of type
per DALY averted) and the input value that least affects results is the cost of type

2 diabetes in the children of women with gestational diabetes (at the bottom, spanning $1,600 to
2 diabetes in the children of women with gestational diabetes (at the bottom, spanning $1,600 to

$1,680 per DALY averted). Multi-way sensitivity analyses are the perfect
$1,680 per DALY averted). Multi-way sensitivity analyses are the perfect

way to summarize the aggregate effect of uncertainty in all inputs. A Monte Carlo simulation, as
way to summarize the aggregate effect of uncertainty in all inputs. A Monte Carlo simulation, as

shown here, calculates the distribution of results from many iterations of the model
shown here, calculates the distribution of results from many iterations of the model

in this example, 20,000 iterations. The histogram shows the frequency of narrow ranges of results.
in this example, 20,000 iterations. The histogram shows the frequency of narrow ranges of results.

It indicates that the most common result is very similar to the base case of $1,640, with
It indicates that the most common result is very similar to the base case of $1,640, with

90% of simulations between $400 and $4,000.
90% of simulations between $400 and $4,000.

23.5 [1]Sensitivity analyses do more than just quantify the impact of uncertainty in
23.5 [1]Sensitivity analyses do more than just quantify the impact of uncertainty in

model input values. They help the analyst understand a model that can get pretty complex.
model input values. They help the analyst understand a model that can get pretty complex.

If results change in predictable ways as input values are widely varied, this is evidence
If results change in predictable ways as input values are widely varied, this is evidence

of the model's sound structure. Conversely, unexpected patterns may reveal technical errors
of the model's sound structure. Conversely, unexpected patterns may reveal technical errors

in the model. Occasionally, such unexpected results uncover interesting dynamics in the
in the model. Occasionally, such unexpected results uncover interesting dynamics in the

CEA. One example of such an interesting dynamic from our work is when the cost of anti-retroviral
CEA. One example of such an interesting dynamic from our work is when the cost of anti-retroviral

therapy for HIV didn't affect the results because it appeared (appropriately) in two
therapy for HIV didn't affect the results because it appeared (appropriately) in two

offsetting parts of the model. Sensitivity analyses also bolster acceptance
offsetting parts of the model. Sensitivity analyses also bolster acceptance

of the CEA. They demonstrate transparency to the reader, and thus build trust in the
of the CEA. They demonstrate transparency to the reader, and thus build trust in the

model's functioning and findings. Indeed, and counter to my original intuition, documenting
model's functioning and findings. Indeed, and counter to my original intuition, documenting

uncertainty via sensitivity analyses tends to foster the audience's emphasis on the base
uncertainty via sensitivity analyses tends to foster the audience's emphasis on the base

case. With SAs, the reader can think only, How certain are these results? With SAs, the
case. With SAs, the reader can think only, How certain are these results? With SAs, the

distraction of unexplored uncertainty is removed, and the reader tends to return to a focus
distraction of unexplored uncertainty is removed, and the reader tends to return to a focus

on the base case. Of course, some SAs reveal massive uncertainty
on the base case. Of course, some SAs reveal massive uncertainty

due to a poorly understood input, and that becomes the main finding. But most CEAs have
due to a poorly understood input, and that becomes the main finding. But most CEAs have

a solid trend in results, and once SAs demonstrate that, the uncertainty slips into the background.
a solid trend in results, and once SAs demonstrate that, the uncertainty slips into the background.

That's all for sensitivity analyses. There's lots more to learn, and eventually to do typically?
That's all for sensitivity analyses. There's lots more to learn, and eventually to do typically?

to 1/3 of total CEA effort is needed to design, conduct, interpret, and respond to SAs.
to 1/3 of total CEA effort is needed to design, conduct, interpret, and respond to SAs.

Up next: timing in CEAs.
Up next: timing in CEAs.