In the beginning of this series, you learned why to do CEAs and how to compare net costs
to health outcomes. Then you learned about DALYs and QALYs, and how to present CEA results
the base case.
In this segment, CEA Sensitivity Analyses, we show you how to quantify the importance
of uncertainty in your analysis.
CEAs have lots of uncertainty. There may be dozens of input variables affecting your estimates
of cost and DALYs, if the model is of typical complexity. Each input has uncertainty. This
may be formal confidence bounds, for example HIV incidence of 1.5 per hundred person-years,
95% confidence interval 1.1 to 2.0 or differences in estimates across studies or a lack of relevant
formal studies, forcing reliance on expert opinion. This uncertainty in inputs can lead
to large uncertainty in CEA results.
Sensitivity analyses are CEAs version of 95% confidence intervals in empirical studies.
Well-designed sensitivity analyses deal systematically with this uncertainty. Each input is considered
in isolation, and in combination with other inputs. Sensitivity analyses can convince
audiences that results are robust that is, that the same basic conclusion holds despite
uncertainty in the precise numerical findings. Or, in contrast, that results depend on the
value of certain inputs. Sensitivity analyses can show that high profile uncertainties such
as intervention effectiveness or the lifetime costs of treating a disease), which many readers
worry about may not affect findings significantly. In sum, sensitivity analyses are mandatory.
But they're also interesting. And fun.
Here"s an efficient and fun way to summarize one-way sensitivity analyses. It's called
a tornado diagram, because of the shape. Each horizontal bar indicates the uncertainty in
the ICER associated with uncertainty in that input. A longer bar means more uncertainty.
Thus, for this analysis of gestational diabetes screening and treatment, the input value that
most affects results is the cost of postpartum care (at the top, spanning $900 to $2,400
per DALY averted) and the input value that least affects results is the cost of type
2 diabetes in the children of women with gestational diabetes (at the bottom, spanning $1,600 to
$1,680 per DALY averted). Multi-way sensitivity analyses are the perfect
way to summarize the aggregate effect of uncertainty in all inputs. A Monte Carlo simulation, as
shown here, calculates the distribution of results from many iterations of the model
in this example, 20,000 iterations. The histogram shows the frequency of narrow ranges of results.
It indicates that the most common result is very similar to the base case of $1,640, with
90% of simulations between $400 and $4,000.
23.5 [1]Sensitivity analyses do more than just quantify the impact of uncertainty in
model input values. They help the analyst understand a model that can get pretty complex.
If results change in predictable ways as input values are widely varied, this is evidence
of the model's sound structure. Conversely, unexpected patterns may reveal technical errors
in the model. Occasionally, such unexpected results uncover interesting dynamics in the
CEA. One example of such an interesting dynamic from our work is when the cost of anti-retroviral
therapy for HIV didn't affect the results because it appeared (appropriately) in two
offsetting parts of the model. Sensitivity analyses also bolster acceptance
of the CEA. They demonstrate transparency to the reader, and thus build trust in the
model's functioning and findings. Indeed, and counter to my original intuition, documenting
uncertainty via sensitivity analyses tends to foster the audience's emphasis on the base
case. With SAs, the reader can think only, How certain are these results? With SAs, the
distraction of unexplored uncertainty is removed, and the reader tends to return to a focus
on the base case. Of course, some SAs reveal massive uncertainty
due to a poorly understood input, and that becomes the main finding. But most CEAs have
a solid trend in results, and once SAs demonstrate that, the uncertainty slips into the background.
That's all for sensitivity analyses. There's lots more to learn, and eventually to do typically?
to 1/3 of total CEA effort is needed to design, conduct, interpret, and respond to SAs.
Up next: timing in CEAs.