Hello, I’m Chana Sacks, executive editor of NEJM Evidence, and this is Stats, STAT!
It’s a big day! You’re a clinical trialist, and today’s the day you begin enrolling
participants in your double-blind, randomized controlled trial of a new sleep medicine.
50 participants will be randomized to the treatment arm, and another 50 to the placebo arm.
Now, imagine that a few days into the trial, one participant assigned to the treatment arm
starts having vivid nightmares. Worried she is experiencing side effects from the study drug,
she decides to skip the medication. Meanwhile, a participant in the placebo arm keeps forgetting
to take his pills. And still another has unexpected travel and leaves the pills at home.
At the end of the study period, when everyone returns for a pill count,
these participants and some others still have most of their pills left.
So what should you do with these participants’ data — and the data from all the others
like them who didn’t stick to the assigned treatment protocol for one reason or another?
This question comes up in clinical trials all the time. While investigators hope for perfect
adherence, reality is messy. Nonadherence happens for lots of reasons — some related to the study
drug (think: side effects) and some not (think: just forgetting to take the pills). In all cases,
investigators must decide how to analyze the data when things don’t go according to plan.
But don’t lose sleep over it! Three common analytic approaches that you
may have heard about are per-protocol, as-treated, and intention-to-treat.
Let’s take a look at each. A per-protocol analysis focuses only
on participants who stuck to the protocol, excluding those who didn’t take the pills,
switched treatment arms, or took the pills differently than prescribed.
It might sound dreamy, but this approach comes with its own problems. First, you might exclude
participants who stopped the treatment because of its side effects — that’s important information!
Second, excluding participants slashes the sample size and reduces statistical power.
Third, a per-protocol analysis introduces bias. Participants who adhere to the protocol may be
very different from those who don’t; by excluding some participants, you lose the
benefits of randomization, effectively turning your experiment into a non-experimental study.
A second option is the as-treated analysis in which participants are analyzed based on the
treatment they actually received, rather than what they were assigned. Imagine the
participant with nightmares lets you know right away she was opting not to take the treatment;
an as-treated approach might lead you to analyze her data as part of the placebo group.
This approach may give you a real-world snapshot of how people decide on treatment options,
but it too eliminates randomization, again essentially giving you a non-experimental study.
On the other hand, in the intention-to-treat analysis, every participant randomized
at the beginning of the trial is analyzed according to their original group assignment,
regardless of adherence. This maintains all the benefits of randomization and keeps the
sample size and similarity between the treatment and placebo groups intact.
What are the downsides? Well, because both trials arms are likely to have participants
who didn’t follow the protocol, the outcomes in the two groups may appear more similar
than they might have been had adherence been better. Keep in mind, an intention-to-treat
analysis estimates the effect of being assigned to treatment vs. placebo — only
with perfect adherence does it estimate the effect of actually taking the medication.
Of these options, the intention-to-treat approach is the only one that preserves the key benefits of
randomization. That’s why it’s the go-to approach in randomized trials — and why you decide to
stick with analyzing all the participants in both arms according to their assigned group.
This also makes clear just how important it is to track the treatments participants actually
receive, instead of just assuming that everything worked like a dream.
There are other advanced statistical methods we could mention,
but for now we’ll let those sleeping dogs lie.