Welcome to Impact Factor, your weekly pinch of commentary on a new medical study, I’m
Dr. F. Perry Wilson.
And this week, I need to tell you about a study with results that are simply unbelievable.
And I don’t mean that like these results are amazing, I mean I don’t believe them.
But I hope that in discussing them I’ll share some insight on how to interpret all
medical studies.
But let’s start with the basics.
This was a randomized trial, appearing in JAMA Internal Medicine, of acupuncture for
the treatment of chronic stable angina in China.
Around 400 patients with stable angina were recruited.
At baseline, they were having an average of around 13 anginal attacks per month.
The participants were randomized into four groups, and here is where it gets interesting.
First, there was a real acupuncture group, where two needles were placed on specific
points - here and here – that are supposed to lie on invisible energy lines – or meridians
- that are beneficial for the heart according to traditional Chinese medicine.
They called this the disease-affected meridian group.
Electric stimulus was applied to the needles as well.
A second group got two electric needles applied to different acupuncture points here and here
– nonaffected-meridians that are not supposed to be beneficial to the heart.
A third group got non-electric needles applied to “sham” points – not associated with
an energy meridian.
And a fourth group got waitlisted – they didn’t get any acupuncture at all.
Anginal attacks decreased in all the groups, but as you can see here they decreased most
in the disease-affected meridian group, then the nonaffected meridian group, then the sham
group, then the waitlist group.
The explanation offered is that these significant differences demonstrate that, in fact, acupuncture
works.
It’s not just the placebo effect – there is something special about sticking needles
in these specific points that reduces angina more than sticking needles anywhere else.
If true, this is a groundbreaking finding – an earth-shattering finding really – that
upends everything we think we understand about physiology.
And that – I can’t believe.
I do not believe there are hidden energy fields in our body that can be manipulated with needles.
I don’t believe that because if there were they could be directly detected by any variety
of modern electromagnetic sensors.
So – how did these results come about?
There are a couple of possibilities.
First, I may be wrong.
My belief that undetectable energy paths do not flow through the body, founded on a mechanistic
understanding of the universe, may be parochial, small-minded, pompous and limited.
I’ve been wrong before and I’ll be wrong again.
But there are other possibilities and we need to consider those before we embrace a hypothesis
that is inconsistent with observation.
The simplest explanation for these findings is that patients were aware of what group
they were in.
The study states that participants were blinded to the intervention, but it is quite possible
that they could figure it out.
Acupuncture is much more popular in China than the US, and participants may have known
that they were getting needles in “heart” associated locations as opposed to non-heart
associated locations.
And of course, the acupuncturist knew.
It is possible that, whether consciously or unconsciously, that information was conveyed
to the participant.
One easy way to test this is to ask the participants, after the trial, to guess what group they
were in.
As far as I can tell, this wasn’t done.
Another possibility is that there was systematic error in measurement or recording of the data.
The point is that this study, and all studies, need to be interpreted not just by the p-value
that pops out at the end, but by the likelihood of the underlying hypothesis.
I am encouraged that the recent CMS proposal to fund acupuncture for patients requires
enrollment in a randomized trial.
I would encourage any such studies to robustly assess the quality of blinding.
If this trial had shown that needles in the disease-affected meridian and non-affected
meridians both helped equally, better than non-electric sham stimulation, I’d have
less trouble believing it.
But ironically the significant differences between the disease-affected meridian group
and the non-affected meridian group in this trial actually undermines all the results
for me.
That’s right.
This study is too good to be true.