recently the website The Hill posted this attention-grabbing headline: "Vegetarians
who drink and smoke a lot are still healthier than meat eaters, big new study finds". wow, that's
a head-turner. is this based on solid evidence? time for some headline bashing. the body of the
article actually doubles down: "research shows that vegetarians are healthier than meat lovers, even if
they are smokers and drinkers". what's this based on? the article says it's according to StudyFind
so let's follow that. it's another website talking about the study. it points to a media release so
let's follow that one. here it is, it describes the findings and says "for full poster click here". so
it's not a published study, it's a poster. a poster is how scientists communicate their data to each
other in conferences and the data on a poster can be quite definitive but it can also be preliminary,
ongoing work. i presented some posters in grad school that ended up in highly cited papers
and others that were never published because things changed with additional experiments
so i don't even think a poster should necessarily be advertised in headlines for the general public
before peer review. very risky. but let's take a look at it. here's the poster. they measure the
levels of different biomarkers, that's markers of things like cholesterol, vitamin D, etc in
vegetarians and meat eaters. some were higher in the vegetarians, some were lower. i actually can't
tell what's statistically significant because they don't report p values but let's give them
the benefit of the doubt and say everything's significant. they conclude the vegetarians had
more favorable biomarkers. even that is arguable because some were better and some were worse but
okay, let's say overall they were better. here's the kicker: the model was adjusted for smoking, alcohol,
etc. that means they tried to remove those factors from the equation to try to isolate the effect of
diet, aside from all the other variables. notice how they described that: "the results were independent
of lifestyle confounding factors". so the biomarker differences they saw are when smoking and drinking
are off the table. now let's work backwards and see how this got increasingly twisted
in that chain of headlines. the press release starts bending the message by saying the findings
are unaffected by smoking and alcohol. that's not really what they showed,
they saw those results when they excluded those factors. we don't know if they're affected
by smoking or not, they don't show the results with and without smoking so we can compare. at
least the press release still refers to the biomarkers. now, the first website reports on it.
they bastardize it even more. they got rid of biomarkers, now they just say
vegetarians are healthier, period. and now it's regardless of how much they drink and smoke
and finally the hill butchers it some more. now they're healthier even if they drink and smoke
a lot. remember, the researchers tried to adjust for smoking and drinking, so their results are looking
at a scenario where alcohol and tobacco are not a factor. that's the opposite of what the headlines
are saying. so this is like that telephone game where the message just gets increasingly twisted
by the end all they're doing is finding an article on a website and recycling it but
making the headline even flashier without ever looking at the original data they're talking about
think about how surreal that is. the final headline is inaccurate on almost everything. it's inaccurate
to say it's when they drink and smoke, the poster didn't show that. it's inaccurate to say they're
healthier, they only measured biomarkers and not all were better on vegetarians
so this is an oversimplification and a leap. it's even misleading to call it a study because
that implies a published peer-reviewed study, not a preliminary poster. now imagine the damage this can
do. at two levels. first, somebody sees this headline and thinks, "oh sweet as long as i don't eat meat i
can smoke like a chimney". and they damage their health. second, they see these sloppy headlines, a
month from now they see a different headline with the opposite message and they just lose confidence,
not just in these websites but in science as a whole. and then we start propagating this idea
that scientists can't agree. now, i'm not blaming bloggers and journalists, they're incentivized to
make things clickable and that's what they did here, at the expense of accuracy. so the
best solution is for us to educate ourselves so we don't fall for this stuff. so the next time you see
that flashy headline, remember: the writer probably didn't read the study, probably didn't even see it,
and is completely focused on getting clicks so take it with all the grains of salt