jump to navigation

Sloppy reporting. June 5, 2013

Posted by ourfriendben in wit and wisdom.
Tags: , , , , ,

Silence Dogood here. If you’ve ever taken a class in statistics, you know that by shaping the parameters of a study, you can make it say whatever you want. (“Three out of four people have blue eyes,” you can announce, if you find three blue-eyed people and pair them up with a brown-eyed person. No matter that blue eyes are in fact the second-rarest and brown eyes the most common.) Sponsored studies are conducted all the time at the behest of corporations with agendas. They put up the money, and scientists who have kids to send to college and mortgages to pay off find themselves reporting that people who eat three Big Macs a day or Monsanto-engineered foods live longer, healthier lives.

This is our reality, and caveat emptor, buyer beware. Of course it makes me sick. But there should be a barrier between the public and research of this kind, and that barrier is journalism. Journalists understand the nature of statistics better than most. They should be watching for the corporate-funded studies that amazingly happen to support corporate greed. And they should be watching for studies that are inherently flawed, even if the flaws are inadvertent, before trumpeting results that are dubious at best and inaccurate at worst.

What set me off on this tirade was an article in yesterday’s Wall Street Journal, “Green Is Good: Vegetarians Live Longer, Study Finds.” You might have thought that I, as a vegetarian, would have been thrilled by this article. But I wasn’t, because the results were flawed, and the reporter didn’t pick up on it.

The good folks conducting the research didn’t compare just any vegetarians with omnivores. They looked at the health history of Seventh-Day Adventists versus the general omnivorous public. Yes, Seventh-Day Adventists are vegetarian. But they also don’t drink, smoke, or take drugs.

For the results to be valid, the scientists conducting the study would have to either compare Seventh-Day Adventists to omnivores who also didn’t smoke, drink, or take drugs, or compare vegetarians in the general population to omnivores in the general population, who might or might not indulge in smoking, drinking, or the like. The only appropriate conclusion from this study is “Seventh-Day Adventists Live Longer, Study Finds.” Shame on the reporter for not calling out such an obvious flaw!

Given the prevalence of such sloppy reporting, it’s up to us to keep our eyes wide open when reading the conclusions drawn by statistical research. Who funded the study? Do the conclusions benefit the entity that sponsored it? Is there some inherent logical flaw in the research, something that would throw off the results, even if no one stands to benefit from them? Put on your Sherlock Holmes deerstalker cap next time you read about a study that “proves” this or that. Maybe it does. But then again, maybe it’s just bad science.

‘Til next time,




1. kickerkim - June 5, 2013

I enjoy ‘bad science’ pick ups, this one is a goody – thanks!

Thanks, Kim! Glad you enjoyed it!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: