Protect yourself from Bad Science

The Pediatric Insider

© 2016 Roy Benaroch, MD

Science, to use the term correctly, isn’t a body of knowledge or a bunch of facts written up on a whiteboard by a crazy-haired professor. It’s a method, or way of figuring out things. Thinking and reading and learning from experts are all important, sure. But real science relies on experimentation.

First, make an educated guess about how something works. Then design an experiment to test your guess (I’m oversimplifying here. Designing the correct experiment takes genius and inspiration). Collect your data carefully, and see if you were right. Then do it again, and again.

Ideally, your experiments and conclusions are written up and published in what are called “peer-reviewed journals,” where fair-minded and educated people review your work to make sure it’s up to par. Then it’s published, and other scientists can see what you’ve done. Your experiments can be repeated and tweaked, maybe confirmed or refuted. In time, the body of what’s become known as “accepted scientific knowledge” grows. And voilà, we’ve landed a man on the moon, unlocked the secrets of the atom, or turned HIV into a treatable infection.

Lather, rinse, repeat. Science isn’t fast, and there are a lot of details that have to be done right. Some experiments, just from bad luck, will lead to dead ends or false conclusions. But it’s clear from the pace of modern progress that The Scientific Method is the best way to figure out how the natural world works.

Still, it’s far from perfect. And a few troubling trends are contributing to a sense that scientists might not always get it right.

These days, anyone can publish anything, either online or in print. I’ve got a few self-published books out there (ie, I paid to print them), along with books that were professionally edited by a genuine publisher (ie, they paid me. Suckers!) Most readers would have a hard time telling the difference. Likewise, blogs and websites—who knows who’s behind most of what’s linked out there? In the past, the quality of scientific publication was guaranteed by a process called “peer-review”, where a new paper was read by several independent experts.

A December, 2015 editorial in the New England Journal of Medicine reviews a growing problem with peer-review: hacking. The authors of a new paper often suggest their own peer reviewers, who sometimes turn out to be fake people with fake e-mails—the “peer review” is actually being done by the authors themselves. Hundreds of articles have been retracted, now that editors know to look for peer review hacking, but it’s likely that thousands more studies already out there are similarly tainted by poor or fraudulent vetting.

But what about the journals themselves? Another huge problem is the proliferation of for-profit, “predatory” journals. They make their money by publishing anything that’s submitted, with zero-to-no editorial review or peer-review. You pay, you play. Some of these are respected journals that have been bought out by shady profiteers. Some are new journals, just made up, that exist only in cyberspace – they have essentially no operating expenses, and publish only online for a fee. Just because you read that something was “published”, doesn’t mean that it’s anything worth reading or paying attention to. Experiments are either entirely faked or done so poorly that they’re worthless – but they’re still published, and unfortunately still reported in the media.

The best defense against the influence of crappy science is for audiences to maintain their skepticism. Something published in a barely-known or predatory journal (yes, there are established lists of these) shouldn’t be turned into headlines or Facebook clickbait. Though everyone is an expert who’s told to “do their own research”, there’s still a place for genuine science journalists and genuine experts to help guide the rest of us through the confusing maze of new information. Don’t believe everything you read, even if it’s couched in sciency-looking terms and looks like it’s been published in a sciency-looking journal.

Science is slow, and far from perfect. But it’s still the best method we’ve got.

Step right up!

Advertisements
Explore posts in the same categories: Pediatric Insider information

Tags: ,

You can comment below, or link to this permanent URL from your own site.

One Comment on “Protect yourself from Bad Science”

  1. science mom Says:

    Thank you for this, I wish more people would remind the world how true knowledge is gained and how to judge what you read for its value.

    I am a policy researcher and have had years of training on how to rank what information can be used as fact and what is just hearsay. I feel parents are particularly prone to believing something a group of moms is talking about, there is so much worry that goes in to parenting these days.

    The vaccine craziness is a perfect example. A fraud study kicked off a movement and now moms think, “if so many are talking about it, there must be something to it.” I’ve even had people say, “you’ve got to trust your instinct above all else, and it just makes sense that vaccines could cause harm.” No, you “instinct” is actually not something to trust if you’ve had no medical training.

    But people don’t want to hear it. I can now send them you thoughts on the matter, written by genuine pediatrician, how helpful.

    Thanks again.

    On Thu, Feb 18, 2016 at 6:41 AM, The Pediatric Insider wrote:

    > Dr. Roy posted: “The Pediatric Insider © 2016 Roy Benaroch, MD Science, to > use the term correctly, isn’t a body of knowledge or a bunch of facts > written up on a whiteboard by a crazy-haired professor. It’s a method, or > way of figuring out things. Thinking and readin” >

    Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: