So Why ARE Most Published Research Findings False? (An Introduction)

Well hello hello! I’m just getting back from a conference in Minneapolis and I’m completely exhausted, but I wanted to take a moment to introduce a new Sunday series I’ll be rolling out starting next week. I’m calling it my “Important Papers” series, and it’s going to be my attempt to cover/summarize/explain the important points and findings in some, well, important papers.

I’m going to start with the 2005 John Ioannidis paper “Why Most Published Research Findings are False“.  Most people who have ever questioned academic findings have heard of this one, but fewer seem familiar with what it actually says or recommends. Given the impact this paper has had, I think it’s a vital one for people to understand.  I got this idea when my professor for this semester made us all read it to kick off our class, and I was thinking how helpful it was to use that as a framework for further learning. It will probably take me 6 weeks or so to get through the whole thing, and I figured this week would be a good time to do a bit of background. Ready? Okay!

John Ioannidis is Greek physician who works at Stanford University. In 2005 he published the paper “Why Most Published Research Findings Are False”. This quickly became the most cited paper from PLOS Medicine, and is apparently one of the most accessed papers of all time with 1.5 million downloads. The paper is really the godfather of the meta-research movement…i.e. the push to research how research goes wrong. The Atlantic did a pretty cool breakdown of Ioannidis’s career and work here.

The paper has a few different sections, and I’ll going through each of them. I’ll probably group a few together based on length, but I’m not sure quite yet how that will look.  However, up front I’m thinking the series will go like this:

  1. The statistical framework for false positive findings
  2. Bias and failed attempts at corrections
  3. Corollaries (aka uncomfortable truths)
  4. Research and Bias
  5. A Way Forward
  6. Some other voices/complaints

I’ll be updating that list with links as I write them.

We’ll kick off next week with that first one. There will be pictures.

Week one is up! Go straight to it here.

 

5 thoughts on “So Why ARE Most Published Research Findings False? (An Introduction)

  1. This hits me so hard, as you can well imagine, after reading Taleb’s (Antifragile) frustration with the medical community ignoring long-term downsides in pursuit of short-term gains in marker numbers. I take 6 medicines every morning, 4 prescribed for metabolic syndrome conditions. Every one of them has been called into question whether it actually does any good. And the statin sometimes causes diabetes, which one of the other 3 treats. The two items non-prescribed but recommended by my PCP, aspirin and Vitamin D, are also controversial. I look at them and wonder “Is this a net gain or net loss?”

    Like

  2. Pingback: So Why ARE Most Published Research Findings False? A Statistical Framework | graph paper diaries

  3. Pingback: So Why ARE Most Published Research Findings False? Bias and Other Ways of Making Things Worse | graph paper diaries

  4. Pingback: So Why ARE Most Published Research Findings False? The Corollaries | graph paper diaries

  5. Pingback: Stats in the News: February 2017 | graph paper diaries

Comments are closed.