A few months ago, I like many others, was surprised to read the article about how Oliver Sacks had made up many of his most famous stories about his patients. Printed in the New Yorker, this article hit many in the scientific community where it hurt. A lot of us had read Sacks books in undergrad or grad school, he was considered an inspiration in both the field of neurology and science communication. Discovering he had made up some of what he wrote about threw his whole body of work in to question.

Unfortunately, the problem didn’t stop there. Two months later the New Yorker published another investigative report on a different famous case study, this one about a baby who allegedly showed signs of opioid exposure from breastfeeding from a mother who was taking codeine. The problem? It was made up. But wait, it gets worse. The original case report was followed by another one published in the Canadian journal Paediatrics & Child Health, and this week they put a label on ALL their case studies in the series stating that they were fictional.

Retraction Watch has the full story here, but it turns out the story for this journal is even more confusing than you’d think. At some points (but not all) the journal actually asked authors to submit fictional case studies for learning purposes. Unfortunately this was never flagged on the published items themselves, so readers were left with the impression they were reading real stories. Adding to the confusion is that the instructions didn’t always say this, so some of these reports were real but are now marked as fictional and that can’t be undone. Confused? Me too.

So why does this matter?

Back when the Sacks thing first came up, the AVI posted on it and Grim asked a really good question in the comments:

It’s far enough outside my area that I’ve never heard of Oliver Sacks before today. Nevertheless, I believe you and bs king when you say that this sort of fraud affects the whole society. How would you characterize the effects it has had, which someone like me might be affect by without being aware of them? It obviously affected your profession and several others, which each exercise influence, for example.

My reply at that time (including some commentary on an earlier Dante comment I’d made) with some parts I think are important bolded:

From my perspective, I took Dante to mean that being a fraud victim (or even merely aware of fraud) erodes trust in society in general, so each new case will keep chipping away at how we all view each other. So if I took Sacks writing at his word and found I was deceived, I will subsequently distrust your writing more merely to ensure I’m not deceived again. It’s a reality of life I guess, but it just strikes me as sad that everything I read my first thought needs to be “is this person fabricating this”? So my comment mostly wasn’t about Sacks himself, but about how much skepticism we need to carry to operate in the world. There’s downsides to that.

For Sacks specifically, case studies are often the first step in medical research. Some diseases or problems are simply uncommon enough that the first person to notice it can’t do a whole study, so they write up a “here’s a weird thing I saw in a patient” report and then see if anyone else has seen it too. This was the first step in discovering HIV for example. Sacks writing was basically case reports, and they influenced a lot of people’s thinking about where the margins of brain function were. These are supposed to be teaching tools so you wouldn’t particularly expect people to make them up….theres no block buster finding on the other end, it’s just one persons “hey I saw some weird others should be aware of in case they see it too”. Sacks books were often assigned in classes because they were so accessible (that’s where I first read him), so a huge number of people were fed erroneous information as part of their learning. Even if it just impacts their idea of how the fringes operate, it’s still incorrect information. That it was done knowingly is even sadder. So overall I agree with AVI, it’s likely not a direct effect, but it means the foundation is shakier than we suspected.

To elaborate further: after working in hospitals for years, I can’t overstate the importance of the anecdotal “you know I saw something like this once”. It’s what separates really experienced clinicians from new ones. Published case reports tend to help that process, and people pass them around to help with their own sense of what to look for. Knowing that an anecdote is a made up example vs a real thing someone saw is pretty critical in clinical practice. We’ve all seen made up examples in textbooks that help explain concepts to us, and those are a fine teaching tool in context. But as the Retraction Watch article ends: “A narrative that is fictional but published in the format of a genuine case report, without disclosure at the time of publication, is functionally indistinguishable from fabrication in the scientific record.”

I will only add to this that case reports are the single hardest thing to fact check. There’s no statistical method to check, no data anomaly to detect. By definition you are talking about something people haven’t seen very often, and the exact details are often covered by medical privacy. Not a great situation.