This week I have to say, we are getting to one of my favorite topics: straight up fraud. Prior to this we have covered a lot of things that can skew the thinking of otherwise good people despite their best efforts, which is the vast majority of issues we run in to, but today we’re going to cover those who intentionally deceived others. Even in the context of the replication crisis, straight up fraud cases make up a very small percentage of the concerns about research findings, but they are still worth focusing on as a potential trouble source.

Before we get started though, I want to mention a somewhat weird thing I’ve noticed over the past few years. I’ve noted that very often when it comes to research, people are often very quick to call human error and/or bias fraud, and then often too slow to call actual fraud, fraud. I have wondered why this is, and my suspicion is that it’s because well intentioned humans who make errors are often very ashamed and may not defend themselves as vigorously, whereas straight up fraudsters are extremely prepared to be challenged and are prepared to be aghast you would ever suspect them of anything. Thus the well intentioned error people seem a lot more “guilty looking” than the fight to death fraudsters.

So with that in mind, let’s talk about what fraud is and isn’t. Fraud is not making a mistake, even if it means you have to retract your study. Admitting you got it wrong and owning up to it is exactly what we want researchers to do. Fraud is also not publishing a faulty study you didn’t question rigorously enough because it matched your pre-existing beliefs, at this point most of us have accidentally share a link to a story that turned out to be false because it just “sounded true”. Fraud isn’t even necessarily only publishing certain outcomes in a study and failing to publish others. Many of these things can teeter towards fraud depending on the circumstances, but most people in their day to day lives will occasionally jump to conclusions or tell stories in ways that benefit them. It’s not a great human weakness, but it’s one we see often. So if those things aren’t fraud, what is? Well in the research world one of the main examples is data falsification. From making up numbers to pretending to have done experiments that never happened, this is an unfortunate reality of some research and it’s only through replication efforts that this can be uncovered.

The wildest example in the research world is actually fairly recent, the sordid tale of Francesca Gino. Gino was a Harvard Business School professor who, amazingly, specialized in “honesty and ethical behavior” research. Back in 2020, a graduate student raised concerns about one of her papers, and then tried to replicate it in 2021. She became suspicious that not only did the study fail to replicate, but the whole set of results seemed wildly implausible. She got some data bloggers involved, and things spiraled from there. To condense a very long story, Gino was eventually put on leave and ultimately her tenure was revoked and she was fired.

What’s interesting, given my second paragraph, is that this all came to the attention of most people because Gino sued both Harvard and the Data Colada bloggers for $25 million saying they were all defaming her. It was actually her own lawsuit that caused Harvard’s internal investigation of her to be released, which made her look incredibly bad. She has alternated between claiming she was the victim of sexism, that it’s all a big mistake and that she was framed. Her coauthors on the other hand, started a website to investigate all of the papers they’d worked with her on to make sure they knew which findings were reliable and which weren’t. While I will note that Gino has defenders still, it’s an interesting story of defensiveness in the face of accusations.

So how does this relate to true crime?

Well, I’d imagine much of the connection would be obvious, but I’d like to point out that in true crime we actually know pretty much from the get go that someone is actually straight up lying. In scientific research, fraud is always a possibility, but probably not more so than in regular human endeavors. It reminds me of the old stats 101 type problem, where you calculate things like “given that the child is a boy, what are the chances his name is John” vs “given that the child’s name is John, what are the chances that he is a boy” and they highlight those are wildly different answers. Here it’s the difference between “given that a scientist published a paper, what are the chances there is fabricated data?” and “given that a bunch of suspects have given different incompatible stories so someone is lying, what are the chances person X is lying”. Why do I point this out? Because as I mentioned at the beginning of this series, for some reason the average person I talk to is more open to hearing that a research study they heard about is wrong than they are to hearing the new true crime podcast they’re listening to is. This makes no sense because crime stories are almost by definition full of liars. One of the first types of lies little kids tell is lies to get themselves out of trouble. If you have even a passing familiarity with the Biblical story of the Garden of Eden, you’ll know that it’s alleged that the first crime humanity itself committed was to attempt to shift the blame for eating the apple. Lying about this stuff is as innate to human nature as it gets. So again, why are we so resistant to being skeptical about these stories when someone puts them on a podcast?

I think there’s a few things skewing our thinking with these. The first I think is that crimes tend to involve a lot of human error from the get go. Witnesses often don’t have the best memories of times/dates/sequences of events, so any attempt to call someone a liar has to be tempered with the frailty of human memory. Additionally, in many crimes, victims are purposely selected because they have pre-existing credibility issues, making things even harder to sort through. In the documentary about the fraudulent results from the Massachusetts state crime lab, a defense attorney notes that the number one risk factor for being falsely accused of a crime is already having a criminal record. Two fraudsters in two different labs got away with filing false drug test results for years in large part because the results mostly impacted known drug dealers.

Interestingly, this applies to any group who comes under fire. I don’t think it’s coincidental that the true crime genre exploded in popularity around the same time as George Floyd/Black Lives Matter gained steam, as “police framed/justice system railroaded innocent person” is perhaps the most popular true crime storyline. Just like having a criminal history does not make you automatically guilty of a crime, police having issues also does not negate the fact that nearly every defendant claims to be framed. There’s actually been some interesting discussion of this in defense attorney circles, with some attorneys arguing that all media that draws attention to the flaws of our justice system is useful, and some maintaining that this type of infotainment does more harm than good. Scott Greenfield, my personal favorite defense attorney/blogger, falls in the latter camp. For this post I went looking for his thoughts on True Crime and was interested that in the years since Serial debuted, he’s gotten even harsher than his initial skepticism. I’d recommend the whole thing, but I love his first three paragraphs that he wrote back in 2023 (bolding mine):

After the podcast Serial became a hit, the phone started ringing. The calls were from journalists, producers, wannabe podcasters, asking whether I had any cases involving a clearly innocent defendant who was abused by the system and ended up convicted and serving a lengthy sentence. Well, of course I did. We all do. But as it turned out, that really wasn’t the story they were interested in.

What they really wanted was a sympathetic defendant, the sort of innocent person people could love, and a simple, clear story of misconduct and abuse that ended with imprisonment. This was where I made the mistake. I had no stories like that, as few defendants were up for beatification before being charged with murder, and while there were arguments for the defense, and complex, messy problems along the way, it wasn’t as if the prosecution didn’t have a case to show they committed the murder.

The sort of post hoc contentions, like witnesses who recanted after they had nothing on the line or jailhouse snitches who say their cellies confessed to them, that true crime producers adored and thought critically valuable were the sort of things judges laughed off, as did I. People lie, all the time, for all sorts of reasons. Why is a post-trial recantation more credible than sworn trial testimony? Defendants bought witness silence or post-trial recantations on occasion. They often claimed innocence all along, even though they were guilty as sin. That’s the nature of criminal defense.

This is a man who makes his living in criminal defense pointing out the rather obvious fact that very few people get to trial without some pretty good evidence that they did it, and that people are constantly lying. If someone claims they lied under oath but now are telling the truth on a podcast, you may want to mark that person down in credibility. So what do we do here? Well, I think we have to approach these things with a huge eye towards fraud, both for the defendant and the podcaster. A few thoughts:

1. Compare the story being told to different sources/established facts: I’ve said it before but I’ll say it again, before you start any documentary or podcast, look for a summary of the facts so you can tell if something’s being left out. Remember that every single person involved, from the defendant to the witnesses to the podcaster, is highly motivated to make themselves look as good as possible. It’s also good to note that wanting your story to be public in and of itself is not a sign of honesty, see my prior comments about Francesca Gino being the one to get her own damning internal investigation released. Some people truly believe facts make them look better than they do.
2. Beware of emotional investment, your own or others: Over a 10 podcast series, you can feel you get to know the host/the subject/whoever, which can lead you to overattribute credibility to them and become less skeptical as time goes by. By the time you finish it can feel mentally awkward to consider someone you’ve come to like a liar. This goes double for podcaster by the way, especially if they got exclusive access to some of the players in the case. I have a rule of thumb that when someone covers a controversial case and interviews someone extensively, then starts hemming and hawing about their opinion while saying “I guess we’ll never know”, they think the person’s guilty. With my local case, we had at least one documentary film maker admit that’s exactly what they did. The burden of highlighting someone’s case just to condemn them is too much for some people.
3. Beware of applying big picture thinking to individual cases: We live in a world where people get raped. This does not mean every individual rape accusation is true. We live in a world where people falsely accuse others of rape. This does not mean every single claim something is a false accusation is true. Unfortunately, there’s an odd thing that happens with true crime I’ll call “true crime as though experiment” where people use a true crime case as a stand in for a bigger issue. This can work in the research world, where research that suggests something similar to prior findings actually can be considered more credible than novel research. But in a crime case? The facts of every single crime still matter on their own. Once a case gets big enough though, a surprising number of people will claim the exact details don’t matter because we need a “bigger conversation”, but good lord imagine if it was you stuck in the middle of things? If your loved one was murdered and someone else decided to fudge some details and portray the murderer sympathetically because they wanted to make a “bigger point”? You’d hate it, we all would. Always consider there are real humans at the center of things, and ask if they signed up to be your morality tale.
4. Remember, people do in fact just make things up and people have been hurt by it: One weird thing I’ve noticed with some true crime assessments is that people will try to play “fair” and give everyone equal credit, like they all are lying a little bit. I think this comes from our natural instinct when we’re adjudicating arguments in our personal life. If you have two friends in a fight and you hear both sides, our instinct tends to be to split the difference and assume both have some points and both are being a little self serving. With many crime stories though, some stories are just incompatible. This happened in my local case, and I was surprised how many people wanted to try to split the difference between two extremely incompatible claims. I ended up having more respect for those who went all in on one side or the other than those who tried to “both sides” two stories that clearly could not coexist.
5. Factual innocence is different from not guilty: As I have throughout this series, I will reiterate that I support the reasonable doubt standard and our justice system. However, I continue to ding some true crime folk for acting like “beyond a reasonable doubt” means the defendant should be given more deference vs every other person involved. As we saw at the height of the #metoo era, a claim of wrongdoing that never enters a courtroom can destroy lives very easily. Not as dramatically as actually being wrongfully convicted in a court of law, but well beyond a level that’s reasonable to accept. It surprises me therefore that so many podcasters take this responsibility so lightly. If you know that one person committed a murder and you spend hours talking about 6 suspects, you should be aware 5 of those people are innocent and you may have just helped ruin their lives. Even Sarah Koenig admitted she’s ashamed of this part of the Serial podcast, that it encouraged people to treat others as pieces in a puzzle to be solved rather than humans who had been through pain. I get the reason people focus on the person on trial, meticulously cataloguing every issue with the case against them, but it’s notable they tend to spend just a few minutes on the weaknesses of the case against alternative suspects, if they mention them at all. This mimics the tactic of defense lawyers who are explicitly there to do this, but I’m surprised it doesn’t weigh heavier on the conscience of those just doing it as a hobby. If the defense lawyer was wrong, he did his job. If you’re wrong, you actually just wrecked somebody’s life for entertainment.
6. Watch how people address the victims: This is a somewhat weird one, but hear me out: the more dismissive a true crime podcast or a suspect is of the loved ones of the deceased, specifically those who could not themselves be suspects, the more I’d question the story. Victims by definition shift the attention away from neatly crafted stories, and thus seem to prompt outsized anger or complete dismissal from those seeking to push a narrative. A good recent example of this is Candace Owens attacking Charlie Kirk’s widow Erika. Owens has stated there was a conspiracy to murder Kirk, and it seems the further she went with the story, the more the grieving widow not asking similar questions annoyed her. Even if you believed Charlie Kirk was killed as part of a bigger conspiracy (I don’t) raging at a young widow would be a weird place to start in making your case. Watch how people treat the undisputed victims, and you’ll get a good insight in to where their focus is.

Ok, that’s all I have for today! Tune in next week when I go over some statistical issues.

To go straight to part 6, click here.