Twice recently I’ve seen minor characters on both sides of the political spectrum claim that they foresaw/predicted some recent event with “eerie precision”, on topics where their predictions had actually appeared (to me at least) only loosely connected to what actually happened.

While I was annoyed by these people, I was more annoyed by the fans of theirs who rushed to agree that it was clear that they had amazing foresight in making their calls. While obviously some of that is just in-group defensiveness, some of them really seem to believe that this person had done something amazing. While none of those fans are people who read my blog, I figured I’d blow off some steam by reminding everyone of two things:

1. Redefining words makes improbably results quite common. In the John Ionnaidis paper “Why Most Published Research Findings Are False”, one of his 6 corollaries for published research is “Corollary 4: The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true.” This is true in research, and more true in political pontificating/predictions. Allowing yourself any latitude at all in redefining words will give you nearly unlimited predictive power after the fact. This is not a “minor quibble”, that’s literally the whole trick.
2. Making lots of predictions has no consequences for a pundits career. Making predictions as a pundit is like playing roulette with the house’s money. You can’t actually lose. In “The Signal and the Noise” (a book which I once owned and have now lost to lending), Nate Silver reviews how often some pundits known for their “amazing predictions” actually make predictions. Answer: a lot. Most of them are wrong. However, the career of the pundit takes almost no hit for those wrong answers, yet skyrockets every time they are right. Thus they have no motivation to stop making crazy predictions, as they lose nothing for a wrong one and gain everything for a right one. When I read TSATN initially, I made this matrix to illustrate:

So yes, next time you see someone make an “amazing prediction”, take a deep breath and ask yourself how much redefining is going on and how many predictions they had to throw out to get to that one that hit. Well, probably not you specifically dear reader, you’re probably fine. This is almost certainly one of those “if you’re thinking about it enough to read this post, you’re probably not the problem” things. Regardless, thanks for letting me get that off my chest.