Author: bs king

Are Conservatives Simple Minded?

/ bs king / 1 Comment

Not too long ago, there was a bit of buzz going on about a study that suggested that liberals and conservatives can both be simple minded. In the past most of the reporting has suggested that when it comes to politics conservatives as a group are less complex thinkers than liberals, so naturally it created a stir. The buzz and the study intrigued me, so I decided to do a bit of a deep dive and sketchnote out what the researchers did.

I got the original study “Are Conservatives Really More Simple-Minded than Liberals? The Domain Specificity of Complex Thinking” by Conway, et al. , and started to read.  One really important note up front: neither I nor the authors suggest that complex thinking is always a sign of correct thinking or even desirable thinking. If you were out with a new acquaintance who told you their views on, say, cannibalism, were complex, you would probably be squicked out. However, since previous research had suggested that conservatives were almost always less complex than liberals, the authors wanted to check that specifically.  Their basic hypothesis was simple: when it comes to complex thinking, topic matters. They conducted 4 studies to test this hypothesis, so the whole thing got a little crowded….but here’s the overview:FullSizeRender

A couple thoughts/notes:

  1. One of those most interesting findings was that complexity dropped as intensity of feeling increased. This causation could go either way….people could feel strongly about things they believe are straightforward, or we could simplify when our feelings are strong. Or it could be both.
  2. It’s interesting to me that they rated both regular college kids and then rated the debates.  That seemed like a nice balance.
  3. When they used regular college kids, they only used people who scored at the higher end for conservatism or liberalism. They did not include people who were in the middle.

Overall, I don’t think this result is particularly surprising. It makes sense that people are not entirely complex or entirely simple. Interesting study, and I look forward to more!

What I’m Reading: February 2016

/ bs king / 3 Comments

An awesome article from two of my favorite statisticians on scientific overreach, power poses, and why we really need to stop quoting studies that don’t subsequently replicate. The day this went up I Tweeted it out, and within an hour someone I follow posted an article they had written that day quoting the original study. Didn’t correct it when I sent them the link either. Harumph.

I have ongoing debates with quite a few friends over appropriate emoji use. This paper should help us out.

A Crusade Against Multiple Regression Analysis. I’m in. Any crusade that has an upfront statistical warning label as end goal is one I can get behind.

What social science reporting gets wrong, according to social scientists. I saw this one then 3 people sent it to me, which makes me feel like I’m headed in the right direction in life.

How you are going to die and at what your chances are of being dead at given ages. Given that I’ve already lived as long as I have, my chances don’t hit 50/50 until I’m 81.

I’ve been playing Guess the Correlation. Current high score is 88.

Totally not in my normal wheelhouse for research I look at, but the Assistant Village Idiot’s take on a study about the origins of fairy tales was fascinating.

This take on the recent debates around nutrition science is a long-ish but fascinating read. Good to see all sides represented at once.

People: Our Own Worst Enemy (Part 8)

/ bs king / 2 Comments

Note: This is part 8 in a series for high school students about reading and interpreting science on the internet. Read the intro and get the index here or go back to Part 7 here.

I love this part of the talk because I get to present my absolute favorite study of all time. Up until now I’ve mostly been covering things about how other people are trying to fool you to get them to your side, but now I’m going to wade in to how we seek to fool ourselves.  That’s why I’m calling this part:

Biased Interpretations and Motivated Reasoning

Okay, so what’s the problem here?

The problem here my friend, is you. And me. Well, all of us really…..especially if you’re smart.  The unfortunate truth is that for all the brain power we put towards things, our application of that brain power can vary tremendously when we’re evaluating information that we like, that we’re neutral towards, or that we don’t like.  How tremendously? Well, in 2013 the working paper “Motivated Numeracy and Enlightened Self-Government“, some researchers decided to ask if people with a rash got better if they used a new skin cream.  They provided this data:

Pt8matrix

The trick here is that you are comparing absolute value to proportion.  More people got better in the “use the skin cream” group, but more people also got worse. The proportion is better for those who did not use the cream (about 5:1) as opposed to those who did use it (about 3:1). This is a classic math skills problem, because you have to really think through what question you’re trying to answer before you calculate, and what you are actually comparing. Baseline about 40% of people in the study got this right.

What the researchers did next was really cool. For some participants, they took the original problem, kept the numbers the same, but changed “patients” to “cities”, “skin cream” to “strict gun control laws” and “rash” to “crime”.  They also flipped the outcome around for both problems, so participants had one of four possible questions.  In one the skin cream worked, in one it didn’t, in one strict gun control worked, in one it didn’t. The numbers in the matrix remained the same, but the words around them flipped.  They also asked people their political party and a bunch of other math questions to get a sense of their overall mathematical ability. Here’s how people did when they were assessing rashes and skin cream:

rashgraph

Pretty much what we’d expect. Regardless of political party, and regardless of the outcome of the question, people with better math skills did better1.

Now check out what happens when people were shown the same numbers but believed they were working out a problem about the effectiveness of gun control legislation:

gunproblem

Look at the end of that graph there, where we see the people with a high mathematical ability. If using their brain power got them an answer that they liked politically, the did it. However, when the answer didn’t fit what they liked politically, they were no better than those with very little skill at getting the right answer.  Your intellectual capacity does NOT make you less likely to make an error….it simply makes you more likely to be a hypocrite about your errors.  Yikes.

Okay, so what kind of things should we be looking out for?

Well, this sort of thing is most common on debates where strong ethical or moral stances intersect with science or statistics. You’ll frequently see people discussing various issues, then letting out a sigh and saying “I don’t know why other people won’t just do their research!”. The problem is that if you believe something strongly already, you’re quite likely to think any research that agrees with you is more compelling than it actually is. On the other hand, research that disagrees with you will look less compelling than it may be.

This isn’t just a problem for the hoi polloi either. I just wrote earlier this week about two research groups who were accusing the other of choosing statistical methods that would support their own pet conclusions. We all do it, we just see it more clearly when it’s those we disagree with.

Why do we fall for this stuff?

Oh so many reasons.  In fact Carol Tarvis has written an excellent book about this (Mistakes Were Made (but Not by Me): Why We Justify Foolish Beliefs, Bad Decisions, and Hurtful Acts) that should be required reading for everyone. In most cases though it’s pretty simple: we like to believe that all of our beliefs are perfectly well reasoned and that all the facts back us up. When something challenges that assumption, we get defensive and stop thinking clearly.  There’s also some evidence that the internet may be making this worse by giving us access to other people who will support our beliefs and stop us from reconsidering our stances when challenged.

In fact, researchers have found that the stronger your stance towards something, the more likely you are to hold simplistic beliefs about it (ie “there are only two types of people, those who agree with me and those who don’t”).

An amusing irony: the paper I cited in that last paragraph was widely reported on because it showed evidence that liberals are as bad about this as conservatives. That may not surprise most of you, but in the overwhelmingly liberal field of social psychology this finding was pretty unusual. Apparently when your field is >95% liberal, you mostly find that bias, dogmatism and simplistic thinking are conservative problems. Probably just a coincidence.

So what can we do about it?

Richard Feynman said it best:

If you want to see this in action, watch your enemy. If you want to really make a difference, watch yourself.

Well that got heavy.  See you next week for Part 9!

Read Part 9 here.

1. You’ll note this is not entirely true at the lowest end. My guess is if you drop below a certain level of mathematical ability, guessing is your best bet.↩

 

 

Grading an Education Infographic

/ bs king / 4 Comments

Welcome to Grade the Infographic, which is pretty much exactly what it sounds like. I have three criteria I’m looking for in my grading: source of data, accuracy of data and accuracy of visuals. While some design choices annoy me, I’m not a designer, couldn’t do any better, and won’t be commenting unless I think it’s skewing the perception of the information. I’m really only focused with what’s on the graphic, so I also don’t assess stats that maybe should have been included but weren’t.  If you’d like to submit an infographic for grading, go here. If you’d like to protest a grade for yourself or someone else, feel free to do so in the comments or on the feedback page.

When I first started doing any stats/data blogging, an unexpected thing happened: people started sending me their infographics.  Despite my repeated assertions that I actually hated infographics, companies seemed to troll the web attempting to find people to post their infographics on various topics.  It gets a little weird because they’re frequently not related to my topics, but apparently I’m not the only blogger who has had this problem.  Long story short, I actually got sent this infographic back in 2013. 

Click to enlarge.

Since one of my favorite readers is a teacher (hi Erin!) who also shares my displeasure with infographics, I thought I’d start off by grading this one. It’s pretty long so I chopped it up in to pieces.  Because I’m a petty despot and all, I actually start with the end. Grading the reference section first is a bit backwards, but it gives me an idea of how much work I’m going to have to do to figure out the accuracy of the rest of it.

Pt1

Oooh, not off to a great start.  The maximum grade you can get from me if you don’t give me a source I can track is in the B range. Giving a website is good, but when it’s as big as the National Center for Education Statistics, it’s also nearly useless.

Pt2

Okay, this is good! That’s a decent selection of countries. Not sure if there was a particular reason, but there doesn’t appear to be any cherry picking going on.

Pt3

Hmmm….this got a little funky. I couldn’t actually locate this source data, though I did locate some from the World Bank that backed up the elementary school numbers. I’m guessing this is real data, but saddened they didn’t let me know where they got it! If you do the work, get the credit! Also, the 4 year gap confused me. Where are 2001 – 2004? It doesn’t look like it particularly matters for this trend, so I only subtracted 2 points for not indicating the gap or better spacing the years.

Pt4

This part broke even.  I was hoping for a year (source again!) but did get some good context about what kind of test this was. That was really helpful, so it got an extra point. The data’s all accurate and it’s from 2011.

Pt5

Oooh, now here’s a bit of a snafu. The graphic said “hours spent studying” which surprised me because that’s 3 hours/day for the US kids. When I found the source data (page 114) it turns out those are actually classroom hours. That made more sense. I docked three points because I don’t think that’s what most people mean by “time spent studying”. It’s not totally wrong, but not totally accurate either. Class hours are normally referred to as such. I felt there was a bit of wiggle room on the definition of “study” though, so I didn’t know it down the 5 points I was going to.

Pt6

Oof. That’s not good. Where did these numbers come from? I went to the OECD report to check out the 2010 numbers, and they were WAY off.

Country Infographic 2010 number OECD 2010 number
United States 88.4% 77%
United Kingdom 82.9% 91%
Spain 64.7% 80%
Germany 86.5% 87%
Sweden 91.1% 75%
South Korea 91.1% 92%
Australia 84.8% No numbers

Now graduation rates have lots of different ways of being calculated (mostly due to differences in what counts as “secondary education”, so it’s plausible those numbers came from somewhere. This is the risk you run when you don’t include sources.

Finalgrade

And there you have it.  Cite your sources!

 

Women, Ovulation and Voting in 2016

/ bs king / 5 Comments

Welcome to “From the Archives” where I revisit old posts  to see where the science (or my thinking) has gone since I put them up originally.

Back in good old October of 2012, it was an election year and I was getting irritated1. First, I was being bombarded with Elizabeth Warren vs Scott Brown for Senate ads, and then I was confronted with this study:The fluctuating female vote: politics, religion, and the ovulatory cycle (Durante, et al), which purported to show that women’s political and religious beliefs varied wildly around their monthly cycle, but in different ways if they were married or single. For single women they claimed that being fertile caused them to get more liberal and less religious, because they had more liberal attitudes toward sex. For married women, being fertile made them more conservative and religious so they could compensate for their urge to cheat.  The swing was wide too: about 20%.  Of note, the study never actually observed any women changing their vote, but compared two groups of women to find the differences. The study got a lot of attention because CNN initially put it up, then took it back down when people complained.  I wrote two posts about this, one irritated and ranty, and one pointing to some more technical issues I had.

With a new election coming around, I was thinking about this paper and wanted to take a look at where it had gone since then. I knew that Andrew Gelman had ultimately taken shots at the study for reporting an implausibly large effect2 and potentially collecting lots of data/comparisons and only publishing some of them, so I was curious how this study had subsequently fared.

Well, there are updates!  First, in 2014, a different group tried to replicate their results in a paper called  Women Can Keep the Vote: No Evidence That Hormonal Changes During the Menstrual Cycle Impact Political and Religious Beliefs by Harris and Mickes.  This paper recruited a different group, but essentially recreated much of the analysis of the original paper with one major addition. They conducted their survey prior to the 2012 election AND after, to see predicted voting behavior vs actual voting behavior.  A few findings:

  1. The first paper (Durante et al) had found that fiscal policy beliefs didn’t change for women, but social policy beliefs did change around ovulation. The second paper (Harris and Mickes) failed to replicate this finding, and also failed to detect any change in religious beliefs.
  2. In the second paper, married women had a different stated preference for Obama (high when low feritility, lower when high fertility), but that difference went away when you looked at how they actually voted. For single women, it was actually the opposite. They reported the same preference level for Obama regardless of fertility, but voted differently based on the time of the month.
  3. The original Durante study had taken some heat for how they assessed fertility level in their work. There were concerns that self reported fertility level was so likely to be inaccurate that it would render any conclusions void. I was interested to see that Harris and Mickes clarified that the Durante paper actually didn’t accurately describe how they did fertility assessments in the original paper, and that they had both ultimately used the same method. This was supposed to be in the supplementary material, but I couldn’t find a copy of that free online. It’s an interesting footnote.
  4. A reviewer asked them to combine the pre and post election data to see if they could find a fertility/relationship interaction effect. When pre and post election data were kept separate, there was no effect. When they were combined, there was.

Point #4 is where things got a little interesting. The authors of the Harris and Mickes study said combining their data was not valid, but Durante et al hit back and said “why not?”. There’s an interesting piece of stat/research geekery about the dispute here, but the TL;DR version is that this could be considered a partial replication or a failure to replicate, depending on your statistical strategy. Unfortunately this is one of those areas where you can get some legitimate concern that a person’s judgement calls are being shaded by their view of the outcome. Since we don’t know what either researchers original plan was, we don’t know if either one modified their strategy based on results. Additionally the “is it valid to combine these data sets” question is a good one, and would be open for discussion even if we were discussing something totally innocuous. The political nature of the discussion intensifies the debate, but it didn’t create it.

Fast forward now to 2015, when yet another study was published: Menstrual Cycle Phase Does Not Predict Political Conservatism. This study was done using data ALSO from the 2012 election cycle3, but with a few further changes.  The highlights:

  1. This study, by Scott and Pound, addressed some of the “how do you measure fertility when you can’t test” concerns by asking about medical conditions that might influence fertility to screen out women whose self reporting might be less accurate. They also ranked fertility on a continuum as opposed to the dichotomous “high” and “low”. This should have made their assessment more accurate.
  2. The other two studies both asked for voting in terms of Romney vs Obama. Scott and Pound were concerned that this might capture a personal preference change that was more about Obama and Romney as people rather than a political change. They measured both self-reported political leanings and a “moral foundations” test and came up with an overall “conservatism” rank, then tracked that with chances of conception.
  3. They controlled for age, number of children, and other sociological factors.

So overall, what did this show? Well, basically, political philosophy doesn’t vary much no matter where a woman is in her cycle.

The authors have a pretty interesting discussion at the end about the problems with Mechanical Turk (where all three studies recruited their participants in the same few months), the differences of measuring person preference (Obama vs Romney) vs political preference (Republican vs Democrat), and some statistical analysis problems.

So what do I think now?

First off, I’ve realized that getting all ranty when someone brings up women’s hormones effecting things may be counterproductive. Lesson learned.

More seriously though, I find the hypothesis that our preferences for individuals may change with hormonal changes more compelling than the hypothesis that our overall philosophy of religion or government changes with our hormones. The first simply seems more plausible to me. In a tight presidential election though, this may be hopelessly confounded by the candidates actual behavior. It’s pretty well known that single women voted overwhelmingly for Obama, and that Romney had a better chance to capture the votes of married women. Candidates know this and can play to it, so if a candidate makes a statement playing to their base, you may see shifts that have nothing to do with hormones of the voters but are an actual reaction to real time statements. This may be a case where research in to the hypothetical (i.e. made up candidate A vs B) may be helpful.

The discussions on fertility measures and statistical analysis were interesting and a good insight in to how much study conclusions can change based on how we define particular metrics.  I was happy to see that both follow up papers hammered on clear and standard definitions for “fertility”. If that is one of  the primary metrics you are assessing, then the utmost care must be taken to assess it accurately, or else the signal to noise ratio can go through the roof.

Do I still think CNN should have taken the story down? Yes….but just as much as I believe that they should take most sensational new social/psych research stories down. If you follow the research for just two more papers, you see the conclusion go from broad (women change their social, political and religious views and votes based on fertility!) to much narrower (women may in some cases change their preference or voting patterns for particular candidates based on fertility, but their religious and political beliefs do not appear to change regardless). I’ll be interested to see if anyone tries to replicate this with the 2016 election, and if so what the conclusions are.

This concludes your trip down memory lane!
1. Gee, this is sounding familiar↩
2. This point was really interesting. He pointed out that around elections, pollsters are pretty obsessive about tracking things, and short of a major scandal breaking literally NOTHING causes a rapid 20 point swing. The idea that swings that large were happening regularly and everyone had missed it seemed implausible to him. Statistically of course, the authors were only testing that there was a difference at all, not what it was….but the large effect should possibly have given them pause. It would be like finding that ovulation made women spend twice as much on buying a house. People don’t change THAT dramatically, and if you find that they do you may want to rerun the numbers.↩
3. Okay, so I can’t be the only one noticing at this point that this means 3 different studies all recruited around 1000 American women not on birth control, not pregnant, not recently pregnant or breastfeeding but of child bearing age, interested in participating in a study on politics, all at the same time and all through Amazon’s Mechanical Turk. Has anyone asked the authors to compare how much of their sample was actually the same women? Does Mechanical Turk have any barriers for this? Do we care? Oh! Yes, turns out this is actually a bit of a problem.↩

Proof: Using Facts to Deceive (Part 7)

/ bs king / 4 Comments

Note: This is part 7 in a series for high school students about reading and interpreting science on the internet. Read the intro and get the index here, or go back to Part 6 here.

Okay, now we come to the part of the talk that is unbelievably hard to get through quickly. This is really a whole class, and I will probably end up putting some appendices on this series just to make myself feel better.  If the only thing I ever do in life is to teach as many people as possible the base rate fallacy, I’ll be content. Anyway, this part is tough because I at least attempt to go through a few statistical tricks that actually require some explaining. This could be my whole talk, but I’ve decided against it in favor some of the softer stuff. Anyway, this part is called:

Crazy Stats Tricks: False Positives, Failure to Replicate, Correlations, Etc

Okay, so what’s the problem here?

Shenanigans, chicanery, and folks otherwise not understanding statistics and numbers. I’ve made reference to some of these so far, but here’s a (not-comprehensive) list:

  1. Changing the metric (ie using growth rates vs absolute rates, saying “doubled” and hiding the initial value, etc)
  2. Correlation and causation confusion
  3. Failure to Replicate
  4. False Positives/False Negatives

They each have their own issues. Number 1 deceives by confusing people, Number 2 makes people jump to conclusions, Number 3 presents splashy new conclusions that no one can make happen again, and Number 4 involves too much math for most people but yields some surprising results.

Okay, so what kind of things should we be looking out for?

Well each one is a little different. I touched on 1 and 2 a bit previously with graphs and anecdotes. For failure to replicate, it’s important to remember that you really need multiple papers to confirm findings, and having one study say something doesn’t necessarily mean subsequent studies will say the same thing. The quick overview though is that many published studies don’t bear out. It’s important to realize that any new shiny study (especially psychology or social science) could turn out to not be reproducible, and the initial conclusions invalid. This warning is given as a boilerplate “more research is needed” at the end of articles, but it’s meant literally.

False positives/negatives are a different beast that I wish more people understood.  While this applies to a lot of medical research, it’s perhaps clearest to explain in law enforcement.  An example:

In 2012, a (formerly CIA) couple was in their home getting their kids ready for school when they were raided by a SWAT team. They were accused of being large scale marijuana growers, and their home was searched. Nothing was found.  So why did they come under investigation? Well it turns out they had been seen buying gardening equipment frequently used by marijuana growers, and the police had then tested their trash for drug residue. They got two positive tests, and they raided the house.

Now if I had heard this reported in a news story, I would have thought that was all very reasonable. However, the couple eventually discovered that the drug test used on their trash has a 70% false positive rate. Even if their trash had been perfectly fine, there was still at least a 50% they’d get two positive tests in a row (and that assumes nothing in their trash was triggering this). So given a street with ZERO drug users, you could have found evidence to raid half the houses.  The worst part of this is that the courts ruled that the police themselves were not liable for not knowing that the test was that inaccurate, so their assumptions and treatment of the couple were okay. Whether that’s okay is a matter for legal experts, but we should all feel a little uneasy that we’re more focused on how often our tests get things right than how often they’re wrong.

Why do we fall for this stuff?

Well, some of this is just a misunderstanding or lack of familiarity with how things work, but the false positive/false negative issue is a very specific type of confirmation bias. Essentially we often don’t realize that there is more than one way to be wrong, and in avoiding one inaccuracy, we increase our chances of different types of inaccuracy.  In the case of the police departments using the inaccurate tests, they likely wanted something that would detect drugs when they were present. They focused on making sure they’d never get a false negative (ie a test that said no drugs when there were). This is great, until you realize that they traded that for lots of innocent people potentially being searched. In fact, since there are more people who don’t use drugs than those who do, the chances that someone with a positive test doesn’t have drugs is actually higher than the chance that they do….that’s the base rate fallacy I was talking about earlier.

To further prove this point, there’s an interesting experiment called the Wason Selection task that shows that when it comes to numbers in particular, we’re especially vulnerable to only confirming an error in one direction. In fact 90% of people fail this task because they only look at one way of being wrong.

Are you confused by this? That’s pretty normal. So normal in fact that the thing we use to keep it all straight is literally called a confusion matrix and it looks like this:

If you want to do any learning about stats, learn about this guy, because it comes up all the time. Very few people can do this math well, and that includes the majority of doctors. Yup, the same people most likely to tell you “your test came back positive” frequently can’t accurately calculate how worried you should really be.

So what can we do about it?

Well, learn a little math! Like I said, I’m thinking I need a follow up post just on this topic so I have a reference for this. However, if you’re really not mathy, just remember this: there’s more than one way to be wrong. Any time you reduce your chances of being wrong in one direction, you probably increase them in another. In criminal justice, if we make sure we never miss a guilty person, we might also increase the number of innocent people we falsely accuse. The reverse is also true. Testings, screenings, and judgment calls aren’t perfect, and we shouldn’t fool ourselves in to thinking they are.

Alright, on that happy note, I’ll bid you adieu for now. See ya next week!

Read Part 8 here.

Coming this February….

/ bs king

As I’ve mentioned in a few comments/conversations around here, one of the main goals of my current blogging kick has been to come up with some more defined project for myself and/or ongoing series. I’ve had quite a bit of fun with the “Intro to Internet Science” posts, and plan to do some other (hopefully) interesting things in the future.

Starting in February, I’m going to roll out a few of these ideas and see what works or at least keeps me entertained. As I start these series I’ll be adding them here, and you can find links to the individual series in the drop down at the top. In addition to a few A few things to look forward to:

Little Miss Probability Distribution: I’ve been obsessed with probability distributions and their relationships with each other, and I need to work that out somehow. Get ready to meet them and see how the get along.

From the Archives: As many reader know, I blogged quite a bit in 2012 and 2013 on all sorts of random issues. Most of that was pre-stats degree, so I’m taking a look in my archives to dig up some old posts and see what I’d say about them now.

Grade an Infographic: Infographics still drive me nuts. I am taking a red pen to them.

Math Words for People Who Like English: I had a really funny conversation with a language obsessed friend about some of the more fun words that exist in the world of math and statistics. I’m going to be highlighting a few of these for her and anyone else who is interested.

Book Suggestions for the Autodidact: I put up a few book lists recently, and I decided I’m going to keep a running list of my favorites for people who want more. You can find it in the bar at the top, or access the ongoing list here. I’ll be changing the number in the title as I add things.

Additionally, I’m going to keep up my R&C posts, where I deep dive/sketch out the different parts of a study either related to my life or the news, and probably keep up the personal life advice column, which keeps cracking me up. As I noted last week, reader questions are always welcome, and can be submitted here.

Proof: Using Facts to Deceive (Part 6)

/ bs king / 1 Comment

Note: This is part 6 in a series for high school students about reading and interpreting science on the internet. Read the intro and get the index here, or go back to part 5 here.

Okay, I’ll be honest here: this part is one of the hardest parts of my talk to cover. The issue I’m going to talk about here is another framing issue, and it has to do with what experts get quoted on what issues and in what proportions. This is a huge issue open to broad interpretations and many legitimate approaches, so I’m going to intentionally tread lightly here.  A large amount about what you feel is deceptive here will depend on what you already believe.  Additionally, when I cover this in the classroom, I have a pretty good idea that there won’t be any raging conspiracy theorists seating in the seats.  Not so on the internet. I’ve been blogging off and on for about a decade now, and you would not believe how many people do key word searches so they can pop in and spew their theories….so forgive me if I speak in generalities. Oh yes, it’s the controversial issue of:

Experts and Balance

Okay, so what’s the problem here?

The problem, in general, is a public misunderstanding about how science works. Not everyone is a scientist, and that’s okay. We often rely on experts to interpret information for us. This is also okay. In the age of the internet though, almost anyone can find an expert or two to back up their own view. Everyone wants to be the first to break a story, and much can get lost in the rush to be on top of the latest and greatest thing. Like, you know, evidence.

Okay, so what kinds of things should we be looking out for?

Well, there are two sides to this coin. The classic logical fallacy here is argumentum ad verecundiam, or “argument from authority”.  Kinda like this guy:

…though my three year old tells me he’s pretty cool.  In all seriousness though, “TV doctors” love to get up and use their credentials to emphasize their points. Their reach can be enormous, but research has found that over half the claims of people like Dr Oz are either totally unsubstantiated or flat out contradicted by the evidence. Just because someone has a certain set of credentials doesn’t mean they’re always right.

 

So if the popular credentialed people aren’t always right, then good old common sense can guide us right? No, sadly, that’s not true either. The flip side of arguing from authority is “appeal to the common man”, where you respect someone’s opinion because they’re not an authority. For medicine you frequently hear this as “the secret doctors don’t want you to know!” or “my doctor said my child was ______, but as a mother my heart knew that was wrong” (side note: remember that mother’s who turn out to be wrong almost never get interviewed). For some people, this type of argument is even stronger than the one above….but that doesn’t mean it’s not fallacious.

So basically, the water gets really murky because almost anyone can claim to know stuff, cant throw out credentials that may or may not be valid or relevant, can throw out research that may or may not be valid, and otherwise sound very compelling. Yeesh.

Complicating matters even further is the idea of balance and false balance. Balance is when a reporter/news cast presents two opposing sides and gives them both time to state their case. False balance is when you give two wildly unequal sides the same amount of time.

All of this can seem pretty reasonable when it comes to hotly debated topics, like say, nutrition and what we should be eating. If you want to pit the FDA vs Gary Taubes vs Carbsane, I will watch the heck out of that. But there are other issues where the debate gets a little harrier, and the stakes get much higher….like say criminal trials. Do you want a psychic on the stand getting time to explain why they think you’re a murderer? Do you want them getting as much time as the forensics experts who say you aren’t?

At some point we have to say it….science does back up certain opinions more than others, and some experts are more reliable than others. Where you draw the line, sadly, probably depends on what you already believe about the topic.

Why do we fall for this stuff?

Well, partially because we should.  On the whole, experts are probably a pretty good bet when it comes to most scientific matters. They may be wrong at times (just ask Barry Marshall), but I have a lot of faith in science on the whole to move forward and self correct. The scientific process is quite literally mans attempt to correct for all of our fallacies in order to move forward based on reality.  It’s a lofty goal, and we’re not always perfect, but it’s start. We listen to experts because as people with more training, more experience and more context than us actually do frequently do better at controlling their biases.

On the other hand, those of us who have been burned might start to love the anti-hero instead. The idea that a lone wolf can take on the establishment is so cool! Because truthfully sometimes the establishment sucks. People are misdiagnosed, treated rudely, and otherwise incorrectly cast aside. Sometimes “crazy” ideas do turn out to be right. Being a contrarian isn’t always the worst way to go….as my favorite George Bernard Shaw quote says “The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.”  Every progress maker is a bit unreasonable at times.

So what can we do about it?

So I may have made it sound like it’s not possible to know anything. That’s not true. Science is awesome, but you have to do a little work to figure out what’s going on. So if you really care about an issue, do a little homework. Read the best and brightest minds that defend the side you don’t agree with. Don’t just read what your side says the other side is saying, read the other side. Read the best of what they have to offer….but check their sources. If they prove to be not-credible on one topic, treat them with suspicion going forward. Scientists should not have to play fast and loose with the truth to get where they need to go. Be suspicious of anyone who does this. Beware of anything that sounds too neat, too clean, too cutting edge. Science and proof move slowly.

Also, follow the money….but remember that works both ways. I work in oncology and there are people who will tell you the treatments we offer are crap and that they have better ones. Evidence to the contrary is dismissed as us not wanting to lose our money. However, the people making these claims frequently make 10 to 20 times what our doctors make. They throw out numbers that represent our whole hospital, while neglecting to mention that their personal income far exceeds any individual employee we have. People make tons of money peddling alternatives.

And if all else fails, just ask a math teacher:

No one’s questioning that a2 + b2 = c2 stuff. Well, except this guy.

Alright, that’s it for part 6….see you next week for part 7!  Read Part 7 here.

Immigration, Poverty and Gumballs

/ bs king / 4 Comments

A long time reader (hi David!) forwarded this video and asked what I thought of it:

It’s pretty short, but if you don’t feel like watching it, essentially it’s a video put out by a group attempting to address whether or not immigration to the US can reduce global poverty.  He uses gumballs to represent the population of people in the world living in poverty (one gumball = one million people), and ultimately concludes that immigration will not solves global poverty.

Now, I’m not the most educated of people when it comes to immigration issues, but I was intrigued by his math based demonstration. At one point he even has gumballs fall all over the floor, which drives home exactly how screwed we are when it comes to fixing global poverty. But do I buy it? Are the underlying facts correct? Is this a good video? Well, lets take a look:

First, some context: Context is frequently missing on Facebook, and it can be useful to know the background of what you’re seeing when there’s a video like this.  I did some digging, so here goes:  The man in the video is Roy Beck, who founded a group called Numbers USA, website here. Their tag line is “for lower immigration levels”, and unsurprisingly, that’s what they want.  The video, and presumably the numbers in it, are from 2010.  I thought the name NumbersUSA sounded ambitious, but I did find they have an “Accuracy Guarantee” on their FAQ page promising they would take down any inaccurate numbers or information. I don’t know if they do it (and they have not responded to my complaint yet), but that was cool to see.

Now, the argument:  To start the video, Mr Beck lays out his argument by quantifying the number of desperately poor people in the world. He clarifies that “desperately poor” is defined by the World Bank standard of “making less than two dollars a day”. He begins to name the number of desperately poor people in various regions of the world, and stacks gumballs to represent all of these regions. The number is heartbreakingly high and it worsens as he continues….but when his conclusion came to about half the globe (3 billion people or 8 larger containers of gumballs) living at that level, I was skeptical. I’ve done some reading on extreme poverty, and I didn’t think it was that high. Well, it turns out it isn’t. It’s actually about 12.7% or 890 million. That’s only about 30% of the number he presents….maybe about 3 containers of gumballs instead of 8.

Given that that the video was older (and that extreme world poverty has been declining since the 1980s) I was trying to figure out what happened, so I went to this nifty visualization tool the World Bank provides. You can set the poverty level (less than $1.90/day or less than $3.10/day) and you can filter by country or region.  Not one of the numbers given is accurate. They haven’t even been accurate recently, as far as I can tell. For example, in 2010, China had 150 million people living on under $2/day.  In the video, he says 480 million, where China was in the year 2000 or so.  For India, he uses 890 million, a number I can’t find ever published by the World Bank.  The highest number they list for India at all is 430 million. The best I can conclude is that the numbers he shows here are actually those living under the $3.10/day level, which seem closer. Now $3.10/day is not rich by any means, but it’s not what he asserted either. He emphasizes the “less than 2 dollars a day” point multiple times.  At that point I figured I wasn’t going to check out the rest of the numbers….if the baseline isn’t accurate, anything he adds to it won’t be either. [Edit: It’s been pointed out to me that at the 2:04 mark he changes from using the $2/day standard to “poorer than Mexico”, so it’s possible the numbers after that timepoint do actually work better than I thought they would. It’s hard to tell without him giving a firm number. For reference, it looks like in 2016 the average income in Mexico is $12,800/year .]  It was at this point I decided to email the accuracy check on his website to ask for clarification, and will update if I hear back. I am truly interested in what happened here, because I did find a few websites that gave similar numbers to his….but they all cite the World Bank and all the links are now broken. The World Bank itself does not appear to currently stand by those statistics.

So did this matter? Well, yes and no. His basic argument is that we have 5.6 billion poor people. That grows every year by 80 million people each year. Subtract out 1 million immigrants to the US each year, and you’re not making a difference.  Even if those numbers are wildly different from what’s presented, the fundamental “1 million immigrants doesn’t make much of a dent in world poverty” probably stands.

But is that the question?

On the one hand, I’ll grant that it’s possible “some people say that mass immigration in to the United States can help reduce world poverty”, as he says to open his video. I do not engage much in immigration debates, but I wasn’t entirely sure that “reduce world poverty” was the primary argument. NumbersUSA puts out quite a few videos on many different topics, so it’s interesting that this one appears to be their most viral.  It currently has almost 3 million views, and most of their other videos don’t have even 1% of that. Given that “solve world poverty” is not one of the stated goals or arguments of the immigration organizations I could find, why was this so shared? I did find some evidence that people argue about immigrants sending money back to their home countries helping poverty, but that is not really addressed in this video. So why did so many people want to debunk an argument that is not the primary one being made?

My guess is the pretty demonstration. I covered in this post about graphs and technical pictures, that these sorts of additions seem to make us think an argument is more powerful than we would have otherwise. In this case, it seems a well demonstrated about magnitude and subtraction is trumping most people’s realizations that this is not arguing a point that is commonly made.

Now if the numbers aren’t accurate, that’s even more irritating (his demonstration would not have looked quite as good if it had 3 containers at the start instead of 8), but I’m not sure that’s really the point. These videos work in two ways, both by making an argument that will irritate people who disagree with you, and by convincing those who agree with you that you’ve answered the challenges you’ve gotten. It’s a classic example of a straw man…setting up an argument you can knock down easily. My suspicion is when you do it with math and a colorful demonstration, it convinces people even more. Not the fault so much of the video maker, as that of the consumer.  While it’s possible Mr Beck will reply to me and clarify his numbers with a better source, it looks unlikely. Caveat emptor.

Got a question/meme/thing you want explained or investigated? I’m on it! Submit them here.

New Feature: Reader Questions

/ bs king / 4 Comments

I’m starting a new feature here that I’ve been doing informally for a while now: reader questions. While I like to amuse myself with my stats based/personal life advice column, I get far more requests for feedback on random things readers come across and want someone to weigh in on.  So….if you have a question, see something irritating on Facebook, or just generally want someone to take a look at the numbers, get in touch here.