The Real Dunning-Kruger Graph

/ bs king / 5 Comments

I’m off camping this weekend, so you’re getting a short but important PSA.

If you’ve hung out on the internet for any length of time or in circles that talk about psych/cognitive biases a lot, you’ve likely heard of the Dunning-Kruger effect. Defined by Wiki as “a cognitive bias wherein persons of low ability suffer from illusory superiority, mistakenly assessing their cognitive ability as greater than it is.”, it’s often cited to explain why people who know very little about something get so confident in their ignorance.

Recently, I’ve seen a few references to it accompanied by a graph that looks like this (one example here):

While that chart is rather funny, you should keep in mind it doesn’t really reflect the graphs Dunning and Kruger actually obtained in their study. There were 4 graphs in that study (each one from a slightly different version of the study) and they looked like this:

Humor:

Logic and reasoning (first of two):

Grammar:

And one more logic and reasoning (performed under different conditions):

So based on the actual graphs, Dunning and Kruger did not find that the lowest quartile thought they did better than the highest quartile, they found that they just thought they were more average than they actually were. Additionally it appears the 3rd quartile (above average but not quite the top), is the group most likely to be clearsighted about their own performance.

Also, in terms of generalizability, it should be noted that the participants in this study were all Cornell undergrads being ranked against each other. Those bottom quartile kids for the grammar graph are almost certainly not bottom quartile in comparison to the general population, so their overconfidence likely has at least some basis.  It’s a little like if I asked readers of this blog to rank their math skills against other readers of this blog….even the bottom of the pack is probably above average. When you’re in a self selected group like that,  your ranking mistakes may be more due to a misjudging of those around you as opposed to just an overconfidence in yourself.

I don’t mean to suggest the phenomena isn’t real (follow up studies suggest it is), but it’s worth keeping in mind that the effect is more “subpar people thinking they’re middle of the pack” than “ignorant people thinking they’re experts”. For more interesting analysis, see here, and remember that graphs drawn in MS Paint rarely reflect actual published work.

 

5 Things You Should Know About Orchestras and Blind Auditions

/ bs king / 6 Comments

Unless you were going completely off the grid this week, you probably heard about the now-infamous “Google memo“.  Written by a (since fired) 28 year old software engineer at Google, the memo is a ten page long document where the author lays out his beliefs about why gender gaps in tech fields continue to exist. While the author did not succeed in getting any policies at Google changed, he did manage to kick off an avalanche of hot takes examining whether the gender/tech gap is due to nature (population level differences in interests/aptitude) or nurture (embedded social structures that make women unwelcome in certain spaces). I have no particular interest in adding another take to the pile, but I did see a few references to the “blind orchestra auditions study” that reminded me I had been wanting to write about that one for a while, to deep dive in to a few things it did or did not say.

For those of you who don’t know what I’m talking about, here’s the run down: back in the 1970s, top orchestras in the US were 5% female. By the year 2000, the were up to almost 30% female. Part of the reason for the change was the introduction of “blind auditions”, where the people who were holding tryouts couldn’t see the identity of the person trying out. This finding normally gets presented without a lot of context, but it’s good to note someone actually did decided to study this phenomena to see if the two things really were related or not. They got their hands on all of the tryout data for quite a few major orchestras (they declined to name which ones, as it was part of the agreement of getting the data) and tracked what happened to individual musicians as they tried out. This led to a data set that had overall population trends, but also could be used to track individuals. You can download the study here, but these are my highlights:

  1. Orchestras are a good place to measure changing gender proportions, because orchestra jobs don’t change. Okay, first up is an interesting “control your variables” moment. One of the things I didn’t realize about orchestras (though may be should have) is that the top ones have not changed in size or composition in years. So basically, if you suddenly are seeing more women, you know it’s because the proportion of women overall is increasing across many instruments. In the words of the authors ” An increase in the number of women from, say, 1 to 10, cannot arise because the number of harpists (a female-dominated instrument), has greatly expanded. It must be because the proportion female within many groups has increased.”
  2. Blind auditions weren’t necessarily implemented to cut down on sexism. Since this study is so often cited in the context of sexism and bias, I had not actually ever read why blind auditions were implemented in the first place. Interestingly, according to the paper written about it, the actual initial concern was nepotism. Basically, orchestras were filled with their conductors students, and other potentially better players were shut out. When they opened the auditions up further, they discovered that when people could see who was auditioning, they still showed preferential treatment based on resume. This is when they decided to blind the audition, to make sure that all preconceived notions were controlled for. The study authors chose to focus on the impact this had on women (in their words) “Because we are able to identify sex, but no other characteristics for a large sample, we focus on the impact of the screen on the employment of women.”
  3. Blinding can help women out Okay, so first up, the most often reported findings: blind auditions appear to account for about 25% of the increase in women in major orchestras. When they studied individual musicians, they found that women who tried out in blind and non-blind auditions were more successful in the blinded auditions. They also found that having a blind final round increased the chances a woman was picked by about 33%. This is what normally gets reported, and it is a correct reporting of the findings.
  4. Blinding doesn’t always help women out One of the more interesting findings of the study that I have not often seen reported: overall, women did worse in the blinded auditions. As I mentioned up front, the study authors had the data for groups and for individuals, and the findings from #3 were pulled from the individual data. When you look at the group data, we actually see the opposite effect. The study authors suggest one possible explanation for this: adopting a “blind” process dropped the quality of the female candidates. This makes a certain amount of sense. If you sense you are a borderline candidate, but also think there may be some bias against you, you would be more likely to put your time in to an audition where you knew the bias factor would be taken out. Still, that result interested me.
  5. The effects of blinding can depend on the point in the process Even after controlling for all sorts of factors, the study authors did find that bias was not equally present in all moments. For example, they found that blind auditions seemed to help women most in preliminary and final rounds, but it actually hurt them in the semi-final rounds. This would make a certain amount of sense….presumably people doing the judging may be using different criteria in each round, and some of those may be biased in different ways than others. Assuming that all parts of the process work the same way is probably a bad assumption to make.

Overall, while the study is potentially outdated (from 2001…using data from 1950s-1990s), I do think it’s an interesting frame of reference for some of our current debates. One article I read about it talked about the benefit of industries figuring out how to blind parts of their interview process because it gets them to consider all sorts of different people….including those lacking traditional educational requirements. With many industries dominated by those who went to exclusive schools, hiding identity could have some unexpected benefits for all sorts of people. However, as this study also shows, it’s probably a good idea to keep the limitations of this sort of blinding in mind. Even established bias is not a consistent force that produces identical outcomes at all time points, and any measure you institute can quickly become a target that changes behavior.  Regardless, I think blinding is a good thing. All of us have our own pitfalls, and we all might be a little better off if we see our expectations toppled occasionally.

5 Nutrition and/or Food Research Blogs I Like to Read

/ bs king

I’ve written a bit here over the years about nutrition research and my own general journey with weight management, but I realized I’ve only really referred in passing to the people who I read when I want to catch up on the field. I figured this was a pretty good time to do a post on that.

  1. For all things vegan: Anyone who followed my old old blog knows that I actually spent several years as a vegan. I eventually gave it up, but I still like to read up what’s going on in the world of plant based nutrition. Ginny Messina (aka the Vegan RD) is a registered dietitian who is a vegan primarily for ethical reasons. As such, she uses her dietitian training to help vegans be as healthy as possible, while also helping lead the charge for veganism to be more evidenced based when they stray out of ethics and in to nutrition claims. She writes critiques of other vegans work if she feels they overstate the evidence, and she even coauthored a book called “Even Vegans Die“. Overall a pretty awesome example of someone who advocates for a particular diet while also adhering to evidence.
  2. For the ancestral health crowd: If you’re paleo or just interested in how our evolutionary history influences how we think about food, Stephan Guyenet is a must read. A neuroscientist who specializes in obesity related research, his research focus is on why we overeat and what we can do about it. His book The Hungry Brain is one of the most well balanced science based nutrition books I’ve ever read, and has received quite a bit of praise for being honest and evidence based.
  3. For deep dives in to the science: There are not many bloggers that I read that make me go “holy crap did this person dig deep in to this paper”, but CarbSane is one blogger who gets that reaction from me on nearly every post. She doesn’t just read papers and give you the gist, she posts tables, cites other literature, and is basically a blog equivalent of a nutritional biochemistry class. She is probably the person most responsible for making me aware of the problem of misreprecitation in nutrition science, because she has the patience, knowledge and wherewithal to figure out exactly what commonly cited papers do and do not say. Oh, and she’s lost over 100 lbs too, so she actually has a good eye for what is and isn’t useful for real people to know. For a taste of what she does, try her posts on the “Biggest Loser Regain Study” that made headlines.
  4. For weight management and health policy: There’s really a bit of a tie here, as I really like both Yoni Freedhoff’s Weighty Matters blog and Darya Rose’s Summer Tomato for this topic.  Freedhoff is a Canadian MD who runs a weight loss center, and he blogs from the medical/health policy perspective. His book “The Diet Fix” covers evidence based ways of making any diet more effective, and he encourages people to take the approach (vegan, low carb, paleo, etc etc) that they enjoy the most. Darya Rose is a neuroscientist who also gives advice about how to make your “healthstyle” more practical and easier to stick to,  and her book “The Foodist” is on the same topic. I like them because they both continuously emphasize that anything too difficult or complicated is ultimately going to be tough to maintain. It’s all about making things easier on yourself.
  5. For those in the greater New Hampshire area: Okay, this ones pretty region specific, but the Co-op News blog from the Hanover Co-op has a special place in my heart. An online version of a newsletter that’s been going since 1936, it features frequent posts from my (dietitian) cousin and my (incredible chef) uncle. It’s a great place to check out if you need advice on anything from using up summer vegetables to figuring out if macaroni and cheese is okay to eat. It also serves to remind me that I should invite myself over to their house more often. That food looks awesome.

Bonus round: if your looking for some one off reads, this week I read this takedown of the science in the vegan documentary “What the Health” and enjoyed it. I also liked this paper that reviewed the (now infamous) Ancel Keys “7 Countries Study” and shed some light on what the original study did and did not say.

Of course if you have a favorite resource, I’d love to hear it!

Follow Up Gazette – the Science Section

/ bs king / 3 Comments

James over at “I Don’t Know, But” had a brilliant idea this week for a journal called “The Follow Up Gazette” (motto: all the things we found out later), that would re-report the news after all the facts were in. His examples were mostly local news, but I would like to throw my hat in the ring to be the editor of the science section. James is of course fully capable of this job himself, but DAMN do I want to do something like this. Let me help James, please.

I’ve been  thinking a lot about this topic, as I had yet another run in with a TED talk recently. We got a question at work from a prospective bone marrow donor asking if we were using a particular collection device in our harvests. None of us had ever even heard of this device, and we were all rather confused where she had gotten her information. A quick google search gave us the answer: there is a TED talk from 8 years ago explaining the device and promising to revolutionize the way marrow harvests are done. Investigating further, we discovered that while this device had gained FDA approval for use in humans,  we couldn’t find any research in humans proving its efficacy, or really any mention of it in the literature past 2009. It’s clear something didn’t go quite as planned, though I’ll be damned if I can find a publicly available record of what. Calling around various people in the field confirmed that no one was using it and that it was not being actively marketed, but we found very few details as to why.

This got me thinking: how would the content of TED talks change if everyone who gave one was required to give an update 5 or 10 years later?

This may seem like a minor point, but I do think it skews our view of science and development of products to hear all of the hype and none of the follow up. Seeing the headline “Brand new drug promises 5 years of extra life for people with horrible disease” juxtaposed with “Actually in practice it was only like 3 months” might help temper our expectations. Alternatively, it may yield that some things were actually shown to be better/safer/whatever than actually thought. My mother recently mentioned that she saw a beautiful house built under power lines, and it hit her that she hadn’t heard a “living under power lines is unhealthy” reference in years. She mentioned that she  assumed that meant that the evidence had shown otherwise, and indeed it has. The Follow-Up Gazette science section would address both sides of this coin, the over hype and the fear mongering. Ideally this would not only educate people in how to consume media, but also encourage media to be slightly more circumspect in their reporting.

James: consider this my application, and thank you in advance for your consideration.

 

Misreprecitation

/ bs king / 1 Comment

A few weeks ago, I wrote a post about a phenomena I had started seeing that I ended up dubbing premature expostulation. I defined this phenomena as “The act of claiming definitively that a person, group or media outlet has not reported on, responded to or comment on an event or topic, without first establishing whether or not this is true. ” Since writing that post, I have been seeing mention of a related phenomena that I felt was distinct enough to merit its own term. In this version, you actually have checked to see what various sources say, enough that you cite them directly, but you misrepresent what they actually say anyway. More formally, we have:

Misreprecitation: The act of directly citing a piece of work  to support your argument, when even a cursory reading of the original work shows it does not actually support your argument.

Now this does not necessarily have to be done with nefarious motives, but it is hard to think of a scenario in which this isn’t incredibly sketchy. Where premature expostulation is mostly due to knee jerk reactions, vagueness and a failure to do basic fact checking, misreprecitation requires a bit more thought and planning. In some cases it appears to be a pretty direct attempt to mislead, in others it may be due to copying someone else’s interpretation without checking it out yourself, but its never good for your argument.

Need some examples? Let’s go!

The example that actually made me think of this was the recent kerfluffle over Nancy MacLean’s book “Democracy in Chains”. Initially met by praise as a leftist take down of right wing economic thought, the book quickly got embroiled in controversy when (as far as I can tell) actual right wing thinkers started reading it. At that point several of them who were familiar with the source material noted that quotes were chopped up in ways that dramatically changed the meaning, and other contextual problems. You can read a pretty comprehensive list of issues here, and overview of the problems and links to all the various responses here, and Vox’s (none to flattering) take here. None of it makes MacLean look particularly good, most specifically because this was supposed to be a scholarly work. When your citations are your strong point, your citations better be correct.

I’ve also seen this happen quite a bit with books that endorse popular diets. Carbsane put together a list of issues in the citations of the low carb book “Big Fat Surprise”, and others have found issues with vegan promoting books. While some of these seem to be differences in interpretation of evidence, some are a little sketchier. Now, as with premature expostulation, some of these issues don’t change the fundamental point….but some do. Overall a citation avalanche is no good if it turns out you had to tweak the truth to get there.

I think there’s three things that cause a particularly fertile breeding ground for misreprecitation: 1) an audience who is sympathetic to your conclusions and 2) an audience who is unlikely to be familiar with the source documents 3) difficulty accessing source documents. That last point may be why books are particularly prone to this error, since you’d have to actually put the book down and go look up a reference. This also may be a case where blogs have the accuracy advantage due to being so public. I know plenty of people who read blogs they don’t agree with, but I know fewer who would buy a whole book dedicated to discrediting their ideas. That increases the chances that no critical person will read your book, they have less recourse once they do read it (notes in the margin aren’t as good as a comments section), and it’s harder for anyone to fact check. Not saying bloggers can’t do it, just thinking they’d be called on it faster.

Overall it’s a pretty ridiculous little trick, as the entire point of citing others work should be to strengthen your argument. In the best case scenario, people could be confused because they misread/failed to understand/copied an interpenetration of the work they read someone else make. In the worst case scenario, they know what they are doing and are counting on their in-group not actually checking their work. Regardless, it needed a name, and now it has one.

Short Takes: Gerrymandering, Effect Sizes, Race Times and More

/ bs king / 1 Comment

I seem to have a lot of articles piling up that I have something to say about, but not enough for a full post. Here’s 4 short takes on 4 current items:

Did You Hear the One About the Hungry Judges?
The AVI sent me an article this week about a hungry judge study I’ve heard referenced multiple times in the context of willpower and food articles. Basically, the study shows that judges rule in favor of prisoners requesting parole 65% of the time at the beginning of the day and 0% of the time right before lunch. The common interpretation is that we are so driven by biological forces that we override our higher order functioning when they’re compromised. The article rounds up some of the criticisms of the paper, and makes a few of its own…namely that an effect size that large could never have gone unnoticed. It’s another good example of “this psychological effect is so subtle we needed research to tease it out, but so large that it noticeably impacts everything we do” type research, and that should always raise an eyebrow. Statistically, the difference in rulings is as profound as the difference between male and female height. The point is, everyone would know this already if it were true. So what happened here? Well,this PNAS paper covers it nicely but here’s the short version: 1) the study was done in Israel  2) This court does parole hearings by prison, 3 prisons a day with a break in between each 3) prisoners who have legal counsel go first 4) lawyers often represent multiple people, and they chose the order of their own cases 5) the original authors lumped “case deferred” and “parole denied” together as one category. So basically the cases are roughly ordered from best to worst up front, and each break starts the process over again. Kinda makes the results look a little less impressive, huh?

On Inter-Country Generalization and Street Harassment
I can’t remember who suggested it, but I saw someone recently suggest that biology or nutrition papers in PubMed or other journal listings should have to include a little icon/picture at the top that indicated what animal the study was done on. They were attempting to combat the whole “Chemical X causes cancer!” hoopla that arises when we’re overdosing mice on something. I would like to suggest we actually do the same thing with countries, maybe use their flags or something. Much like with the study above, I think tipping people off that we can’t make assumptions things are working the same way they work in the US or whatever country you hail from. I was thinking about that when I saw this article from Slate with the headline “Do Women Like Being Sexually Harassed? Men in a New Survey Say Yes“. The survey has some disturbing statistics about how often men admit to harassing or groping women on the street (31-64%) and why they do it (90% say “it’s fun”), but it’s important to note it surveyed men exclusively in the Middle East and Northern Africa. Among the 4 countries, results and attitudes varied quite a bit, making it pretty certain that there’s a lot of cultural variability at play here. While I thought the neutral headline was a little misleading on this point, the author gets some points for illustrating the story with signs (in Arabic) from a street harassment protest in Cairo. I only hope other stories reporting surveys from other countries do the same.

Gerrymandering Update: Independent Commissions May Not be That Great (or Computer Models Need More Validating)
In  my last post about gerrymandering, I mentioned that some computer models showed that independent commissions did a much better job of redrawing districts than state legislatures did. Yet another computer model is disputing this idea, showing that they aren’t. To be honest I didn’t read the working paper here and I’m a little unclear over what they compared to what, but it may lend credibility to the Assistant Village Idiot’s comment that those drawing district maps may be grouping together similar types of people rather than focusing on political party. That’s the sort of thing that humans of all sorts would do naturally and computers would call biased. Clearly we need a few more checks here.

Runner Update: They’re still slow and my treadmill is wrong
As an update to my marathon times post, I recently got sent this websites report that  showed that US runners for all distances are getting slower. They sliced and diced the data a bit and found some interesting patterns: men are slowing down more than women and slower runners are getting even slower. However, even the fastest runners have slowed down about 10% in the last two decades. They pose a few possible reasons: increased obesity in the general population, elite runners avoiding races due to the large numbers of slower runners, or in general leaving to do ultras/trail races/other activities. On a only tangentially related  plus side, I thought I was seriously slowing down in my running until I discovered that my treadmill was incorrectly calibrated to the tune of over 2 min/mile.  Yay for data errors in the right direction.

 

 

A Pie Chart Smorgasbord

/ bs king / 1 Comment

This past week I was complaining about pie charts to a friend of mine, and I was trying to locate this image to show what I was complaining about:

Source.

I knew I had come across this on Twitter, and in finding the original thread, I ALSO discovered all sorts of people defending/belittling the lowly pie chart. Now I generally fall in the anti-pie chart camp, but these made me happy. I sourced what I could find a source on, but will update if anyone knows who I should credit.

First, we have the first and best use for a pie chart:

No other chart represents that data set quite as well.

Sometimes though, you feel like people are just using them to mess with you:

Source.

Sometimes the information they convey can be surprising:

But sometimes the conclusions are just kind of obvious:

And you have to know how to use them correctly:

They’re not all useless, there are some dramatic exceptions:

If you want more on pie charts, try these 16, uh, creative combinations, or read why they’re just the worst here.

Premature Expostulation

/ bs king

In my last post, I put out a call for possible names for the phenomena of people erroneously asserting that some ideological opponent hadn’t commented on a story without properly verifying that this was true. Between Facebook and the comments section I got a few good options, but the overall winner was set up by bluecat57 and perfected by the Assistant Village Idiot: Premature Expostulation. I have to admit, expostulation was one of those words I only sort of knew what it meant, but the exact definition is great for this situation “to reason earnestly with someone against something that person intends to do or has done; remonstrate:” Therefore, the definition for this phrase is:

Premature Expostulation: The act of claiming definitively that a person, group or media outlet has not reported on, responded to or comment on an event or topic, without first establishing whether or not this is true. 

Premature expostulation frequently occurs in the context of a broader narrative (they NEVER talk about thing X, they ALWAYS prioritize thing Y) , though it can also occur due to bad search results, carelessness, inattention, or simply different definitions of what “covered the story” means. If someone is discussing a news outlet they already don’t like or you are not familiar with, be alert.  It’s easy to miss a statement from someone if you don’t frequent what they write or don’t keep up with them.

To note, premature expostulation is a specific claim of fact NOT subjective opinion. The more specific the claim, the more likely it is (if proven wrong) to be premature expostulation. Saying a story was “inadequate” can cause endless argument, but is mostly a matter of opinion. If you say that a news outlet “stayed silent” however, showing that they ran even one story can disprove the claim.

I think there’s a lot of reasons this happens, but some of the common ones I see seem to be:

  • Search algorithm weirdness/otherwise just missing it. Some people do quick searches or scans and just simply miss it. I have speculated that there’s some sort of reverse inattentional blindness thing going on where you’re so convinced you’ll see something if it’s there that you actually miss it.
  • Attributing a group problem to an individual. I can’t find it right now, but I once saw a great video of a feminist writer who was on a panel get questioned by an audience member why she had hypocritically stayed silent on a particular issue it seems she should have commented on. It turns out she actually had written columns on the issue and offered to send them to him. Poor kid had no idea what to do. Now I suspect at the time there were feminist writers being breathtakingly hypocritical over this issue, but that didn’t mean all of them were.  Even if there were hundreds of feminist writers being hypocritical, you still should double check that the one you’re accusing is one of them before you take aim.
  • Attributing an individual problem to a group Sometimes a prominent figure in a group is so striking that people end up assuming everyone in the group acts exactly as the one person they know about does.
  • Assuming people don’t write when you’re not reading When I had a post go mini-viral a few months ago, I got a huge influx of new people who had never visited this blog. I got many good comments/criticisms, but there were a few that truly surprised me. At least a few people decided that the biggest problem I had was that I never took on big media outlets and that I only picked on small groups, or that I was never talked about statistics that might challenge something liberals said. Now regular readers know this is ridiculous. I do that stuff all the time. For whatever reason though, some people assumed that the one post they read of mine somehow represented everything I’d ever written. That’s a personal anecdote, but we see this happen with other groups as well. During the gay marriage debate I once had a friend claim that Evangelicals never commented on straight divorce. Um, okay. No. You just don’t listen to them until they comment on something you are upset by, then you act like that’s all they ever say.
  • The emotional equivalency metric If someone doesn’t feel the same way you do, they must not have seen the story the way you have. Therefore they can’t have covered the story until they mirror your feelings.

I’m sure there are other ways this comes up as well, feel free to leave me your examples.

 

Sharing Your Feelings

/ bs king / 18 Comments

Yesterday morning during some random Twitter scrolling, I saw two interesting tweets in my feed that seemed a bit related. The first was one complaining about a phenomena that has been irritating the heck out of me recently :

//platform.twitter.com/widgets.js

If the embed doesn’t work, here’s the link. The first shot is some text from a Pacific Standard article about Lisa Durden’s firing. In it, the author claims that “In contrast to other free speech-related controversies on college campuses, there has been almost no media coverage of Durden’s ouster.” The Google news search however shows a different story….in fact many media outlets have covered the story.

Now this type of assertion always seems a little surprising to me for two reasons:

  1. We have absolutely unprecedented access to what people and news outlets are/are not reporting on, and any claim like this should be easy to verify.
  2. It’s an easy claim to modify in a way that makes it a statement of opinion, not fact. “there has been far less media outrage” would seem to preserve the sentiment without being a statement of fact.

Once I started thinking about it, I felt like I heard this type of assertion made quite frequently. Which of course got me wondering if that sort of hyper-attention was part of the phenomena. I think everyone knows the feeling of “I heard one reference to this issue/unusual word/obscure author and now I have seen it 5 places in two days”. I got to wondering….could a related (but opposite) phenomena happen when it came to people you disagreed with saying things? Were people purposefully ignoring or discounting reporting from outlets that didn’t fit their narrative, or were they actually not hearing/registering things that were getting said?

I started wondering further when in one recent case, a writer for the Federalist actually Tweeted out the links to her search results that “proved” the New York Times wasn’t covering a story about NSA abuses under Obama. However, the NYTs had actually covered the story (they broke it actually), and clicking on her links shows that their story was among the results she had been scanning over. She issued a correction Tweet a few hours later when someone pointed that out, which makes me doubt she was really trying to deceive anyone. So what made her look at the story and not see it?

Well, this brings me to the second Tweet I saw, which was about a new study about the emotional drivers of political sharing across social networks. I don’t have access to the full text of the paper, but two interesting findings are making headlines:

  1. For the issues studied (gun control, same-sex marriage, climate change), including moral-emotional language in your headline increased sharing by 20%
  2. This sharing increase occurred almost exclusively in your in-group. Liberals and conservatives weren’t sharing each others stories.

I’m speculating wildly here, but I wonder if this difference in the way we share stories contributes to perceptions that the other side is “not talking” about something. When something outrages my liberal (or conservative) friends, the same exact article will show up in my news feed 10 times. When the opposing party comments on it/covers it, they almost never share the same exact story, they comment/share different ones. They only comment on the same story when they oppose the coverage.

For example, in the NSA case above, the story that got Mollie Hemingway looking at search results was titled “Obama intel agency secretly conducted illegal searches on Americans for years.”. The ones she missed in the NYTs results was “N.S.A. Halts Collection of Americans’ Emails About Foreign Targets” and “How Trump’s N.S.A. Came to End a Disputed Type of Surveillance“. Looking at those 3 headlines, it’s easy to see why you could miss they were all talking about the same thing. At the same time, if you’re going to claim that a story isn’t being reported, you need to double check that it’s not just your feelings on the story that aren’t being mirrored.

And also lest I be a hypocrite here, I should talk about the time I committed this error because I failed to update my information. Back in February I made that error, claiming that TED didn’t update their webpage to reflect the controversy with Amy Cuddy’s research. I was right the first time I claimed it and wrong the second time. I could have sworn I rechecked it, but I either didn’t recheck when I thought I did, or I simply didn’t see the correction that got added. Was it because I was looking for a more dramatic correction, bold letters or some other sort of red flag? Yeah, I’d say that was part of it. TED does not appear nearly as concerned about the controversy as I am, but that doesn’t mean they failed to talk about it.

I need a name for this one I think.

Statisticians and Gerrymandering

/ bs king / 2 Comments

Okay, I just said I was blogging less, but this story was too interesting to pass without comment. A few days ago it was announced that the Supreme Court had agreed to hear a case about gerrymandering, or the practice of redrawing voting district lines to influence the outcome of elections. This was a big deal because previously the court has only heard these cases when the lines had something to do with race, but had no comment on redraws that were based on politics. The case they agreed to hear was from Wisconsin, and a lower court found that a 2011 redistricting plan was so partisan that it potentially violated the rights of all minority party voters in the affected districts.

Now obviously I’ll leave it to better minds to comment on the legal issues here, but I found this article on how statisticians are getting involved in the debate quite fascinating. Obviously both parties want the district lines to favor their own candidates, so it can be hard to cut through the noise and figure out what a “fair” plan would actually look like. Historically, this came down to just two parties bickering over street maps, but now with more data available there’s actually a chance that both gerrymandering and the extent of gerrymandering can be measured.

One way of doing this is called the “efficiency gap” and is the work of Eric McGhee and Nicholas Stephanopolous, who explain it here. Basically this measures “wasted” votes, which they explain like this:

Suppose, for example, that a state has five districts with 100 voters each, and two parties, Party A and Party B. Suppose also that Party A wins four of the seats 53 to 47, and Party B wins one of them 85 to 15. Then in each of the four seats that Party A wins, it has 2 surplus votes (53 minus the 51 needed to win), and Party B has 47 lost votes. And in the lone district that Party A loses, it has 15 lost votes, and Party B has 34 surplus votes (85 minus the 51 needed to win). In sum, Party A wastes 23 votes and Party B wastes 222 votes. Subtracting one figure from the other and dividing by the 500 votes cast produces an efficiency gap of 40 percent in Party A’s favor.

Basically this metric highlights unevenness across the state. If one party is winning dramatically in one district and yet losing in all the others, you have some evidence that those lines may not be fair. If this is only happening to one party and never to the other, your evidence grows. Now there are obvious responses to this….maybe some party members really are clustering together in certain locations….but it does provide a useful baseline measure. If your current plan increases this gap in favor of the party in power, then that party should have to offer some explanation. The author’s proposal is that if the other party could show a redistricting plan that had a smaller gap, the initial plan would be considered unconstitutional.

To help with that last part, two mathematicians have created a computer algorithm that draws districts according to state laws but irrespective of voting histories. They then compare these hypothetical districts “average” results to the proposed maps to see how far off the new plans are. In other words, they basically create a normal distribution of results, then see how the current proposals line up. To give context, of the 24,000 maps they drew for North Carolina, all were less gerrymandered than the one the legislature came up with. When a group of retired judges tried to draw new districts for North Carolina, they were less gerrymandered than 75% of the computer models.

It’s interesting to note that some of the most gerrymandered states by this metric are actually not the ones being challenged. Here are all the states with more than 8 districts and how they fared in 2012. The ones in red are the ones facing a court challenge. The range is based on plausible vote swings:

Now again, none of these methods may be perfect, but they do start to point the way towards less biased ways of drawing districts and neutral tests for accusations of bias. The authors note that the courts currently employ simple mathematical tests to evaluate if districts have equal populations: +/- 10%.  It will be interesting to see if any of these tests are considered straightforward enough for a legal standard. Stay tuned!