Eating Season

Happy almost Thanksgiving! Please enjoy this bit of trivia I recently stumbled on about American food consumption patterns during this time of year! It’s from the book “Devoured: From Chicken Wings to Kale Smoothies – How What We Eat Defines Who We Are” by Sophie Egan.

From page 173:

A few paragraphs later, she goes a bit more in depth about what happens to shopping habits (note: she quotes the embattled Cornell Food and Brand lab, but since their data matches another groups data on this, I’m guessing it’s pretty solid):

I had no idea that “eating season” had gone so far outside the bounds of what I think of as the holiday season. Kinda makes you wonder if this is all just being driven by winter and the holidays are just an excuse.

On a related note, my capstone project is done/accepted with no edits and I will probably be putting up some highlights about my research in to food insecurity and health habits on Sunday.

Happy Thanksgiving!

5 Interesting Things About IQ Self-Estimates

After my post last week about what goes wrong when students self-report their grades, the Assistant Village Idiot left a comment wondering about how this would look if we changed the topic to IQ. He wondered specifically about Quora, a question asking/answering website that has managed to spawn its own meta-genre of questions asking “why is this website so obsessed with IQ?“.

Unsurprisingly, there is no particular research done on specific websites and IQ self-reporting, but there is actually some interesting literature on people’s ability to estimate their own IQ and that of those around them. Most of this research comes from a British researcher from the University College London, Adrian Fuhrman.  Studying how well people actually know themselves kinda sounds like a dream job to me, so kudos to you Adrian. Anyway, ready for the highlights?

  1. IQ self estimates are iffy at best One of the first things that surprised me about IQ self-estimates vs actual IQ was how weak the correlation was. One study found an r=.3, another r=.19.  This data was gathered from people who first took a test, then were asked to estimate their results prior to actually getting them. In both cases, it appears that people are sort of on the right track, but not terrific at pinpointing how smart they are. One wonders if this is part of the reason for the IQ test obsession….we’re rightfully insecure about our ability to figure this out on our own.
  2. There’s a gender difference in predictions Across cultures, men tend to rank their own IQ higher than women do, and both genders consistently rank their male relatives (fathers, grandfathers and sons) as smarter than their female relatives (mothers, grandmothers and daughters). This often gets reported as male hubris vs female humility (indeed, that’s the title of the paper), but I note they didn’t actually compare it to results. Given that many of these studies are conducted on psych undergrad volunteers, is it possible that men are more likely to self select when they know IQ will be measured? Some of these studies had average IQ guesses of 120 (for women) and 127 (for men)….that’s not even remotely an average group, and I’d caution against extrapolation.
  3. Education may be a confounding factor for how we assess others One of the other interesting findings in the “rate your family member” game is that people rank previous generations as half a standard deviation less intelligent than they rank themselves. This could be due to the Flynn effect, but the other suggestion is that it’s hard to rank IQ accurately when educational achievement is discordant. Within a cohort, education achievement is actually pretty strongly correlated with IQ, so re-calibrating for other generations could be tricky.  In other words, if you got a master’s degree and your grandmother only graduated high school, you may think your IQ is further apart than it really is. To somewhat support this theory, as time has progressed, the gap between self rankings and grandparent rankings has closed. Interesting to think how this could also effect some of the gender effects seen in #2, particularly for prior generations.
  4. Being smart may not be the same as avoiding stupidity One of the more interesting studies I read looked at the correlation between IQ self-report and personality traits, and found that some traits made your more likely to think you had a high IQ. One of these traits was stability, which confused me because you don’t normally think of stable people as being overly high on themselves. When I thought about it for a bit though, I wonder if stable people were defining being “smart” as “not doing stupid things”.  Given that many stupid actions are probably more highly correlated with impulsiveness (as opposed to low IQ), this could explain the difference. I don’t have proof, but I suspect a stable person A with an IQ of 115 will mostly do better than an unstable person B with an IQ of 115, but person A may attribute this difference to intelligence rather than impulse control. It’s an academic distinction more than a practical one, but it could be confusing things a bit.
  5. Disagreeableness is associate with higher IQs, and self-perception of higher IQs  Here’s an interesting chicken and egg question for you: does having a high IQ make you more disagreeable or does being disagreeable make you think you have a higher IQ? Alternative explanation: is some underlying factor driving both? It turns out having a high IQ is associate both with being disagreeable and being disagreeable is associated with ranking your IQ as higher than others. This probably effects some of the IQ discussions to a certain degree….the “here’s my high IQ now let’s talk about it” crowd probably really is not as agreeable as those who want to talk about sports or exchange recipes.

So there you have it! My overall impression from reading this is that IQ is one of those things where people don’t appreciate or want to acknowledge small differences. In looking at some of the studies of where people ranking their parents against each other, I was surprised how many were pointing to a 15 point gap between parents, or a 10 point gap between siblings. Additionally, it’s interesting that we appear to have a pretty uneasy relationship with IQ tests in general. Women in the US for example are more likely to take IQ tests than men are but less likely to trust their validity. To confuse things further, they are also more likely to believe they are useful in educational settings. Huh? I’d be interested to see a self-estimated IQ compared to an actual understanding of what IQ is/is not, and then compare that to an actual scored IQ test. That might flesh out where some of these conflicting feelings were coming from.

Grade Prediction by Subject

I saw an interesting study this week that seems to play in to two different topics I’ve talked about here: self reporting bias and the Dunning-Kruger effect.

The study was “Examining the accuracy of students’ self-reported academic grades from a correlational and a discrepancy perspective: Evidence from a longitudinal study“, and it took a look at how accurate students self-reported grades were. This is not the first time someone has looked at this, but it did add two key things to the mix: non-US test scoring and different academic subjects over different years of school. The students surveyed were Swiss, and they were asked to report their most recent grade in 4 different subjects. This was then compared to their actual most recent grade. The results were pretty interesting (UR=under-report, OR=Over-report, T1-T3 are years of school):

Unsurprisingly, kids were much more likely to over-report than under-report. Since most of the differences were adding a half point or so (out of 6), one wonders if this is just a tendency to round up in our own favor. Interestingly, a huge majority of kids were actually quite honest about their ability….about 70% for most years. The authors also noted that the younger kids were more likely to be honest than the older kids.

I think this is a really interesting example of how self-reporting biases can play out. It’s easy to think of bias as something that’s big and overwhelming, but studies like this suggest that most bias is small for any given individual. A rounding error here, and accidental report of your grade from last semester….those are tiny for each person but can add up over a group. I suspect if we looked at those older students who reported their grades as inaccurately high, we would discover that they had gotten high grades in previous years. There does seem to be a bias towards reporting your high water mark rather than your current status….kinda like the jock who continues to claim they can run a 5 minute mile long after they cease to be able to do so.

The phenomena is pretty well known, but it’s always interesting to see the hard numbers.

Millenials and Communism

I was perusing Twitter this past weekend when I started to see some concerning headlines float by.

Survey: 1 in 2 millennials would rather live in a socialist or communist country than capitalist one

Millenials think socialism would make a great safe space

Nearly 1 In 5 Millennials Consider Joseph Stalin And Kim Jong Un ‘Heroes’

While I could see a survey of young people turning up with the socialism result, that last headline really concerned me. At first I thought it was just a case of “don’t just read the headline“, but all the articles seemed to confirm the initial statistic. AOL said “a lot of them see Joseph Stalin and Kim Jong Un as “heroes.”” Fox News hit on my discomfort when they said “The report also found that one in five Americans in their 20s consider former Soviet dictator Joseph Stalin a hero, despite his genocide of Ukrainians and Orthodox priests. Over a quarter of millennials polled also thought the same for Vladimir Lenin and Kim Jong Un.”

Seriously?

While I know polls frequently grab headlines by playing on people’s political ignorance, this seemed to go a step beyond that. I had trouble wrapping my head around the idea that anyone in the US could list Stalin, Lenin or Jong-Un as a hero, let alone 20-25%. I had to go see what question prompted such an odd set of results.

The overview of the poll results is here, and sure enough, the question that led to the results is worded a little differently than the article. Here’s the screenshot from the report, blue underlines/boxes are mine:

I think the “hero for their country” part is key. That asks people to assess not just their own feelings, but what they know about the feelings of a whole other country.

Interestingly, I decided to look up Kim Jong-un’s in-country approval rating, and some defectors put it as high as 50%.  According to one poll, 38% of Russians consider Josef Stalin to be the “most outstanding person” in world history. You could certainly debate if those polls had problems in wording, sample or other methodology, but the idea that a 25 year old in the US might see a headline like that and conclude that Russians really did like Stalin doesn’t seem outside the realm of possibility. Indeed, further down the report we find out that only 6% of millenials in the US state that they personally have a favorable view of Stalin. That’s lizard people territory folks.

In this case, it appears the polling company was actually pretty responsible about how they reported things, so it’s disappointing that further reports dropped the “in their country” piece. In my ongoing quest to name different biases and weird ways of skewing data, I’m now wondering what to name this one. What do you call it when someone asks a poll question in a way that encompasses a variety of scenarios, then the later reports shorten the question to make it sound like a different question was answered? I’m gonna work on this.

Daylight Saving (is not the worst of evils)

Well hi there! At this point on Sunday, I’m going to assume you’ve remembered that your clock should have been set back last night. With the advent of cell phones and auto-updates, I suspect the incidence of “showing up to church an hour early because no one remembered daylight saving time” has dropped precipitously since I was a kid.

Growing up, daylight saving time was always the subject of some debate in my house. My dad is a daylight saving time defender, and takes a lot of joy in pointing out to people that no matter how irritated you are by the time change, not changing the time would be even more annoying.

To support his point, I found this site that someone posted on Facebook rather interesting. It’s by a cartographer, and it lets you see the impact of Daylight Saving on the different regions of the country. It also lets you monkey around with different schemes….eliminate daylight saving vs impose it permanently vs keep the status quo…and see what impact they’d have on the sunrise/sunset times. (Note: he created it in 2015, so some numbers may not reflect the 2017 time changes)

My Dad’s point was always that daylight saving blunts the extremes, so I tried out a few different schemes to see how often they made the sunrise very early vs very late. For example, here’s how many days the sun would rise before 5am in different regions if we keep things status quo vs eliminate daylight saving vs always use it:

If you go to the actual website and hover, you can get the exact number of days those colors represent. If we did away with daylight saving, my region of the country would have over 120 days of pre-5am sunrises. I’m an early riser, but that seems a little much even for me.

Here’s how it would effect post-8pm sunsets:

So basically my Dad was right. If you want lots of early sunrises, push to abolish daylight saving. I think most people sort of know that’s what the time change thing is all about, but it is interesting to see exactly how many early sunrises we’re talking about. When you consider that the sky starts to lighten half an hour before sunrise, you realize that getting rid of daylight saving is signing yourself up for a LOT of early morning sunshine.

I think the real PR problem here is that the time changes happen so far away from the extremes that people forget that it’s really designed to help mitigate situations that would occur several months later. I think there’s a new bias name in here somewhere.

The Weight of Evidence

I’ve been thinking a lot about the law and evidence this week, for 3 reasons:

First, this article my lawyer father sent me about the Supreme Court’s aversion to math. It reviews a case about gerrymandering  I’ve mentioned before, and the attempts of statisticians/computer guys to convince the court that their mathematical  models are worth using. While the case hasn’t been decided yet, some researchers were fairly annoyed at how reflexively some of the justices dismissed the models presented, and their invocation of the “gobbledygook” doctrine.

Second was this article I stumbled on that discussed an effort to fact-check supreme court decisions, and found a rather alarming number of them contain factual errors. This one was concerning for two reasons: some of the errors actually appeared to be related to the ultimate decision and some of the errors appear to have come from the Justices doing their own research.

Finally, this article about yet another evidence scandal in my state. Apparently our state lab has been systematically withholding evidence of failed breathalyzer calibrations, calling in to question hundreds of DUI convictions. This is not an aberration…for those of you not from around here, Massachusetts has been on a bad run with our state crime/forensics lab. This is our 3rd major scandal in the past few years, and we now have the dubious distinction of being cited in every report about the problems with forensics.

This got me thinking about a few things:

  1. The line between gobbledygook and “good idea, needs work” is often familiarity. In reading some of the Supreme Courts skepticism of mathematical models and contrasting it with the general acceptance of forensics despite serious concerns, it’s hard not to think that this has something to do with familiarity. Forensics is a science that was quite literally built to support the criminal justice system, whereas computer modeling was built to support….well, all sorts of things. I suspect that’s why one gets more scrutiny than the other.
  2. Mathematical models have to simplify and/or those who build them have prioritize explaining them to people who are not on their side The new wave of mathematical models is intriguing, exciting, and a little bit frightening all at once. Complexity is necessary at times, but ultimately can be used to hide assumptions and get your way. The justices on the Supreme Court know this, and their first suspicion is going to be that all that math is just there to hide something. Anyone hoping to build a model that effects policy should probably keep in mind that for everyone they impress, they will make someone else suspicious. As with any argument, trying it out on someone not inclined to agree with you will teach you a lot about where the holes might be.
  3. Lawyers need to learn more about statistics This one has been the subject of many long talks with my Dad. Unless they were required to take it for their undergrad degree, many lawyers can get through their whole higher ed career without touching a stats class. This seems like a gap to me, especially now that so much of the evidence they’re seeing requires some knowledge of probability and evidence. I’ve mentioned before that doctors struggle with the concept of false positives and false negatives and base rates,  and it seems clear many people in law enforcement do as well.  With all the new types of evidence out there, it seems like this is a gap.
  4. The Supreme Court needs a fact checker Seriously. Are you really telling me there’s not one clerk out there who would be willing to just read through the decisions and find citations for each stat? Or better yet, someone who’d read through each briefing filed with the court and error check them before they got to the Justices? In the case the article cited, the stat in question wasn’t a common controversial one (the % of workplaces that drug tested employees), but the answer provided (88%) apparently had no source at all. I feel like of all groups, the Supreme Court should have figured out how to get this stuff screened out before it biases them.

I am thinking there’s a presentation in here somewhere. If you have any more good articles, send them my way!