5 Interesting Things About IQ Self-Estimates

After my post last week about what goes wrong when students self-report their grades, the Assistant Village Idiot left a comment wondering about how this would look if we changed the topic to IQ. He wondered specifically about Quora, a question asking/answering website that has managed to spawn its own meta-genre of questions asking “why is this website so obsessed with IQ?“.

Unsurprisingly, there is no particular research done on specific websites and IQ self-reporting, but there is actually some interesting literature on people’s ability to estimate their own IQ and that of those around them. Most of this research comes from a British researcher from the University College London, Adrian Fuhrman.  Studying how well people actually know themselves kinda sounds like a dream job to me, so kudos to you Adrian. Anyway, ready for the highlights?

  1. IQ self estimates are iffy at best One of the first things that surprised me about IQ self-estimates vs actual IQ was how weak the correlation was. One study found an r=.3, another r=.19.  This data was gathered from people who first took a test, then were asked to estimate their results prior to actually getting them. In both cases, it appears that people are sort of on the right track, but not terrific at pinpointing how smart they are. One wonders if this is part of the reason for the IQ test obsession….we’re rightfully insecure about our ability to figure this out on our own.
  2. There’s a gender difference in predictions Across cultures, men tend to rank their own IQ higher than women do, and both genders consistently rank their male relatives (fathers, grandfathers and sons) as smarter than their female relatives (mothers, grandmothers and daughters). This often gets reported as male hubris vs female humility (indeed, that’s the title of the paper), but I note they didn’t actually compare it to results. Given that many of these studies are conducted on psych undergrad volunteers, is it possible that men are more likely to self select when they know IQ will be measured? Some of these studies had average IQ guesses of 120 (for women) and 127 (for men)….that’s not even remotely an average group, and I’d caution against extrapolation.
  3. Education may be a confounding factor for how we assess others One of the other interesting findings in the “rate your family member” game is that people rank previous generations as half a standard deviation less intelligent than they rank themselves. This could be due to the Flynn effect, but the other suggestion is that it’s hard to rank IQ accurately when educational achievement is discordant. Within a cohort, education achievement is actually pretty strongly correlated with IQ, so re-calibrating for other generations could be tricky.  In other words, if you got a master’s degree and your grandmother only graduated high school, you may think your IQ is further apart than it really is. To somewhat support this theory, as time has progressed, the gap between self rankings and grandparent rankings has closed. Interesting to think how this could also effect some of the gender effects seen in #2, particularly for prior generations.
  4. Being smart may not be the same as avoiding stupidity One of the more interesting studies I read looked at the correlation between IQ self-report and personality traits, and found that some traits made your more likely to think you had a high IQ. One of these traits was stability, which confused me because you don’t normally think of stable people as being overly high on themselves. When I thought about it for a bit though, I wonder if stable people were defining being “smart” as “not doing stupid things”.  Given that many stupid actions are probably more highly correlated with impulsiveness (as opposed to low IQ), this could explain the difference. I don’t have proof, but I suspect a stable person A with an IQ of 115 will mostly do better than an unstable person B with an IQ of 115, but person A may attribute this difference to intelligence rather than impulse control. It’s an academic distinction more than a practical one, but it could be confusing things a bit.
  5. Disagreeableness is associate with higher IQs, and self-perception of higher IQs  Here’s an interesting chicken and egg question for you: does having a high IQ make you more disagreeable or does being disagreeable make you think you have a higher IQ? Alternative explanation: is some underlying factor driving both? It turns out having a high IQ is associate both with being disagreeable and being disagreeable is associated with ranking your IQ as higher than others. This probably effects some of the IQ discussions to a certain degree….the “here’s my high IQ now let’s talk about it” crowd probably really is not as agreeable as those who want to talk about sports or exchange recipes.

So there you have it! My overall impression from reading this is that IQ is one of those things where people don’t appreciate or want to acknowledge small differences. In looking at some of the studies of where people ranking their parents against each other, I was surprised how many were pointing to a 15 point gap between parents, or a 10 point gap between siblings. Additionally, it’s interesting that we appear to have a pretty uneasy relationship with IQ tests in general. Women in the US for example are more likely to take IQ tests than men are but less likely to trust their validity. To confuse things further, they are also more likely to believe they are useful in educational settings. Huh? I’d be interested to see a self-estimated IQ compared to an actual understanding of what IQ is/is not, and then compare that to an actual scored IQ test. That might flesh out where some of these conflicting feelings were coming from.

3 More Examples of Self Reporting Bias

Right after I put up my self reporting bias post last week, I saw a few more examples that were too good not to share. Some came from commenters, some were random stories I came across, but all of them could have made the original list. Here you go:

  1. Luxury good ratings Commenter Uncle Bill brought this one up in the comments section on the last post, and I liked it. The sunk cost fallacy  says that we have a hard time abandoning money we’ve already spent, and this kicks in when we have to say how satisfied we are with our luxury goods. No one wants to admit a $90,000 vehicle actually kind of sucks, so it can be hard to figure out if the self reported reliability ratings reflect reality or a desired reality.
  2. Study time Right after I put my last self reporting bias post up, this study came across my Twitter feed. It was a study looking in to “time spent on homework” vs grades, and initially it found that there was no correlation between the two. However, the researchers had given the college students involved pens that actually tracked what they were doing so they double checked the students reports. With the pen-measured data, there actually was a correlation between time on homework and performance in the class. It turned out that many of the low performing kids wildly overestimated how much time they were actually spending on their homework, much more so than the high performing kids. This bias is quite possibly completely unintentional….kids who were having a tough time with the material probably felt like they were spending more time than they were.
  3. Voter preference I mentioned voter preference in my Forest Gump Fallacy post, and I wanted to specifically call out Independent voters here. Despite the name and the large number of those who self identify as such, when you look at voting patterns many independent voters are actually what they call “closet partisans”. Apparently someone who identifies as Independent but has a history of voting Democrat is actually less likely to ever vote GOP than someone who identifies as a “weak Democrat”.  So Independent is a tricky group of Republicans who don’t want to say they’re Republicans, Democrats who don’t want to say they’re Democrats, 3rd party voters, voters who don’t care, and voters who truly have no party affiliation. I’m sure I left someone out, but you can see where it gets messy. This actually also effects how we view Republicans and Democrats, as those groups are normally polled based on self identification. By removing the Independents, it can make one or both parties look like their views are changing, even if the only change is who checked the box on the form.

If you see any more good ones, feel free to send them my way!

International data – beware the self reporting

Maybe it’s just because the Olympics are on, but I’ve run in to a few interesting international statistics lately that gave me pause.

The first was regarding infant mortality.  After Aaron Sorkin’s new show The Newsroom incorrectly reported that the US was 178th in infant mortality (really, you think there are 177 countries you’d rather give birth in?), I went looking for the infant mortality listings across the world.  The US does not typically do very well in terms of other industrialized countries.  
There are a few interesting reasons for that….we have a much larger population than most of the countries that beat us, and it’s spread out over a much larger area.  Our care across areas/populations tends to be more uneven, states vary wildly on issues like access, health insurance, prenatal care, etc. Our records however, tend to be meticulous….there is very little doubt that we capture nearly all infant mortality that actually occurs.  This combination can put the US at a huge disadvantage in these statistics (10-30% according to the best published studies).
This raises the point of why Cuba tends to beat us.  Now, realistically speaking, if you or someone you love had to give birth, would you seriously pick Cuba over the US?  Would anybody?  And yet they look safer given the data….which is all self reported.  I have no problems believing that Singapore outranks us, but I’m skeptical of any country that might have an agenda.  Worldwide, there is actually very little consensus on what is a “live birth”, and the US tends to use the “any sign of life” definition.  
On the other end of the spectrum, I saw this piece recently on gun control.  I’ve covered misleading gun stats before (suicides are often combined with homicides to get “death by gun violence” numbers).  One of the interesting facts the article above points out is that internationally, gun deaths are only counted when it’s civilian on civilian violence.  This is certainly fine in the US…I would think we wouldn’t want to count every time the police had to open fire, but in countries with, um, more questionable police tactics, this could cause some skewing (Syria was cited as one such example).  
Data is hard enough to pin down when you know the sources have no vested interest in misleading you….international rankings will never be free from such bias.