5 Things About Precognition Studies

Several months ago now, I was having dinner with a friend who told me he was working on some science fiction based on some interesting precognition studies he had heard about. As he started explaining them to me and how they was real scientific proof of ESP, he realized who he was talking to and quickly got sheepish and told me to “be gentle” when I ended up doing a post about it. Not wanting to kill his creative momentum, I figured I’d delay this post for a bit. I stumbled on the draft this morning and realized it’s probably been long enough now, so let’s talk about the paranormal!

First, I should set the stage and say that my friend was not actually wrong to claim that precognition has some real studies behind it. Some decent research time and effort has been put in to experiments where researchers attempt to show that people react to things that haven’t happened yet. In fact the history of this work is a really interesting study in scientific controversy and it tracks quite nicely with much of the replication crisis I’ve talked about. This makes it a really interesting topic for anyone wanting to know a bit more about the pluses/minuses of current research methods.

As we dig in to this, it helps to know a bit of background: Almost all of the discussions about this are referencing a paper by Daryl Bem from 2011, where 9 different studies were run on the phenomena. Bem is a respected psychological researcher, so the paper made quite a splash at the time. So what did these studies say and what should we get out of them, and why did they have such a huge impact on psychological research? Let’s find out!

  1. The effect sizes were pretty small, but they were statistically significant Okay, so first things first….let’s establish what kind of effect size we’re talking about here. For all 9 experiments the Cohen’s d was about .22. In general, a d of .2 is considered a “small” effect size, .5 would be moderate, .8 would be large. In the real world, this translated in to participants picking the “right” option 53% of the time instead of the 50% you’d expect by chance.
  2. The research was set up to be replicated One of the more interesting parts of Bem’s research was that he made his protocols publicly available for people trying to replicate his work, and he did this before he actually published the initial 2011 paper. Bem particularly pointed people to experiments #8 and #9, which showed the largest effect sizes and he thought would be the easiest to replicate. In these studies, he had people try to recall words off of a word list, writing down those they could remember. He then gave them a subset of those words to study more in depth, again writing down what they could remember. When they looked back, they found that subjects had recalled more of their subset words than control words on the first test. Since the subjects hadn’t seen their subset words at the time they took the first test, this was taken as evidence of precognition.
  3. Replication efforts have been….interesting. Of course with interesting findings like these, plenty of people rushed to try to replicate Bem’s work. Many of these attempts failed, but Bem published a meta-analysis stating that on the whole they worked. Interestingly however, the meta-analysis actually analyzed replications that pre-dated the publication of Bem’s work. Since Bem had released his software early, he was able to find papers all the way back to 2001. It has been noted that if you remove all the citations that pre-dated the publication of his paper, you don’t see an effect. So basically the pre-cognition paper was pre-replicated. Very meta.
  4. They are an excellent illustration of the garden of forking paths. Most of the criticism of the paper comes down to something Andrew Gelman calls “The Garden of Forking Paths“. This is a phenomena in which researchers make a series of tiny decisions as their experiments and analyses progress, which may add up to serious deviation from the original results. In the Bem study for example, it has been noted that some of his experiments actually used two different protocols, then combined the results. It was also noted that the effect sizes got smaller as more subjects were added, suggesting that the number of subjects tested may have fluctuated based on results. There are also decisions so small you mostly wouldn’t notice. For example, in the word recall study mentioned above, word recall was measured by comparing word lists for exact matches. This meant that if you spelled “retrieve” as “retreive”, it didn’t automatically give you credit. They had someone go through and correct for this manually, but that person actually knew which words were part of the second experiment and which were the control words. Did the reviewer inadvertently focus on or give more credit to words that were part of the “key word” list? Who knows, but small decisions like this can add up. There were also different statsticall analyses performed on different experiments, and Bem himself admits that if he started a study and got no results, he’d tweak it a little and try again. When you’re talking about an effect size of .22, even tiny changes can add up.
  5. The ramifications for all of psychological science were big It’s tempting to write this whole study off, or to accept it wholesale, but the truth is a little more complicated. In a thorough write-up over at Slate, Daniel Engber points out that this research used typical methods and invited replication attempts and still got a result many people don’t believe is possible. If you don’t believe the results are possible, then you really should question how often these methods are used in other research. As one of the reviewers put it “Clearly by the normal rules that we [used] in evaluating research, we would accept this paper. The level of proof here was ordinary. I mean that positively as well as negatively. I mean it was exactly the kind of conventional psychology analysis that [one often sees], with the same failings and concerns that most research has”. Even within the initial paper, the word “replication” was used 23 times. Gelman rebuts that all the problems with the paper are known statistical issues and that good science can still be done, but it’s clear this paper pushed many people to take good research methods a bit more seriously.

So there you have it. Interestingly, Bem actually works out of Cornell and has been cited in the whole Brian Wansink kerfluffle, a comparison he rejects. I think that’s fair. Bem has been more transparent about what he’s doing, and did invite replication attempts. In fact his calls for people to look at his work were so aggressive, there’s a running theory that he published the whole thing to make a point about the shoddiness of most research methods. He’s denied this, but that certainly was the effect. An interesting study on multiple levels.

5 Things About IQ Errors in Intro Psych Textbooks

A few months ago I did a post on common errors that arise when people try to self-estimate their IQ.  One concern I sort of covered at the time was that many people may not truly understand what IQ was. For example, there seems to be a tendency to confuse educational attainment with IQ, which is likely why many of us think our grandparents were not nearly as smart as we are.

I was thinking about this issue this past week when I saw a newly published study called “What Do Undergraduates Learn About Human Intelligence? An Analysis of
Introductory Psychology Textbooks“. As the study suggests, the authors took a look at intro psych textbooks to see what they say about IQ, and how well it aligns with the actual published research on IQ. So what did they find? Let’s take a look!

  1. Most of what undergrads learn about intelligence will be learned in intro psych. To back up the premise of the study, the authors looked at the topics covered in psych programs around the country. They determined that classes on intelligence were actually pretty rare, and that the primary coverage the topic got was in intro psych. Once they’d established this, they were able to pull the 30 most popular intro psych textbooks, and they chose to analyze those. Given the lack of subsequent classwork and the popularity of the textbooks used, they estimate that their study covers a huge proportion of the formal instruction/guidance/learning on intelligence that goes on in the US.
  2. The percent of space dedicated to discussing intelligence has dropped The first research question the authors wanted to look at was how much space was dedicated to explaining IQ/intelligence research to students. In the 80s, this was 6% of textbook space, but now it’s about 3-4%. Now it’s possible that this is because textbooks got longer (and thus the percent dropped), or it could be that the topic got de-emphasized. Regardless, an interesting note.
  3. IQ Fallacies were pretty common The list of possible IQ “fallacies” was drawn from two sources. The first was from this article by Gottfredson et al, which was published after “The Bell Curve” came out and had 52 signatories who wanted to clear up what current research on IQ said. The second paper was a statement from the American Psychological Association, also in response to the publicity around the Bell Curve. They used these two papers to generate the following list:  The most common fallacies they found were #2, 3 4 and 6. These were present in 8 books (2 and 3) and 6 books (4 and 6) respectively. Interestingly, for #3 they specifically clarified that they only called it a fallacy if someone asserted that you could raise IQ by adding a positive action as opposed to eliminating a negative action. Their example was that lead poisoning really does provably lower IQ, but fish oil supplements during pregnancy have not been proven to raise IQ. The initial two papers explain why these are viewed as fallacies.
  4. Briefs discussions led to inaccuracies In addition to fallacies, the authors also took a look at inaccuracies, questionable theories, and the proportionate amount of time authors spent looking at various topics. Many of the textbooks committed the errors of citing part of the story, but not the full story. For example, it was noted that testing bias was well covered, but not the efforts that have been made to correct for testing bias. Some textbooks went so far as to say that all IQ tests required you to speak English, where as nonverbal tests have been available as far back as 1936. Additionally, some theories of intelligence that have not born out well (Gardner’s theory of multiple intelligences and Sternberg’s triarchic theory of intelligence) were two of the most discussed topics in textbooks, but did not include a discussion of the literature supporting those vs the g theory of intelligence. I imagine the oversimplification issue is one that affects many topics in intro textbooks, but this does seem a bit of an oversight.
  5. Overall context of intelligence scores was minimized Despite good proof that intelligence scores are positively correlated with various good outcomes, the most surprising finding was that several textbooks said directly that IQ only impacted education and had little relevance to every day life (4 textbooks). This directly contradicts most current research, and also a certain amount of common sense. Even if IQ only helped you in academia, having a degree helps you in many other areas of life, such as income and all the advantages that brings.

Overall this was a pretty interesting paper, especially when they gave examples of the type of statements they were talking about. Reading the statement from the APA and comparing it to the textbooks was rather interesting, as it shows how far it is possible it is to drift from consensus if you’re not careful.

Additionally, the authors cited some interesting work to show that some popular public misconceptions around IQ are directly mirrored in the intro psych textbooks errors. Overall I think the point is well taken that intro to anything textbooks should be given a lot of scrutiny in making sure their claims are factual before being assigned.

5 Things About that “Republicans are More Attractive than Democrats” Study

Happy Valentine’s Day everyone! Given the spirit of the day, I thought it was a good time to post about a study Korora passed along a few days ago called “Effects of physical attractiveness on political beliefs”, which garnered a few headlines for it’s findings that being attractive was correlated with being a Republican. For all of you interested in what was actually going on here, I took a look at the study and here’s what I found out:

  1. The idea behind the study was not entirely flattering. Okay, while the whole “my party is hotter than your party” thing sounds like compliment, the premise of this study was actually a bit less than rosy. Essentially the researchers hypothesized that since attractive people are known to be treated better in many aspects of life, those who were more attractive may get a skewed version of how the world works. Their belief/experience that others were there to help them and going to treat them fairly may cause them to develop a “blind spot” that caused them to believe people didn’t need social programs/welfare/anti-discrimination laws  as much as less attractive people might think.
  2. Three hypotheses were tested Based on that premise, the researchers decided to test three distinct hypotheses. First, that attractive people were more likely to believe things like “my vote matters” and “I can make a difference”, regardless of political party. Second, they asked them about ideology, and third partisanship. I thought that last distinction was interesting, as it drew a distinction between the intellectual undertones and the party affiliation.
  3. Partisans are more attractive than ideologues. To the shock of no one, better looking people were much more likely to believe they would have a voice in the political process, even when controlled for education and income. When it came to ideology vs partisanship though, things got a little interesting. Attractive people were more likely to rate themselves as strong Republicans, but not necessarily as strong conservatives. In fact in the first data set they used (from the years 1972, 1974 and 1976) only one year should any association between conservatism and attractiveness, but all 3 sets showed a strong relationship between being attractive and saying you were a Republican. The later data sets (2004 and 2011) show the same thing, with the OLS coefficient for being conservative about half (around .30) of what the coefficient for Republicanism was (around .60). This struck me as interesting because the first headline I saw specifically said “conservatives” were more attractive, but that actually wasn’t the finding. Slight wording changes matter.
  4. We can’t rule out age cohort effects When I first saw the data sets, I was surprised to see some of the data was almost 40 years old. Then I saw they used data from 2004 and 2011 and felt better. Then I noticed that the 2004 and 2011 data was actually taken from the Wisconsin Longitudinal Study, whose participants were in high school in 1957 and have been interviewed every few years ever since. Based on the age ranges given, the people in this study were born between 1874 and 1954, with the bulk being 1940-1954. While the Wisconsin study controlled for this by using high school yearbook photos rather than current day photos, the fact remains that we only know where the subjects politics ended up (not what they might have been when they were young) and we don’t know if this effect persists in Gen X or millenials. It also seems a little suspect to me that one data set came during the Nixon impeachment era, as strength of Republican partisanship dropped almost a whole point over the course of those 4 years. Then again, I suppose lots of generations could claim a confounder.
  5. Other things still  are higher predictors of affiliation. While overall the study looked at the effect of attractiveness by controlling  for things like age and gender, the authors wanted to note that those other factors actually still played a huge role. The coefficients for the association of Republican leanings with age (1.08) and education (.57) for example  were much higher than attractiveness the coefficient for attractiveness (.33). Affinity for conservative ideology/Republican partisanship was driven by attractiveness (.37/.72) but also by income (.60/.62) being non-white (-.59/-1.55) and age (.99/1.45). Education was a little all over the place…it didn’t have an association with ideology (-.06), but it did with partisanship (.94). In every sample, attractiveness was one of the smallest of the statistically significant associations.

While this study is interesting, I would like to see it replicated with a younger cohort to see if this was a reflection of an era or a persistent trend. Additionally, I would be interested to see some more work around specific beliefs that might support the initial hypothesis that this is about social programs. With the noted difference between partisanship and ideology, it might be hard to hang your hat on an particular belief as the driver.

Regardless, I wouldn’t use it to start a conversation with your Tinder date. Good luck out there.

5 Things About the GLAAD Accelerating Acceptance Report

This past week a reader contacted me to ask what I thought of a recent press release about a poll commissioned by GLAAD for their “Accelerating Acceptance” report. The report struck me as pretty interesting because the headlines mentioned that in 2017 there was a 4 point drop in LGBT acceptance, and I had actually just been discussing a Pew poll that showed a 7 point jump in the support for gay marriage in 2017. 

I was intrigued by this discrepancy, so I decided to take a look at the report (site link here, PDF here), particularly since a few of the articles I read about the whole things seemed a little confused about what it actually said. Here are 5 things I found out:

  1. The GLAAD report bases comfort/acceptance on reaction to seven different scenarios In order to figure out an overall category for each person, respondents were asked how comfortable they’d feel with seven different scenarios. The scenarios were things like “seeing a same sex couple holding hands” or “my child being assigned an LGBT teacher”. Interestingly, respondents were most likely to say they’d be uncomfortable if they found out their child was going to have a lesson in school on LGBT history (37%), and they were least likely to say they’d be uncomfortable if an LGBT person was at their place of worship (24%).
  2. The answers to those questions were used to assign people to a category Three different categories were assigned to people based on the responses they gave to the previous seven questions. “Allies” were respondents who said they’d be comfortable in all 7 situations. “Resisters” were those who said they’d be uncomfortable in all 7 situations. “Detached supporters” were those whose answers varied depending on the situation.
  3. It’s the “detached supporter” category that gained people this year. So this is where things got interesting. Every single question I mentioned in #1 saw an increase in the “uncomfortables” this year, all by 2-3%. While  that’s right at the margin of error for a survey this size (about 2,000 people), the fact that every single one went up by a similar amount give some credence to the idea that it’s an uptick. To compound that point, this was not driven by an uptick of people responding they were uncomfortable in every situation, but actually more people saying they were uncomfortable in some situations but not others:
  4. The percent of gay people reporting discrimination has gone up quite a bit. Given the headlines, you’d think the biggest finding of this study would be the drop in the number of allies for LGBT people, but I actually thought the most striking finding was the number of LGBT people who said they had experienced discrimination. That went from 44% in 2016 to 55% in 2017, which was a bigger jump than other groups: That red box there is the only question I ended up with. Why is the 27% so small? Given that I saw no other axis/scale issues in the report, I wondered if that was a typo. Not the biggest deal, but curiosity inducing nonetheless.
  5. Support for equal rights stayed steady For all the other findings, it was interesting to note that 79% of people continue to say they support equal rights for LGBT people. This number has not changed.

So overall, what’s going on here? Why is support for gay marriage going up, support for equal rights unchanged, but discrimination reports going up and individual comfort going down? I have a few thoughts.

First, for the overall “comfort” numbers, it is possible that this is just a general margin of error blip. The GLAAD survey only has 4 years of data, so it’s possible that this is an uptick with no trend attached. Pew Research has been tracking attitudes about gay marriage for almost 20 years, and they show a few years where a data point reversed the trend, only to change the next year. A perfectly linear trend is unlikely.

Second, in a tense political year, it is possible that different types of people pick up the phone to answer survey questions. If people are reporting similar or increased levels of support for concrete things (like legal rights) but slightly lower levels of comfort around people themselves, that may be a reflection of the polarized nature of many of our current political discussions. I know my political views haven’t changed much in the past 18 months, but my level of comfort around quite a few people I know has.

Third, there very well could be a change in attitudes going on here. One data point does not make a trend, but every trend starts with a data point. I’d particularly be interested in drilling in to those discrimination numbers to see what types of discrimination were on the uptick. Additionally, the summary report mentions that they’ve changed some of the wording (back in 2016) to make it clearer that they were asking about both LGB and T folks, which makes me wonder if the discrimination is different between those two groups. I wasn’t clear from the summary if they had separate answers for each or if they just mentioned each group specifically, so I could be wrong about what data they have here.

Regardless, the survey for next year should shed some light on the topic.

5 Things About the Perfect Age

When people ask me to explain why I got degrees in both family therapy and statistics, my go to answer is generally that “I like to think about how numbers make people feel.” Given this, I was extremely interested to see this article in the Wall Street Journal this weekend, about researchers who are trying to figure out what people consider the “perfect” age.

I love this article because it’s the intersection of so many things I could talk about for hours: perception, biases, numbers, self-reporting, human development, and a heavy dose of self-reflection to boot.

While the researchers haven’t found any one perfect age, they do have a lot of thought provoking commentary:

  1. The perfect age depends on your definition of perfect Some people pick the year they had the most opportunities, some the year they had the most friends, some the years they had the most time, others the year they were the happiest, and other the years they had a lot to reflect on. Unsurprisingly, different definitions lead to different results.
  2. Time makes a difference Unsurprisingly, young people (college students) tend to say if they could freeze themselves at one age, it would be sometime in their 20s. Older people on the other hand name older ages….50 seems pretty popular. This makes sense as I suspect most people who have kids would pick to freeze themselves at a point where those kids were around
  3. Anxiety is concentrated to a few decades One of the more interesting findings was that worry and anxiety were actually most present between 20 and 50.  After 50, well-being actually climbed until age 70 or so. The thought is that generally that’s when the kids leave home and people start to have more time on their hands, but before the brunt of major health problems hits.
  4. Fun is also concentrated at the beginning and end of the curve Apparently people in the 65 to 74 age range report having the most fun of any age range, with 35 to 54 year olds having the least. It’s interesting that we often think of young people as having the “fun” advantage due to youth and beauty, but apparently the “confusion about life” piece plays a big part in limiting how fun those ages feel. Sounds about right.
  5. How stressed you are in one decade might dictate how happy you are in the next one This is just me editorializing, but all of this research really makes me wonder how our stress in one decade impacts the other decades. For example, many parents find the years of raising small children rather stressful and draining, but that investment may pay off later when their kids are grown. Similar things are true of work and other “life building” activities. Conversely, current studies show that men in their 20s who aren’t working report more happiness than those in their cohort who are working….but one suspects by age 40 that trend may have reversed. You never know what life will throw at you, but even the best planned lives don’t get their highs without some work.

Of course after thinking about all this, I had to wonder what my perfect age would be. I honestly couldn’t come up with a good answer to this at the moment, especially based on what I was reading. 50 seems pretty promising, but of course there’s a lot of variation possible between now and then. Regardless, a good example of quickly shifting opinions, and how a little perspective tweak can make a difference.

5 Things to Know About Hot Drinks and Esophageal Cancer

Fun fact: according to CNN, on New Year’s Day 90% of the US never got above freezing.

Second fun fact: on my way in to work this morning I passed an enormous fire burning a couple hundred yards from where the train runs. I Googled it to see what was happened and discovered it was a gas main that caught on fire, and they realized that shutting the gas off (normal procedure I assume) would have made thousands of people in the area lose heat. With temps hitting -6F, they couldn’t justify the damage so they let the fire burn for two days while they figured out another way of putting it out.

In other words, it’s cooooooooooold out there.

With a record cold snap on our hands and the worst yet to come this weekend, I’ve been spending a lot of time warming up. This means a lot of hot tea and hot coffee have been consumed, which reminded me of a factoid I’d heard a few months ago but never looked in to. Someone had told me that drinking hot beverages was a risk factor for esophageal cancer, but when pressed they couldn’t tell me what was meant by “hot” or how big the risk was. I figured this was as good a time as any to look it up, though I was pretty sure nothing I read was going to change my behavior. Here’s what I found:

  1. Hot means HOT When I first heard the hot beverage/cancer link, my first thought was about my morning coffee. However, I probably don’t have to worry much. The official World Health Organization recommendation is to avoid drinking beverages that are over 149 degrees F. In case you’re curious, Starbucks typically servers coffee at 145-165 degrees, and most of us would wait for it to cool for a minute before we drank it.
  2. Temperature has a better correlation with cancer than beverage type So why was anyone looking at beverage temperature as a possibly carcinogen to begin with? Seems a little odd, right? Well it turns out most of these studies were done in part to rule out that it was the beverage itself that was causing cancer. For example, quite a few of the initial studies noted that people who drank traditional Yerba Mate had higher esophageal cancer rates than those who didn’t. The obvious hypothesis was that it was the Yerba Mate  itself that was causing cancer, but then they noted that repeated thermal injury due to scalding tea was also a possibility. By separating correlation and causation, it was determined that those who drink Yerba Mate (or coffee or other tea) at lower temperatures did not appear to have higher rates of esophageal cancer. Nice work guys.
  3. The risk has been noted in both directions So how big a risk are we looking at? A pretty sizable one actually. This article reports that hot tea drinkers are 8 times as likely to get esophageal cancer as those who drink tea at lower temperatures, and those who have esophageal cancer are twice as likely to say they drank their tea hot before they got cancer. When assessing risk, knowing both those numbers is important to establish a strong link.
  4. The incidence rate seems to be higher in countries that like their beverages hot It’s interesting to note that the US does not even come close to having the highest esophageal cancer rates in the world. Whereas our rate is about 4.2 per 100,000 people, countries like  Malawi have rates of 24.2 per 100,000 people. Many of the countries that have high rates have traditions of drinking scalding hot beverages, and it’s thought that combining that with other risk factors (smoking, alcohol consumption, poverty and poorly developed health care systems) could have a compounding effect. It’s not clear if scalding your throat is a risk in and of itself or if it just makes you more susceptible to other risks, but either way it doesn’t seem to help.
  5. There is an optimum drinking temperature According to this paper, to minimize your risk while maximizing your enjoyment, you should serve your hot beverages at exactly 136 degrees F. Of course a lot of that has to do with how quickly you’ll drink it and what the ambient temperature is. I was pretty impressed with my Contigo thermos for keeping my coffee pretty hot during my 1.5 mile walk from the train station in -3 degrees F this morning, but lesser travel mugs might have had a problem with that. Interestingly I couldn’t find a good calculator to track how fast your beverage will cool under various conditions, but if you find one send it my way!

Of course if you really want to cool a drink down quickly, just move to Fairbanks, Alaska and throw it in the air:

Stay warm everyone!

5 Interesting Resources for Snowflake Math Lessons

Happy National Make a Paper Snowflake Day (or National Make Cut Out Snowflakes Day for the purists)!

I don’t remember why I stumbled on this holiday this year, but I thought it would be a really good time to remind everyone that snowflakes are a pretty cool (no pun intended) basis for a math lesson. My sister-in-law teaches high school math and informs me that this is an excellent thing to give kids to do right before winter break. I’m probably a little late for that, but just in case you’re looking for some resources, here are some good ones I’ve found:

  1. Khan Academy Math for Fun and Glory  If you ever thought the problem with snowflake cutting is that it wasn’t technical enough, then this short video is for you. Part of a bigger series that is pretty fun to work through, this video is a great intro to how to cut a mathematically/anatomically(?) correct snowflake.
  2. Computer snowflake models There’s some interesting science behind computer snowflake models, and this site takes you through some of the most advanced programs for doing so. It seems like a fun exercise, but apparently modeling crystal growth has some pretty interesting applications. Gallery of images here, and an overview of the mathematical models here.
  3. Uniqueness of snowflakes Back in the real world, there’s an interesting and raging debate over the whole “no two snowflakes are alike” thing. According to this article,  “Yes, with a caution”, “Likely but unprovable” or “it depends on what you mean by unique” are all acceptable answers.
  4. Online snowflake maker If you’re desperate to try out some of the math lessons you just learned but can’t find your scissors, this online snowflake generator has you covered.
  5. Other winter math If you’re still looking for more ideas, check out this list of winter related math activities. In addition to snowflake lessons around symmetry, patterns and Koch snowflakes, they have penguin and snowman math.

Happy shoveling!