I’ve gotten a few links lately on the topic of “observer effects”/confirmation biases, basically the idea that the process of observing a phenomena can actually influence the phenomena you’re observing. This is an interesting issue to grapple with, and there’s a lot of misconceptions out there, so it seemed about right for a blog post.

First up, we have a paper on the Hawthorne effect. The Hawthorne effect was originally a study done on factory workers (in the Hawthorne factory) in order to see how varying their working conditions  improved their productivity. What the researchers found was that changing basically anything in the factory work environment ended up changing worker productivity. This was so surprising it ended up being dubbed “the Hawthorne effect”. But was it real?

Well, likely yes, but the initial data was not nearly as interesting as reported. For several decades it appeared to have been lost entirely, but it was found again back in 2011. The results were published here, and it turns out most of the initial effect was due to the fact that all the lighting conditions were changed over the weekend, and the productivity was measured on Monday. No effort was made to separate the “had a day off” effect from the effect of varying the conditions, so the 2011 paper attempted to do that. They found subtle differences, but nothing as large as originally reported. The authors state they believe the effect is probably real, but not as dramatic as often explained.

Next up, we have this blog post that summarizes the controversy over the “Pygmalion effect“. (h/t Assistant Village Idiot). This another pretty famous study that showed that when teachers believed they were teaching high IQ children, the children’s actual IQs ended up going up. Or did they? It turns out there’s a lot of controversy over this one, and like the Hawthorne effect paper the legend around the study may have outpaced its actual findings. The criticisms were summed up in this meta-analysis from 2005:

1. Self-fulfilling prophecies in the classroom do occur, but these effects are typically small, they do not accumulate greatly across perceivers or over time, and they may be more likely to dissipate than accumulate
2. Powerful self-fulfilling prophecies may selectively occur among students from stigmatized social groups
3. Whether self-fulfilling prophecies affect intelligence, and whether they in general do more harm than good, remains unclear
4. Teacher expectations may predict student outcomes more because these expectations are accurate than because they are self-fulfilling.

I find the criticisms of both studies interesting not because I think either effect is completely wrong, but because these two studies are so widely taught as definitively right. I double checked the two psych textbooks I have laying around and both mention these studies positively, with no mention of controversy. Interestingly, the Wikipedia pages for both go in to the concerns….score one for editing in real time.

Finally, here’s an observation effect study I haven’t seen any criticism of that has me intrigued “Mind Over Milkshakes: Mindsets, Not Just Nutrients, Determine Ghrelin Response” (h/t Carbsane in this post). For this study, the researchers gave people a 380 calorie milkshake two weeks in a row, and measured their ghrelin response to it. The catch? In one case it was labeled as a 620 calorie “indulgent” milkshake, and in the other case it was labeled as a 120 calorie “sensible” shake. The ghrelin responses are seen below:

This is a pretty brilliant test, as everyone served as their own control group. Each person got each shake once, and it was the same shake in each case. Not sure how large the resulting impact on appetite would be, but it’s an interesting finding regardless.

Overall I think the entire subject of how viewing things can change reality is rather fascinating. For the ghrelin example in particular, it’s interesting to see how a hormone none of us could consciously manipulate can still be manipulated by our expectations. It’s also interesting to see the limitations of what can be manipulated. For the Pygmalion effect, it’s found that if the teachers know the kids for at least 2 weeks prior to getting IQ information, there is actually no effect whatsoever. Familiarity appears to breed accurate assessments I suppose. All of this seems to point to the idea that observation does something, but the magnitude of the change may not be easy to predict. Things to ponder.