5 Things

5 Things About Ambiverts

/ bs king / 1 Comment

Okay, so after writing 5 Things about Introverts and 5 Things About Extroverts, it has come time for me to talk about MY people: the ambiverts. Sometimes referred to as an introverted extrovert or an extroverted introvert, ambiverts are the people who don’t really fit either mold.  So what’s up with this category? Is it a real thing? If it is real is it a good thing? Let’s take a look!

  1. Ambiversion has been around for a while Okay, so when I first heard about ambiversion, I thought it was a made up thing. Apparently though Carl Jung actually did write about this category when he originally developed the introvert/extrovert scale, though he didn’t name it. According to the Wall Street Journal, the name came about in the 1940s. And to think, I was just blaming Buzzfeed.
  2. Most people are probably ambiverts If you think of introversion and extroversion as a spectrum of traits, ambiverts are the ones in the middle. It makes sense that most people would be there, though the exact percentage is a little in question: some say 1/3rd of all people, some say 2/3rds. The exact percentage is probably in question because it depends where you draw the line. If you’re 40-60% extroverted, does that make you an ambivert, or is it 35-65%? Regardless, it’s probably not a small number.
  3. The Big 5 recognizes them, Myers Briggs not so much One of the reasons ambiversion doesn’t get much press is because Myers Briggs (the 500 lb gorilla in the personality testing room) doesn’t really recognize it. Where the Big 5 Personality Scale is based on a sliding scale and generally recognizes “low” “moderate” and “high” scores, Myers Briggs insists on binary classifications.
  4. The ability to recognize both sides is probably helpful Not a lot of research has been done in to ambiversion, but the little that has been done suggests good things. When studying salespeople, it was found that ambiverts actually made more money than either introverts or extroverts. The researchers think this is because they can work with both types of people and adapt their style more easily to fit the customer. Obviously there would still be a social intelligence aspect to this, but the ability to vary the approach does seem to have it’s benefits.
  5. The need for both types of recharging can lead to burnout In my previous posts, I asserted that introverts want people to pay more attention to their strengths, and extroverts want people to pay less attention to their faults. Reading through the things written about ambiverts, I realized that their biggest problem seemed to be paying attention to themselves. If you know you need quiet to recharge, that’s straightforward. If you know you need noise, that’s also straightforward. However, if it kind of depends, you have to make a judgment call…..and you very well could be wrong. A lot.

So there you have it! Research in this area is clearly a little light, but I still think it’s interesting to think about how we classify these things. Also, fun fact I learned after writing this….there apparently is an introverted, ambivert and an extroverted facial type:

The article was a little unclear on how good the correlation between facial structure and actual personality type was, but it did raise some questions about the chicken and egg nature of how others perceive us. If someone looks like an extrovert are they more likely to be treated like one and therefore become one? Or is there some “extrovert gene” that determines both? Since all introversion/extroversion measures are self reported it’s hard to know, but it’s an interesting thought. Now I’m gonna go look in the mirror and figure out which type of face I have.

5 Things About Extroverts

/ bs king / 3 Comments

Last week I gave a run down of all the interesting stuff I found out about introverts, so naturally this week is going to be about extroverts. Since extroverts are the opposite of introverts, much of what I said last week still applies (or applies in reverse): extroverts tend to need more stimulation from their environment. While this is often phrased as “they get their energy from people”, that’s not entirely true. Being extroverted does not mean social interaction trumps sleep, food, water, or that you can’t get sick of people (all things I’ve heard people claim). So what is true of extroverts? Let’s take a look:

  1. I’ve been spelling “extrovert” wrong, and apparently Jung would be annoyed Before I wrote my post last week, I tried to look up “extrovert vs extravert” to see which was the correct spelling. It turns out that the debate about this runs a little deeper than I thought. While my spellchecker insists that “extrovert” is correct, Carl Jung (the guy who invented the whole concept) felt strongly it should be “extravert”. This was based on the Latin root and the actual definition he was going for. I’m going to stick with the one that makes my spell checker calmer, but it’s worth noting that we probably should be using “extravert”.
  2. There may be two types of extroverts Just like introversion, it turns out extroversion may not be a monolith. The two types (agenetic and affiliative) are described here, but basically they boil down to “social leadership” and “social warmth”. The first one has a lot to do with going after rewards, and the second one just wants to hang out with everyone. They are correlated, but some people have more of one than the other. Think the person who wants to be in charge of every group vs the person who just wants to be in every group.
  3. The success of extroverts is kinda bimodal Despite all the rumors that being an extrovert is some sort of cultural ideal, it turns out it’s actually kind of a mixed bag. For example, if you go to Urban Dictionary and type in “introvert” and you get a thoughtful description of what an introvert is. Try the same with the word “extrovert” and you get “asshole who doesn’t know how to shut their goddamn mouth“.  I’m serious. In fact 5 out of the top 7 definitions of extroverts slam extroverts. Interestingly, 5 of the top 7 definitions of introverts ALSO slam extroverts. If the chronic complaint from introverts is that their strengths go unnoticed, then the equivalent extrovert complaint would probably be that their faults get a little too noticed. This makes a lot of sense….having attention on you is great if you’re good at something, but probably worse for you if you’re bad at something. Interestingly, this plays out with things like leadership. Leaders are more likely to be extroverts, but if you control for social skills there actually isn’t an extrovert advantage.
  4. Some extrovert “benefits” are just circular reasoning Okay, so here’s the extrovert bias introverts so often complain about. Many of the supposed benefits of being an extrovert come not from actual benefits, but by using some of the definition for extroversion as a definition for other things. For example, for years it was noted that extroverts were happier than introverts. Then it was finally noted that many of the tests that measured happiness did so by asking things like “do you have a lot of friends?”, which is also a question used to determine if you’re an extrovert or not.  This works in the negative direction too. When you hear criticisms of extroverts, it’s often things like “they hog the spotlight” (random example here). However “do you like to be the center of attention” is a pretty frequently used question on personality tests, and it makes complete sense that people who say “yes” to that would end up spending more time as the center of attention than those that say “no”. I think this is important because sometimes I hear this get referenced as though personality tests were objective neurological tests, but they are really all rating and self assessment. The same answers that landed you in one category or another tend to persist even when you’re not taking the test.
  5. Test taking is often biased against them So if personality and psychological tests favor extroverts, then extroverts must really love test taking, right? Well, not all tests. It turns out that our most common testing environments (ie quiet rooms with no ambient noise) actually are biased against extroverts. Because of their need for stimulation, some research has found that extroverts actually perform better on tests when there is noise present. Unsurprisingly, introverts are the opposite, and ambiverts are in the middle: In school settings this is an obvious disadvantage, but in real life may explain why some professions end up extrovert dominated. In many settings, you actually will have to make your toughest calls while there is a lot of noise and chaos around you. By the way, there’s a rather persistent rumor (normally stated in the form of “introverts think more deeply”) that extroverts are less intelligent than introverts. Actually the most recent research says extroverts have a tiny advantage here, but the correlation on that is pretty shaky, and depends heavily on exactly how intelligence is measured.  There’s some suggestion that the high IQ (>160) may lean introvert, but that’s a really small slice of the population and wouldn’t be enough to move the dial.

So there you have it! Next week I may try to take on ambiverts, who can’t make up their mind about anything.

5 Things About Introverts

/ bs king / 3 Comments

I am fascinated by personality testing. Myers-Briggs, Big 5, Enneagram, Buzzfeed quiz, yes please. I’ll take them. There’s something about assigning humanity to little boxes that just, I don’t know, appeals to me. Maybe that’s the ENTJ in me, or my moderate conscientiousness, or the fact that according to this quiz I’m a sea monster. Given this, I realized it was high time I did a bit of a research roundup on some of the better known facets of personality testing. This week I’m taking on introverts, and if all goes well next week will be extroverts.

A few things up front: first, on introvert/extrovert scales, I score right in the middle. This makes me one of the dreaded “ambiverts” who apparently can’t make up their minds. Second, while the definition of introvert is sometimes a little lacking (see point #1 below) it’s generally defined as someone who gets their energy from being alone. With the rise of the internet, introverts started kind of having a moment, and there’s been a rash of books/memes/Buzzfeed lists about how unappreciated they all are. So what’s going on here, and what does the research say?

  1. Introversion doesn’t always have a definition One of the first rather odd things I learned about introverts is that the most commonly used academic  definition is….”not an extrovert”. For example, in the Big Five Personality scale “introversion” is not technically a trait but “low extraversion” is. This may not seem like a big deal, but it can mean that we are lumping different things under “introvert” that may not necessarily be similar to one another. As introversion has become more trendy, I have seen more and more people lump normal social or physical limitations under “introversion”. For example, a rather extroverted friend of mine recently announced she was pretty sure she was actually an introvert. When asked why she thought this, she mentioned that she had been out 3 different nights the week before and that by the weekend she had been too exhausted to go to another party. When I inquired if maybe this was simply lack of sleep, she responded “but extroverts get their energy from people, so I should have been fine!”.  No. People get their energy from rest. Almost no one can substitute human interaction for sleep too often and feel good about it. Wanting to sleep isn’t “introverted” merely because you’re not socializing while you do it.
  2. There may be 4 types of introversion  When psychologists started actually looking in to introversion, they developed a theory that there may actually be 4 types of behavior we’ve been lumping under “introvert”: social introversion, thinking introversion, anxious introversion and restrained introversion. This was a helpful list for me, as I am moderately socially introverted (I prefer small groups), highly introverted in my thinking, but I have very little social anxiety and I’m not very restrained. Thus it makes sense that I strongly resonate with some descriptions of introversion and not others. The social anxiety piece can also be important to recognize as a separate category. I have a few friends who thought they were introverted when they were in high school only to discover that they really just didn’t like their classmates. While most introverts fight the perception that introversion = shyness, it’s probably good to note that most shy or socially anxious people will end up self selecting as introverts.
  3. Stimulation matters The four categories mentioned in #2 are still in the research phase, but there are other ways of looking at introversion as well. Some of the very first literature on introversion (and extroversion) actually defined it as an aversion to (or need for) extra environmental stimulation.   I like this framing a bit better than the social framing, because it includes things like loud noises or fast music or why coffee only helps extroverts (basically it increases your sensitivity to stimulation, which is the last thing most introverts need when they’re trying to get things done). This explains why I’ve occasionally had introverted coworkers complain that I talked to much, even when I was studiously avoiding talking to them, or why an introvert I mentioned this to always has to tell her (extrovert) husband to shut the TV off. Social situations may not be taxing because of social issues, but rather just the stimulation of hearing so many people talk.
  4. This can lead to some judginess With all the recent attention on introverts in the workplace, it’s interesting to note that there’s some evidence that  introverts actually judge extroverts more harshly than the other way around. In some studies done by Florida State University, they found introverted MBA students were more likely to give low marks to extroverted students, recommended they get lower bonuses, and declined to recommend them for promotions. This was true even when they manufactured the scenarios and controlled for performance. The extroverts in the study awarded bonuses/promotions/high marks much more in line with actual performance on the tasks and did not penalize introverts. The researchers hypothesize that due to the stimulation issue (#3) introverts may just have a harder time working with extroverts regardless of their competence. I also have to wonder if there’s a bit of the Tim Tebow Fallacy going on here….with all the press about how extroverts do better in business, many introverts (especially in MBA programs, as these research subjects were) could feel that by marking extroverts down they are balancing the scales a bit. We don’t know how this works in the general population, but it is worth keeping in mind.
  5. Introverts may (wrongly) think they’re the minority  There’s a bit of confusion over what percentage of the population is introverted….which is not particularly surprising when you consider the weird definitions we considered in #1-#3. At this point though, most estimates put it at about 50% (unless you consider “ambivert” a category). So why do introverts tend to feel outnumbered?  Well, it’s a statistical quirk called the majority illusion. Basically, because extroverts are more likely to have lots of friends, people are more likely to be friends with lots of extroverts. This artificially skews the perception of the numbers, and leaves people with the impression that they know more extroverts because there are more extroverts. So introverts, take heart. There are more of you out there than you think.

Come back next week and we’ll take a look at extroverts!

On Outliers, Black Swans, and Statistical Anomolies

/ bs king / 1 Comment

Happy Sunday! Let’s talk about outliers!

Outliers have been coming up a lot for me recently, so I wanted to put together a few of my thoughts on how we treat them in research. In the most technical sense, outliers are normally defined as any data point that is far outside the expected range for a value. Many computer programs (including Minitab and R) automatically define an outlier as a point that lies more than 1.5 times the interquartile range outside the interquartile range as an outlier. Basically any time you look at a data set and say “one of these things is not like the others” you’re probably talking about an outlier.

So how do we handle these? And how should we handle these? Here’s a couple things to consider:

  1. Extreme values are the first thing to go When you’re reviewing a data set and can’t review every value, almost everyone I know starts by looking at the most extreme values. For example, I have a data set I pull occasionally that tells me how long people stayed in the hospital after their transplants. I don’t scrutinize every number, but I do scrutinize every number higher than 60. While occasionally patients stay in the hospital that long, it’s actually equally likely that some sort of data error is occurring. Same thing for any value under 10 days….that’s not really even enough time to get a transplant done. So basically if a typo or import error led to a reasonable value, I probably wouldn’t catch it. Overly high or low values pretty much always lead to more scrutiny.
  2. Is the data plausible? So how do we determine whether an outlier can be discarded? Well the first is to assess if the data point could potentially happen. Sometimes there are typos, data errors, someone flat out misread the question, or someone’s just being obnoxious. An interesting example of implausible data points possibly influencing study results was in Mark Regenerus’ controversial gay parenting study. A few years after the study was released, his initial data set was re-analyzed and it was discovered that he had included at least 9 clear outliers….including one guy who reported he was 8 feet tall, weighed 88 lbs, had been married 8 times and had 8 children. When one of your outcome measures is “number of divorces” and your sample size is 236, including a few points like that could actually change the results. Now, 8 marriages is possible, but given the other data points that accompanied it, they are probably not plausible.
  3. Is the number a black swan? Okay, so lets move out of run of the mill data and in to rare events. How do you decide whether or not to include a rare event in your data set? Well….that’s hard to. There’s quite a bit of controversy recently over black swan type events….rare extremes like war, massive terrorist attacks or other existential threats to humanity. Basically, when looking at your outliers, you have to consider if this is an area where something sudden, unexpected and massive could happen to change the numbers. It is very unlikely that someone in a family stability study could suddenly get married and divorced 1,000 times, but in public health a relatively rare disease can suddenly start spreading more than usual. Nicholas Nassim Taleb is a huge proponent of keeping an eye on data sets that could end up with a black swan type event, and thinking through the ramifications of this.
  4. Purposefully excluding or purposefully including can both be deceptive In the recent Slate Star Codex post “Terrorist vs Chairs“, Scott Alexander has two interesting outlier cases that show exactly how easy it is to go wrong with outliers. The first is to purposefully exclude them. For example, since September 12th, 2001, more people in the US have been killed by falling furniture than by terrorist attacks. However, if you move the start line two days earlier to September 10th, 2001, that ratio completely flips by an order of magnitude. Similarly, if you ask how many people die of the flu each year, the average for the last 100 years is 1,000,000. The average for the last 97 years? 20,000.  Clearly this is where the black swan thing can come back to haunt you.
  5. It depends on how you want to use your information Not all outlier exclusions are deceptive. For example, if you work for the New York City Police Department and want to review your murder rate for the last few decades, it would make sense to exclude the September 11th attacks. Most charts you will see do note that they are making this exclusion. In those cases police forces are trying to look at a trends and outcomes they can affect….and the 9/11 attacks really weren’t either. However, if the NYPD were trying to run numbers that showed future risk to the city, it would be foolish to leave those numbers out of their calculations. While tailoring your approach based on your purpose can open you up to bias, it also can reduce confusion.

Take it away Grover!

5 Possible Issues With Genomics Research

/ bs king / 1 Comment

Ah, fall. A new school year, and a new class on my way to finish this darn degree of mine. This semester I’m taking “Statistical Analysis of Genomics Data”, and the whole first was dedicated to discussing reproducible research. As you can imagine, I was psyched. I’ve talked quite a bit about reproducible research, but genomics data has some twists I hadn’t previously considered.  In addition to all the usual replication issues, here are a few issues that come up when you try to replicate genomics studies:

  1. Different definitions of “raw data” In the paper “Repeatability of published microarray gene expression analyses” John Ioannidis et al attempted to reproduce one figure from 18 different papers that used microarray data. They succeeded on 2 of them. The number one reason for failure to replicate? Not being able to access the raw data that was used. In most cases the data had been deposited (as required by the journal) but it had not really been reviewed to see if it was just summary data or even particularly identifiable. Six out of 18 research groups had deposited data that you couldn’t even attempt to use, and other groups had data so raw it was basically useless. Makes me shudder just to think about it.
  2. Large and unwieldy data files Even in papers where the data was available, it was not always useable. Ioannidis et al had trouble reproducing the about 8 papers due to unclear data decisions. Essentially the files and data were there, but they couldn’t figure out how someone actually had waded through them to produce the results they got. To give you a picture of how big these data files are, my first homework for this class required a “practice” file that was 20689×37….or almost 800,000 data points. Unless that data is very well labeled, you will have trouble recreating what someone else did.
  3. Non-reproducible workflow Anyone who’s ever attempted to tame an unweildy data set knows it’s a trek and a half. I swear to god I have actually emerged from my office sweating after one of those bouts. That’s not so terrible, but what can kick it to the seventh circle of hell is finding out there was an error in the data set and now you have to redo the whole thing. In 8 of the papers Ioannidis et al looked at, they couldn’t figure out what the authors actually did to generate their figures. Turns out, sometimes authors can’t figure out what they did to generate their figures….which is why we end up with videos like this:

    All that copy/pasting and messing around is just ASKING for an error.

  4. Software version changes Another non-glamorous way things can get screwed up: you update your software part way through and the original stuff you wrote gets glitchy. This is an enormous headache if you notice it, and a huge issue if you don’t. 2 of the papers Ioannidis et al looked at didn’t include software version and couldn’t be reproduced. R is the most commonly used software for things like this and it’s open source, so updates aren’t always compatible with each other.
  5. Excel issues Okay, so you loaded your data, made a reproducible workflow, figured out your version of R, and now you are awesome right? Not necessarily. It turns out that Excel, one of the most standard computer programs on the planet, can seriously screw you up. In a recent paper, it was discovered that 20% of all genomics papers with Excel data files had inadvertently converted gene names to either dates or floating point numbers. This almost certainly means those renamed genes didn’t end up being included in the final research, but what effect that had is unknown. Sadly, the rate of this error is actually increasing by about 15%. Oof.

I am tempted to summarize all this by saying “Mo’ Data Mo’ Problems”, but…..no, actually, that sounds about right. Any time you can’t actually personally review all the data, you are putting your faith in computer systems and the organization of the files. Good organization is key, and it’s hard to focus on that when you’re wading through data files. Semper vigilans.

5 Fun Intersections of Math and Birthdays

/ bs king / 1 Comment

Well hi there! Guess what? Today’s my birthday! Hurray for another trip around the sun! This of course reminded me of all the great birthday related math things that are out there, so I thought I’d go ahead and put a few together just for kicks:

  1. The birthday paradox A classic problem in statistics, the birthday paradox asks some form of the question “If you have 23 people in a room, what are the chances at least two of them have the same birthday?”. The answer is 50%, and it’s not really a paradox but just a thing people have trouble understanding. Better Explained has a nice breakdown of the problem here, which reminds readers that exponents are hard and that part of you immediately focused on your own birthday. As always, the chances of something happening somewhere are higher than any particular thing happening to you. My favorite way of viewing this problem came from the Assistant Village Idiot, who explains it by asking people to imagine they’re throwing darts randomly at squares and inquiring how long  they think it would take before two darts wind up in the same square.
  2. Cheryl’s birthday I love when math puzzles go viral, and Cheryl’s birthday was a pretty good one. If you missed it, here it is: And here is the Guardian’s explanation of the answer.
  3. Common birthdays This graphic of common birthdays is both an interesting infographic and a cautionary tale of using ordinal data on the uninitiated. Basically, the author put together a visual representation/heat map of the most common birthdays by rank (as in 1-366), had it go viral, then had people complaining to him that it “wasn’t accurate”.  He was rightfully irritated since it was just something he’d done for fun, but it’s a good reminder to fully think through visuals you see on the internet and to read the original sources for proper context.
  4. When is the old/young tipping point? Well, if you define “old” vs “young” as “when is over half the global population younger than me”, the 538 says the tipping point is somewhere in your late 20s. If you’re limiting yourself to just the US though, you have until you’re 37. Nathan Yau has a great visual here, and you can break it down by gender.
  5. And of course, one of the most compelling statistical truths of all time: 

Can’t argue with that one. I’m gonna go have some cake.

5 Interesting Things Research Tells Us About Internet Trolls

/ bs king

I got in an interesting discussion this weekend with some folks about internet trolls. One person had made an offhanded comment about “anonymity bringing out the worst in people”, and was surprised when I informed them that the current research didn’t really support them in that. Depending on your perspective I am either the best or worst person to ever get in one of these discussions with, because I decided to do a little roundup of the current research on internet trolls. Hang in there with me, and you get a bonus SHEEPLE at the end:

  1. Defining trolling is actually kind of hard While most of us would say that trolling is a sort of “you know it when you see it” issue, the definitions used actually vary a bit. For example this study found trolls by asking participants directly “do you like to troll”, this study just counted Tweets that contained “bad words”, this study had researchers read through individual posts and had rank their “trollishness”, and this study had researchers track whole comment histories of banned forum users. None of those are necessarily wrong, but they all will catch slightly different sets of behavior and groups of commenters.
  2. Trolls who cause chaos online also like to cause chaos for researchers.  To the surprise of no one, those who admit they like to troll online also get a kick out of messing with researchers. When Whitney Phillips was writing her book about trolling, she tried to interview self professed trolls to see what motivated them. Unfortunately they kept making up stories then hanging up on her. That makes you wonder about research where people had to self define as trolls, like this widely reported study that said trolls tend to be sadists. Are trolls really sadists, or do those who say “yes I like to troll” also like to answer “yes” to questions on sadism quizzes? And is answering “yes” as a joke substantively different from answering “yes” in all seriousness?
  3. Who gets targeted is a complicated question One of the issues that arises due to the different definitions of trolls (see #1) is the question of “who gets targeted”. At this point “trolling” can be used to define anything from irritating but benign behavior to criticism of all types to abuse, threats and harassment. With so many varieties of trolling, figuring out who the targets are can be more difficult than it first appears. For example, this British marketing group found that male celebrities got twice the Twitter harassment as female celebrities. To note, the standard for “harassment” used there was a “bad word” filter, and the number or content of the celebrities Tweets were not rated. Given that Piers Morgan, Ricky Gervais, and Katie Hopkins ended up as the top three receivers of abuse, content appears to matter. Anyway, Cathy Young has this to say about the gender breakdown and Amanda Hess replied with this. We do know that young people (18-24) are the most likely to have problems, and there is a gender difference in type of harassment. From Pew Research:This is all age ranges:  Note: all of those terms were self defined and self reported, and there was no controlling for where those things occurred. In other words, people being called offensive names out of the blue in an innocuous situation were counted the same as someone calling you a name in the middle of a heated debate.
  4. Real names don’t necessarily help. Nearly as long as trolls have been discussed, people have been mentioning the enabling role of anonymity. A recent study suggests that may be less important than we think. A recent study of German social media showed that using real names frequently made people more hostile, not less. It turns out that the social signaling/credibility gained from online posts actually can empower people to get meaner. Oh boy.
  5. Controlling your own emotions might actually help The most common advice dispensed on this whole topic is of course “don’t feed the trolls”.  However, it can be a little tough figuring out what that means. When these researchers tried to create a predictive algorithm to see if they could identify trolls by their first ten posts on a site, they discovered that trolls tend to escalate when they have posts unfairly deleted. In order to find “unfair” deletions, they blinded an assessor to the identity of the poster, and asked them if it was offensive or not. It turns out that trolls really were more likely to have inoffensive posts deleted, and that their postings worsened significantly after that happened. Now of course this may have been the goal….moderators who are sick of someones posts entirely may get capricious with the hopes that they’ll get so mad they’ll leave, but it is an interesting insight. Also interesting from the paper: trolls comment more often but in fewer threads, they have worse overall writing quality, and they get more responses than other users. Yup, designed to irritate.

Unfortunately none of my research turned up any guidance on how likely this is to happen:

Stay safe out there.

5 Examples of Bimodal Distributions (None of Which Are Human Height)

/ bs king / 1 Comment

Of all the strange things about statistics education in the US (and other countries for all I know) is the way we teach kids about the bimodal distribution. A bimodal distribution is a set of data that has two peaks (modes) that are at least as far apart as the sum of the standard deviations. It looks like this:

CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=641362

It’s an important distribution to know about, because if your data looks like this, your calculations for the average are going to be totally useless. For the distribution above for example, we’d get an average of (around) zero, which would tell us nearly nothing about the data itself, and would completely miss both peaks. So far so good. However, when this is taught in stats classes, the “real world” example most kids are given is human height….and human height is not bimodal. Bummer.

Given that it’s the start of the school year and all, I thought it would be a good time to provide teachers with some new examples. Now, depending on the underlying data set you might use, some of these examples may not make the “peaks separated by the length of the combined standard deviations” cutoff either…..but at least you’ll be wrong in new ways. That’s got to count for something, right?

  1. Starting salaries for lawyers On average new lawyers do well. In reality there are big winners and losers in the whole “getting a good job after graduation” game, and it shows in the salary distributions. Read the Above The Law complaint here.
  2. Book prices Book prices cluster around different price points, depending on whether your looking at paperbacks or hardcovers as God Plays Dice explains. If the gap between paperback and hardcovers isn’t wide enough for you, imagine you could pull price data for every book available on Amazon.com. You’d end up with a two modes, one for regular books and one for textbooks.
  3. Peak restaurant hours If you plotted a histogram of when every customer entered a restaurant on a given day, you’d end up with a bimodal distribution around 2 points: lunch and dinner. This type of histogram also tends to appear when you map road usage (morning and afternoon rush hours) and residential water/electricity usage (before and after work).
  4. Speed limits This one I actually couldn’t find much data on, but I’m guessing if you mapped out all the speed limits on every mile of road in the US (or maybe just your state), your distribution would end up clustered around 30/35 and then again around 60/65. Basically highways or regular roads. This distribution would also have the additional wrinkle of skewing differently based on whether we used miles of road or number of roads, but that’s a different matter entirely.
  5. Disease patterns There’s a rather fascinating two part blog post  by Jules J Berman that discusses bimodal cancer patters here and here. Basically these are cancers that appear similar but tend to hit rather different ages groups. For example Karposi’s sarcoma hits young men with AIDS and older men who do not have AIDS, and Berman argues that seeing these patterns should give us important clues about the diseases themselves. Possible explanations from Berman’s post:  1. Multiple environmental causes targeting different ages 2. Multiple genetic causes with different latencies 3. Multiple diseases classified under one name 4. Faulty or insufficient data 5. Combinations of 1,2,3 and 4.

Bimodal distributions are also a great reason why the number one rule of data analysis is to ALWAYS take a quick look at a graph of your data before you do anything. As you can see from the above examples, the peaks almost always contain their own important sets of information, and must be understood both separately and together in order be understood at all.

So what’s your favorite non-human height example?

5 Things I’ve Learned From Reading About Problems in Physics

/ bs king

One of my favorite things about getting an engineering degree was the amount of basic science classes I had to take. It gave me at least a dilettante’s knowledge of quite a few scientific fields, and I’ve always enjoyed using that background to keep at least half an eye on other scientific fields. Of all of those fields, my particular favorite is physics. I always loved physics in that “I’m so glad to see you, but let’s just be friends” kind of way, and I try to make sure I read at least a book or two a year about it.

A few months ago I read Lee Smolin’s book “The Trouble With Physics“, and was intrigued to read a breakdown of some of the current (well, ten years ago now) problems in the field. It got me pretty stressed out about string theory, which is not a problem I had expected to have that week. I digress. Anyway, this physics anxiety got a little worse when James over at I Don’t Know But posted about how physics needed some new ideas, and then he left me this link about the rather embarrassing 750 GeV diphoton excess incident. He compared the whole debacle to priming studies, which seemed fair. Anyway, since blogging is the primary way I deal with my science and statistics related anxiety problems, I thought I’d put together a post on why I actually love reading about issues in physics.  Ready? Let’s go!

  1. Reading outside your field gives you a new perspective on errors  Most of my working experience is in healthcare, and one of my degrees is in a psych field. When you’re familiar enough with your field, it can be pretty easy to figure out what all the most common errors are. Since professions tend to attract people who think similarly, it stands to reason that fields will all have certain errors they are particularly susceptible to. Reading outside your normal field is a good way of realizing what problems are actually pretty universal, which ones you may never have thought of, and (ideally!) how other fields have dealt with some problems. Additionally, it’s really easy to see the issues in fields that tend to capture headlines (psych, nutrition, etc), while other fields that are less accessible can seem like they don’t have any problems. Reminding yourself this isn’t true is kind of reassuring.
  2. Statistical noise is a problem for everyone One of the reasons I went in to statistics in the first place was the allure of how many different fields had to use it. At the time I loved the idea of learning a topic that basically every single discipline had to use. I still do. The link James mentioned originally was about a topic I won’t even pretend I can explain (750 GeV diphoton excess) but focused on a problem I’m REALLY familiar with: over-interpertation of statistical noise. Yeah, basically theoretical physicists published about 500 papers on a phenomena that appeared to be true but then didn’t replicate.  Oops. In their defense though, it was a really large anomaly in an area that was theoretically plausible and that they’d had success with before, which is pretty much the perfect storm for confirmation bias.
  3. So is failing to check basic assumptions. If I had to make a complaint about the way we teach  statistics to kids, I would argue that the biggest error we make is not emphasizing to them how important it is to check basic assumptions. Textbooks are always reminding you that you have to make sure assumptions x, y and z hold true before you can use certain equations….then they just let you assume all those things for the rest of the class and send you on your merry way. The real world doesn’t work like this.  That was evident back in May when a couple of retraction notices came out from the New Journal of Physics. There was no intentional misconduct, but the authors had assumed the data was symmetrical without checking that assumption. In Smolin’s book, he discusses a few fundamental string theory assumptions (mentioned in the second column on page 2 of this review) that didn’t actually have experimental evidence behind them, despite most people assuming otherwise.
  4. The goal is to push the limits. In my priming studies post, I mentioned that pushing the limits and studying the fringes of a field is a feature, not a bug. That sentiment is echoed in this interesting article about “The Data That Threatened to Break Physics“. It discusses the struggle of a researcher to cope with completely unexpected results that run contrary to conventional wisdom. In the case of superluminal neutrinos, the results turned out to be the fault of a faulty cable, but the lead researcher quite rightly asks what people thought he should have done differently. Suppressing a potentially controversial result is not really something we want to encourage, and the upshot of that may be that we end up with retractions. To quote the lead researcher: “The worst data are better than the best theory. If you look for reasonable results, you would never make a discovery, or at least you will never make an unexpected discovery”.
  5. Even when you stop studying people, you can’t get out of dealing with people. At it’s heart, science is as much about bias management as it is about discovery. It is really difficult to do much of the latter if you don’t do the former. In Smolin’s book, two of the most fascinating chapters were “How Do You Fight Sociology?” and “How Science Really Works” (covered a bit in this review). Smolin reviews how tenure, grant related politics and even just plain old ego and groupthink can influence what scientific theories get money and attention. All this occurs without any outside social pressure, since of course it doesn’t matter to most lay people if string theory is true or not. Smolin proposes that to counter this, universities should reserve some money/positions for those who are actually quite polarizing in their work. He proposes that we invest in scientific ideas like many stock market investors work: put most of your money in safe things, but put some of it on ideas that look a little nuts. Nicholas Nassim Taleb famously calls this “the black swan approach”. I’ve heard worse ideas.

So there you have it, and if you have any good physics book recommendations, I’m always looking!

 

5 Interesting Reasons Priming Studies Go Wrong

/ bs king

Last week, commenter Christopher B left an interesting comment on my post about masculinity threats and voting that made me realize I wanted to do a bigger post on priming studies in general. Priming studies have come under a lot of fire in the past few years, and they have the unfortunate distinction of being called (by some) the “poster child for doubts about the integrity of psychological research“. So what’s going on here? What are these studies and why do they go wrong so often?

Well, as Christopher B pointed out, it’s not because priming isn’t a thing. Priming is typically defined as “an implicit memory effect in which exposure to one stimulus (i.e., perceptual pattern) influences the response to another stimulus“. In other words, something you see or do at one point unconsciously biases you to act differently at a later date. Some of these could be pretty straightforward. If you see a list of words that containing the word “dog” and then someone asks you to name an animal that starts with the letter w, you will probably be more likely to say “wolf” than “walrus”. Lots of marketers attempt to use priming-like effects to get people to buy more or differently than they would have otherwise. There’s even some efforts to see if getting alcoholics to physically (well, in the form of video games) practice pushing away drinks helps lead to lower rates of relapse. I think most of us would accept that your brain does have a bit of an auto-suggest type system, and most people would accept it can probably be manipulated subtly. So where’s the problem? Well, in addition to the p-value and replication issues I’ve raised before, here’s some other reasons things have gone haywire:

  1. A lot of work gets done at the edges The examples I’ve given above are pretty straightforward, much more straightforward than most of the priming studies that get attention. It’s unsurprising that most researchers aren’t as interested in obvious and straightforward effects, but rather increasingly subtle and indirect effects. For example, in the study I talked about last week, the researchers didn’t ask men to consider a world where women reigned supreme, but rather asked “who makes more money, you or your wife?” The effects they’re interested in are subtle and subconscious, and obviously there’s a limit to how far that can be stretched. Finding that limit is part of the goal. Unfortunately, the edges of any phenomena are going to be those most susceptible to signal and noise problems, and priming researchers got in the habit of casting a broad net at the edges of their conceptual field. Let’s just say that if your field ends up lending itself to parody this pointed, you may want to take a step back.
  2. Primes themselves are subject to bias There’s a great paper on priming studies out of Stanford called “Why many priming results don’t (and won’t) replicate: A quantitative analysis” that points out a lot of logistical reasons priming studies don’t work. One of the more interesting issues they raise is that it’s really freaking hard to actually establish how strong a prime is, and the choices are made by things that are obvious to the researcher, not necessarily the subjects. For example, the most famous priming study primed undergrads with words associated with the elderly like “Florida” or “sentimental”. The authors of the quantitative analysis paper pointed out that the frequency of those words being associated with “elderly people” has actually been decreasing in the past several decades. So basically things that will be “obvious” associations to a 40 year old researcher may not be as obvious to their 18 year old students. To give a more run of the mill example of this, think of celebrity names. If I ask you to name an actor whose first name is “Alan”, many baby boomers might say “Alda”, whereas younger  Harry Potter fans may say “Rickman”. This issue also explains why these studies don’t tend to replicate in other languages.
  3. Age of subjects matters In addition to the word choice bias, there’s some good evidence that our susceptibility to priming may actually change as we age. When attempts have been made to actually create new word association relationships for people, age is a confounder:ageandprimingImage from the quantitative analysis paper. The authors in that paper propose that this will translate in to young people being much more susceptible to subtle primes, with older people only responding to more direct ones. This age discrepant behavior is not always accounted for.
  4. Experimenters can prime just as well as their actual primes One of the main blows to priming studies came when a group of researchers attempted to replicate the “hear words about old people/subsequently walk more slowly” study. In a study called “Behavioral Priming: It’s All in the Mind, but Whose Mind?”, researchers found that priming the researcher to believe the subjects had been primed to walk more slowly caused the participants to walk more slowly. In fact the researcher’s belief made a bigger difference than the priming itself:Turns out subjects aren’t the only ones susceptible to subtle and unconscious biases. You can read the original studies author rather grouchy response to the whole thing here, and Andrew Gelman’s eyeroll back here.
  5. The field did attract an unfortunate number of frauds. Maybe it was due to the headline grabbing nature of many of these priming studies, but there have been some absolutely audacious fraud cases in priming research. Diederik Staples published over 20 big studies with  made up data. Dirk Smeesters also had seven. Lawrence Sanna is up to 8. Is this worse than other fields? Maybe, or maybe it’s just that these studies tended get a lot of attention. It’s not so much the fraud that casts a shadow, but the alarming realization that so many made up studies got through without question. This has led to calls for standards involving immediate replication attempts and other measures to stop bad research before it starts.

Now keep in mind, all of these reasons are over and above the normal file drawer effect and p-hacking that all fields face. Hopefully this gives you a little insight in to a few of the less obvious ways these studies can go wrong, and will trigger you to think about these things when you hear the word “prime”….see what I did there????