Advice Column

In Praise of Confidence Intervals

/ bs king / 2 Comments

This past week I was having a discussion with my high school teacher brother about an experiment his class was running and appropriate statistical methods for analyzing the data. We were discussing using the chi square statistic to compare data from an in class calorimetry experiment to the expected/published values (this is the point where the rest of my family wandered off), and he asked what other statistical analysis his kids could do that might help them understand their results. I mentioned that I was a big fan of confidence intervals for understanding data like this, and started to rattle off my reasons. While testing that produces a p-value is more commonly used in scientific work, I think for most people confidence intervals are more intuitive to use and should be worked in to the mix. Since we were talking about all this at around 7am (prior to both the second cup of coffee AND a trip out to move the cows at his farm), I figured I’d use my blog post today to more thoroughly lay out a few reasons confidence intervals should be more widely used (particularly by teachers) and provide a few helpful links.

The foundations of my argument comes from a paper published a few years ago called “Why the P-value culture is bad and confidence intervals a better alternative“, which gets in to the weeds on the stats, but makes a good overall case for moving away from a reliance on p-values and towards a focus on confidence intervals. The major reasons are:

  1. Confidence intervals use values you already have p-values and confidence intervals are mathematically similar and take the same basic variables in to account: number of observations (n) and variation of those observations (standard error or SE). More observations and lower variability within those observations generally are considered good things. If you can get a p-value, you can get a confidence interval.
  2. Confidence intervals give you more information than the p-value Where a p-value tells you just the statistical significance of the difference, the confidence interval tells you something about the magnitude of the difference. It gives an upper and lower limit, so you get a sense of what you’re really see. For kids learning about variation, I also thinks this can give them a better sense of how each of their experimental values affects the overall outcome. For example, if you’re doing a calorimetry experiment and know that the expected outcome is 100, and your class of 30 kids gets an average of 98 with a standard deviation of 5, you would tell them that p=.03 and thus this a significant difference. Using the confidence interval however, you would give them the range 96.2 to 99.8. This gives a better sense of how different the difference really is, as opposed to just accepting or rejecting a binary “is there a difference” assumption.
  3. Confidence intervals are more visual. The paper I mentioned above has a great figure with it that illustrates what I’m talking about:On this graph you can draw lines to show not just “is it different” but also “when do we really care”. I think this is easier to show kids than just a p-value by itself, as there’s no equivalent visual to show p-values.
  4. It’s easier to see the effect of sample size with a confidence interval. For the calorimetry experiment mentioned above, let’s show what happens if the class is different sizes, all with the same result of 98 with a standard deviation of 5:
    n 95% Confidence interval p-value
    10 94.9-101.1 .2377
    15 95.5-100.5 .1436
    20 95.8-100.2 .0896
    25 96-100 .0569
    30 96.2-99.8 .0366

    I think watching the range shrink is clearer than watching a p-value drop, and again, this can easily be converted in to a graph. If you’re running the experiment with multiple classes, comparing their results can also help show kids a wider range of what the variation can look like.

  5. Confidence intervals reiterate that some variation is to be expected. One of the harder statistical concepts for people to grasp is how predictable a bit of unpredictability really is. For some things we totally get this (like our average commute times), but for other things we seem to stumble (like success of medical treatments) and let outliers color our thinking. In the calorimetry experiment, if 1 kid gets 105 as a value, confidence intervals make it much easier to see how that one outlier fits in with a bigger picture than a single p-value.

So there you go. Confidence intervals are a superior way of presenting effect size, significance of the finding, and are easy to visualize for those who have trouble with written numbers. While they don’t do away with all of the pitfalls of p-values, they really don’t add any new pitfalls to the mix, and they confer some serious benefits for classroom learning.  I used Graphpad to quickly calculate the confidence intervals I used here, and they have options for both summary and individual data.

No One Asked Me: Moving Sucks

/ bs king / 2 Comments

“Why does moving suck?”

-two of my cousins, while cleaning out their childhood home in 90 degree heat

 

Ugh, moving. Yeah, that definitely sucks. Get back to packing.

 

You’re still here.

 

Okay, seriously. You need to pack.

 

Okay, fine. Twist my arm, I’ll distract you.

 

So the thing about moving, is that it sucks.  It’s bad for several notable reasons, on which I am more than happy to expound:

1. Time spent packing and moving is like some sort of horrible geometric progression that theoretically approaches finished but feels like it’s never going to get there.

So a pretty typical geometric progression looks like this:

16jul15pic1

The stuff of dreams or nightmares, depending on your perspective

Which could translate in to percent of move completed, like this:

16jul15pic2

Behold the angst of the x-axis

Now if you take that first series, there’s a somewhat fundamental mathematical rule that tells us this will eventually happen:

16jul15pic3

The “…” literally means “and we go on like this FOR EV ER.

Nice equation, but you know what it’s telling us about getting to that perfect ending?  YOU NEED AN INFINITE NUMBER OF TERMS BEFORE YOU GET THERE.  To me, this means that no matter how beautiful the picture in your head of how organized you’ll be or how nicely you’re going to leave everything looking, you’ll eventually have to start calling .9875 close enough and just start shoving things in trash bags and labeling them with Band-aids and sharpies.1  

2. Moving is all the household chores at one time.

Household chores are a pretty highly researched area, especially when it comes to how fairly we divide them up.  The lousy thing about moving is that it turns your life in to one big several day long whirl of all the household chores, and none of it feels fair.  You have to clean and organize and sort and probably do laundry.  Then you have to do it all over again when you get where you’re going. Ugh.

Household chores just don’t bring happiness.  Neither does moving, at least for women2.

3. Moving is probably just another term in a bigger equation of lousy things.

One of the interesting things about looking in to the research on moving is that the process itself is most commonly studied when it’s attached to other things like divorce, childhood, aging, etc.  For example, the Holmes and Rahe stress scale3
assigns a general sort of “stress number” to all sorts of different life events. The value assigned for “changing residence” is fairly small (20), but if you look at that list, it’s really uncommon to change residences without ticking at least a few of the other boxes. 

For example, the last time I moved, I was pregnant (40) and not unrelatedly, we got a new family member shortly thereafter (39).  We also took on a mortgage (31) and changed our living conditions (25).  The magic number for Holmes and Rahe (in terms of predicting stress related health events) is 150….and in 2 months our household racked up a 155.  That’s enough to put both my husband and I at an elevated risk for a serious health breakdown over the next two years.  Now let’s look at my last 3 moves:

16jul15pic4

I wanted to add “in law trouble” for 29 to that last one, but I actually really like my in laws and had fun living with them. Take that Holmes-Rahe!

So two out of the three of them actually got me above the 150 level in a very short period of time.  When you consider that the Holmes/Rahe scale is actually supposed to cover the last year of your life, you can see why a move can  impact your whole year pretty quickly.  Additionally, keep in mind that neither of my “bad” moves were really that uncommon.  Changes in family status (marriage, babies, divorce, etc) and just wanting or needing more space/better neighborhood/to own a home are far and away the most common types of moves.  In fact, if you look at the Census Bureau list of the most common reasons for moving, it reads like….well, like the Holmes and Rahe stress scale.  People move when things change and changes are stressful.  People then associate moving with stress, and we all come to the accurate conclusion that it sucks.

Put another way, it’s not so much that moving sucks, is just that it’s a seemingly endless series of chores and housework that is almost always associated with a level of stress that is so bad it can make you sick.

Glad I could clear that up.

 

1. This is definitely a thing I have done.↩
2. To be fair, most of the moves studied in that study were family moves, possibly initiated by the man changing jobs. This can skew things a bit. Whatever. I hate moving.↩
3. The Wikipedia page also has an example of a version for non-adults, defined as anyone who doesn’t feel like the first list understands their problems well enough.↩

No One Asked Me: Love at First Sight

/ bs king / 1 Comment

Would you believe in a love at first sight?  Yes I’m certain that it happens all the time.

-John Lennon and Paul McCartney (cowriters)

 

This week’s question comes from a little known group called “The Beatles”.  It’s from their song “With a Little Help From My Friends“1, and the sentiment was raised by my friend John when we were discussing relationships.  Now John’s a little bit of a hopeless romantic pragmatic idealist, so the idea of love at first sight kind of appeals to him.  But does it exist?  And more importantly, does it really happen all the time?  Let’s take a look!

Alright, let’s be honest here…the question of whether or not you can really fall in love at first sight is one typically addressed by philosophical debates, not statisticians. Literally everyone has an opinion on this, and often a strong one. It’s a question that inspires all sorts of crazy debates, tons of movies, countless songs, and a mildly disturbing yet rather watchable reality show.  I’m not a philosopher and I’m not getting in to all of that “what is love” junk2, but I can tell you in the dating market it’s kind of a guy thing.  In a user survey done by Match.com, they found that about 60% of men believed in it, and 40% said it had happened to them.  For women, those numbers were about 50% and 30%, respectively.  Those numbers would suggest that John and Paul were on to something, as it certainly seems to be a pretty common occurrence.  But is that the whole story?

What jumps out at me as a I pondered this question was a concept known as the toupee fallacy.  This seems to be one of those questions where the facts we’re not seeing might be as important as the ones we are seeing.  I’m concerned that there’s some silent evidence at play here, and we may be missing a few things.  Namely, we’re not seeing how often people think they’ve fallen in love at first sight, only to be quickly disappointed.  Whatever this feeling or moment we are talking about is only gets counted if it works.  Here, let me illustrate:

09jul15pic1

It all looks so easy, doesn’t it?

So pretty much everyone we meet falls in one of those 4 boxes.  When we talk about love at first sight though, we often only talk about it in the context of those two red boxes, ie people who wind up together.  What we can’t forget about is that blue box there…those we meet, feel an instant attraction to that never pans out.  Here’s the same information put another way:

09jul15pic2

Possible Stalker/Type 1 error is my new band name.

Now what we’re generally going for in life is either the box in red or the box in black.  In stats terms the red ones are true positives (falling in love with someone who loves you) and the black are true negatives (not falling in love with someone who doesn’t love you).  The other two boxes are actually what we’d call Type 1 errors and Type 2 errors….ie, the chance that we make the wrong call initially.  If we presume the null hypothesis is that most people don’t love us3, we can call the box in blue, our type 1 error and the box in green our type 2 error.  In love, we almost always prefer Type 2 errors….in other words, we want to find out we loved someone when we didn’t realize it rather than fall in love with someone who doesn’t like us.

But what influences the number of people who fall in each box?  Well, for that we have to take a look at the words that make up both of our conditions.

 

Let’s start with “end up together”.

During the discussion that prompted this question, John and I were specifically chatting about people who end up married.  Now, in 2015, this may not be a great metric to go by.  Many people who are in love do not get married, date or cohabitate for much longer than past generations, or otherwise define their loving relationships differently.  The point is not to cover every possible scenario, but rather to remind people that the more narrowly you define “end up together” the less likely it is to happen from a strictly statistical point of view.  For example, in the numbers I gave in the beginning of the post, 40% of men said they had fallen in love at first sight…but these were men participating in a survey for singles on a dating website.  Of course some of those men could have been widows, but the rest of them either ended the relationship that started with love at first sight, or had it ended for them.  Does this count?  Some will say yes, others will say no.  Your standards will influence how many people are covered by that first row.

Now, “didn’t end up together” seems more straightforward, but it actually can also cover a range of scenarios.  I made a joke in my table about someone who falls in love with someone who doesn’t love them being a stalker, but that’s not the whole story.  Most of us have met someone who we thought was awesome….for 5 seconds until they opened their mouths.  Or until part way through the first date.  Or two weeks later when you saw their massive teddy bear collection.  You get the picture.  The point is, not ending up with someone can mean a whole lot of things from “they were taken” to “we decided we were better as friends”.  How broadly you define this will also determine how often people fall in this category.

Love At First Sight (LAFS)

Alright, lets move on to love at first sight.  How are we defining this and how often is it happening?  Well, this one can get interesting.  LAFS is one of those things people tend to define by saying things like “if you have to ask, it didn’t happen”.  You know it when you see it.  This makes it ripe for hijinks and chicanery, which I’ll get in to in a minute.  In it’s most basic sense though, everyone seems to agree it’s some sort of overwhelming feeling of attraction bordering on feeling magnetically pulled towards a person.  How broadly you define this, and how often you think this has happened to you already are going to effect the number of people in that box.

So now that we’ve got some definitions, let’s put some generally fictitious numbers in those boxes.  Let’s say you’ve met about 1000 people in the generally correct age/gender/orientation that you’re attracted to.  Here’s what happened with them.  You’ve dated about 20, and twice think you felt something that could have been LAFS.  One of those worked out, one didn’t.  Your percentages are here.  We get these numbers:

09jul15pic3

There’s a Taylor Swift song somewhere in here.

So unfortunately, the chances are kind of small.  You can run the numbers for your own life, but my guess is it will be pretty small there too.

But John and Paul promised me! You said they were on to something! 

Okay, you got me.  So what’s going on here?

Well, the answer is really that we don’t actually often think of this in the terms I put above.  We’re not evaluating our own lives and our own chances, we’re trying to go off of other people’s experiences.  We are not calculating overall probabilities like I did above, we’re doing conditional probabilities.  No one asks people what happened when they didn’t find love, we ask them what happens when they did find love.  In stats this is a huge difference.  We just went from a regular probability to a conditional probability.  Basically, it’s the difference between these two equations:

09jul15pic4

P here is “probability” and the rest is about how I’m totally not bitter.  

That first equation gives us a .01% chance, and the second one gives us a 5% chance, using the numbers above.  That’s 500 times higher!

And this is assuming everyone’s being honest about who they’re putting in what box.  Spoiler alert: they’re not.

Most of this isn’t intentional though.  It’s just that as humans, we don’t tend to remember all the details of good events.  In fact, our memories of bad events are much stronger and typically more detailed.  So when two people fall in love and things work out, they will likely not really remember the moments of doubt or insecurity that may have actually been present in the beginning of their relationship.  They will retell the story more amusingly and more positively than the actual events may have warranted.  This is so prevalent in fact that it is actually considered a hallmark of a healthy relationship. We can infer then that by only talking to people in happy relationships, we may actually be overestimating how many people met and “just knew”4.  That’s why research on this is so sparse….the data confounds itself.

Ugh, well that’s not great news.

No, and it gets worse.  When John and Paul claimed that this happened all the time, they were likely right….but that won’t help you.  For example, let’s say that 1 out of 1000 people every year are likely to experience un-exaggerated, for real, LAFS with someone they stay with.  That’s about 25,000 people a year in the USA.   That’s 67 a day.  You will almost certainly know some of these people….but they may not ever be you.  Bummer.

Got any more good news?

Well yeah, actually, I do!  See, the thing is, LAFS may not even be the ideal here.  There’s actually some interesting evidence that people who date for longer stay married longer5.  Apparently it’s long engagements that threaten marital stability, not long dating periods.  So while those in the LAFS/stay together box may get a lot of attention, the ones in the no LAFS/stay together box may be quietly outdoing them. Also, when finding true love, most people really are more interested in the exponential distribution, not the Poisson distribution6. In other words, we’re not so concerned about the number of events, but rather how long we have to wait for it! Once you find the one, you probably won’t care so much how it happens, and evidence suggests that you and your beloved will keep altering your story bit by bit until it’s worthy of it’s own movie with the attractive Hollywood folks of your choice.  You’ll get there.  May your W = time to first event be short, and your moment generating function be beautiful.

 

 

1. Weird fact I learned about this song while researching this post: the first line was originally “what would you do if I sang out of tune, would you throw ripe tomatoes at me?” but Ringo made them change it when he realized their rabid fans might take it seriously. ↩
2. Baby don’t hurt me.↩
3. Okay, emo kid.↩
4. If you ever want to see this in action, find a friend who you knew pre and post divorce. If you know the story of how they met their ex, it’s really interesting to ask them again after their divorce. It is almost guaranteed the story will have changed, gotten briefer or otherwise be a bit altered. Do NOT point this out to them. Don’t ask me how I know this.↩
5. Some of this data is kinda old…marriage and dating practices have changed rapidly over the last few decades. Caveat emptor.↩
6. Our love is anything but a normal distribution!↩

No One Asked Me: Farting in Public

/ bs king

I’m a guy and I try my very best to not fart at all because if I did in around anyone, I’d be so humiliated I’d probably never come to school again or never go where I did again. Especially if someone was close and may have heard. Is it unhealthy to hold in farts? Do you guys think it feels good to fart? I’m sorry, I know it’s a disgusting question but I’m just curious on what you guys think of it.

-Anonymous Teen

Found at Yahoo Answers

Alright AT,  I’m gonna skip over some of the touchy feely does-farting-feel-good stuff here, and go straight for the crux of the matter: it’s not going to hurt your health to hold in farts, but it might give you a few cramps and be kind of uncomfortable.

I’m not letting you off that easily though, because I’m a little concerned about your anxiety level here. No one’s saying you have to be Peter Griffin, but  I’m not sure I’ve ever heard of someone being this concerned about farting in public1.  Would it make you feel better if we ran some numbers here?  That always calms me down.

The simplest way of drawing out your fears is through a Venn diagram, like this:

02Jul15blog

My circles aren’t great. I need a compass.

In other words, you’re not scared of farting, you’re not scare of being in public, you’re scared of farting AND being in public at the same time.

So the first thing we have to do is to assess the scope of the problem aka the size of the circles.  For the purposes of this exercise, I’m going to assume that other than the anxiety you’re  healthy.  If you read anything here that makes you think you are farting more often or with a worse smell than what’s normal, you may want to talk to your doctor.  Alright, let’s start with the farting.   Now the average person is going to fart about 10-20 times in a 24 hour period, with at least some of those coming at night.  In this study, the median number during waking hours was 8, so lets start with that.  This works out to about one fart for every two hours of wakefulness.  However, that same study told us that it’s very likely this isn’t spread out evenly.  You’re more likely to fart in the hour after you eat.  So what would happen if we drew out a little table of what a hypothetical day of farting might looking like?  Well, in stats we call this a probability density function or PDF, and we can draw it like this:

All the values should be multiplied by 8, if that's the average we're rolling with here.

All the values should be multiplied by 8 to get the total per hour, if that’s the average we’re rolling with here.

If you’d like some better data, you can track yourself for a few days and see if you have any unique patterns.  In a case study about a terribly unfortunate man who was farting 30-130 times a day, his journal looked like this2.  Now this may be segmented out a little too much to be useful, so lets collapse that down a bit:

02Jul15pic3

Red = Danger danger Will Robinson

So if you follow this schedule, we’re looking at 6 or 7 farts in public.

Now a few things jump out at me here.  The first is that spike in the morning could definitely be moved a bit if you ate breakfast earlier, causing that first wave to happen before you get to school.  Also, there’s some evidence we could mitigate that spike after lunch if you started trying to avoid having gas producing foods.  Another tip from that list is to avoid gum or soda all day, as those can both lead to farting3.  If you want to get even better, you could make sure you wander off by yourself for a bit right after dinner.

Making those changes, we can shift the brackets in a bit, and we may be down to dealing with only 4 or 5, most right after lunch. Nice!  Now you’re dealing with a lot less stress, and you have some idea about when you might need to run to the bathroom or use one of these tricks to hide the sound or these pads to hide the smell.

Good luck.

 

1. Not gonna name names or anything, but there’s a few people in my life I might like you to talk to when we’re done here.↩
2. Spoiler alert: don’t go to his house around 8am.↩
3. The fact that you might be able to control this a bit is good news for you, but it’s why I couldn’t assume independence and thus use a Poisson distribution which sucks because I LOVE POISSON DISTRIBUTIONS.↩

 

No One Asked Me: Yesterday’s Weather

/ bs king

“I always dress for yesterday’s weather.”
-my brother

Okay so that’s not really a question.  In my defense though, my brother’s got a philosophy degree, which means most of what he says is an attempt to provoke a reaction, make a grand statement about life, explain his more questionable dating choices, or to get more attention, though not necessarily in that order.  Anyway, he posed this statement to me recently, then arched his eyebrow.  It’s possible I was supposed to take that as an opportunity to extrapolate some deeper meaning about his relationship with his ex-girlfriend, but instead I got curious.  If you really did always dress for yesterday’s weather, how often would this be okay?

It turns out this is one of those interesting stats questions that you can sort of come up with an answer for, but you have to make all sorts of assumptions to get there.   I did some poking around, and here are the parameters I figured I’d have to work with:

  1. You are perfectly rational.  Now this may not be a great assumption1, but it’s one we have to go with if we hope to get anywhere.  The problem with this is that people, especially those of us in northern climates, tend to start rebelling against winter every year. It’s a pretty well documented phenomena that some time around March/April people in northern climates just say “screw it” to the coat/gloves/scarf thing.  I don’t totally know how to take this in to account, but it’s something to keep in mind.
  2. You are like me.  It appears at least some types of cold/heat perception are pretty heritable, so when in doubt I assumed you’d act exactly like I do.  Hey, it worked in middle school.
  3. You modify clothes approximately every ten degrees (Fahrenheit).  This one was actually remarkably hard to find data about.  The problem is that apparently our bodies make lousy thermometers, and we have a remarkable spread of preferences.  The most consistent breakdowns I could find were actually on running or other outdoor sport sites, and they seem to support my “ever 10 degrees” hypothesis.  Apparently that’s where you can measure an impact on performance.
  4. You live in Boston. Yeah, you don’t.  Never have actually.  But I do, and the data’s actually stored for a while.
  5. Being stuck in the rain without an umbrella will bug you, but having an umbrella you don’t need won’t.  Umbrellas are like towels.  Always good to carry one.
  6. You don’t use an umbrella or other rain gear if it’s snowing.  Because snow’s not mean like that and you already have a jacket on and you’d look silly, that’s why.

 

Alright, with those out of the way, lets talk data.  I found a handy site called Weather Underground that actually keeps detailed archives of the weather.  From there I pulled all the data for Boston from Jan 1st, 2010 to June 22nd, 20152.  After that I measured a few things:

  1. How often the daily high temperature changed from one day to the next by more than 10 degrees in either direction
  2. How often the average daily temperature changed from one day to the next by more than 10 degrees in either direction
  3. How often a clear day was followed by a rainy day.

Basically if any of those three changes occurred, I assumed that you ended up dressed incorrectly.  It’s not perfect…the rain could have happened overnight for example, but it’ll get us in the ballpark.  I knocked off a few values because of fluctuations that fell in to either of the extremes (ie under 25 degrees or over 80 degrees).  Essentially if the day before was 85 and the next day was 96, I assumed you still dressed the same way.   At that point we normally resort to things like swimming or staying inside as opposed to clothing changes.  I did not account for changes in the daily low, as those usually happen at night, and the average picks up those changes. Based on all of this you ended up about 65.5% accurate.  Not bad!

Okay, so what went wrong on the other days?  Well, of the days you got wrong, here’s what tripped you up:

Temperature Changed: 49%

It Rained: 38%3

It Rained AND the Temperature Changed: 13%

Cool!   Now what if we wanted to know your luckiest month?  Well I have that too!

Month % of days you are properly dressed
August 75%
February 71%
July 70%
September 70%
October 68%
January 66%
November 66%
December 65%
June 63%
April 60%
May 60%
March 59%

So you’re actually headed in to a pretty good stretch here!  July’s almost here and August is really your month. At the very least you have some time to kill before March.  Use it wisely, and feel free to put this data on your LinkedIn/Facebook/Match.com profile.  It’s sure to impress.

You’re welcome.

 

1. At least that’s what mom said when I mentioned it to her.↩
2. Hey, happy birthday!↩
3. Interestingly, that means if you took my advice and always carried an umbrella, your accuracy would go up to almost 78%.  Things to consider. ↩

No One Asked Me: Height Differences

/ bs king

Do you think the height difference is too much if a girl is 4’11 and a guy is 5’8?
Just wondering what other people’s thoughts are… My friend thinks it’s too much of a difference.
-Anonymous

Found on Yahoo! Answers

Okay Anonymous, let’s talk this out.  There’s a few different ways of looking at this type of problem, and figuring out how much of a height difference is “too much”.  The first thing you should know is that height for males and females follow two different, but similar, normal (gaussian) distributions.   They look like this:

Aw cute, they're holding hands...er, tails!

Aw cute, they’re holding hands…er, tails!

That’s what it looks like when two normal distributions are similar in shape but have different averages.  For the most part they stay on their own sides, but there’s some overlap.  Now, some people1 will tell you this is a good example of a bimodal distribution, but there’s some controversy about that2, so tread lightly, Anonymous. Those same people will also tell you this means it looks like a camel….

This camel would like to point out that comparisons to him are ALSO a potentially inaccurate analogy and yet no one's writing papers about HIM. He has feelings too you know.

This camel would like to point out that comparisons to his humps are ALSO a potentially inaccurate analogy and yet no one’s writing papers about that issue. He has feelings too you know.

….maybe you should avoid this analogy too. Regardless of what we call it or how we describe it, you’ve noticed this before and you know what it means – most women will wind up with men who are taller than them.  Quite handily for your question, people love to track these distributions as much as Stats 101 profs love to use them as examples, so we have a nice data set from 2007-2008 here.  We’ll work off of that.  I like this data set because it’s kind of cute, and height is measured rather than self reported, which means it’s likely more accurate than a lot of the other more shifty looking data sets out there.

Now Anon (can I call you Anon?), I’m going to assume from your question that you’re on the young side.  I don’t know if you’re pre-pubescent or post, but just know that if either of you are younger than 15 or 16, there’s still a shot one of you could grow a bit more.  For the purposes of this exercise though, let’s assume you’re both in the 20-29 range.  While technically adult height ranges could be anywhere from 21 inches to 8’3”, we know that in reality over 99%3 of adult  men you meet will be between 5’3” and 6’4”, and 99% of adult women will be between 4’10” and 6′ tall4.

Since you didn’t give your gender, we’re going to look at this from two different angles.

If you are the girl:

If you’re our 4’11” girl, you may not want to go ruling out a 9 inch height difference so quickly.  Only 33.1% of men are shorter than this, so it’s actually more like than not that you’ll meet a guy whose even taller than 5’8”….and thus more than 9” taller than you.

If you are the guy:

Well now the story kind of changes.  If you’re our guy here, your chances of meeting a girl shorter than 4’11” are actually pretty small….only 2.6% of women are shorter than this.  Thus nearly every woman you meet will be less than 9 inches taller than you.  You’ll actually meet women taller than you 5 times as often as you’ll meet a woman shorter than 4’11”.

If you just want to fit in:

Alright, so now that we told you your chances, lets take this from another angle.  What happens if you are not concerned about you personally, so much as you’re concerned about everyone else.  How often do people, in general, wind up with someone 9 inches taller than them or more?  Well, for that we have to use some slightly different data.  When I mentioned the bimodal controversy up there a while back, I linked to another nifty data set that gave me some more information:  mean and standard deviation of male and female heights.  Those are cool because now I can blatantly exploit their good nature to calculate how likely it is that we will find a woman 9″ shorter than her male partner.  To do this we….you know what?  You are not going to be interested.  If you want to see the calculations, go here.  I want to show you on of the equation I used though, because it’s initially kind of funny looking but when you see it in action you find yourself oddly attracted to it, like a Yugo or Benedict Cumberbatch:

The Yugo literally looks like a bad drawing of a car came to life

The Yugo literally looks like a bad drawing of a car came to life

Take that baby out for a spin and it tells you that there’s a 17% chance that he’s more than 9 inches taller than her.

BUT YOU’VE BEEN LIED TO, BY ME AND POSSIBLY BY EVERYONE:

Alright Anon, I gotta come clean.  I’ve been lying to you, and I’m sorry.  In my haste to impress you, I totally made a few things up that I shouldn’t have.  It’s hard for me to say this, but here we go: I never should have presumed that any of this was random. I was trying to make life easier on myself by making some assumptions, but I can’t do that to you now that we’re friends and all.  All the calculations I just did presume that men and women just get thrown together without anyone ever thinking anything or having preferences.  That’s completely not true and you and I both know it.  People like what they like,  and what they like frequently includes height preferences, at least enough to mess with my calculations.  As a little bit bonus of life advice, any time anyone shows you a statistic, it’s a good idea to try and figure out what assumptions they made on their way to getting it.  Everyone makes assumptions to get their math to be a little easier5, but sometimes those assumptions make our results way less useful.

So what’s the real story if we don’t presume everything’s random? Well, Fivethirtyeight.com did some good math on this a few months ago while answering a question about how often in real life a man would be shorter than his partner here, and they linked to an interesting study that suggested in the real world, 9 inch (or close) height differences would occur in 30% of couples. That’s even more than the 17% we came up with up earlier, and a few more calculat.  So you’re not alone Anon, not even close.  Everything else being equal, the height difference shouldn’t be a problem, and it definitely won’t be that unusual.  In fact, given how common it is, you may want to consider that one of your friends has a crush on whoever you’ve got your eye on, and is trying to talk you out of it so they can have him/her to themselves.  I’m unimpressed with their advice here, and the math agrees with me.  Good luck and god speed you crazy kids you.

 

1. Up to and including every stats 101 prof ever. ↩
2. This paper ends with one of the most amusing conclusions of all time. Essentially they conclude that a. male/female height distributions are not really technically bimodal but b. there’s no other good quick classroom demonstration they can think of to illustrate the concept and c. readers should think of one and tell them so we can stop using this↩
3. 99% sounds like a lot, but keep in mind that at least in the USA this leaves nearly 2.5 million adults outside of these ranges. 1% of a large number is a LOT.↩
4. Tangentially related bonus fact:Right around 5’7” we hit a kind of magical crossover place, where it’s equally likely that men and women will be that height.  Put another way, if I were to say something like “my cousin is 5’7””, and you were trying to guess my cousin’s gender based only on that statement, you’d have to guess completely at random.↩
5. If you think I’m bad, don’t even get me started on physicists.↩