Signs signs everywhere signs

Well, it appears that either there was no systematic bias against Republicans in the polls, or Nov 6th just happened to be the wrong time of the month for the Republicans.

My mother was with me on election night, and she mentioned being quite surprised that New Hampshire wasn’t a closer race (52-46 for Obama), and even more surprised that Maggie Hassan beat Ovide Lamontagne by as wide a margin as she did (55-42).  Apparently the polls had showed a closer race, and many people she knew were convinced that bias meant the Republicans were actually leading.

I ended up driving back to New Hampshire with her, and I started to see where some of the problem had come up.  At least on the route I take, the roads were COVERED in Romney/Ryan and Lamontagne signs.  They outnumbered Obama/Biden and Hassan signs by quite a bit.

I was reflecting that I’ve heard that’s the point of signs….to give the impression that there is a majority for one candidate, and that you are going against all of your neighbors if you vote otherwise.  I wondered how many people saw those signs and had at least some of that influence there opinions of the polls.  There can’t be that many people voting for the other guy….I see hundreds of signs every morning that say otherwise.

This is yet another example of where proxy markers can fail.  Political signs along major routes reflect the dedication of a few, not necessarily the opinion of the many.

Election Eve and Polling Bias

Well it’s election eve and Nate Silver is still predicting an Obama win….with the caveat that it is possible that if Romney wins it will mean nearly all state polling might be biased against Republicans.

I don’t think he was saying this to be glib, or ruling the possibility out.  He actually goes quite in depth as to where he thinks error could occur.

To me though, this brought up an interesting point…..what do we do if it’s true?  If nearly all swing state polls are saying Obama, and they break Republican, we will have to do quite a bit of reworking of our polling system.  But that’s not what this post is about.

This post is actually about a rather entertaining comment I saw in a discussion about this.  Why haven’t there been more concentrated efforts to skew polls?  Essentially, if you live in a swing state and hate political advertising, why not start a movement to get people in your state to all answer the same candidate to obscure the fact that it was a battleground state and reduce the number of dollars spent there?

This sounds wacky, but how many people would really have to buy in to this to make a difference?

Let’s take my home state of New Hampshire.  As of January, there were about 770,000 registered voters.  As of today, polls show they are tied for Obama and Romney.  From what I can find, even the best polls only have a 10% response rate, and many are at 2 to 5%.  The UNH Granite State Poll is widely reported and only surveys 500 people.  It seems it would not take many people making an effort to answer their phones and state they are for a particular candidate to start to skew things.  Even if word got out, it would introduce enough uncertainty in to the polls to confuse the heck out of the political consultants and the media…and wouldn’t that at least be entertaining for the rest of us?

It’s not like this is unprecedented….it was tried with Sanjaya on American Idol and there were rumors about Bristol Palin on Dancing With the Stars.  Those efforts took far more people than it would take to skew the polls in a small state like New Hampshire.  With 58% of adults using Facebook to get political information, it shouldn’t be too hard to mobilize people….just like Twitter was used to start chants at the Boston Garden during the playoffs last year.

This is the danger of big data.  While data driven decision making is awesome, it’s also hackable.  I’m just curious what the back up plan is if polls don’t work any more.

Friday Fun links 11-2-12

A history of film, in one graph.

With election day coming, should you make sure you’re voting for the candidate whose positions you most agree with?  It’s a good quiz, with both yes/no options or more nuanced opinions…also lets you rank how important certain issues are.  I was happy to see that I’m in 97% agreement with my candidate of choice for president, and my Senate choice aligned with my beliefs too, though not as strongly.

Heard about this on Tim Ferriss’s blog….they’re billing it the “Manhattan Project to End Fad Diets“.  I’ll be following this.

A new study shows an increasing danger for men in our time….Tie Retraction Syndrome.

Science Ink….a compilation of geeky tattoos.

Data visualization…the beauty of simplicity

With the rise of creative data visualization, I’ve heard some commentary lately regarding the tendency of some of these creations to be high on the visuals but low on the data.  While intense data visuals may look amazing, they can give the impression that complex charts are the only effective ones.

While watching a TED talk this morning, I saw a chart that reminded me this is not so.  It was a talk by an ICU doctor, Peter Saul, and the charts showed the four ways people die.  Excuse the poor resolution, it’s a screenshot of the video:

If it’s tough to read, it essentially graphs function (of the patient as a whole) vs time.  The four ways he tracks are (clockwise from upper left):
  1. Sudden Death
  2. Terminal Illness
  3. Organ Failure
  4. Frailty
I thought these graphs were extremely effective at illustrating a (for most people) unfamiliar concept quickly.  
Definitely a “picture’s worth a thousand words” type of graph.

A week like no other

With the hurricane and all, this week got weird in a hurry.  My darling husband is stuck in Chicago, so I chose to ride out the storm at a hotel with my in laws.  Turns out this is the hotel the electric company puts it’s on call employees in, so I’m thinking we’re keeping power.

Anyway, with all the record setting weather, I thought this post from statschat was particularly interesting.

Essentially, it backs up my previous gripes that people don’t often accurately report what they spend their time on.  They included this graph from the Washington Post:

Essentially it categorizes how much people’s reported hours differ from actual hours, and compares that to a more specific question of “how many hours did you work last week”.  When you ask people for a specific week, they answer more accurately.  I thought it was interesting that people who work fewer hours actually tend to underestimate how much they work, as opposed to those who work long hours.  My guess is those at the lower end are not as driven to impress and thus don’t worry about their number as much, whereas anyone putting in a long week wants full credit.  
I appreciated the comments on the Post article.  Many people pointed out that work hours and personal hours are getting more and more intertwined making these estimates much harder.  If I spend an hour at night working on emails in front of the TV, is that work time or TV time?  If I do work on the train ride home is that work time or commute time?  I can’t be the only one asking these questions, and I do wonder how these surveys are capturing these things.
Regardless, statschat had a good comment on the concept of people “lying” about their hours: 

The Washington Post article that provided the graph says that people who claim to work long hours are “lying”, but it’s more complicated than that.  Presumably these are people who ‘typically’ work long hours but reasonably often have to leave work ‘early’ to handle some part of the rest of their lives.  Conversely, the people at the low end of the distribution may have a regular part-time job that provides their ‘usual’ hours of work, but fairly often have over-time or additional jobs so that the average week has more work than a ‘usual’ week.   They aren’t lying, they just aren’t answering the question you thought you wanted to ask.

The concept of people answering what they think you’re asking or responding to different wording with different answers is something all survey makers should keep in mind.

Anyway, I’m sure for all my east coast readers, this will not be a “typical” week….no exaggeration needed.  Stay safe everyone!

Argh argh argh

The AVI left me an interesting link on my last post on famous social psychology studies that have not been replicated.  It’s good reading….they include the famous study that found that teacher’s expectations being self fulfilling (ie kids achievement went up or down based on how smart the teacher thought they were).  That was interesting to me, as I’ve heard that study quoted many times, and never heard that larger studies had failed to replicate it.

Anyway, as I was reading that article, a headline for another article floated across the top of the screen “Sleeping more than 7 hours or less than 5 1/2 hours has been found to decrease longevity”.

No.

No.

No.

I don’t even have to read the article to tell you no study found any such thing.

The only way you could actually prove that is to randomize three groups, force one to sleep more than 7 hours, one to sleep between 7 and 5.5 hours, and one less than 5.5 hours per night (for the rest of their lives) and then see how long they lived.  No one did that.  We know no one did this.

Sure enough I clicked on the article and found that people who reported getting more than 7 hours of sleep/night were 12% more likely to do within 6 years than those who got slightly less (again, with the raw numbers the 12% increase might not be that impressive….how many otherwise healthy people died in the 6 year time period to begin with?).  So there is a correlation, but no one proved what caused it.  The most obvious caveat is that people who are sick might sleep more.

Why oh why do people still write headlines like this?  I can see it when it’s on the front page of Yahoo.com or something, but shouldn’t Psychology Today have slightly higher standards?

Sigh.

Lord of the Rings Statistics

Four posts in two days?  This is what happens when the little one starts sleeping in 7 hour stretches.

Anyway, this one was too good to pass up….a statistical breakdown of various aspects of Lord of the Rings.

More thoughts on voting and non publication bias

The more I think about the study I commented on yesterday, the more irritated I am they didn’t include a control group (either women over 50 or women on hormonal birth control) to give some context to their claims.

Of course then the results might not have been as stark, and this means they either would have chosen not to publish, or it wouldn’t have been accepted for publication.  It’s crucial to keep in mind that study authors are under no compulsion to publish any results they don’t like.  Obviously, this can skew what gets out there.  Apparently there are laws that actually require this reporting for drug trials, but an audit found only 20% compliance in the US.
Ben Goldacre is currently waging quite the campaign trying to get pharmaceutical companies to live up to the laws that require them to publish info on ALL of their clinical trials, not just the ones that produce flattering results.  This comes in conjunction with his new book Bad Pharma that has apparently caused quite a stir (it’s not out yet in the US….but it will be in January…in case you wondered what to get me for Christmas).
I suggest reading some of his blog posts if you want a crash course in publication bias and why it’s so harmful to us.  The quick example of course is the study on hormones and voting….do you really think if a study came out showing that women’s menstrual cycles did not effect their voting that it would be published?  Journals wouldn’t find it interesting, and researchers who base their careers on finding ovulation/behavior links would likely not even submit it.  
In the last chapter of his book Bad Science, Goldacre takes the media to task for this.  He documents how the most sensational science stories are almost never given to science writers in the interest of making a better story.  He then calls out journalists (by name) in the UK who published stories calling for more research on vaccine/autism links, while subsequently failing to report when such research was done (and came up with no link).  
If you haven’t read anything by him, I highly recommend it.

Technical Clarification

I was feeling a bit ranty in my last post about the women/hormones study, but I decided it needs a slightly more academic treatment.  Despite CNN yanking the story, I managed to find the original study and read the whole thing.

A few points:

  1. All the participants were paid via Mechanical Turk for their participation.  This gave me pause.  Depending on how this was set up, I was curious how they verified that people didn’t give some of their answers just to qualify to get paid.  
  2. The study did not follow individual women and show them to be fluctuating.  The study compared groups of women at high and low fertility times and reported their differences.
  3. The measured political attitudes excluded all fiscal views (because those didn’t change much) and focused only on social views.  
  4. The single women assessed for political affiliation had a median income of $15000-25000/year, whereas the married women had incomes of $35000-$50000/year.  Interestingly, in the discussion section, this difference is considered relatively small and inconsequential.
  5. While the study (and articles) mention that they surveyed 275 women for the first experiment, they later clarify that they tossed out nearly half of them because they couldn’t reasonably determine where they were in their cycle.  The second study started at around 500 and got whittled down the 300.  This means the groups being compared were about 75 people each in the first study and 150 each in the second.  
  6. The groups were not controlled for anything.  Those income ranges are so big you could drive a truck through them, and nothing was said about what states people came from. 
  7. No woman under 44 was counted, nor were any of them asked if they planned on voting. 
Overall, I was less weirded out by this study when I saw the authors.  They are all pretty hard core evolutionary psych folks, and pretty much believe everything people do is hooked to mating opportunity (interesting, this includes religion.  Apparently women become religious to either stop themselves from cheating or to attempt to impose a social order on others that will keep their mates faithful).  Take a look at Kristina Durante’s publishing history and you’ll see why they never even looked at how any other variables might influence anything.  Truthfully, they saw a link where they had already decided there was a link.  
With sample sizes as small as they were from a very specific group (people seeking out paid work on the internet), a control for region or income would have been helpful.  Additionally, the group studied (18-44) is the least likely group to vote.  Even beyond their reproductive years, women still tend to vote Democrat…so there’s that.  I also thought it interesting that there was no control for historic voting behavior…if women who voted for Obama in 2008 were more likely to change their vote in conjunction with specific times of month, it might have been more interesting.  As is though, we have no idea if there’s a real shift in individuals or if it’s just the groups they picked (a interesting number of their p values did not reach the level of statistical significance, the results were reported in the article without this caveat).  

Data so bad even CNN took it down

After living in New Hampshire for my entire upbringing, moving to Massachusetts when I was 18 was a bit of a surprise.  Why you ask?  Because my goodness are election years more peaceful here.

For those readers who aren’t from New England, New Hampshire residents are some of the most harassed people in the nation when it comes to presidential elections.  Between the first in the nation primary and swing state status, the amount of effort people put in to trying to find out what New Hampshirites are going to do on election day is staggering.  Massachusetts on the other hand is reliably blue, so everyone pretty much leaves us alone (Exception: the Scott Brown/Elizabeth Warren face off is really harshing my mellow this year).

Anyway, as a woman who both strongly believes it’s her civic duty to vote and who puts a lot of thought in to her vote every 4 years, I was a bit surprised to see a story on CNN yesterday about how women voted with their hormones.  The link actually goes to Jezebel there because the “science” was so bad that CNN actually took the story down. 

Essentially, the research claimed that during “that time of the month” women felt sexier.  This led single women to want more social services (because they apparently were worried they wouldn’t be able to help but get pregnant with a random partner).  Married women on the other hand apparently overcompensated and wanted to vote Republican because they….I don’t know.  I really couldn’t follow the convoluted reasoning of how feeling sexy or not influenced your vote.

To note, this was an internet survey done by a marketing research person.  It also apparently found that women’s level of religiousness varied based on monthly cycle.

The sheer weirdness of saying political party and religious affiliation, two of the deepest and most profound beliefs people have, is based on a few fluctuating hormones (of course only in women….I mean, have you ever heard of testosterone influencing men?  I don’t think so) is just so reductionist it’s bizarre.

It also of course leaves out post menopausal women, women who are on hormone regulating birth control, and ignores better research that shows women in committed relationships are already more likely to be conservative.  Oh, and it totally leaves out anyone voting for a third party candidate.

I bring this up not just because it was a bad story and because it actually got taken down, but also because it’s part of a larger phenomena of journalists inflating the effect of small differences to write a better story.  I am really stunned how many times in the past week I’ve seen stories about “why Obama/Romney isn’t getting as much support as he should”.  The author then goes on to talk about some line of reasoning that supposedly explains why their candidate would be creaming the other guy if it weren’t for the influence of the small factor that they and only they are acknowledging.

News flash to the media:  most people not voting for your candidate are voting the way they are because they don’t agree with him, or your party, or because they like the other candidate or party better.  Stop belittling large portions of the population while trying to prove otherwise.

*Gets off soapbox*
Thank you for your time.