Month: June 2012

Outcome metrics and the research we do not do

/ bs king / 3 Comments

I’ve spent most of last week at work trying to perfect a grant proposal that pretty much everyone in our program has to sign off on.  On Thursday, Friday and today there was a great deal of discussion about what metrics we could use to measure our outcomes, should we get funding.

It’s actually not an easy question, as the project we’re working on is a general good thing (patient education) designed to address a multitude of issues, as opposed to something more targeted.

Watching half a dozen people go back and forth about all this got me thinking about how often it is taken for granted that somewhere out there is a definition for “success” in various topics.

When I took a child development class in grad school, I remember in one of the first classes someone asked what the best parenting methods were.  Our professor replied that there really couldn’t be a consensus, because no one could agree on what would qualify as a success.  He proceeded to use religion as an example:  for parents of strong religious persuasion, a child who grew up a financially successful atheist would not necessarily be what they were going for.  Conversely, secular atheist parents might be distressed at a strong religious conversion.

There are probably scores of good studies that could have been done on parenting methods if we actually had a definition of success we could all agree on.  Too frequently, I think people overlook this point.  The reason so many strange fads in parenting can get going is because it is really really hard to prove anyone right or wrong.  Even if you try, you might just wind up with the dodo bird verdict.

If you can’t agree on where you’re going, you most certainly can’t tell people how to get there.  The studies you don’t do are often as important as the studies you do.

New Nassim Taleb

/ bs king

Apparently Nassim Taleb has a new book due out in November.

Farnam Street has a bit up from him that I liked quite a bit regarding how we process excessive data, most often to our detriment.

Best quote:

If you want to accelerate someone’s death, give him a personal doctor.

Cutting and pasting OR always check the source data

/ bs king / 2 Comments

I’ve mentioned before that I don’t like infographics.

Normally this is because the infographic itself is misleading, but today I found an equally hideous incarnation of this.

It all started over at feministing.com, where I was greeted with this graph:

This pretty much set off my alarm bells immediately.  I had quite a few questions about all of this, as the graph obviously said very little about the methodology.  Who was included?  How did they account for gaps in years worked?  Most importantly, did they control for profession?

I clicked on the link provided, which took me to this blog post on the New York Times website.  It shows the same picture as above, with an intro of the following two sentences:

We’ve written before about how the gender pay gap grows with age. Generally speaking, the older a woman is, the wider the gap between what she earns and what her male counterpart earns.

I was struck by that phrase “male counterpart”.  Were we really talking about counterparts here?  I was curious again about the profession question.  It struck me that many female dominated professions are actually “terminal” professions….i.e. the job you enter can remain pretty unchanged for years: teachers, nurses, therapists, etc.  On the other hand, many male dominate professions have far more steps on the ladder, which would be a pretty non-sexist explanation for the continued growth seen throughout the decades.

With this in mind, I went to find the methodology for the graph.  I not only found the methodology, but the rest of the infographic.

As it turns out, the profession issue was directly addressed on the original….but it was completely edited out in subsequent reprints.  Profession does have an effect on earnings growth, and the original captured that.  I’m a little concerned about how far this graphic went without all of the important qualifying information they took care to include.

 Interestingly, the NYT columnist did actually write a more comprehensive article on the topic 2 years ago that she linked to in this article, but I’m surprised she didn’t do a recap.  With the ease of transport of info on the web, I don’t think the cut and paste job is an okay thing to do.  It sets up less diligent bloggers to merely reprint, and it undermines the original work.  Someone out there is quoting this right now, having no idea that they’re missing 2/3rds of the information.

Bad data, bad.

Friday Fun Links 6-1-12

/ bs king / 1 Comment

Last time I did a list of fun links, my most curmudgeonly reader informed me they weren’t fun enough.

Fine, I’ll try again.

I don’t even attempt to touch on economic stats on this blog.  Frankly, they make me dizzy.  However, I’m excited to see that George Mason’s Stats.org is launching an Econostats website soon.  Their regular site is pretty darn good for going behind the headlines, and I’m hoping this one will be too.  Here’s a post from the guys who will be running it.

If you’re looking for summer reading for your local math/logic puzzles nerd, this might be a good choice.  Even for those not on the job market it looks fairly interesting. (Fixed the link)

Nate Silver feels about acronyms what I feel about infographics.

I’ve been trying to improve my data visualization skills lately, and I’ve been noticing huge variances in examples on the web.  Thus I liked reading this proposal for creating three different categories: data visualization, data illustration, and data art.

Speaking of data art, I bought David McCandless’s book, which is very pretty, very fun, and answers the burning question “what can facebook teach us about peak break-up times?”

Facebook breakups not of interest to you?  Maybe you’re a tennis fan?  Watching the French Open?