Heteroskedasticity

Retractions, while sometimes necessary, are never as good as the real thing

/ bs king / 2 Comments

Since starting this blog, I’ve become quite the fan of the website Retraction Watch.

One of the more interesting ongoing stories has been the number of retractions from Dipak Das, the UCONN researcher who faces massive misconduct charges for fabricating data in his research about the health benefits of red wine.

His current retraction count stands at 13 papers, with 145 counts of misconduct being investigated.

While the role of his work in the field is contested, one can’t debate that his results were widely reported and certainly helped with the public perception that red wine is good for you.  Thus, I found it interesting that Jezebel was running an article at the same time about the further proof that red wine is good for you.  In the background they mention some of the studies that Das did, that have since been retracted.  Not that this is necessarily their fault….recently it was found that only a quarter of retracted articles in online databases carry a retraction notice, and this drops to 5% if you look at downloadable PDFs.

People have complained about this with newspapers for years….large headlines, little tiny retractions…but with the ever increasing retraction rate and the centrality of the internet, this is liable to get worse before it gets better.

Economic Data, and why I don’t talk about it

/ bs king / 1 Comment

I find it really hard to even comment on economic data on this blog.  It’s based on so many assumptions and there are so many different numbers that can be included or excluded that critiquing it is a combination of trying to shoot fish in a barrel and trying to catch a greased pig.

Not my idea of a good time.

Anyway, BD Keller linked to an excellent post today that is way more articulate than I about why evidence based monetary policy is so hard to come by.

On economic experimental models:

Think of a good experimental design: randomised control variables, holding everything else constant, etc. Now think of the worst possible experimental design. Imagine something that engineers or psychologists might dream up over beers for a laugh, or to illustrate what not to do. That’s what economists face. It’s as if our lab assistants (the fiscal and monetary authorities) were deliberately trying to make our (economists’) lives as hard as possible. They do this, of course, not to spite us, but to try to make everyone else’s lives as easy as possible. To get a good experimental design for economists, both the fiscal and monetary authorities would need to be malevolent.

Makes sense, but given this, I do wish they’d stop saying their predictions with such authority.

Does egg = cigarette?

/ bs king / 2 Comments

Oh CNN, your headlines make me sad sometimes.

Is eating egg yolks as bad as smoking?

No.  No it is not.  The study you’re reporting on does in fact claim that eating egg yolks accelerate heart disease about 2/3rds as much as smoking does, but acceleration of heart disease is not actually the health problem smoking is most known for.  But you know that.  Sigh.

Not that I’m buying the study anyway.  They asked people, who already had heart disease, to report their egg yolk consumption over the course of their lives.  How accurately can you recall your average egg yolk consumption over the course of your life?  Additionally, people who have heart disease have most likely been told to cut down on consumption of saturated fat and cholesterol.  Those still eating more eggs have likely heard this advice, and disregarded it.  What are the chances that they’re disregarding other advice as well?  Lastly, it does not appear the study asked about the consumption of any other food, meaning egg consumption could actually just be co-occuring with the consumption of something else that was even worse.  Surveys that ask only about very specific foods tend to see what they want to see.

So basically, another correlation/causation issue here, combined with those terrible consumption recollection surveys, with a sprinkle obnoxious headline writing.   Yeehaw.

Now THAT’s how you write a science headline

/ bs king / 3 Comments

“Babies Shun Altruism, Prefer Bouncing”

Speaking of replication of results, this study failed to substantiate the idea that 10 month old babies had a moral code.  Turns out that the their preference for “helpful” robots was based less on the fact that the robots were helpful, and more on the fact that they bounced.

I’m sort of curious how many of the original study authors were parents.  I’ve only been a mom for 19 days and even I could tell you that babies like bouncy things more than discussions about man’s existential angst.  The 2 AM feeding helps you figure these things out pretty quickly.

For fun, I decided to conduct my own n=1 experiment and to present my son with a survey regarding his preference for robots in general and their morals in particular.  I thought it was a fairly well crafted survey.

I think my findings are best summarized with this picture:

I think that should be good enough for any number of social psychology journals.  

Replication of results

/ bs king / 2 Comments

I haven’t talked much here about reproducing results of studies, mostly because most studies have so much to pick at from the outset that it doesn’t matter.  However, a new initiative is looking to highlight reproducibility in scientific studies, and encourage independent verification of results.

I think this is pretty cool.

Right now, journals tends to value originality of research over anything else, and that can lead to incentive problems.  In fact, the woman who helped start the replication initiative did so after she had trouble finding someone to publish a paper that called in to question her own previous research when she was unable to replicate it.

Repeating results is supposed to be one of the fundamentals of the scientific method.  Good to see it finally getting it’s due.

Anti-conservative bias and social psychology

/ bs king / 5 Comments

My most popular blog post of all time was the one I did on conservative trust in the scientific community vs retraction rates.   I called it “Paranoia is just good sense if people really are out to get you” because I had a suspicion (confirmed when I ran the data) that conservatives might actually be behaving rationally when they said they trusted science less, given the ever increasing retraction rates in prominent journals.

Now, a new study shows that this distrust of the scientific community is even more well founded than I originally thought.

In a survey conducted by two self proclaimed liberals, it was found that there is heavy evidence that conservatives are being systematically discriminated against in the field of social psychology.  What unnerved the authors even more is that this was not a case where people were hiding their bias:

To some on the right, such findings are hardly surprising. But to the authors, who expected to find lopsided political leanings, but not bias, the results were not what they expected.
“The questions were pretty blatant. We didn’t expect people would give those answers,” said Yoel Inbar, a co-author, who is a visiting assistant professor at the Wharton School of the University of Pennsylvania, and an assistant professor of social psychology at Tilburg University, in the Netherlands.
He said that the findings should concern academics. Of the bias he and a co-author found, he said, “I don’t think it’s O.K.”

The study isn’t available yet, so I can’t say I’ve read the nuances.  Still, it’s hard for me to believe two liberal authors would have attempted to skew the results in this direction.  Conservatives have claimed this bias exists for years (look no further than the ethics complaint lodged against Mark Regnerus for proof), and will no doubt find nothing shocking about the results.  For liberals to have to face what this means however, that’s something new.  Even in the comments on this article, the vitriol is surprising, with many saying that conservatives are so out of touch that it is an ethical responsibility to keep them out of fields like social psychology.

Yikes.

It is much to my chagrin that social science gets lumped in with harder science, but since findings in this field are so often reported in the media, it makes sense to take them in to account.  We have a vicious cycle here now where some fields are dominated by one party, who then do studies that slam the other party, then accuse that party of being anti-science when they don’t agree with the results.  This is crazy.  The worst thing that can happen to any scientific research is too much consensus….especially when it involves moving targets like social psychology.  With 40% of the population identifying as conservative, how can we leave those perspectives out?  Everyone, liberal and conservative, should be troubled by these findings.  Those untroubled by this should take a good look at themselves and truly ask the question “what am I so afraid of?”.

Growth charts and tiny babies

/ bs king / 4 Comments

This is another post that reflects my current life situation, but it highlighted some pretty interesting issues with data tables.

This issue is particularly interesting to me because I delivered via unplanned/urgent c-section, in part because of some abnormal measurements found during a routine ultrasound.  We had to have quite a few follow up consults and testing (among other things, they actually had to assess for achondroplasia – better known as the major cause of dwarfism)*.

Given this, my mother thought I’d find this Wall Street Journal article on baby growth charts interesting.  Essentially, baby growth charts were set several decades ago based on a population that’s different from what we have now.  The CDC does not want to readjust the charts, as it would make obesity look more normal than they think it should, and this is causing a situation where a high number of children are measuring “off the charts”.

It’s an interesting situation when you realize that 95th percentile doesn’t actually mean “larger than 95% of children of the same age” but rather “larger than 95% of children the same age 40 years ago”.

Additionally, it also points out that the CDC growth chart is based largely on formula fed babies, who grow slightly differently from breast fed babies.  So at the same time Mayor Bloomberg is pushing breastfeeding, doctors are potentially telling parents their children need formula to speed their growth up to match a chart that only tracks where they would be if they had done formula to begin with (this is why state mandated health policy drives me nuts so often….you solve one aspect while leaving several causes unadressed).

As the availability of testing goes up, we have to be particularly vigilant to make sure our standards charts keep up as well.  Otherwise we routinize unnecessary testing and freak out new parents.  And from personal experience, I can say that’s just not nice.

*It was ruled unlikely, though apparently we can’t get a definitive no until he actually starts growing, or not as the case may be.  There’s no genetic history of it in my family or the husband’s, though we are both on the short side.  In this case, us being short is actually a positive….it means the abnormalities are more likely natural variations.  Our genetic consult doctor was hilariously terrible though….she suggested if we wanted more information about the condition we watch the reality TV show about it (Little People Big World).  Then she said it was unlikely, but maybe we should still watch the show.  She ended it all with a comment about how it was never good when genetics doctors had too much to say, so we should be happy she wasn’t talking too much.  I don’t think she was very self aware.  

Rich Mom Poor Mom

/ bs king / 3 Comments

I have a sleeping baby in my lap, so you’ll forgive me if I have a one track mind.

Yesterday we met with a nurse who let us know that in Sweden, they have now set minimums for skin to skin contact between mom and babies during hospital stays.  If you don’t do the minimum, you pay the hospital bill.  This morning, in my first perusal around the internet in a few days, I see that Mayor Bloomberg is trying to find ways of encouraging new mother’s to breastfeed.

A note on research regarding babies and various practices in infancy:  Babies are a lot of work.  I realize I’m preaching to the choir on this, as many of my readers have successfully raised quite a few children, but it’s true.  Many of the practices that show lots of benefits for babies (skin to skin contact, breastfeeding, etc) take even more time than the alternatives.  While I believe these things are good for babies on their own, all data collected on these practices will be complicated by the fact that parents who engage in them tend to have more time, resources, and support than those who don’t.  Pushing these practices on those who are already particularly stressed may not have as profound an outcome as it did in the study, as the groups went from self selecting to random.

Something to think about for the policy makers.

Sorry, I’ve been reading over a lot of hospital literature and getting mildly annoyed.  I think that means the pain medication has worn off.  Nurse!

International data – beware the self reporting

/ bs king / 2 Comments

Maybe it’s just because the Olympics are on, but I’ve run in to a few interesting international statistics lately that gave me pause.

The first was regarding infant mortality.  After Aaron Sorkin’s new show The Newsroom incorrectly reported that the US was 178th in infant mortality (really, you think there are 177 countries you’d rather give birth in?), I went looking for the infant mortality listings across the world.  The US does not typically do very well in terms of other industrialized countries.  
There are a few interesting reasons for that….we have a much larger population than most of the countries that beat us, and it’s spread out over a much larger area.  Our care across areas/populations tends to be more uneven, states vary wildly on issues like access, health insurance, prenatal care, etc. Our records however, tend to be meticulous….there is very little doubt that we capture nearly all infant mortality that actually occurs.  This combination can put the US at a huge disadvantage in these statistics (10-30% according to the best published studies).
This raises the point of why Cuba tends to beat us.  Now, realistically speaking, if you or someone you love had to give birth, would you seriously pick Cuba over the US?  Would anybody?  And yet they look safer given the data….which is all self reported.  I have no problems believing that Singapore outranks us, but I’m skeptical of any country that might have an agenda.  Worldwide, there is actually very little consensus on what is a “live birth”, and the US tends to use the “any sign of life” definition.  
On the other end of the spectrum, I saw this piece recently on gun control.  I’ve covered misleading gun stats before (suicides are often combined with homicides to get “death by gun violence” numbers).  One of the interesting facts the article above points out is that internationally, gun deaths are only counted when it’s civilian on civilian violence.  This is certainly fine in the US…I would think we wouldn’t want to count every time the police had to open fire, but in countries with, um, more questionable police tactics, this could cause some skewing (Syria was cited as one such example).  
Data is hard enough to pin down when you know the sources have no vested interest in misleading you….international rankings will never be free from such bias.

Too hot to hire?

/ bs king / 3 Comments

….or why psych undergrads would make lousy hiring managers.

I saw this study pop up on Instapundit, and while the number of “that happens to me all the time” jokes are infinite, I’m pretty sad this study got mentioned at all.  Here’s the Router’s recap:

Attractive women faced discrimination when they applied for jobs where appearance was not seen as important. These positions included job titles like manager of research and development, director of finance, mechanical engineer and construction supervisor.

Oh the sad sad existence of beautiful women.  To work so hard on your career and then get denied a job because you’re too attractive.  Now, out of curiosity, exactly how many women got rejected from these jobs for this study?

None.

This study didn’t study women or men actually applying for jobs.  They studied what happens when you give a bunch of psych undergrads a huge stack of pictures, a list of job titles and say “sort these pictures in to groups of who you think would be most qualified for a job based solely on the pictures“.  Seriously, that’s what they did.  Read the full study here.

It turns out that when you ask 65 undergrads (mostly women) to rank a whole bunch (204) of photos of people using no criteria other than what they look like, people might judge other people based on what they look like.  There was some lovely statistical analysis in here, but at no point did they attempt to prove that asking a 20 year old (who presumably had no first hand knowledge about any of the fields other than psych) to sort a picture reflected at all what goes on in hiring offices.

In fact, this is what the “practical implications” section of the paper said:

Although the findings reported here demonstrate the “what is beautiful is good” and “beauty is beastly” effects, it is important to address the likelihood of such stereotypes influencing actual employment decisions. For example, in situations where there is a high cost of making a mistake, as would be the case for a hiring decision, one would expect the decision maker to rely more on individuating information, rather than on stereotypes about physical appearance. However, it is important to note that the bias for the physically attractive, unlike other stereotypes, seems to impact impression formation in a broader range of circumstances. Recent meta-analyses suggest that the what is beautiful is good effect is pervasive, even when the perceiver has additional information about the target   (Hosoda et al., 2003; Langlois et al., 2000). Attractiveness may influence decision  making at a subconscious level, where exposure to an attractive individual elicits positive feelings in the decision maker, causing him or her to judge the target more favorably (Eagly et al., 1991). Moreover, in situations where a decision maker is under a high cognitive load or under time pressure, he or she may be more likely to rely on stereotypes (Fiske & Taylor, 1991; Pendry & Macrae, 1994).

So there is some proof that people favor attractive people no matter what, but no similar proof that they might discriminate against an attractive person if they had real world information.  Which leads me to get a little weirded out by quotes like this from the researcher (in interviews, not the article):

“In every other kind of job, attractive women were preferred,” said Johnson, who chided those who let stereotypes affect hiring decisions. 

Putting aside the fact that equality in this case appears to mean that everyone should prefer attractive people….what hiring managers was she chiding?  The ones she never studied?  Since the largest bias against attractive women was found when the mostly female undergrads were asked about who was qualified for male dominated fields….does that say more about what men think about women in non traditional fields, or what women think about women in non traditional fields?

While I’m sure that physical appearance does make a difference in hiring practices, I would have loved to see a little more time dedicated mimicking the real world before announcing that women were facing discrimination in certain professions.  To allow these results to be propagated as proof of what goes on at legitimate companies is a bit of a stretch, and points the finger at people who never even got asked what they would do.