A reader sent along an article recently with a provocative headline “U.S. not in top 60 nations for equality for working women“, along with a request for my opinion. As a working woman I was of course immediately interested in which 60ish countries were ahead of us, so I took a look. The article itself is based on a report from the World Bank called “Women, Equality and the Law 2019” which can be found here. It looks at laws in 8 different areas that impact women’s equality in the workplace, and assigns countries a score based on whether they have them or not. The areas looked at were:

• Going places: constraints on freedom of movement
• Starting a job: analyzes laws affecting women’s decision to work
• Getting paid: Measures laws and regulations affecting women’s pay
• Getting married: Assesses legal constraints
• Having children: Examines laws affecting women’s ability to work after having children
• Running a business: Analyzes constraints to women starting and running businesses
• Managing assets: Considers gender differences in property and inheritance
• Getting a pension: Assesses laws affecting the size of a woman’s pension

Given this approach, it seems that the initial headline was a little misleading as to what this study actually found. This study looks only at the existence of laws, not whether they are enforced or not or actual equality for women. Headline writers gonna headline, I guess. The article itself was better, as it covered the basics of the report and why legal equality is important. Snark aside, I would absolutely prefer to live in a country that gave me the legal right to sign a contract or inherit property over one that didn’t have such a law, even if the law was imperfectly enforced. So where did the US fall down? Well, we had a “no” on 6 different questions, which gave us a score of 83.75. These were in three different categories:

• Getting paid: “Does the law mandate equal renumeration for work of equal value”, the US got a “no”
• Getting a pension: “Does the law establish explicit pension credits for periods of childcare?”, the US got a “no”
• Having children, we had “no” on 4 questions:
  • Is there paid leave of at least 14 weeks available to women?
  • Does the government pay 100% of maternity leave benefits, or parental leave benefits (where maternity leave is unavailable)?
  • Is there paid paternity leave?
  • Is there paid parental leave?

Now this struck me as interesting. Our biggest ding was in the one category where they also asked about men’s legal rights, i.e. paternity leave. In other words, the US got marked as unequal under the law because men and women were equal under the law, but in the wrong direction. I can see where they were going with this, but it’s an interesting paradox. I was curious about their justification for putting these laws up there along with the other ones, so I read the explanation they provided. From page 6 of the report: “Women are more likely to return to work if the law mandates maternity leave (Berger and Waldfogel 2004). Though evidence on the impact of paternity and parental leave is mixed, parental leave coverage encouraged women to return to work in the United Kingdom, the United States and Japan (Waldfogel, Higuchi and Abe 1999).” To note, this was the only criteria they included where they explicitly stated the evidence was mixed. Now as part of this report, the World Bank had stated that legal equality was correlated with equal incomes and equal workforce participation, and showed this graph to support its claim: Now this graph struck me as interesting because while it does show a nice correlation (that they explicitly remind everyone may not equal causation), the correlation for countries scoring above the mid-70s is much less robust. While workforce participation for women in countries earning a perfect score is very high (left chart), it’s interesting to note that the pay ratio for men and women in those countries goes anywhere from the 50 to 80% range. I looked up the individual numbers for the 6 countries getting a perfect score (Belgium, Denmark, France, Latvia, Luxembourg and Sweden) and the US, and found this. The labor force participation ratio and the F/M pay ratio are from the reports here. I added the other metrics they put under “Economic Opportunity and Participation” just for fun.

So out of the 6 countries with perfect scores in legal equality, 2 have overall economic participation/opportunity scores higher than the US, 1 is equal, and 3 are lower. Many countries that have high scores end up with high numbers of women in the workforce, but low numbers in positions of power (2nd and 3rd rows from the bottom). The US and Latvia led the pack with women in higher profile jobs, even with the US having (relatively) low workforce participation.

Oh, and in case you’re wondering how Luxembourg got a perfect income ratio score but not a perfect equal pay score: incomes for the ratio calculation were capped at $75k. Luxemborg has a very small population (600,000) and a very high average income (about $75K), so it really kinda broke the calculation. Including incomes above $75k would probably have changed that math.

Now, I want to be clear: I am not saying legal equality isn’t important. It is. But once you get beyond things like “are women allowed to legally go out at night” and in to things like “do pensions give explicit credit for child care”, the impact on participation is going to vary a lot. Different cultures have different pluses and minuses, so the same culture that gives extensive maternity leave may not end up encouraging women to go for the highest professional jobs. In cultures with low relative incomes like Latvia, women may use their legal equality to get better paying jobs more often. This confirms the pattern we see in engineering and science degrees….the most gender equal countries are not the ones producing the most female science grads.

Still, the hardest thing about calculating equality is probably relative income. While Latvia may have more women in higher level jobs, most women would prefer the US average income of $43k to the Lativian $20k. While many of the other top countries are OECD countries, some of the ones scoring the same as the US were not. Even with equal legal situations, I’d imagine that life for women in the Bahamas, Kenya, Malawi and the US are very different. That doesn’t make the metric wrong, it just means equality can be very different depending on where the overall median is.

One final thought: it struck me as I was researching these countries exactly how big the US is in comparison. If you add up the populations of the 6 countries with perfect scores, you get about 96 million people. This is less than a third of the 325 million people who live in the US. Large countries tend to be less equal than small ones, and I do wonder how much of that is simply supporting a large and disparate population. The US is the third largest country in the world by population, and the first one with a higher legal equality score than us is #10 on the list, Mexico with 38% of our population and a score of 86.25. The next one to have a higher score is #17 (Germany) then France. To test my theory, I put together a graph of population size vs WBL index. Note: it’s on a log scale because otherwise China and India just kinda dwarf everything else.

So basically no country with a population over 100 million has gotten over a score of 86. Speculating on why is a little out of my wheelhouse, but I think it’s interesting.

And with that I’ve gone on long enough for today, but suffice it to say I find global statistics and inter-country comparisons fascinating!

The AVI sent this along, and man do I feel seen.  Source.

This is an utterly bonkers NSFW article about how alarmingly easy it is to fake your credentials if you talk about sex, try this one.  Seriously, is there any other field where you could get this much media attention with this little media scrutiny? Anything more political would have been looked in to by opponents, and anything more boring probably wouldn’t have gotten media attention.

I took this creativity test and got  76.32. My breakdown was pretty accurate. My best moments of creativity come when I’m trying to improve the status quo  (curiosity – my highest), but I don’t really go much further once the problem is fixed (persistence – my lowest). Applied laziness is kinda my super power.

Big news in the meta science world last week, when Ben Goldacre (of Bad Science and Bad Pharma fame) released some new studies about ethical standards in scientific publishing. The studies called “COMPare: a prospective cohort study correcting and monitoring 58 misreported trials in real time” and “COMPare: Qualitative analysis of researchers’ responses to critical correspondence on a cohort of 58 misreported trials” wanted to look at what happened when study authors didn’t follow the ethical or quality standards that the journals they published in set forward. The first paper looked at the journals response to issues that were pointed out, the second looked at the response of the paper authors themselves. Goldacre and his team found these papers, found the errors, then pointed them out to the journals to watch what happened. Unfortunately, it went less well than you would hope.

Before we get in to the results though, I want to give a bit of context to this. Over the past few years, there’s been a lot of debate over how to ethically “call out” bad publication patterns. The Calling BS guys have a whole section about this in their class “The Ethics of Calling Bullshit”, which I wrote about here. To highlight concerns about scientific publishing people have published fake papers, led replication efforts, and developed statistical tools to try to ferret out bad actors. In all of these cases, concerns have been raised about the ethics of each approach. People have complained about mob mentalities or picking on individual researchers, taking advantage of people’s trust, or using these things to advance their own careers.  “Science is already self-correcting” the complaint goes “no need to make a bigger deal out of it”.

I have to think Goldacre had this in mind when he designed this study. His approach is fascinating in that it actually shares the blame between journals and authors, and also focuses heavily on the ability of people to respond to criticism. Journals tend to point to their ethical/quality standards when proving that they are concerned about quality of studies they publish, but it is often unclear how those standards are actually enforced. Additionally, issues with a journals standards or enforcement are a big deal with a widespread impact. Finding a study author who made a mistake or committed fraud is great, but still only impacts the person in question. Finding out a journal has a systemic issue can have ripple effects to hundreds of studies, and a whole field of research. To highlight this fact, Goldacre and his team specifically looked at some of the biggest journals out there: the New England Journal of Medicine (NEJM), Journal of the American Medical Association (JAMA), Annals of Internal Medicine, British Medical Journal (BMJ) and Lancet. No small fish in this pond.

In the first study, the journals and their responses were the focus. Goldacre and his team looked at 67 trials and found 58 had issues.  The metrics they were looking for were simple: did the papers publish their publicly available pre-trial registration outcomes, or did they explain any changes from their original plan. These are the basic requirements laid out by the CONSORT guidelines (found here) which all the journals said they endorse.  Basic findings:

• Only 40% of their letters were published
• JAMA and NEJM published NONE of the letters they received
• Most  letters were published online only
• Letters that were published in the hard copy journals were often delayed by months

Now the more concerning findings were grouped by the researchers in to themes:

• Conflicts with CONSORT standards Despite saying they endorsed the CONSORT standards, when instances of non-compliance were pointed out to them the journals said they didn’t really agree with the standard or think it was necessary
• Timing of pre-specification/registries in general: Several journals objected that actually the trial pre-registrations were done too early or were too unreliable to go by.
• Rhetoric: This was my favorite category. This is where Goldacre et al put complaints like “space constraints prevented us from adding a reason why we changed our outcome metric”, with a note that mentioned that they had plenty space to add new and interesting outcomes. They also got some “we applaud your goal but think you’re going about this poorly”.
• Journal processes: This one was weird too. Journals clarified that they asked authors to do things by the book, despite Goldacre et al showing that the authors weren’t actually doing those things. Odd defense.
• Placing responsibility on others: Sometimes the journals claimed it was actually up to the reader to go check the preregistration. Sometimes they said it was the preregistration databases that were wrong, not them. The Lancet didn’t reply at all and just let the authors of the paper in question respond.

The paper goes on to also summarize the criticism they got from journal editors once reporters started asking. Just scroll down to the tables in the paper here to read all the gory details. The summary of the responses for individual journals was pretty interesting too:

• NEJM: Published no letters, said they never required authors to adhere to CONSORT. Provided journalists with a rebuttal they had not sent to Goldacre et al.
• JAMA: Published no letters, said they didn’t have enough detail to find the errors. Goldacre points out they have a word limit, and that they linked to their full complaints on their website.
• Lancet: Published almost every letter, but its editors didn’t reply to anything.
• BMJ: Published all letters and issued a correction for one study out of 3
• Annals: Got in a weird fight with the COMPare folks that has its own timeline in the paper

Overall, the results seem to suggest that there are still a lot of work to be done in getting journals to adhere to clear and transparent standards. They suggested that something like CONSORT should perhaps have a list of those who “endorse” the standards and those who agree to “enforce” the standards.

They also noted that they were actually quite surprised by the number of responses that they got saying that trial pre-registrations were inaccurate or not useful, because they noted that journals were actually one of the driving forces behind getting those set up. The idea that they were useless/not their problem was a very troubling rewrite of history.

Interestingly, the COMPare folks noted that for all the back and forth, they had a feeling their findings might actually be making a difference. They plan on doing a follow up study to see if anything’s changed. Something about knowing people are watching does tend to

Alright, I’ve gone on a bit on this one, I’ll wait until next week to review the second paper.

A few weeks ago, the Assistant Village Idiot was at my house and noted, with pleasure, that one of his favorite books (Albion’s Seed) was on my bookshelf. Sheepishly, I had to confess that the copy on my shelf was actually one he had lent me “about a year ago”.

He apparently had not remembered this, but remarked that he had bought a new copy as he was unclear what had happened to the previous one. This is a very AVI problem to have, but he also remarked that he was pretty sure the book had been lent closer to two years ago.

Being properly chagrined at this conversation, I promptly did two things:

1. Began reading the book immediately, which I hadn’t quite gotten to yet
2. Looked up the email exchange we had about the book, to figure out when exactly he had lent it to me. October 2017 – 16 months, for those keeping track at home.

In other words, my one year was generous, but I’m not quite so bad as two years. Also, if you lend me a book, you may want to put a time limit on it.

Anyway, I was thinking of this today because I got to an interesting part in the book that had to do with the cultural insights that come from noting how people tweak the truth.

For those of you who have never read Albion’s Seed, the basic premise is that different parts of the US were settled by people from different parts of England, and that led to cultural differences that persist until the current day. I’m still in the part about New England, but they note that the Puritans who settled here highly valued those who lived to older ages. This case is proved in part by the fact that the census data showed that after a certain age, more people claimed to be older than they actually were. Instead of having a relatively even spread between ages like 69, 70 and 71, they found that more people said they were 70, and fewer said they were 69:

This is the reverse of what we see today, where people tend to say they are a little younger than they are. This trend actually did not remain constant with the Puritans, rather the age bias only came in to play in the late 50s:

This struck me as fascinating, because we so often think of lying on a survey as a bad thing. However, when you have access to comparison data, people lying on surveys can actually be helpful. Since everyone tends to exaggerate in a direction that makes them look better, figuring out what direction people lie in can actually give you good insight in to what a culture values.

Things to consider.

Under the weather this week, but I wanted to post this excellent graph from Twitter, showing what different Cohen’s d effect sizes mean in practice for populations:

I love this because despite the efforts of many many people, every time you see some sort of “group x is different from group y” type assertion, you still see people claiming that this either:

1. Can’t be true because they know someone in group y who is more like group x or
2. This  is completely true and every from group x is superior to every member of group y

Both are mistakes. For a more detailed look, there’s a visualization tool here that shows what these translate in to for random superiority. In other words, if you pick one person from each group randomly, what is the chance that the one from the higher group will actually have an outcome above the other person?

• For d=.2, it’s 55.6%
• For d=.5, it’s 63.8%
• For d=.8, it’s 71.4%
• For d=2, it’s 92.1%

This is good to keep in mind, as Cohen’s d is not an overly intuitive statistic for most people. Visualizations are good to help see quickly what these differences might mean on the population level.

Not the most groundbreaking point, but one that seems to bear repeating!

As long as humans have been around and arguing with each other, there have always been disputes about what the interpretation of events. During these disputes, I’d imagine that people wished we had ways of capturing events in real time, believing that this would eliminate disagreements. If everyone could work from a shared set of facts, then of course we would end disputes, right?

While I’m sure that’s what I would have thought if I’d lived 100 years ago, our recent age of ubiquitous cell phone cameras and real time Twitter updates has taught all of us that things are not so simple. Every time we see an example of this, I think of one of my favorite words: parallax. Defined as “the apparent displacement of an observed object due to a change in the position of the observer“, it reminds me that our perception of things sometimes depends not only on the object itself, but also where you’re standing.

If this is true of physical objects, then of course emotional situations up the ante. In the best of circumstances human communication can be prone to difficulties, and differences in perception can complicate things enormously. While the promise of technology is often that it will improve communication, the reality is that it often just creates new opportunities for differences in perception.

I bring this up because I had a really interesting example of this in my personal life recently, when the same Twitter thread led to two really different conclusions.

In the middle of one of the many many Twitter controversies of the past few weeks, I noticed some rather high profile people reacting to a Twitter thread from someone I knew was an acquaintance of my brother. Their reactions were not kind, and she was generally getting kinda dragged. He and I had talked about the issue previously before I knew she had jumped in, so I texted him the thread as an example of what I considered a Bad Opinion.

Basically while opining on the issue of the day, this woman had tried to make point X, but in the process had (IMHO) completely minimized counterpoint Y to a comical extent. My opinion was shared by others, who were mocking her for it.

When my brother and I talked a few days later, I mentioned the incident, and was surprised to find he disagreed with me. He said he didn’t at all see that she had minimized point Y, and in fact had emphasized point Y with points Y₁ and Y₂. At this point I got confused….I hadn’t seen either of those points made. My mind spun a bit. My brother and I have been arguing for years, and I knew he wouldn’t make something like that up. I was in the car so I couldn’t check the Twitter thread, but I started to wonder how I had missed the points she had made. Had I been scrolling too fast? Had I been projecting? Had I jumped to conclusions? I admitted to my brother that it was possible I’d missed something, and agreed that if I had I’d misjudged his friend.

Later that night it was still bothering me, so I went back to my text and reread the Twitter thread. I read all 31 Tweets she had sent, and discovered that neither point Y₁ or Y₂ were in there. Now I was really confused. Like I said, my brother is one of my oldest sparring partners. We’ve been arguing for decades, and at this point I know he would never fabricate a point like that. I also knew that he would have mentioned it if he’d seen it elsewhere. So what the heck had happened?

As I pondered this, I reflexively hit the button “show more replies” to see if I could see some of the reactions he had mentioned and found….there were 4 more Tweets in the thread. Apparently at some point after she had sent the one she labelled 31/31 and said “thanks for listening”, she had added on a few other points. I’m not sure, but I think that because of the time delay between the initial Twetts and the add ons, Twitter hid those Tweets from anyone who accessed the thread directly. Since she had indicated it was the end of the thread, no one reading it that way would have known to go looking for other Tweets. When I clarified this with my brother, he mentioned that when I’d sent him the thread he hadn’t actually clicked on it, he’d just gone directly to her Twitter feed. Since Twitter shows things in reverse chronological order, he had read her added on points first, and then read everything else through that lens. No wonder we’d ended up with different opinions.

I was very struck by this whole thing, as it got me wondering how often this happens in our everyday lives. We believe we’re seeing the same thing due to the promises of technology, but the way it’s presented to us skews our reading. If my brother and I didn’t have years of good faith arguing behind us, I would likely not have been so curious about our different perceptions. If we weren’t both so interested in how information gets presented, we may not have cared or considered how our way of accessing the information had colored our subsequent reading of it. Our belief that we were both seeing the same thing might have actually impeded our communication rather than helped it.

I don’t have a good answer for how to get around this, but it’s good to keep in mind as more disputes are started and perpetuated online. While eliminating some pitfalls, technology does create new ones on a much grander scale. Just because you’re looking at the same thing doesn’t always mean you’re seeing the same thing.