Five Reasons Not to Use a Blog Post as a Reference

Recently I had a discussion with a friend from childhood who is now a teacher. She had liked my “Intro to Internet Science” series, and we were discussing the possibility of me coming and chatting with her AP chemistry class about it. We were discussing time frames, and she mentioned it might be best to come in April when the kids started writing their thesis. “Every year they get upset I won’t let them use blog posts instead of peer-reviewed journal articles.” she said.

Oh boy. As a long time blogger who likes to think she’s doing her part to elevate the discourse, let me say this clearly: NEVER CITE A BLOG POST AS A PRIMARY SOURCE. Not even mine.  Here’s why:

Anybody can be a blogger. One of the best things about blogging is that it’s an incredibly easy field to enter. It takes less than 15 minutes to set up a blogger or WordPress account and get started. It takes about $20 to register a custom domain name. This is awesome because you can hear lots of voices on lots of topic you wouldn’t have otherwise had access too.  This is also terrible because there are lots of voices on lots of topics you wouldn’t have otherwise had to deal with.

Nothing stops people from fabricating credentials, using misleading titles or just flat out making stuff up. Don’t believe me? Health and wellness blogger Belle Gibson built an enormous empire based on her “I cured my cancer through whole foods” schtick…..only to have it revealed she never had cancer and had no idea what she was talking about.

Peer review isn’t perfect, but any deception perpetrated in published papers will have taken a huge amount of time to pull off.  Simply out of laziness, that means there will be less outright fraud (although it does still happen).

No one checks bloggers before we hit publish. Like many bloggers, I do most of my blogging late at night, early in the morning or on weekends. I have a full time job, a husband, a child, and I take classes. I’m tired a lot. Despite my best intentions, sometimes I say things poorly, let my biases slip in, or just do my math wrong1. I happen to have smart commenters who call me out, but it’s plausible even they miss something.

I try to adhere to a general blogger code of conduct and provide sources/update mistakes/be clear on my biases when I can, but I will not always be perfect. No one will be. With peer-reviewed papers, you know MANY people looked at the papers before they went to press. Doesn’t make them perfect, but it does mean they’ll far less likely to contain glaring errors before publication.

Also, good bloggers talking about a scientific paper will ALWAYS cite the primary source so you can find it and see for yourself. Here’s a few rules for assessing how they did that.

Blog posts can mislead. While many bloggers are driven by nothing more than a desire to share their thoughts with the world, many are doing it for money or other motivations. Assuming that blog posts are actually marketing tools until they prove otherwise.  I wrote a whole 10 part series on this here, but suffice it to say there are many ways blog posts can deceive you or make things sound more convincing than they are.

Science changes, but the internet is forever. Even if you find a good solid blog post from a thoughtful person who cited sources and knew what they were talking about, you’re still not out of the woods. The longer the internet sticks around, the more things will outdate or need updating, even if they were right at the time the author wrote them. I’ve started a series where I go back to posts I wrote back in 2012/2013 and update them with new developments, but nothing will stop Google from pulling them up in search results as is.

Using blog posts robs you of a good chance to learn how to read scientific papers. Reading scientific papers is a bit of an art form, and it takes practice. Learning how to find critical information, how to figure out what was done well (or not at all!), and doing more than just reading the press release can take some practice. Everyone has a slightly different strategy, and you’re not going to find the one that works for you unless you read a lot of them. If you’re still at the point in your life where you have external motivations to read papers (like, say, a teacher requesting that you do it), take advantage of that. It’s a skill you’ll value later, one of those “you’ll thank my when you’re older” things.

In conclusion: One of my favorite blog taglines ever is from Scott Greenfield’s Simple Justice blog “Nothing in this blog constitutes legal advice. This is free. Legal advice you have to pay for.” Same goes for science blogging. If it’s free, you get what you pay for.

1. I’m actually perfect, but I figured I’d throw the hypothetical out there.

5 rules for reading "scientific papers" online

This weekend I stumbled in to a very dark place on the internet.

It wasn’t political, oh no, it was something much much scarier, with far more demeaning adherents.

It was the world of parenting theory.

I’m not going to name the theory, because quite frankly, I’m afraid they’d find me….but lets just say it’s considered pretty fringe by most people.  Don’t tell them that though, they went after critics (and even people just asking questions about tiny modifications) with a viciousness that made CNN commenters look rational.

Anyway, as a response to many questions, they kept linking to a paper by a doctor who was part of their movement to prove any and all points.  Finally, I decided to go see what all the fuss was about.

I won’t link to it here, because I am seriously scared of these folks, but the paper got me thinking about how often this sort of thing appears on the internet, and things people should look for when consuming “scientific research” that shows up on random websites.  Many readers of this blog are probably already good at this, but for those who may not be so savvy, here we go:

  1. Remember that pretty much anything can call itself a “scientific paper”. When I clicked on the link above, I was surprised to find it was really just an opinion piece by a doctor.  While I’ll take it at face value that she actually exists and has the credentials she claimed, this was not a peer reviewed paper published in a journal.  That’s fine, but calling it a “scientific paper” definitely gave it a little more credit than it deserved….it’s like calling something in the supermarket “natural”.  It sounds good, and you think you know what it means, but there’s really no standard for it.
  2. Beware people who jump around a lot.  I don’t mean to be picky, but for the life of me I couldn’t figure out what the point of this paper was.  There was no thesis statement, just lots of disjointed “some critics say this” followed by some anecdotes and then the next topic.  It doesn’t sound that bad, but believe me, it was.  When I finished I went back through and realized that she was giving just enough rebuttal to raise doubt, but moving on before she had to make a full argument.  Any one of her comebacks would have fallen apart if it had been responded to, but by jumping around with her topics it made you forget what you were objecting to.  Haven’t we all had an in person argument like this?  Where the person never responds to a counter argument, they just throw another objection at you?
  3. Follow up on citations.  I had a few minutes to kill while I was reading this paper, so I decided to actually follow the citation trail she gave for her points.  It was amazing how few of them said what she said they said.  Many of them were close, but a few were downright deceptive.  This was uncovered merely by reading the abstract.  I can only conclude they were either mistaken links or the author thought no one would follow up.
  4. Don’t presume the citation supports the entire sentence.  This was the most deceptive part of the paper.  At least twice there was a sentence that said something like “You should do A because A leads to B which helps with C and that prevents D from happening, as researcher X has proved”  Now none of these were intuitive connections, all were based on her particular strategy.  When I followed up on researcher X’s work, all the paper said was that C prevented D….which was the least controversial statement.  She offered no actual proof that her methods A and B would actually increase C…but her links implied it was there.
  5. Watch out for nitpicking. I know, I’m the last one who should call someone out for nitpicking other people’s studies.  However, this is a favorite technique of those trying to prove a point.  They take opposing research, point out a weakness, then proclaim the study invalid because of this.  The problem is that some of it doesn’t actually make studies invalid.  Sometimes it’s minor things like “those studies on gender pay gap didn’t include transgendered persons so they don’t really prove anything!”.  In this case she faulted the opposition for using too narrow of a definition for one of her recommended practices, and then two paragraphs later mentioned that the narrow definition was really the one she advocated for.
While I stumbled on these in parenting theory, I’ve seen similar on nutrition/lifestyle websites in particular.  I’m sure there are other examples and other rules.  Would love to hear your thoughts.