Should papers that are clearly biased be published?

26 Jul 2017

You may have heard the recent news about how playing American football leads to brain injury. At least that’s what the headlines are saying. Headlines such as the click-baity “111 N.F.L. Brains. All But One Had C.T.E.” from the New York Times and the apocalyptic “The CTE Study That Could Kill Football” from Forbes. But if you read the articles, you’ll quickly learn that the study only includes brains that were donated to a program studying repetitive head trauma. If you take a look at Table 3 in the original article, it’s pretty clear that the majority of the individuals whose brains were donated exhibited signs of cognitive decline. In other words, these brains were coming from individuals who were suspected of having possible brain damage. In the words of the article’s primary author, Dr. Ann McKee,

“Families don’t donate brains of their loved one unless they’re concerned about the person. So all the players in this study had - on some level, were symptomatic. So that leaves you with a very skewed representation.”

The headlines coming from NPR, CNN, and ESPN were all worded in such a way as to imply that the study was limited in scope.

While the news articles varied in their headlines, at least they all mentioned the bias involved in the study in the body of the story. Unfortunately, however, these disclaimers don’t show up until after the article gives a summary of the research. If someone doesn’t finish reading the article, they are going to miss the disclaimer and go about their day believing this to be a comprehensive study. In contrast to the news reports, the original article makes it crystal clear from the beginning that they were not able to conduct their study on a representative population.

So all this brings me to my question.

“Should papers that are clearly biased be published?”

Let’s first examine why they should be published. Even though this article is limited to only those brains which were donated on suspicion of brain damage, the fact that the majority did have brain damage is convincing evidence that this is an area worthy of research. Whether or not football is to blame for the brain damage remains to be seen, but this study hints that such may be the case.

Why shouldn’t biased articles be published? I think an obvious reason would be the headlines that this particular article generated. This study is being used as click-bait and as a sensational headline. There is a big push for people to be educated, and maybe the sites using this article as click-bait will say that is what they are doing, but it is clear that the results of this study are being exaggerated when you factor in the bias. But how the article is portrayed in the media is out of the hands of the authors. Even a perfectly designed and executed study could be used in such a way. On a more scientific note, it is hard to say whether or not the results of this study are meaningful due to the bias. The prevalence of brain damage might not actually be higher in NFL players than in the rest of the population, or it could be due to some other factor besides football. We just can’t say with the data that’s been collected.

If news companies were better with their headlines and with how they summarize research articles, I don’t think it would be too big of a deal if an article is published which discloses any potential bias. Those involved with the study did make a finding and it is a good starting point for future research. There are some fields where bias is unavoidable, and I think research involving human brains is definitely one of those fields. It is always better to avoid any sort of bias in your research, but recognizing your sources of bias is the next best thing. Unfortunately, scientific articles are often difficult even for scientists to read and understand, and to further add to the issue, articles are often locked behind paywalls. Since these articles can be inaccessible to the average person, we rely on news sources to distill their findings into easy to understand news articles.

So what is one to do? Since eliminating bias completely isn’t feasible, my recommendation is to always try to find the original scientific article and read it critically to make sure you are understanding what is actually going on. If you are unable to do that, try to find the same story from a few different sources to try to “average” things out.