"I wish we could all go on a plane to Disney World" slowly changes to
"A fish swims on time to Disney World"....
and so on. Before you know it, Disney World isn't even mentioned.
Why does that happen? Is is because we are hard of hearing, or because we hear only what we want to hear?
This "broken telephone problem" is a rampant problem in the news media as well as in science. People's background and view of the world affect how they receive information, and scientists are not immune. Our background, training, and belief systems affect our presumably logical reasoning. This is one reason why the scientific method is so important: it gives scientists a framework for formulating, testing, and evaluating hypotheses. The peer review process is a related system of checks and balances that helps results of experiments become accepted knowledge through the repetition of experiments and evaluation by qualified colleagues. Results become conclusions, which through repetition, become facts, and eventually theories.
Almost all current scientific writing draws on past scientific facts and theories to provide background for, justify, compare or contrast with the experiments being presented or proposed. Scientists receiving funding support from a granting agency need to justify the necessity of the supported project and how it will advance scientific knowledge or provide answers to questions for the common good. Justification of an idea can come through a clear and logical explanation, but most scientists use the results and conclusions of previously published research to justify similar research or conclusions. The more scientific literature I read on a common topic, the more it reminds me of the law field, where lawyers or even Supreme Court justices use logic to evaluate a present situation and apply the conclusions of a similar decision from the past.
Thomas Kuhn, philosopher of science, coined the term "paradigm shift" (http://www.molwick.com/en/scientific-methods/041-scientific-methodology.html)
Drawing from the knowledge and conclusions of past research and building upon it is vitally important for the advancement of science, but this system will only work efficiently if the logical links between past and current research are accurate. These logical links go something like this:
Past paper: Hypothesis -> Test of hypothesis -> Results -> Conclusions (support/do not support hypothesis) ->
New paper (generally in introduction and/or discussion section): Evaluation of past paper's hypothesis
-> Critique of whether methods were appropriate, results were reasonable, conclusions were supported by results -> Synthesis of past paper with other past papers -> Translation and application to new problem or the next step -> Justification for new research
That is a lot of logical links. A lot of people in the game of telephone. A lot of potential for broken wires. There is even more potential for breakage when this process is sped up.
Ten or twenty years ago, the publishing process was faster and there were a lot fewer papers being published. Therefore, the conclusions of one paper could be addressed reasonably by a similar study a year or two later, and there was some dialogue between scientists in the literature. Even ten years ago when I started my MS degree and was writing my prospectus, I had to visit the library to photocopy the papers I looked up by title in a database, but I could copy and read the major papers on the topic in a day or two. Over the course of my MS, I added maybe 5-10 papers to my thesis that were recent publications that had come out since my prospectus. I had some time to assimilate the information I read, formulate my own research questions, conduct the experiments, and write up the results as they related to previous work in the field. Now I am writing my prospectus for my PhD, and every time I search the literature (ahem, Google Scholar), there are 5-10 more papers for me to read. Do I read them all? No. Could I possibly? If I did nothing else, maybe!
What happens when scientists are overloaded with reading in the field, reading in related other fields, and sitting in front of Google Scholar with 30+ recent abstracts in front of them while they write the introduction to their paper? Do they individually download, read, and evaluate each of these studies before drawing conclusions that can be addressed in their own manuscript? What if there are some older "classic" papers in the field that should be acknowledged in the introduction, but that are not online? What if your institution has gotten rid of most of its paper journal collection to save space, in favor of all electronic journal databases? What if some of the papers needed are in a journal not supported by your institution's database? Would you get an interlibrary loan for those 30+ papers? Would you pay cash for the electronic version?
What if the papers you do have cite that older paper as evidence for the phenomenon you are looking to describe, but you can't find that older paper?
I'll tell you what many scientists do. They cheat. They either (1) ignore the difficult-to-find paper that keeps coming up in the database and look for a different paper to cite, (2) use just the abstract from the paper to draw a conclusion that is cited, or (3) use the secondary source (someone who cites the primary data collector) and either use the secondary source's reference, or misattribute the data to the secondary source rather than the primary source.
It seems like an innocent (though admittedly lazy) process, but it can result in real problems. Exaggeration of conclusions, giving credit to the wrong authors, even completely getting it wrong by mis-reading or mis-quoting a sentence or two. But is this really cheating? Is it lying? Is it misconduct? Probably not in the same way as falsifying data or plagiarism, but it does cause similar problems in the scientific process. It causes erosion and breaks in the logical chain that advances scientific knowledge. It causes paradigms to be created that are not supported by reason. It causes misunderstanding and mistrust of science by the general public. It can result in policy decisions based upon consensus belief rather than data.
My husband, Rob Condon, is working on a paper with colleagues addressing the "broken telephone" phenomenon as it applies to the paradigm of jellyfish blooms increasing worldwide. The belief in this paradigm by the public as well as the scientific community is astounding, considering the lack (so far) of scientific data on the subject.
Some other students and I have been getting frustrated at the number of broken logical links and misattribution (AKA "citation trail") we have been finding during our literature searches, so we have been talking about starting a published list of the broken telephone offenders that we find. Maybe it will bring a little more recognition to the topic.
In conclusion:
- Don't be lazy
- Go ahead and skim the abstracts, but if you're putting something in your paper, make sure you've read it thoroughly
- Remember that the author you are citing might very well be your reviewer
- (On the other hand) Many, many papers play the broken telephone game and make mistakes, and the only way to keep from magnifying this misinformation is by going to the original source and evaluating their data and their conclusions for yourself
- Getting caught would be really embarrassing and you don't want to end up on our List of Broken Telephone Shame.
I would like to acknowledge my advisor Ruth Carmichael for encouraging lively discussion on this topic in her Scientific Communication course, and Kelly Robinson for her Facebook status update inspiring me to finally write something about this pet peeve!
Please submit examples of misattribution, misquotation, exaggeration, and other breaks in the scientific telephone line in the comments below! The more specific you are, the better. Thanks!
