We might assume that a paper has to exist to be referenced. But Kroonenberg and Harzing (2017) discovered a non-existent “phantom” paper that has been cited more than 400 times:
Van der Geer, J., Hanraads, JAJ., Lupton, RA. ([2000]2010). The art of writing a scientific article. J Sci Commun. 163(2):51–59
The phantom reference was part of a style guide used by Elsevier to illustrate how to format a reference for particular journals – it was a hypothetical example. Kroonenberg and Harzing attributed the accumulation of large numbers of citations to this phantom to a naive error by researchers who had used a styling template to produce their reference lists.
I followed this up. By 2019, this phantom reference had been cited more 480 times. Most citations came from conference abstracts, but 79 derived from peer-reviewed journal papers. Of these, 13 papers were connected together through references, and in all 12 accessible papers, this reference was being used to support the claim that a compound, rutin, could dilute the blood, reduce capillary permeability, and lower blood pressure. Here, we find an example of the propagation of error through chains of citation.
To explore this, I used the Web of Science and its ‘Cited Reference’ function; which lets researchers search for variants of particular references in the bibliographies of all indexed papers. After downloading all citing papers and their references and constructing a citation network, I looked for ‘clusters’ of papers. A citation network is one way of analysing how papers knit together through their references by representing all papers as vertices (circles in the below diagram) and all references as edges (directed arrows that point from the citing to the cited paper). Below, the phantom reference is depicted as the large black circle, its size reflecting the number of citations it received from papers in this network. At the top, in blue, are papers that referenced this phantom but which did not reference any other paper that had also referenced the phantom. These, it would seem, made the style sheet error proposed by Kroonenberg and Harzing. At the bottom of the network, however, in red, are 13 papers that referenced both the phantom and other papers that had also referenced it.
I then examined the 13 papers that shared references with other papers citing the phantom. I positioned each citing paper vertically by year of publication and horizontally based on their citation relations. The most cited documented is by Sun et al. (2008), who probably made the style-sheet error described previously. However, the papers published after this repeated the claim that Sun et al. had made, and used the same reference.
For this claim-specific citation network, I retrieved all accessible papers (12 in total – one was behind a pay wall that required me to spend more than $100 dollars to access the paper, which seemed an excessive cost to check for an error). I read all the papers, and copied exactly how each had referenced this phantom. In nearly every paper, the claim that rutin has certain health benefits was being justified by reference to this phantom alone.
For example, in Sun et al. (2008), we find:
“Rutin is often used as a therapeutical medicine with the functions in a wide range of circulatory problems, which can dilute the blood, reduce capillary permeability and lower blood pressure [1]”
Here, [1] points to the phantom reference. Two years later, in Yang et al. (2010), the phantom accompanied a very similar claim:
“For example, it [rutin] can be applied to the treatment of diseases, such as capillary bleeding by diluting the blood, reducing capillary permeability and lower blood pressure [6]”
Yes, [6] points to the phantom. Similar passages are found in the other papers:
Qu et al. (2011) – ““It [Rutin] can be applied to the treatment of diseases, such as capillary bleeding by diluting the blood, reducing capillary permeability and lower blood pressure [38]”.*
Wang et al. (2012) – “Rutin is a kind of flavonoid glycoside, called as vitamin P, which can dilute the blood, reduce capillary permeability, and lower blood pressure [26]” [Phantom is reference 26]
Zhou et al. (2012) – “It has been reported that rutin has the pharmacological actions of reducing capillary permeability, lowering blood pressure [3]” [Phantom is reference 3]
Yang et al. (2014) – “…rutin has many physiological functions, such as diluting the blood, reducing capillary permeability, as well as lowering blood pressure [1]” [Phantom is reference 1]
Gao et al. (2016) – “Rutin has many physiological functions, such as diluting the blood, reducing capillary permeability and lowering blood pressure [5]” ([Phantom is reference 5]
Saritha et al. (2017) – “Rutin is used to lower the blood pressure and it can also reduce the capillary permeability of the blood vessel through which some small molecules will flow [7,8]” [Phantom is reference 7]
In 2013, the claim made a brief transition into something a little more extravagant in a paper by Pinar et al. (2013): “It [Rutin] has clinically relevant functions, such as antibacterial, antioxidant, antihypertensive, anti-inflammatory, antitumor and anti-ageing. Furthermore, it can be used in the treatment of diseases such as capillary bleeding by diluting the blood, reducing capillary permeability and lower blood pressure [7-12]” [Phantom is reference 10]
For Miao et al. (2014), “Rutin has been also isolated from plants and used clinically as therapeutic medicine to reduce capillary permeability, alleviate pain, and lower blood pressure [4,5]" [Phantom is reference 5]
According to Zhu et al. (2015), “Rutin…is a kind of bioactive flavonoid glycosides, which is often used as anti-tumor, anti-inflammatory, anti-oxidants, etc. It is also used as a therapeutical medicine with the functions in a wide range of circulatory problems, which can lower blood pressure, reduce capillary permeability and dilute the blood [1]” [Phantom is reference 1]
It seems that papers copied references and claims from previous papers without checking the referenced paper.
Error propagation and referencing practices
In 2003, Simkin and Roychowdhury analysed errors in the references to papers (e.g. author name, page number, publication date etc.). Errors in a reference happen from time to time, as is only to be expected. However, the repetition of the same errors in papers by different authors suggests that authors might be copying references from other papers. By examining their propagation, and fitting a model in which authors copied references from other papers at some uniform rate, Simkin and Roychowdhury estimated that “only about 20% of citers read the original”.
In a later publication, these authors proposed a model, known as the random-citing scientists model, based on a simple rule – “When a scientist writes a manuscript, he picks several random recent papers, cites them, and also copies some of their references” (2007 p.1661). By this, they show that the model’s predicted citation distribution fit empirical citation distributions well and could explain other features of the dynamics of citations.
We will explore this paper and other models of citation dynamics another time. Before signing off, I’d like to emphasise that the phantom reference’s title was the “The art of writing a scientific article”…
Author: Rhodri Leng | Date: February 2020| Edited 09/10/2020
*Note of thanks to Adam Strandberg @The_Lagrangian for spotting an embarrassing error.
Notes
Harzing, A. (2017) “The mystery of the phantom reference”. Blog post. Stable URL: https://harzing.com/publications/white-papers/the-mystery-of-the-phantom-reference. Accessed: 15.04.2018
Sun, W., Yang, M., Li, Y., et al. (2008). Electrochemical behavior and determination of rutin on a pyridinium-based ionic liquid modified carbon paste electrode. J Pharm Biomed Anal. 48(5):1326–31
Yang, S., et al. (2010). Gold nanoparticles/ethylenediamine/carbon nanotube modified glassy carbon electrode as the voltammetric sensor for selective determination of rutin in the presence of ascorbic acid. J Electroanal Chem. 645(2): 115-122
Qu, LB , Yang, SL. (2011). Application of Carbon Nanotubes Modified Electrode in Pharmaceutical Analysis, in: Naraghi, D.M. (Ed.), Carbon Nanotubes. IntechOpen, Rijeka. https://doi.org/10.5772/17277
Wang, X., et al. (2012). Sensitive voltammetric determination of rutin at a carbon nanotubes-ionic liquid composite electrode. J Solid State Electrochem. 16: 2815–2821
Yang, S. et al. (2014). Decoration of graphene modified carbon paste electrode with flower-globular terbium hexacyanoferrate for nanomolar detection of rutin. Electrochimica Acta. 144: 268-274
Gao, Y., Wang, L., Zhang, Y., et al. (2016) Greenly synthesized graphene with l-glutathione-modified electrode and its application towards determination of rutin. RSC Advances. 6(96):P94024-94032
Saritha, D., et al (2017). A Sensitive determination of Rutin at Fe3O4@SiO2 [FS-MCPE] - A Voltammetric Study. Analytical & Bioanalytical Electrochemistry 9(4): 506-520
Pınar, P., et al. (2013). Voltammetric behavior of rutin at a boron-doped diamond electrode. Its electroanalytical determination in a pharmaceutical formulation. Open Chemistry
Miao, D., et al. (2014). Supersensitive electrochemical sensor for the fast determination of rutin in pharmaceuticals and biological samples based on poly(diallyldimethylammonium chloride)-functionalized graphene. J Electroanal Chem.
Zhu, X., et al. (2015). Multi-walled Carbon Nanotubes-Aminate Reduced Graphene Oxide Modified Glassy Carbon Electrode as the Voltammetric Sensor for Sensitive Electrochemical Determination of Rutin. Int J Electrochem Sci. 10: 9192-9204
Simkin, MV., Roychowdhury, VP. (2003). Read before you cite! Complex Syst. 14:269–74
Simkin, MV., Roychowdhury, VP. (2007). A mathematical theory of citing. J Am Soc Inf Sci. 58(11):1661–73