Hazel Muir, "Scientists exposed as sloppy reporters" (2002)

Scientists exposed as sloppy reporters

A cunning statistical study has exposed scientists as sloppy reporters. When they write up their work and cite other people's papers, most do not bother to read the original.

The discovery was made by Mikhail Simkin and Vwani Roychowdhury of the University of California, Los Angeles, who study the way information spreads around different kinds of networks.

They noticed in a citation database that misprints in references are fairly common, and that a lot of the mistakes are identical. This suggests that many scientists take short cuts, simply copying a reference from someone else's paper rather than reading the original source.

To find out how common this is, Simkin and Roychowdhury looked at citation data for a famous 1973 paper on the structure of two-dimensional crystals. They found it had been cited in other papers 4300 times, with 196 citations containing misprints in the volume, page or year. But despite the fact that a billion different versions of erroneous reference are possible, they counted only 45. The most popular mistake appeared 78 times.

The pattern suggests that 45 scientists, who might well have read the paper, made an error when they cited it. Then 151 others copied their misprints without reading the original. So for at least 77 per cent of the 196 misprinted citations, no one read the paper.

Spread like weeds

Still, you might think that the scientists who cited the paper correctly had been more dutiful about reading it. Not so, say Simkin and Roychowdhury. They modelled the way misprints spread as each new citer finds a reference to the original source in any of the papers that already cite it.

The model shows that the distribution of misprinted citations of the 1973 paper could only have arisen if 78 per cent of all the citations, including the correct ones, were "cut and pasted" from a secondary source. Many of those who got it right were simply lucky.

The problem is not specific to this paper, the researchers say. Similar patterns of errors cropped up in a dozen other high-profile papers they studied. The trouble is that researchers trust other scientists to repeat the key message of a paper correctly. This means that when misconceptions take root, they spread like weeds.

Simkin and Roychowdhury promise that between them they read all the references listed in their own paper including a book by Sigmund Freud. Their advice to other scientists is "read before you cite".