Anchoring effects in the assessment of papers

In this study, we empirically study the assessment of cited papers within the framework of the anchoring-and-adjustment heuristic. We are interested in the question whether citation decisions are (mainly) driven by the quality of cited references.

In 2022, we undertook a first survey of corresponding authors with an available email address in the Web of Science database. The authors were asked to assess the quality of papers that they cited in previous papers. Some authors were assigned to three treatment groups that received further information alongside the cited paper: citation information, information on the publishing journal (journal impact factor), or a numerical access code to enter the survey. The control group did not receive any further information.

Our results show that the quality assessments of papers seem to depend on the citation impact information of single papers. The other information (anchors) such as an arbitrary number (an access code) and journal impact information did not play a (important) role in the assessments of papers.

The study was preregistered. An article with the results was published in March 2023.

Between December 20203 and March 2024, a second survey was conducted in which we can better control for the actual quality of the papers than it was possible in the in the first survey. The second survey was also preregistered.

Research team: Lutz Bornmann¹, Christian Ganser², Alexander Tekles^1,2 (first survey only)

^{1 Science Policy and Strategy Department, Administrative Headquarters of the Max Planck Society, Munich,
Germany, 2 Department of Sociology}

^{Bild von Mudassar Iqbal auf Pixabay}