A quantitative analysis of peer review

Azzurra Ragone, Katsiaryna Mirylenka, Fabio Casati, Maurizio Marchese

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

In this paper we focus on the analysis of peer reviews and reviewers behaviour in a number of different review processes. More specifically, we report on the development, definition and rationale of a number of appropriate quantitative metrics to assess the processes main properties. We then apply the proposed metrics and analysis framework to data sets from conference evaluation processes and we discuss the results implications and their eventual use toward improving the analyzed peer review processes. A number of unexpected results were found, in particular: (1) the low correlation between peer review outcome and impact in time of the accepted contributions and (2) the presence of a high level of randomness in the analyzed peer review processes.

Original languageEnglish
Title of host publicationProceedings of ISSI 2011 - 13th Conference of the International Society for Scientometrics and Informetrics
EditorsEd Noyons, Patrick Ngulube, Jacqueline Leta
PublisherISSI, Leiden University and University of Zululand
Pages724-736
Number of pages13
ISBN (Electronic)9789081752701
Publication statusPublished - 2011
Externally publishedYes
Event13th Conference of the International Society for Scientometrics and Informetrics, ISSI 2011 - Durban, South Africa
Duration: 4 Jul 20117 Jul 2011

Publication series

NameProceedings of ISSI 2011 - 13th Conference of the International Society for Scientometrics and Informetrics
Volume2

Conference

Conference13th Conference of the International Society for Scientometrics and Informetrics, ISSI 2011
CountrySouth Africa
CityDurban
Period4.7.117.7.11

ASJC Scopus subject areas

  • Modelling and Simulation
  • Management Science and Operations Research
  • Computer Science Applications
  • Applied Mathematics
  • Statistics and Probability

Fingerprint Dive into the research topics of 'A quantitative analysis of peer review'. Together they form a unique fingerprint.

  • Cite this

    Ragone, A., Mirylenka, K., Casati, F., & Marchese, M. (2011). A quantitative analysis of peer review. In E. Noyons, P. Ngulube, & J. Leta (Eds.), Proceedings of ISSI 2011 - 13th Conference of the International Society for Scientometrics and Informetrics (pp. 724-736). (Proceedings of ISSI 2011 - 13th Conference of the International Society for Scientometrics and Informetrics; Vol. 2). ISSI, Leiden University and University of Zululand.