TY - GEN
T1 - A quantitative analysis of peer review
AU - Ragone, Azzurra
AU - Mirylenka, Katsiaryna
AU - Casati, Fabio
AU - Marchese, Maurizio
PY - 2011
Y1 - 2011
N2 - In this paper we focus on the analysis of peer reviews and reviewers behaviour in a number of different review processes. More specifically, we report on the development, definition and rationale of a number of appropriate quantitative metrics to assess the processes main properties. We then apply the proposed metrics and analysis framework to data sets from conference evaluation processes and we discuss the results implications and their eventual use toward improving the analyzed peer review processes. A number of unexpected results were found, in particular: (1) the low correlation between peer review outcome and impact in time of the accepted contributions and (2) the presence of a high level of randomness in the analyzed peer review processes.
AB - In this paper we focus on the analysis of peer reviews and reviewers behaviour in a number of different review processes. More specifically, we report on the development, definition and rationale of a number of appropriate quantitative metrics to assess the processes main properties. We then apply the proposed metrics and analysis framework to data sets from conference evaluation processes and we discuss the results implications and their eventual use toward improving the analyzed peer review processes. A number of unexpected results were found, in particular: (1) the low correlation between peer review outcome and impact in time of the accepted contributions and (2) the presence of a high level of randomness in the analyzed peer review processes.
UR - http://www.scopus.com/inward/record.url?scp=85083494681&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85083494681&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85083494681
T3 - Proceedings of ISSI 2011 - 13th Conference of the International Society for Scientometrics and Informetrics
SP - 724
EP - 736
BT - Proceedings of ISSI 2011 - 13th Conference of the International Society for Scientometrics and Informetrics
A2 - Noyons, Ed
A2 - Ngulube, Patrick
A2 - Leta, Jacqueline
PB - ISSI, Leiden University and University of Zululand
T2 - 13th Conference of the International Society for Scientometrics and Informetrics, ISSI 2011
Y2 - 4 July 2011 through 7 July 2011
ER -