Exploring and understanding citation-based scientific metrics

Mikalai Krapivin, Maurizio Marchese, Fabio Casati

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

This paper explores citation-based metrics, how they differ in ranking papers and authors, and why. We initially take as example three main metrics that we believe significant; the standard citation count, the more and more popular h-index, and a variation we propose of PageRank applied to papers (called PaperRank), that is appealing as it mirrors proven and successful algorithms for ranking web pages. As part of analyzing them, we develop generally applicable techniques and metrics for qualitatively and quantitatively analyzing indexes that evaluate content and people, as well as for understanding the causes of their different behaviors. Finally, we extend the analysis to other popular indexes, to show whether the choice of the index has a significant effect in how papers and authors are ranked. We put the techniques at work on a dataset of over 260 K ACM papers, and discovered that the difference in ranking results is indeed very significant (even when restricting to citation-based indexes), with half of the top-ranked papers differing in a typical 20-element long search result page for papers on a given topic, and with the top researcher being ranked differently over half of the times in an average job posting with 100 applicants.

Original languageEnglish
Pages (from-to)59-81
Number of pages23
JournalAdvances in Complex Systems
Volume13
Issue number1
DOIs
Publication statusPublished - Feb 2010
Externally publishedYes

Fingerprint

Websites

Keywords

  • Citation analyses
  • PageRank
  • Scientometrics

ASJC Scopus subject areas

  • Control and Systems Engineering
  • General

Cite this

Exploring and understanding citation-based scientific metrics. / Krapivin, Mikalai; Marchese, Maurizio; Casati, Fabio.

In: Advances in Complex Systems, Vol. 13, No. 1, 02.2010, p. 59-81.

Research output: Contribution to journalArticle

Krapivin, Mikalai ; Marchese, Maurizio ; Casati, Fabio. / Exploring and understanding citation-based scientific metrics. In: Advances in Complex Systems. 2010 ; Vol. 13, No. 1. pp. 59-81.
@article{3d4781cef2214893a4aba20c69ae9bd8,
title = "Exploring and understanding citation-based scientific metrics",
abstract = "This paper explores citation-based metrics, how they differ in ranking papers and authors, and why. We initially take as example three main metrics that we believe significant; the standard citation count, the more and more popular h-index, and a variation we propose of PageRank applied to papers (called PaperRank), that is appealing as it mirrors proven and successful algorithms for ranking web pages. As part of analyzing them, we develop generally applicable techniques and metrics for qualitatively and quantitatively analyzing indexes that evaluate content and people, as well as for understanding the causes of their different behaviors. Finally, we extend the analysis to other popular indexes, to show whether the choice of the index has a significant effect in how papers and authors are ranked. We put the techniques at work on a dataset of over 260 K ACM papers, and discovered that the difference in ranking results is indeed very significant (even when restricting to citation-based indexes), with half of the top-ranked papers differing in a typical 20-element long search result page for papers on a given topic, and with the top researcher being ranked differently over half of the times in an average job posting with 100 applicants.",
keywords = "Citation analyses, PageRank, Scientometrics",
author = "Mikalai Krapivin and Maurizio Marchese and Fabio Casati",
year = "2010",
month = "2",
doi = "10.1142/S0219525910002487",
language = "English",
volume = "13",
pages = "59--81",
journal = "Advances in Complex Systems",
issn = "0219-5259",
publisher = "World Scientific Publishing Co. Pte Ltd",
number = "1",

}

TY - JOUR

T1 - Exploring and understanding citation-based scientific metrics

AU - Krapivin, Mikalai

AU - Marchese, Maurizio

AU - Casati, Fabio

PY - 2010/2

Y1 - 2010/2

N2 - This paper explores citation-based metrics, how they differ in ranking papers and authors, and why. We initially take as example three main metrics that we believe significant; the standard citation count, the more and more popular h-index, and a variation we propose of PageRank applied to papers (called PaperRank), that is appealing as it mirrors proven and successful algorithms for ranking web pages. As part of analyzing them, we develop generally applicable techniques and metrics for qualitatively and quantitatively analyzing indexes that evaluate content and people, as well as for understanding the causes of their different behaviors. Finally, we extend the analysis to other popular indexes, to show whether the choice of the index has a significant effect in how papers and authors are ranked. We put the techniques at work on a dataset of over 260 K ACM papers, and discovered that the difference in ranking results is indeed very significant (even when restricting to citation-based indexes), with half of the top-ranked papers differing in a typical 20-element long search result page for papers on a given topic, and with the top researcher being ranked differently over half of the times in an average job posting with 100 applicants.

AB - This paper explores citation-based metrics, how they differ in ranking papers and authors, and why. We initially take as example three main metrics that we believe significant; the standard citation count, the more and more popular h-index, and a variation we propose of PageRank applied to papers (called PaperRank), that is appealing as it mirrors proven and successful algorithms for ranking web pages. As part of analyzing them, we develop generally applicable techniques and metrics for qualitatively and quantitatively analyzing indexes that evaluate content and people, as well as for understanding the causes of their different behaviors. Finally, we extend the analysis to other popular indexes, to show whether the choice of the index has a significant effect in how papers and authors are ranked. We put the techniques at work on a dataset of over 260 K ACM papers, and discovered that the difference in ranking results is indeed very significant (even when restricting to citation-based indexes), with half of the top-ranked papers differing in a typical 20-element long search result page for papers on a given topic, and with the top researcher being ranked differently over half of the times in an average job posting with 100 applicants.

KW - Citation analyses

KW - PageRank

KW - Scientometrics

UR - http://www.scopus.com/inward/record.url?scp=77951672267&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77951672267&partnerID=8YFLogxK

U2 - 10.1142/S0219525910002487

DO - 10.1142/S0219525910002487

M3 - Article

AN - SCOPUS:77951672267

VL - 13

SP - 59

EP - 81

JO - Advances in Complex Systems

JF - Advances in Complex Systems

SN - 0219-5259

IS - 1

ER -