Assessment of the NIH-supported relative citation ratio as a measure of research productivity among 1687 academic neurological surgeons

Restricted access

OBJECTIVE

Publication metrics such as the Hirsch index (h-index) are often used to evaluate and compare research productivity in academia. The h-index is not a field-normalized statistic and can therefore be dependent on overall rates of publication and citation within specific fields. Thus, a metric that adjusts for this while measuring individual contributions would be preferable. The National Institutes of Health (NIH) has developed a new, field-normalized, article-level metric called the “relative citation ratio” (RCR) that can be used to more accurately compare author productivity between fields. The mean RCR is calculated as the total number of citations per year of a publication divided by the average field-specific citations per year, whereas the weighted RCR is the sum of all article-level RCR scores over an author’s career. The present study was performed to determine how various factors, such as academic rank, career duration, a Doctor of Philosophy (PhD) degree, and sex, impact the RCR to analyze research productivity among academic neurosurgeons.

METHODS

A retrospective data analysis was performed using the iCite database. All physician faculty affiliated with Accreditation Council for Graduate Medical Education (ACGME)–accredited neurological surgery programs were eligible for analysis. Sex, career duration, academic rank, additional degrees, total publications, mean RCR, and weighted RCR were collected for each individual. Mean RCR and weighted RCR were compared between variables to assess patterns of analysis by using SAS software version 9.4.

RESULTS

A total of 1687 neurosurgery faculty members from 125 institutions were included in the analysis. Advanced academic rank, longer career duration, and PhD acquisition were all associated with increased mean and weighted RCRs. Male sex was associated with having an increased weighted RCR but not an increased mean RCR score. Overall, neurological surgeons were highly productive, with a median RCR of 1.37 (IQR 0.93–1.97) and a median weighted RCR of 28.56 (IQR 7.99–85.65).

CONCLUSIONS

The RCR and its derivatives are new metrics that help fill in the gaps of other indices for research output. Here, the authors found that advanced academic rank, longer career duration, and PhD acquisition were all associated with increased mean and weighted RCRs. Male sex was associated with having an increased weighted, but not mean, RCR score, most likely because of historically unequal opportunities for women within the field. Furthermore, the data showed that current academic neurosurgeons are exceptionally productive compared to both physicians in other specialties and the general scientific community.

ABBREVIATIONS AANS = American Association of Neurological Surgeons; h-index = Hirsch index; NIH = National Institutes of Health; PhD = Doctor of Philosophy; RCR = relative citation ratio.
Article Information

Contributor Notes

Correspondence Nitin Agarwal: University of Pittsburgh Medical Center, Pittsburgh, PA. agarwaln@upmc.edu.INCLUDE WHEN CITING Published online January 31, 2020; DOI: 10.3171/2019.11.JNS192679.Disclosures The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.
Headings
References
  • 1

    Adler REwing JTaylor P: Citation statistics: a report from the International Mathematical Union (IMU) in Cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS). Stat Sci 24:1142009

    • Search Google Scholar
    • Export Citation
  • 2

    Agarwal NClark SSvider PFCouldwell WTEloy JALiu JK: Impact of fellowship training on research productivity in academic neurological surgery. World Neurosurg 80:7387442013

    • Search Google Scholar
    • Export Citation
  • 3

    Aoun SGBendok BRRahme RJDacey RG JrBatjer HH: Standardizing the evaluation of scientific and academic performance in neurosurgery—critical review of the “h” index and its variants. World Neurosurg 80:e85e902013

    • Search Google Scholar
    • Export Citation
  • 4

    Bland CJCenter BAFinstad DARisbey KRStaples JG: A theoretical, practical, predictive model of faculty and department research productivity. Acad Med 80:2252372005

    • Search Google Scholar
    • Export Citation
  • 5

    Bornmann LDaniel HD: The state of h index research. Is the h index the ideal way to measure research performance? EMBO Rep 10:262009

    • Search Google Scholar
    • Export Citation
  • 6

    Bornmann LDaniel HD: What do we know about the h index? J Am Soc Inf Sci Technol 58:138113852007

  • 7

    Carpenter CRCone DCSarli CC: Using publication metrics to highlight academic productivity and research impact. Acad Emerg Med 21:116011722014

    • Search Google Scholar
    • Export Citation
  • 8

    Eloy JASvider PFCherla DVDiaz LKovalerchik OMauro KM: Gender disparities in research productivity among 9952 academic physicians. Laryngoscope 123:186518752013

    • Search Google Scholar
    • Export Citation
  • 9

    Harzing AWAlakangas S: Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison. Scientometrics 106:7878042016

    • Search Google Scholar
    • Export Citation
  • 10

    Hirsch JE: An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A 102:16569165722005

  • 11

    Hutchins BIYuan XAnderson JMSantangelo GM: Relative citation ratio (RCR): a new metric that uses citation rates to measure influence at the article level. PLoS Biol 14:e10025412016

    • Search Google Scholar
    • Export Citation
  • 12

    Khan NRThompson CJTaylor DRVenable GTWham RMMichael LM II: An analysis of publication productivity for 1225 academic neurosurgeons and 99 departments in the United States. J Neurosurg 120:7467552014

    • Search Google Scholar
    • Export Citation
  • 13

    Lee JKraus KLCouldwell WT: Use of the h index in neurosurgery. Clinical article. J Neurosurg 111:3873922009

  • 14

    Pagel PSHudetz JA: H-index is a sensitive indicator of academic activity in highly productive anaesthesiologists: results of a bibliometric analysis. Acta Anaesthesiol Scand 55:108510892011

    • Search Google Scholar
    • Export Citation
  • 15

    Rad AEBrinjikji WCloft HJKallmes DF: The H-index in academic radiology. Acad Radiol 17:8178212010

  • 16

    Rock CBPrabhu AVFuller CDThomas CR JrHolliday EB: Evaluation of the relative citation ratio, a new National Institutes of Health–supported bibliometric measure of research productivity, among academic radiation oncologists. J Am Coll Radiol 15 (3 Pt A):4694742018

    • Search Google Scholar
    • Export Citation
  • 17

    Spearman CMQuigley MJQuigley MRWilberger JE: Survey of the h index for all of academic neurosurgery: another power-law phenomenon? J Neurosurg 113:9299332010

    • Search Google Scholar
    • Export Citation
  • 18

    Surkis ASpore S: The relative citation ratio: what is it and why should medical librarians care? J Med Libr Assoc 106:5085132018

  • 19

    Svider PFPashkova AAChoudhry ZAgarwal NKovalerchik OBaredes S: Comparison of scholarly impact among surgical specialties: an examination of 2429 academic surgeons. Laryngoscope 123:8848892013

    • Search Google Scholar
    • Export Citation
  • 20

    Tomei KLNahass MMHusain QAgarwal NPatel SKSvider PF: A gender-based comparison of academic rank and scholarly productivity in academic neurological surgery. J Clin Neurosci 21:110211052014

    • Search Google Scholar
    • Export Citation
TrendMD
Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 433 433 433
Full Text Views 100 100 100
PDF Downloads 64 64 64
EPUB Downloads 0 0 0
PubMed
Google Scholar