Academic impact and rankings of neuroendovascular fellowship programs across the United States

Ashish SonigDepartments of Neurosurgery and
Department of Neurosurgery, Gates Vascular Institute at Kaleida Health; and

Search for other papers by Ashish Sonig in
Current site
Google Scholar
PubMed
Close
 MD, MS, MCh
,
Hussain ShallwaniDepartments of Neurosurgery and
Department of Neurosurgery, Gates Vascular Institute at Kaleida Health; and

Search for other papers by Hussain Shallwani in
Current site
Google Scholar
PubMed
Close
 MD
,
Bennett R. LevyDepartment of Neurosurgery, Gates Vascular Institute at Kaleida Health; and

Search for other papers by Bennett R. Levy in
Current site
Google Scholar
PubMed
Close
,
Hakeem J. ShakirDepartments of Neurosurgery and
Department of Neurosurgery, Gates Vascular Institute at Kaleida Health; and

Search for other papers by Hakeem J. Shakir in
Current site
Google Scholar
PubMed
Close
 MD
, and
Adnan H. SiddiquiDepartments of Neurosurgery and
Radiology, Jacobs School of Medicine and Biomedical Sciences, and
Toshiba Stroke and Vascular Research Center, University at Buffalo, State University of New York;
Department of Neurosurgery, Gates Vascular Institute at Kaleida Health; and
Jacobs Institute, Buffalo, New York

Search for other papers by Adnan H. Siddiqui in
Current site
Google Scholar
PubMed
Close
 MD, PhD
Full access

OBJECTIVE

Publication has become a major criterion of success in the competitive academic environment of neurosurgery. This is the first study that has used departmental h index–and e index–based matrices to assess the academic output of neuroendovascular, neurointerventional, and interventional radiology fellowship programs across the continental US.

METHODS

Fellowship program listings were identified from academic and organization websites. Details for 37 programs were available. Bibliometric data for these programs were gathered from the Thomson Reuters Web of Science database. Citations for each publication from the fellowship's parent department were screened, and the h and e indices were calculated from non–open-surgical, central nervous system vascular publications. Variables including “high-productivity” centers, fellowship–comprehensive stroke center affiliation, fellowship accreditation status, neuroendovascular h index, e index (h index supplement), h10 index (publications during the last 10 years), and departmental faculty-based h indices were created and analyzed.

RESULTS

A positive correlation was seen between the neuroendovascular fellowship h index and corresponding h10 index (R = 0.885; p < 0.0001). The mean, median, and highest faculty-based h indices exhibited positive correlations with the neuroendovascular fellowship h index (R = 0.662, p < 0.0001; R = 0.617, p < 0.0001; and R = 0.649, p < 0.0001, respectively). There was no significant difference (p = 0.824) in the median values for the fellowship h index based on comprehensive stroke center affiliation (30 of 37 programs had such affiliations) or accreditation (18 of 37 programs had accreditation) (p = 0.223). Based on the quartile analysis of the fellowship h index, 10 of 37 departments had an neuroendovascular h index of ≥ 54 (“high-productivity” centers); these centers had significantly more faculty (p = 0.013) and a significantly higher mean faculty h index (p = 0.0001).

CONCLUSIONS

The departmental h index and analysis of its publication topics can be used to calculate the h index of an associated subspecialty. The analysis was focused on the neuroendovascular specialty, and this methodology can be extended to other neurosurgical subspecialties. Individual faculty research interest is directly reflected in the research productivity of a department. High-productivity centers had significantly more faculty with significantly higher individual h indices. The current systems for neuroendovascular fellowship program accreditation do not have a meaningful impact on academic productivity.

ABBREVIATIONS

ACGME = Accreditation Council for Graduate Medical Education; CAST = Committee on Advanced Subspecialty Training; DNV = Det Norske Veritas; h10 index = h index for publications in the last 10 years; h index-dept = departmental h index; h index-neuroendovascular = h index for the neuroendovascular fellowship; JC = Joint Commission; UCLA = University of California, Los Angeles; UCSF = University of California, San Francisco.

OBJECTIVE

Publication has become a major criterion of success in the competitive academic environment of neurosurgery. This is the first study that has used departmental h index–and e index–based matrices to assess the academic output of neuroendovascular, neurointerventional, and interventional radiology fellowship programs across the continental US.

METHODS

Fellowship program listings were identified from academic and organization websites. Details for 37 programs were available. Bibliometric data for these programs were gathered from the Thomson Reuters Web of Science database. Citations for each publication from the fellowship's parent department were screened, and the h and e indices were calculated from non–open-surgical, central nervous system vascular publications. Variables including “high-productivity” centers, fellowship–comprehensive stroke center affiliation, fellowship accreditation status, neuroendovascular h index, e index (h index supplement), h10 index (publications during the last 10 years), and departmental faculty-based h indices were created and analyzed.

RESULTS

A positive correlation was seen between the neuroendovascular fellowship h index and corresponding h10 index (R = 0.885; p < 0.0001). The mean, median, and highest faculty-based h indices exhibited positive correlations with the neuroendovascular fellowship h index (R = 0.662, p < 0.0001; R = 0.617, p < 0.0001; and R = 0.649, p < 0.0001, respectively). There was no significant difference (p = 0.824) in the median values for the fellowship h index based on comprehensive stroke center affiliation (30 of 37 programs had such affiliations) or accreditation (18 of 37 programs had accreditation) (p = 0.223). Based on the quartile analysis of the fellowship h index, 10 of 37 departments had an neuroendovascular h index of ≥ 54 (“high-productivity” centers); these centers had significantly more faculty (p = 0.013) and a significantly higher mean faculty h index (p = 0.0001).

CONCLUSIONS

The departmental h index and analysis of its publication topics can be used to calculate the h index of an associated subspecialty. The analysis was focused on the neuroendovascular specialty, and this methodology can be extended to other neurosurgical subspecialties. Individual faculty research interest is directly reflected in the research productivity of a department. High-productivity centers had significantly more faculty with significantly higher individual h indices. The current systems for neuroendovascular fellowship program accreditation do not have a meaningful impact on academic productivity.

Publication has become a major criterion of success in the competitive academic environment of neurosurgery. Research productivity has been linked with the ability to rise to the upper echelons in the realm of academics. In fact, tenure-track appointment is directly elated to the number of scientific publications of an individual.8,10,12 Citation analysis is a central tool of bibliometric analysis, which is a statistical method used to quantitatively analyze scholarly publications. An individual is the usual subject of a bibliometric analysis. However, the analysis can be extended to include a department, group, or journal. Several methods have been employed to evaluate the productivity of an individual or a department, the commonest of which is the h index.8,10,12 Proposed by a physicist from the University of California, San Diego, Jorge E. Hirsh, the h index is “the number of papers with citation number ≥ h as a useful index to characterize the scientific output of a researcher.”6 Another index that supplements the h index is the e index, which was proposed by Zhang.13 The e index accounts for not only all publications that are part of the h index but also the cumulative citations of these publication. Thus, it provides information on ignored citations. Other indices that have been used are the g index3 and the m quotient.8

The number of neuroendovascular and neurointerventional fellowship training programs in the US has increased in recent years.7 In this scenario, in which more than 50 neurosurgical programs offer these fellowships, it is of paramount importance to assess the academic productivity of such training programs and to know if accreditation with an authority or board has an impact on academic productivity. In the present study, our primary objective was to assess the academic output of neuroendovascular and neurointerventional programs across the continental US using h and e index–based matrices. To the best of our knowledge, this is the first study in which departmental academic productivity for this subspecialty has been assessed. An earlier study of pediatric neurosurgery focused on the h indices of individual faculty members.9 This method may not be reliable, as faculty can move from one program to another, and thus the results of such an analysis may not reflect the true productivity of the fellowship program. Currently, few neurosurgical fellowships are accredited by a formal standardized system. Our secondary aim was to determine if a difference exists in terms of the academic productivity of fellowships with and without accreditation and with and without affiliation with a comprehensive stroke center.

Methods

Listings for neuroendovascular, endovascular surgical neuroradiology, neuroendovascular surgery, and interventional neuroradiology fellowship programs were obtained from the websites of the fellowship directories of the American Association of Neurological Surgeons (http://www.aans.org/grants%20and%20fellowships/fellowship%20directory.aspx), Accreditation Council for Graduate Medical Education (ACGME) (https://apps.acgme.org/ads/public/reports/report/1), and the Committee on Advanced Subspecialty Training (CAST) of the Society of Neurological Surgeons (www.societyns.org/fellowships/NeuroEndovascularNeurosurgery.asp). Only those programs for which a description of a neuroendovascular, endovascular surgical neuroradiology, neuroendovascular surgery, or interventional neuroradiology fellowship was available on the website of the parent department or institution were included in the final analysis (Appendix 2). Thus, institutions offering only a noninterventional stroke fellowship were excluded. Additionally, information was gathered on the CAST or ACGME ac creditation status of the fellowship program and the affiliation of the parent department or institution with comprehensive stroke center certification (Det Norske Veritas [DNV; http://dnvglhealthcare.com/hospitals?search_type=and&q=&c=20806&c=19200&prSubmit=Search] or the Joint Commission [JC; [http://www.strokecenter.org/trials/centers?utf8=L&search=14209]).

Bibliometric data for all years through May of 2016 were gathered from the Thomson Reuters Web of Science citation database (http://login.webofknowledge.com). Because neuroendovascular training closely involves other departments and neurointerventionists can have a training background in radiology or neurology and neurosurgery, we used the following search strings: neurol*surg*(neurological surgery), neurosurg*(neurosurgery), neurolog*(neurology), neuroend ovasc*(neuroendovascular), neuroradiol*(neuroradiology), cerebrovasc*(cerebrovascular), and stroke*(stroke). These search strings were attached to the institute's address strings (Appendix 1).10 Thus, a cumulative h index was generated, which is referred to as the departmental h index (h index-dept) (Table 1).

TABLE 1.

Neuroendovascular fellowship rankings based on the neuroendovascular fellowship h index

InstitutionLocationCAST/ACGME AccreditationAffiliation w/Comprehensive Stroke Center (JC/DNV)Departmental h IndexNeuroendovascular Fellowship h Indexh10 Indexe IndexNo. of Neuroendovascular-Trained FacultyHighest h Index of FacultyMean h Index of Department FacultyMedian h Index of Faculty
Stanford UniversityStanford, CANoYes18710339121.3545327.5024
UCSFSan Francisco, CANoNo1758240115.2467437.5036
UCLALos Angeles, CANoYes142774885.7155428.6022
University at BuffaloBuffalo, NYYesYes72714469.1446740.2539.5
Johns Hopkins UniversityBaltimore, MDYesYes177643374.2153013.8014
Washington UniversitySt Louis, MOYesNo192632389.0334438.0040
St. Joseph's Hospital (Barrow)Phoenix, AZYesNo115623480.0123838.0038
University of PittsburghPittsburgh, PANoYes111593586.2642719.0017.5
NewYork-Presbyterian/Cornell UniversityNew York, NYNoYes148572482.4634421.0011
Cleveland ClinicCleveland, OHYesYes115542780.9732115.6720
Duke University HospitalDurham, NCNoYes116472049.3122617.0017
University of Texas–HoustonHouston, TXNoYes123472257.7132311.336
Emory UniversityAtlanta, GAYesYes135472561.533109.009
University of MinnesotaMinneapolis, MNYesYes85432683.453119.009
University of AlabamaBirmingham, ALNoYes73382246.7622218.5018.5
University of Illinois at ChicagoChicago, ILNoYes57371340.9752717.4013
Brigham & Women's HospitalBoston, MANoNo206342043.0622518.0018
Yale UniversityNew Haven, CTNoYes16932840.3238438.3320
Thomas Jefferson UniversityPhiladelphia, PAYesYes82312652.6333024.3325
Roosevelt Hospital (Mount Sinai)New York, NYNoYes107311540.0043618.5017
New York UniversityNew York, NYNoYes10930549.0021518.0018
University of California, San DiegoSan Diego, CANoYes61281857.654199.007
University of Wisconsin–MadisonMadison, WIYesYes51271734.7431511.3315
Indiana UniversityIndianapolis, INNoYes68271241.5131512.3314
Rush UniversityChicago, ILYesYes108261336.1932116.3320
Rutgers New Jersey Medical SchoolNewark, NJYesYes77261932.2821713.5013.5
Medical University of South CarolinaCharleston, SCNoYes54262036.5841811.7511
University of Tennessee/Semmes-Murphey ClinicMemphis, TNYesYes6822830.13396.339
Mayo Clinic JacksonvilleJacksonville, FLNoYes41221522.493119.679
University of Florida–GainesvilleGainesville, FLYesYes39191324.65333159
Cedars-Sinai Medical CenterLos Angeles, CAYesYes63181433.4142414.5013
University of MiamiMiami, FLYesNo5116932.9821211.0011
Hartford HospitalHartford, CTNoYes24161523.12375.336
Albany Medical CenterAlbany, NYNoNo6015923.9732517.6723
University of UtahSalt Lake City, UTYesYes64151019.18483.252.5
The Ohio State UniversityColumbus, OHYesYes7314714.9731411.0011
Penn State Hershey Medical CenterHershey, PAYesNo5712424.91394.333

Calculation of the Neuroendovascular h and e Indices

Each publication citation included in h index-dept was screened, and the h index for the neuroendovascular fellowship (referred to as h index-neuroendovascular) was calculated from citations in the CNS vascular domain, excluding those that were limited to microneurosurgical nuances or microneurosurgical surgical anatomy. Similarly, the h index-neuroendovascular for citations during the last 10 years (referred to as h10 index) and the h index of individual neuroendovascular-trained faculty members at each parent department were calculated. Multiinstitutional data were equally distributed among all included institutes. For example, all institutions that were attributed to the authors of a publication received equal credit for that publication in the calculation of the h index for the department.

Next, h index-neuroendovascular was subjected to quartile analysis. Neuroendovascular fellowship programs falling in the ≥ 75th percentile were labeled as “high-productivity” centers in terms of academic works. Correlation analysis was done for these high-productivity centers with other parameters, including affiliation with a comprehensive stroke center, number of faculty members, accreditation status, and “highest” h index of neuroendovascular-trained faculty members from each department.

To calculate the e index, we used the formula13 below:
article image
where citj are the citations received by the jth paper (j refers to the serial number when the citations are arranged in descending order of the number of times cited) and denotes excess citations within the h core (the h core comprises all articles within the h index).13 Figure 1 illustrates the calculation of the neuroendovascular h, h10, and e indices.
FIG. 1.
FIG. 1.

Steps involved in the computation of the departmental h index, neuroendovascular fellowship h index, and e index.

Calculation of Faculty h Index

The faculty h index was calculated from the Thomson Reuters Web of Science citation database (http://login.webofknowledge.com). This index was calculated using the author's name, irrespective of his or her academic affiliation.

Statistical Analysis

The Shapiro-Wilk test was performed to determine if each of the continuous variables in our study was normally distributed. The continuous variables were h index-neuroendovascular, h10 index-neuroendovascular, number of faculty members, mean h index of faculty, median h index of faculty, and highest h index of faculty. Correlations between these variables were evaluated using the Spearman's rank correlation coefficient. The Wilcoxon-Mann-Whitney test was used to compare the distributions of the continuous variables by group. The chi-square test or Fisher's exact test was used to evaluate the relationship between categorical variables (i.e., affiliation with a comprehensive stroke center, accreditation status, and higher productivity center). When the p value was < 0.05, the 2 variables under evaluation were considered to be significantly related. SAS (version 9.4, SAS Institute) was used for the statistical analysis.

Results

Rankings of Neuroendovascular Fellowships Based on the h Indices

The search revealed a total of 54 fellowship programs, of which details for 37 fellowships were available on the parent institute or department websites. These 37 programs were included in our analysis (Table 1).

The median h index-neuroendovascular was 31 (mean 38.86 ± 22.04; range 12–103). The median of the h10 index-neuroendovascular was 19 (mean 20.32 ± 10.92; range 4–48). The mean number of neuroendovascular faculty members in each department was 3.27 ± 0.962 (range 2–6). Stanford University, University of California, San Francisco (UCSF), University of California, Los Angeles (UCLA), University at Buffalo, and Johns Hopkins University had the top 5 rankings based on h index-neuroendovascular. The h index-neuroendovascular indices for these institutions were 103, 82, 77, 71, and 64, respectively.

Correlation Analysis of Neuroendovascular Fellowship h Indices With Faculty h Indices

Correlation analysis (Table 2) showed a strong positive correlation between the neuroendovascular fellowship h index and its h10 index (rs = 0.885; p < 0.0001). Similarly, the mean, median, and highest h indices of the neuroendovascular departmental faculty were positively correlated with the neuroendovascular fellowship h index (rs = 0.662, p < 0.0001; rs = 0.617, p < 0.0001; and rs = 0.649, p < 0.0001 respectively). There was a significant correlation between the number of faculty members and the neuroendovascular h index (rs = 0.408; p = 0.006) and h10 index (rs = 0.441; p = 0.003). The mean, median, and highest h indices of neuroendovascular departmental faculty were also correlated with the h10 index of the fellowship (rs = 0.568, p < 0.0001; rs = 0.569, p < 0.0001; and rs = 0.541, p < 0.0001, respectively).

TABLE 2.

Correlation coefficients between neuroendovascular fellowship program–related bibliometric variables

VariableFellowship h Indexh Index From Previous 10 Yrs (p value)No. of Faculty Members (p value)Highest h Index of Faculty (p value)Mean h Index of Faculty (p value)Median h Index of Faculty (p value)
Neuroendovascular fellowship h index1.000000.885 (<0.0001)*0.408 (0.006)*0.649 (<0.0001)*0.662 (<0.0001)*0.617 (<0.0001)*
Neuroendovascular h index from last 10 yrs1.000000.441 (0.003)*0.541 (<0.0001)*0.568 (<0.0001)*0.569 (<0.0001)*
No. of faculty1.000000.409 (0.006)*0.188 (0.132)0.117 (0.246)
Highest h index of faculty1.000000.894 (<0.0001)*0.685 (<0.0001)*
Mean h index of faculty1.0000.910 (<0.0001)*
Median h index of faculty1.000

Significant p value.

Comparison of Neuroendovascular Fellowship h Indices With CAST/ACGME Accreditation and With Affiliation to a JC/DNV Stroke Center

Of the 37 fellowships, 18 (48.6%) were accredited by CAST or ACGME, and 30 (81.1%) were affiliated with a JC- or DNV-accredited stroke center. There was no significant association between the neuroendovascular fellowship h indices and CAST/ACGME accreditation or JC/DNV-accredited stroke center affiliation (Table 3).

TABLE 3.

Neuroendovascular fellowship and faculty h indices compared by accreditation and stroke center affiliation status and p values (Wilcoxon-Mann-Whitney test)

Affiliation StatusNo. of Neuroendovascular Fellowship ProgramsMeanSDMedianMinMaxp Value
CAST/ACGME Accreditation Status
  Fellowship h index
    No1942.5323.6434.0015.00103.000.2238
    Yes1835.0020.1626.5012.0071.00
  h index from last 10 yrs
    No1921.0511.6820.005.0048.000.7726
    Yes1819.5610.9218.004.0044.00
  No. of faculty
    No193.421.123.002.006.000.3340
    Yes183.110.763.002.005.00
JC/DNV Stroke Center Affiliation Status
  Fellowship h index
    No740.5728.2734.0012.0082.000.824
    Yes3038.4720.8931.0014.00103.00
  h index from last 10 yrs
    No719.8513.5520.004.0040.000.904
    Yes3020.4310.8218.505.0048.00
  No. of faculty
    No73.001.413.002.006.000.417
    Yes303.330.843.002.005.00

On the basis of accreditation status, there was no statistically significant difference in the mean or median of the neuroendovascular fellowship h index and h10 index (Table 3). Similarly, on the basis of an affiliation with a JC- or DNV-accredited comprehensive stroke center, there was no statistically significant difference in the mean or median values of those indices.

Comparison of Neuroendovascular Fellowships Based on Productivity

On the basis of the quartile analysis, the 75th percentile of the neuroendovascular fellowship h index was 54. Departments with an h index-neuroendovascular of ≥ 54 were labeled as “high-productivity” centers. Ten (27.0%) centers formed this cohort: Stanford University, UCSF, UCLA, University at Buffalo, Johns Hopkins University, Washington University, St. Joseph's Hospital (Barrow), University of Pittsburgh, NewYork-Presbyterian/Cornell University, and Cleveland Clinic. In addition, Stanford University had the highest e index (121.35) (Table 1).

The mean number of neuroendovascular faculty members in the department was significantly higher in neuroendovascular fellowship programs with higher productivity (p = 0.013) (Table 4). Moreover, higher productivity fellowship programs had significantly higher mean and median neuroendovascular faculty h indices (p ≤ 0.0001 and p ≤ 0.0001). Also, significantly higher productivity was seen in the last 10 years (p < 0.0001). However, higher productivity was not associated with the CAST/ACGME accreditation of the fellowship or its affiliation with a JC/DNV-accredited stroke center (p = 0.7140 and p = 0.295, respectively).

TABLE 4.

Neuroendovascular fellowship and faculty h indices compared by productivity group (Wilcoxon-Mann-Whitney test)

High ProductivityNo. of Neuroendovascular Fellowship ProgramsMeanSDMedianMinMaxp Value
h index from previous 10 yrs
  No2715.006.3415.004.0026.00<0.0001*
  Yes1034.708.3334.5023.0048.00
No. of faculty
  No273.040.763.002.005.000.013*
  Yes103.901.204.002.006.00
Highest h index of faculty
  No2720.9614.8918.007.0084.00<0.0001*
  Yes1045.2017.1544.0021.0074.00
Mean h index of faculty
  No2713.767.0312.333.2538.33<0.0001*
  Yes1027.9310.1328.0513.8040.25
Median h index of faculty
  No2712.875.8613.002.5025.00<0.0001*
  Yes1026.2011.1523.0011.0040.00

Significant p value.

Discussion

To the best of our knowledge, we have performed the most detailed bibliometric evaluation to date on the neuroendovascular subspecialty. Our study is unique for several reasons. First and foremost, department productivity was analyzed by specifically looking at the subspecialty of interest and using address and subject matter search strings (Appendix 1) to extract publication citations. By screening the citations for publications (Fig. 1) that comprised the parent department's h index, we created a new h index (i.e., h index-neuroendovascular) that was based on relevant neuroendovascular publications. This methodology can be used to study and analyze most fellowship programs in neurosurgery or other specialties across the US. Ponce and Lozano10 compared the academic productivity of American and Canadian neurosurgical training programs using departmental address search strings and concluded that h indices can be useful for comparing academic output across neurosurgery departments. However, their study was limited to neurosurgery and did not include any subspecialty.

Klimo et al.9 used the Scopus (https://www.scopus.com/) and Google Scholar (http://scholar.google.com/) bibliometric databases to profile the academic productivity of individual pediatric neurosurgeons. Their study was mainly an individual authorship–based study in which the h index of a pediatric neurosurgeon was calculated according to his or her name. Those authors also evaluated pediatric department productivity for a period of 5 years. The study did not focus on publications pertaining to pediatric neurosurgery alone. For example, a pediatric neurosurgeon might have coauthored several manuscripts on different topics of interest, but those citations would still be part of his or her h index. Each department was ranked based on the cumulative h and e indices of its current faculty members; thus, the method may not reflect the department's productivity. Additionally, faculty members may change from one department to another, which is another aspect that was not into taken into account during the ranking of the program.

Accreditation and Productivity

The accreditation of neurosurgical fellowships is a “hot” topic in view of a recent increase in the number of programs that offer neuroendovascular or neurointerventional fellowships.4 An argument was made to halt fellowship training because of a decrease in the volume of procedures and the saturation of larger centers with neurointerventionists.4 It was feared that graduating fellows would accept appointments at community hospitals with lower procedure volumes and untrained support staff.

Unlike other specialties (such as obstetrics and gynecology, cardiology, vascular surgery, and surgical oncology) where almost all fellowships are accredited by the ACGME, fellowships in neurosurgery mostly are either unaccredited or accredited by the Accreditation Council for Pediatric Neurosurgical Fellowships (for pediatric neurosurgery fellowships) or CAST, which offers accreditation of fellowship programs in the subspecialties of neurocritical care, cerebrovascular neurosurgery, neuroendovascular surgery, spinal neurosurgery, neurosurgical oncology, pediatric neurosurgery, peripheral nerve neurosurgery, and stereotactic and functional neurosurgery. Few fellowships in neurosurgery are ACGME accredited.

In the present study on neurosurgical departments with a neuroendovascular fellowship program, we analyzed if academic productivity correlates with accreditation status (ACGME/CAST). Fourteen fellowships were accredited by CAST, 4 by ACGME, and 2 by both. There was no significant difference in the academic productivity of unaccredited versus accredited programs (p = 0.70). In fact, only 5 of 10 programs in the higher productivity cohort are accredited (Table 1). At present, the accreditation system for endovascular training is fragmented, in part because it is a field populated by 3 different specialties: neurosurgery, neurology, and neuroradiology. Furthermore, there is no mandate for academic productivity by any of the participating societies, which stands in contrast to the standard for residency training programs that have research criteria that need to be met prior to the graduation of their residents. To meet the needs of this rapidly evolving field, including the evaluation of indications, techniques, and devices, in addition to expanding horizons to new paradigms, research should be embraced as a foundational element of accredited programs. This would serve both to advance and justify the field, as well as to further raise the bar for those who wish to enter it. Our study is a step in that direction. The methods can be used by accreditation bodies to extract the research productivity of endovascular fellowship programs and monitor pre-established goals.

Affiliation With a Stroke Center

The year 2015 saw the publication of 5 randomized controlled trials that favored mechanical thrombectomy plus intravenous tissue plasminogen activator over medical management with intravenous tissue plasminogen activator alone for large-vessel occlusion in the setting of anterior circulation stroke.1,2,5–7 These trials endorsed the superiority of endovascular treatment over medical management alone. As a result, endorsement by the DNV or JC for comprehensive stroke center status became important for an institution. A recent study has shown that the direct admission of patients to a comprehensive stroke center (as opposed to the transfer of patients from another facility to such a center) is associated with better outcomes and lower hospitalization costs.11

The present study showed that departments affiliated with a JC- or DNV-certified stroke center did not have a significantly higher number of neuroendovascular faculty members (p = 0.417). The small number of nonaffiliated centers (7 of 37 centers) may explain the lack of significance. Moreover, there is a requirement for a minimum of 2 trained neurointerventionists to staff a certified comprehensive stroke center, which was met by all affiliated and nonaffiliated centers (Table 1). However, association with a JC- or DNV-certified stroke center did not increase the academic productivity of the department. Even though certification as a primary stroke center started in the year 2003 (https://www.jointcommission.org/certification_for_comprehensive_stroke_centers/), comprehensive stroke center certification is a recent phenomenon. The DNV announced its first comprehensive stroke center in the year 2012 (http://dnvglhealthcare.com/releases/dnv-healthcare-introduces-comprehensive-stroke-center-certification). Because the h index depends on the number of times a paper is cited, it will take some time before the impact of faculty volume is reflected in academic productivity at these comprehensive stroke centers.

h Index-Neuroendovascular and its h10 and e Indices

In our study, we were able to extract publications specific to the subspecialty of interest and compute the h10 index. The h index-neuroendovascular correlated significantly with the number of neuroendovascular faculty members (p = 0.04), the highest h index of the faculty (p < 0.0001), and its mean (p < 0.0001).

If one looks at the subgroup analysis of high-productivity centers (Table 4), these centers had a significantly higher h10 index and number of neuroendovascular faculty members. The highest and mean faculty h indices were significantly greater at these centers, suggesting more resources for research are provided at these centers or that these centers attract faculty members with better research profiles. Further, larger programs have more faculty members and can spread call and case responsibilities. This might facilitate more “protected time” for research and academic endeavors.

The h10 index depicts academic productivity regardless of the longevity of the program, and thus it negates the natural advantage of older fellowship programs. Our study showed that h10 correlated significantly with h index-neuro endovascular (p < 0.0001) and the mean, median, and highest h index of neuroendovascular faculty members (Table 2).

The h index bibliometric has been criticized for its inability to truly reflect the citation prowess of an individual or an institution. For example, 2 institutions can have the same h index, yet one of these may have manuscripts that have been cited much more often than its h index indicates. These unaccounted for or excess citations can be calculated in the e index. Thus, if 2 institutions have the same h index and one of them has a higher e index, it would mean that the institution with the higher e index had more citations.

The e index needs to be interpreted along with the h index; hence, we did not perform a separate statistical analysis of the e index. In our study, Stanford University had the highest neuroendovascular h and e indices, and UCLA had the highest h10 index.

Study Limitations

This is the first attempt to rank fellowship programs on the basis of the h indices that specifically pertained to specialty of interest: in this instance, the neuroendovascular subspecialty. Such an analysis is prone to errors, as the denominator was based on the address strings of the parent institution or department. It is possible that a completely different abbreviated address could be used by the department for publication purposes. It is also possible that the Thomson Reuters Web of Science portal may not harbor all of the articles published by the department.

The h index considers overall citations rather than the importance of single contributions, neglects the quality of the content of the publications, and does not consider the context of the citation. It is also influenced by the accuracy of the citation database used for its calculation.

Future Perspectives

This study included neuroendovascular fellowship programs in the continental US. The pattern of research productivity at centers across the globe is different because each country has its own process for accreditation and affiliation. Moreover, information for fellowships outside the scope of our study is lacking at present. Once more information is available on websites, the methods enumerated in this manuscript can be used to analyze neuroendovascular fellowship programs across the globe.

In addition, the methods applied here can be used as a benchmark for future academic productivity by determining if a correlation exists between the individual productivity of the graduates of the fellowship program and that of the parent department. The data can be extrapolated into various subsets of publication types: clinically oriented (patient outcomes and technology advancement) and basic science oriented (bench research and preclinical in vivo and in vitro work).

Conclusions

On the basis of the institution's address and subject matter strings, a departmental h index can be calculated. Further analysis of the publications that make up the h index could reveal the inclination of a department toward one particular subspecialty and can be used to accurately calculate the h index of a particular specialty. Our analysis was restricted to the neuroendovascular subspecialty, but the methodology can be extended to other neurosurgical subspecialties. The research interests of individual faculty members are directly reflected in the research productivity of a department. The current systems for the accreditation of neuroendovascular fellowship programs do not have a meaningful impact on academic productivity. Publication prowess based on the above-mentioned methods could be given consideration during the process of the accreditation of neuroendovascular fellowships in the future.

Acknowledgments

The authors thank Tingting Zhuang, MA, from the University at Buffalo Department of Biostatistics for statistical analysis, Adrienne R. Doepp, BA, MLS, from the A. H. Aaron Health Sciences Library at Buffalo General Medical Center for assistance with the search strategy, and Debra J. Zimmer from the University at Buffalo Department of Neurosurgery for editorial assistance.

Disclosures

The authors report the following. Dr. Siddiqui has financial interests in Buffalo Technology Partners Inc., Cardinal Health, International Medical Distribution Partners, Medina Medical Systems, Neuro Technology Investors, StimSox, and Valor Medical. He serves as a consultant to Amnis Therapeutics Ltd., Cerebrotech Medical Systems Inc., CereVasc LLC, Codman & Shurtleff Inc., Corindus Inc., Covidien (acquired by Medtronic), GuidePoint Global Consulting, Lazarus (acquired by Medtronic), Medina Medical (acquired by Medtronic), Medtronic, MicroVention, Neuravi, Penumbra, Pulsar Vascular, Rapid Medical, Rebound Medical, Reverse Medical (acquired by Medtronic), Silk Road Medical Inc., Stryker, The Stroke Project Inc., Three Rivers Medical Inc., Cerebrotech Medical Systems Inc., and W. L. Gore & Associates. He is a principal investigator or serves on the National Steering Committee for the following trials: Covidien SWIFT PRIME, LARGE, Medtronic SWIFT DIRECT, MicroVention CONFIDENCE trial, MicroVention FRED trial, Penumbra 3D Separator, Penumbra COMPASS, Penumbra INVEST, and POSITIVE Trial. He is a member of the board of the Intersocietal Accreditation Committee.

Author Contributions

Conception and design: Sonig. Acquisition of data: Siddiqui, Sonig, Shallwani. Analysis and interpretation of data: all authors. Drafting the article: Sonig. Critically revising the article: all authors. Reviewed submitted version of manuscript: all authors.

Supplemental Information

Online-Only Content

Supplemental material is available with the online version of the article.

Appendices 1 and 2. https://thejns.org/doi/suppl/10.3171/2016.9.JNS161857.

References

  • 1

    Berkhemer OA, Fransen PS, Beumer D, van den Berg LA, Lingsma HF, Yoo AJ, et al.: A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med 372:1120, 2015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 2

    Campbell BC, Mitchell PJ, Kleinig TJ, Dewey HM, Churilov L, Yassi N, et al.: Endovascular therapy for ischemic stroke with perfusion-imaging selection. N Engl J Med 372:10091018, 2015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3

    Egghe L: Theory and practice of the g-index. Scientometrics 69:131152, 2006

  • 4

    Fiorella D, Hirsch JA, Woo HH, Rasmussen PA, Shazam Hussain M, Hui FK, et al.: Should neurointerventional fellowship training be suspended indefinitely?. J Neurointerv Surg 4:315318, 2012

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5

    Goyal M, Demchuk AM, Menon BK, Eesa M, Rempel JL, Thornton J, et al.: Randomized assessment of rapid endovascular treatment of ischemic stroke. N Engl J Med 372:10191030, 2015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6

    Hirsch JE: An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A 102:1656916572, 2005

  • 7

    Jovin TG, Chamorro A, Cobo E, de Miquel MA, Molina CA, Rovira A, et al.: Thrombectomy within 8 hours after symptom onset in ischemic stroke. N Engl J Med 372:22962306, 2015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8

    Khan NR, Thompson CJ, Taylor DR, Venable GT, Wham RM, Michael LM II, et al.: An analysis of publication productivity for 1225 academic neurosurgeons and 99 departments in the United States. J Neurosurg 120:746755, 2014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9

    Klimo P Jr, Venable GT, Khan NR, Taylor DR, Shepherd BA, Thompson CJ, et al.: Bibliometric evaluation of pediatric neurosurgery in North America. J Neurosurg Pediatr 14:695703, 2014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10

    Ponce FA, Lozano AM: Academic impact and rankings of American and Canadian neurosurgical departments as assessed using the h index. J Neurosurg 113:447457, 2010

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 11

    Sonig A, Lin N, Krishna C, Natarajan SK, Mokin M, Hopkins LN, et al.: Impact of transfer status on hospitalization cost and discharge disposition for acute ischemic stroke across the US. J Neurosurg 124:12281237, 2016

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 12

    Spearman CM, Quigley MJ, Quigley MR, Wilberger JE: Survey of the h index for all of academic neurosurgery: another power-law phenomenon?. J Neurosurg 113:929933, 2010

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 13

    Zhang CT: The e-index, complementing the h-index for excess citations. PLoS One 4:e5429, 2009

Supplementary Materials

  • Collapse
  • Expand
  • View in gallery
    FIG. 1.

    Steps involved in the computation of the departmental h index, neuroendovascular fellowship h index, and e index.

  • 1

    Berkhemer OA, Fransen PS, Beumer D, van den Berg LA, Lingsma HF, Yoo AJ, et al.: A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med 372:1120, 2015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 2

    Campbell BC, Mitchell PJ, Kleinig TJ, Dewey HM, Churilov L, Yassi N, et al.: Endovascular therapy for ischemic stroke with perfusion-imaging selection. N Engl J Med 372:10091018, 2015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 3

    Egghe L: Theory and practice of the g-index. Scientometrics 69:131152, 2006

  • 4

    Fiorella D, Hirsch JA, Woo HH, Rasmussen PA, Shazam Hussain M, Hui FK, et al.: Should neurointerventional fellowship training be suspended indefinitely?. J Neurointerv Surg 4:315318, 2012

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5

    Goyal M, Demchuk AM, Menon BK, Eesa M, Rempel JL, Thornton J, et al.: Randomized assessment of rapid endovascular treatment of ischemic stroke. N Engl J Med 372:10191030, 2015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6

    Hirsch JE: An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A 102:1656916572, 2005

  • 7

    Jovin TG, Chamorro A, Cobo E, de Miquel MA, Molina CA, Rovira A, et al.: Thrombectomy within 8 hours after symptom onset in ischemic stroke. N Engl J Med 372:22962306, 2015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 8

    Khan NR, Thompson CJ, Taylor DR, Venable GT, Wham RM, Michael LM II, et al.: An analysis of publication productivity for 1225 academic neurosurgeons and 99 departments in the United States. J Neurosurg 120:746755, 2014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9

    Klimo P Jr, Venable GT, Khan NR, Taylor DR, Shepherd BA, Thompson CJ, et al.: Bibliometric evaluation of pediatric neurosurgery in North America. J Neurosurg Pediatr 14:695703, 2014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10

    Ponce FA, Lozano AM: Academic impact and rankings of American and Canadian neurosurgical departments as assessed using the h index. J Neurosurg 113:447457, 2010

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 11

    Sonig A, Lin N, Krishna C, Natarajan SK, Mokin M, Hopkins LN, et al.: Impact of transfer status on hospitalization cost and discharge disposition for acute ischemic stroke across the US. J Neurosurg 124:12281237, 2016

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 12

    Spearman CM, Quigley MJ, Quigley MR, Wilberger JE: Survey of the h index for all of academic neurosurgery: another power-law phenomenon?. J Neurosurg 113:929933, 2010

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 13

    Zhang CT: The e-index, complementing the h-index for excess citations. PLoS One 4:e5429, 2009

Metrics

All Time Past Year Past 30 Days
Abstract Views 1087 0 0
Full Text Views 2834 885 51
PDF Downloads 1485 737 45
EPUB Downloads 0 0 0