Morbidity after intracranial tumor surgery: sensitivity and specificity of retrospective review of medical records compared with patient-reported outcomes at 30 days

View More View Less
  • 1 Departments of Anesthesiology and
  • 3 Neurosurgery and
  • 5 National Competence Centre for Ultrasound and Image-Guided Therapy, St. Olavs University Hospital; and 
  • 2 Departments of Circulation and Medical Imaging and
  • 4 Neuroscience, Norwegian University of Science and Technology, Trondheim, Norway
Full access

OBJECT

Published outcome reports in neurosurgical literature frequently rely on data from retrospective review of hospital records at discharge, but the sensitivity and specificity of retrospective assessments of surgical morbidity is not known. The aim of this study was to elucidate the sensitivity and specificity of retrospective assessment of morbidity after intracranial tumor surgery by comparing it to patient-reported outcomes at 30 days.

METHODS

In 191 patients who underwent surgery for the treatment of intracranial tumors, we evaluated newly acquired neurological deficits within the motor, language, and cognitive domains. Traditional retrospective discharge data were collected by review of hospital records. Patient-reported data were obtained by structured phone interviews at 30 days after surgery. Data on perioperative medical and surgical complications were obtained from both hospital records and patient interviews conducted 30 days postoperatively.

RESULTS

Sensitivity values for retrospective review of hospital records as compared with patient-reported outcomes were 0.52 for motor deficits, 0.4 for language deficits, and 0.07 for cognitive deficits. According to medical records, 158 patients were discharged with no new or worsened deficits, but only 117 (74%) of these patients confirmed this at 30 days after surgery. Specificity values were high (0.97–0.99), indicating that new deficits were unlikely to be found by retrospective review of hospital records at discharge when the patients did not report any at 30 days. Major perioperative complications were all identified through retrospective review of hospital records.

CONCLUSIONS

Retrospective assessment of medical records at discharge from hospital may greatly underestimate the incidence of new neurological deficits after brain tumor surgery when compared with patient-reported outcomes after 30 days.

ABBREVIATIONRCT = randomized controlled trial.

OBJECT

Published outcome reports in neurosurgical literature frequently rely on data from retrospective review of hospital records at discharge, but the sensitivity and specificity of retrospective assessments of surgical morbidity is not known. The aim of this study was to elucidate the sensitivity and specificity of retrospective assessment of morbidity after intracranial tumor surgery by comparing it to patient-reported outcomes at 30 days.

METHODS

In 191 patients who underwent surgery for the treatment of intracranial tumors, we evaluated newly acquired neurological deficits within the motor, language, and cognitive domains. Traditional retrospective discharge data were collected by review of hospital records. Patient-reported data were obtained by structured phone interviews at 30 days after surgery. Data on perioperative medical and surgical complications were obtained from both hospital records and patient interviews conducted 30 days postoperatively.

RESULTS

Sensitivity values for retrospective review of hospital records as compared with patient-reported outcomes were 0.52 for motor deficits, 0.4 for language deficits, and 0.07 for cognitive deficits. According to medical records, 158 patients were discharged with no new or worsened deficits, but only 117 (74%) of these patients confirmed this at 30 days after surgery. Specificity values were high (0.97–0.99), indicating that new deficits were unlikely to be found by retrospective review of hospital records at discharge when the patients did not report any at 30 days. Major perioperative complications were all identified through retrospective review of hospital records.

CONCLUSIONS

Retrospective assessment of medical records at discharge from hospital may greatly underestimate the incidence of new neurological deficits after brain tumor surgery when compared with patient-reported outcomes after 30 days.

ABBREVIATIONRCT = randomized controlled trial.

No more than 10% of studies in 3 major neurosurgical journals in 2010 were prospective,13 and only around 3% of neurosurgical studies are randomized9—most of these comparing drugs, not surgical procedures. Even in neurosurgical randomized controlled trials (RCTs), blinded outcome investigation is seen in only 39% of studies.9 Consequently, retrospective case series still form the main basis for benchmarking risks and benefits of surgery and for comparing results between different techniques or centers. The lack of standardized outcome measures and follow-up times may potentially hamper comparisons across studies, centers, and populations. For softer and more subjective outcome measures, such as morbidity and complications, the tradition of unblinded retrospective assessment of surgeons’ own data may be especially prone to bias. In a large meta-analysis of studies on glioma surgery,3 the risk of acquired deficits was 20.1% in the 2 RCTs, whereas in 88 observational, mainly retrospective case series, deficits were observed in only 4.5% of patients.10 Also in publications on management of unruptured intracranial aneurysms, studies of high methodological quality present higher risks of unfavorable outcome than low-quality studies.6 The question remains whether reviewing medical records in retrospect provides a realistic impression of surgical morbidity or merely shows the tip of the iceberg.

In a study population of 191 patients who underwent planned resection of intracranial tumors, we assessed postoperative morbidity based on retrospective review of medical records compared with morbidity reported by the patients themselves.

Methods

In this registry-based study, we included 191 (46.8%) of 409 adult patients (age ≥ 18 years) who underwent craniotomy for the treatment of intracranial tumors at Trondheim University Hospital, Norway, in the period from September 9, 2011, to October 10, 2013. Our department serves a geographically defined catchment region. Only patients with confirmed intracranial tumors who gave their written informed consent and who later provided follow-up data were included in the study.

A flowchart of the inclusion process, with reasons for noninclusion, is shown in Fig. 1.

FIG. 1.
FIG. 1.

Flowchart of the inclusion process and reasons for noninclusion.

Data collection was approved by the Regional Ethical Committee for Health Region Middle Norway and adhered to the guidelines of the Helsinki Declaration. Newly acquired or worsened motor deficits, language deficits, and cognitive deficits were included as outcome measures. Traditional (i.e., retrospective) outcome data were obtained by thorough review of available documentation from hospital medical records (electronic). Referral documentation (from other hospitals or general practitioners), admission notes (by neurosurgical registrars), and surgery reports (by operating neurosurgeons) were reviewed to record preoperative function/deficits. Postoperative documentation, such as notes and reports by neurosurgical doctors, neurologists, and/or oncologists, including discharge summaries, were closely reviewed to assess function at discharge. Prospective data on patient-reported new or worsened deficits were collected by one of 2 research nurses in a structured phone interview 30 days postoperatively (median 30 days, range 23–52 days). Proxies (close relatives) were consulted in cases involving patients with severe communication problems or cognitive deficits. The patients were asked for “limb weakness” or “palsy” that was new or had worsened after surgery. If the answer was “yes,” the nature of the new deficit (arm or leg weakness/hemiparesis or -paralysis) was recorded. For recording cognitive worsening or language deficits, the patients were asked for “new/worsened cognitive/memory problems (yes/no)” after surgery and for “new/worsened speech problems (yes/no)” after surgery. In cases in which patients expressed that they were unsure about the existence of new or worsened deficits, the response was classified as “no.”

Data on acquired neurological deficits as registered at discharge and new neurological deficits as reported by the patients themselves at 30 days postsurgery were dichotomized (new or worsened deficits, yes/no) and compared for sensitivity and specificity. Additionally, we analyzed postoperative complications registered in the hospital medical records within the first 30 days after surgery and compared the results on a case by case basis with complications as reported by the patients at 30 days.

Statistical Analyses

All statistical analyses were performed using SPSS Statistics 21.0. The statistical significance level in tests of normality was set to p < 0.05. Q-Q plots were used to test for normal distribution of data. Descriptive statistics are presented. Sensitivity and specificity were calculated from 2×2 contingency tables.

Sensitivity and specificity are measures of test properties, and, in this context, patient-reported outcomes were set as the “true value,” while outcome data from the hospital records served as the clinical test with either positive or negative results.

Results

As seen in Table 1, the mean age in our patient cohort was 54 years, and 48% of the patients were female. Significant preoperative comorbidity as indicated by American Society of Anesthesiologists (ASA) Class 3 and Charlson Comorbidity Index > 5 was seen in 24% and 8% of patients, respectively. Prior to surgery, 171 patients (90%) were able to care for themselves, as indicated by Karnofsky Performance Status scores ≥ 70.

TABLE 1.

Clinical and demographic characteristics of the 191 patients included in our analysis

CharacteristicValue
Age, mean ± SD54 ± 13
Female, n (%)92 (48)
Histopathology, n (%)
 Low-grade glioma36 (19)
 High-grade glioma58 (30)
 Meningioma56 (29)
 Metastasis28 (15)
 Other*13 (7)
Preop KPS score
 Median (range)90 (40–100)
 ≥70, n (%)171 (90)
Preop symptoms, n (%)
 Seizures37 (19)
 Cognitive deficits29 (15)
 Language deficits18 (9)
 Motor deficits21 (11)
Preop CCI
 Median (range)0 (0–8)
 >5, n (%)16 (8)
ASA class, n (%)
 16 (3)
 2140 (73)
 345 (24)
LOS in days, median (range)4 (1–23)
Primary operation, n (%)127 (67)

ASA = American Society of Anesthesiologists; CCI = Charlson Comorbidity Index; KPS = Karnofsky Performance Status; LOS = length of hospital stay.

Other: ependymoma, 2; blood vessel tumor, 7; nerve sheath tumor, 2; epidermoid, 1; osteosarcoma, 1.

Some patients presented multiple symptoms on admission.

No New Deficits

New or worsened deficits as documented in medical records compared with patient reports are summarized in Table 2. According to medical records, 158 patients were discharged with no new or worsened deficits, but only 117 (74%) of these patients confirmed this.

TABLE 2.

Patient-reported new or worsened neurological deficits at 30 days after surgery compared to documentation by clinicians at discharge (n = 191)

DescriptionType of New/Worsened Neurological Deficit
MotorLanguageCognitive
Agreement btwn hospital records & pt reports
 No new deficit153160157
 New deficit(s)17122
 Sum, agreement170 (89%)172 (91%)159 (83%)
Discordance btwn hospital records & pt reports
 Deficit reported by patients only161829
 Deficit reported by clinicians only513
 Sum, discordance21 (11%)19 (9%)32 (17%)
Sensitivity of retrospective review of hospital records0.520.400.07
Specificity of retrospective review of hospital records0.970.990.98

Pt = patient.

Motor Deficits

Medical records documented 22 patients with new or worsened motor deficits after surgery. In 16 of the patients who reported new or worsened motor deficits at follow-up, there was no corresponding documentation in hospital medical records. In 5 patients for whom new or worsened motor deficits were registered in medical records at discharge, the patients denied the development or worsening of the deficit during the follow-up interview 30 days post-operatively. The specificity of medical record documentation compared with the patients’ experience was 0.97, but sensitivity was only 0.52.

Language Deficits

Medical records documented new or worsened language deficits in 13 patients after surgery but failed to do so in 18 patients who themselves reported new deficits at follow-up. One patient denied any history of language deficits at follow-up despite documentation of new language deficit in the medical record. For language deficits, the specificity of retrospective review of medical records as compared with patient-reported outcomes was 0.99, but sensitivity was only 0.4.

Cognitive Deficits

Medical records documented new or worsened cognitive deficits in 5 patients after surgery, but failed to report new cognitive problems in 29 patients who reported new or worsened cognitive deficits at follow-up. Three patients denied any new or worsened cognitive deficits at follow-up despite documentation of new or worsened cognitive deficits in the medical records. For cognitive deficits the specificity of retrospective review of medical records as compared with patient-reported outcomes was 0.98, but sensitivity was only 0.07.

For all 3 evaluated deficit categories, Fig. 2 depicts the difference in reporting between hospital records at discharge and patient-reported outcome at 30 days.

FIG. 2.
FIG. 2.

New or worsened deficit documented in discharge hospital records (HR) compared with patient-reported outcome at 30 days (PRO).

Perioperative Complications

Comparison of hospital records and patient-reported data showed that serious complications such as postoperative intracranial hematomas, perioperative stroke, hy-drocephalus, meningitis, and deep surgical-site infections, were all documented in hospital medical records within the first 30 days (Table 3). Two superficial wound infections were reported by the patients at 30 days and not documented in the hospital records as occurring within the first 30 days after surgery, but were treated and documented in the hospital records more than 30 days post-operatively. Six urinary tract infections were reported by patients at 30 days but were not registered in the hospital records.

TABLE 3.

Perioperative complications documented in hospital records and reported by patients

ComplicationDocumented in Hospital Records at DischargeDocumented in Hospital Records during 1st 30 DaysDocumented in Hospital Records or Reported by Pts at 30 Days
Postop intracranial hematoma (subdural, epidural, cavity)444
Deep surgical-site infection (meningitis, subdural empyema osteomyelitis)244
Superficial wound infection013
Other infection (UTI, pneumonia)151622
Periop stroke366
Postop hydrocephalus, shunt/drain failure133
CSF leak266
Subdural hygroma022
Any complication27 (14.1%)42 (22.0%)50 (26.2%)
No complications164 (85.9%)149 (78.0%)141 (73.8%)

UTI = urinary tract infection.

Discussion

Retrospective review of hospital records at discharge is a traditional and widely used method of assessing acquired deficits in the neurosurgical literature. In the present study, we compared this to patient-reported outcomes at 30 days. We found that assessments based on retrospective reviews of medical records may greatly underestimate postoperative neurological deficits. Sensitivity of retrospectively obtained data from hospital records was low when compared with the patients’ perspectives at 30 days after surgery. Especially cognitive problems seemed to be largely underestimated by review of discharge data. This is not surprising, because subtle, yet important changes in cognitive function might not even occur to the patients themselves during the first days after surgery, and, consequently, not to the surgeon either. The same may apply to language deficits, although to a considerably smaller degree, with a sensitivity value of 0.40 for the retrospective review of hospital medical records. The discordance rate of 11% between the surgeons’ reports on new motor deficits compared with the patients’ does, at first glance, seem acceptable. Nevertheless, the sensitivity of the surgeon’s assessment of new motor deficits was no more than 0.52. For this category of deficits as well, it has to be considered that problems with fine motor skills such as writing, crafting, or other manual work may not always be detected by a customary neurological assessment at the end of a hospital stay. The relatively high specificity values we obtained for retrospective review of medical records indicate that new deficits were unlikely to be documented in the hospital records at discharge when the patients did not report any at 30 days.

In gastrointestinal surgery, a systematic review revealed that anastomosis leak was defined in 54 different ways in the literature.1 Similar studies are lacking in neurosurgery, but for softer outcome measures such as surgical morbidity and even extent of resection, there is no consensus on outcome classification. Therefore, it is no surprise that the heterogeneity of the systematic review on intraoperative mapping by De Witt Hamer and colleagues was almost 100% for early deficits and 87% for late deficits.3 When outcomes are measured systematically and prospectively, as for example in the Glioma Outcome Project2 and the 5-ALA (5-aminolevulinic acid) study,11 surgical morbidity is strikingly much higher than commonly reported in retrospective case series. Together with the known publication bias,12 the heterogeneity and, as seen, the insensitivity of retrospective outcome assessment is clearly problematic for benchmarking results, comparing results, and for meta-analyses in neurosurgery.10

The sensitivity and specificity of retrospective reviews of medical records is likely to depend considerably on local clinical routines and time of follow-up. Some authors define early and late outcome assessment,4,5,8 but there are large variations in timing, assessing profession (e.g., surgeon, neuropsychologist), assessment methods (neurological examination, neuropsychology examination, rating scales), and assessed domains. Regardless, there is a need for consensus on how and when to assess surgical morbidity following brain tumor surgery. Landriel Ibañez7 and colleagues have suggested a classification system of surgical and medical complications (including new deficits) in neurosurgery, but unfortunately, this system has not been widely embraced yet. Whether patient-reported measures are an optimal solution can be debated. Advantages of standardized and validated patient-reported measures include prospective data collection and reduced impact of diverging local traditions for outcome assessment. Another advantage of patient-reported data is limiting personal interest bias that might affect results when surgeons assess the outcome of their own operations.

In our study population, major medical and surgical complications (excluding new deficits) were all identified through retrospective review of hospital records within the first 30 days, almost two-thirds already at discharge. A few urinary tract infections were reported by patients only. Retrospective review of hospital records at 30 days may give a realistic impression of the complication rate.

The nonstandardized method and timing of assessment of neurological functions following brain tumor surgery is clearly problematic. Neurological function is not stable after surgery, and subtle deficits can be difficult to detect during a routine gross examination. We believe that a scheduled, detailed, and standardized neurological evaluation at 30 days after surgery might have yielded higher sensitivity values than the present retrospective review of medical records at discharge. Still, high sensitivity compared with patient-reported results is not necessarily the most crucial aspect of an outcome measure. To avoid confusion and to facilitate comparison of outcomes, reliability and validity are by far most important. Unfortunately, validated measures of neurosurgical morbidity are lacking for intracranial surgery.

Strengths/Weaknesses

In the present study, the population-based referral is a strength that ensures a well-defined and representative cohort. However, some patients did not consent to participate, possibly due to a wish to deal with their severe diagnosis without having to meet the additional demands from participation in a study. Some patients were lost to follow-up, mainly due to administrative errors and, in 3 cases, death. There may have been inconsistencies in reporting from cognitively impaired patients. Also, external validity of our findings depends on local clinical routines of assessing and documenting surgical results in medical records. In our hospital, postoperative outcome assessment at discharge is documented based on a nonstandardized clinical neurological assessment, without the use of a classification system. It has to be considered that a defined classification system or other specialists like neuropsychologists or speech therapists might have unveiled a greater part of the less-obvious deficits already at the time of discharge. Nevertheless, we believe that our clinical practice is in accordance with data published in many, if not most neurosurgical case series, which enhances the relevance of our results.

Conclusions

Retrospective assessment of medical records at discharge from hospital may greatly underestimate the incidence of new neurological deficits after brain tumor surgery when compared with patient-reported outcomes after 30 days. Although routines for when and how to assess surgical morbidity vary between institutions and publications, we believe our findings are relevant for a disconcertingly large proportion of the neurosurgical outcome literature.

Author Contributions

Conception and design: all authors. Acquisition of data: Drewes, Sagberg, Jakola, Solheim. Analysis and interpretation of data: Drewes. Drafting the article: Drewes, Solheim. Critically revising the article: all authors.

References

  • 1

    Bruce J, , Krukowski ZH, , Al-Khairy G, , Russell EM, & Park KG: Systematic review of the definition and measurement of anastomotic leak after gastrointestinal surgery. Br J Surg 88:11571168, 2001

    • Search Google Scholar
    • Export Citation
  • 2

    Chang SM, , Parney IF, , McDermott M, , Barker FG II, , Schmidt MH, & Huang W, : Perioperative complications and neurological outcomes of first and second craniotomies among patients enrolled in the Glioma Outcome Project. J Neurosurg 98:11751181, 2003

    • Search Google Scholar
    • Export Citation
  • 3

    De Witt Hamer PC, , Robles SG, , Zwinderman AH, , Duffau H, & Berger MS: Impact of intraoperative stimulation brain mapping on glioma surgery outcome: a meta-analysis. J Clin Oncol 30:25592565, 2012

    • Search Google Scholar
    • Export Citation
  • 4

    Gulati S, , Jakola AS, , Nerland US, , Weber C, & Solheim O: The risk of getting worse: surgically acquired deficits, perioperative complications, and functional outcomes after primary resection of glioblastoma. World Neurosurg 76:572579, 2011

    • Search Google Scholar
    • Export Citation
  • 5

    Konglund A, , Rogne SG, , Lund-Johansen M, , Scheie D, , Helseth E, & Meling TR: Outcome following surgery for intracranial meningiomas in the aging. Acta Neurol Scand 127:161169, 2013

    • Search Google Scholar
    • Export Citation
  • 6

    Kotowski M, , Naggara O, , Darsaut TE, , Nolet S, , Gevry G, & Kouznetsov E, : Safety and occlusion rates of surgical treatment of unruptured intracranial aneurysms: a systematic review and meta-analysis of the literature from 1990 to 2011. J Neurol Neurosurg Psychiatry 84:4248, 2013

    • Search Google Scholar
    • Export Citation
  • 7

    Landriel Ibañez FA, , Hem S, , Ajler P, , Vecchi E, , Ciraolo C, & Baccanelli M, : A new classification of complications in neurosurgery. World Neurosurg 75:709715, 2011

    • Search Google Scholar
    • Export Citation
  • 8

    Sanai N, , Martino J, & Berger MS: Morbidity profile following aggressive resection of parietal lobe gliomas. J Neurosurg 116:11821186, 2012

    • Search Google Scholar
    • Export Citation
  • 9

    Schöller K, , Licht S, , Tonn JC, & Uhl E: Randomized controlled trials in neurosurgery—how good are we?. Acta Neurochir (Wien) 151:519527, 2009

    • Search Google Scholar
    • Export Citation
  • 10

    Solheim O, , Jakola AS, & Unsgård G: Scientific alchemy and proposed gold standards of care. World Neurosurg 82:e566e567, 2014

  • 11

    Stummer W, , Reulen HJ, , Meinel T, , Pichlmeier U, , Schumacher W, & Tonn JC, : Extent of resection and survival in glioblastoma multiforme: identification of and adjustment for bias. Neurosurgery 62:564576, 2008

    • Search Google Scholar
    • Export Citation
  • 12

    Syin D, , Woreta T, , Chang DC, , Cameron JL, , Pronovost PJ, & Makary MA: Publication bias in surgery: implications for informed consent. J Surg Res 143:8893, 2007

    • Search Google Scholar
    • Export Citation
  • 13

    Yarascavitch BA, , Chuback JE, , Almenawer SA, , Reddy K, & Bhandari M: Levels of evidence in the neurosurgical literature: more tribulations than trials. Neurosurgery 71:11311137, 2012

    • Search Google Scholar
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

Contributor Notes

Correspondence Christina Drewes, Norwegian University of Technology and Science, Department of Circulation and Medical Imaging, Postboks 8905, Trondheim 7491, Norway. email: christina.drewes@ntnu.no.

ACCOMPANYING EDITORIAL 10.3171/2015.3.JNS142959.

INCLUDE WHEN CITING Published online August 7, 2015; DOI: 10.3171/2014.12.JNS142206.

Disclosure The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

  • View in gallery

    Flowchart of the inclusion process and reasons for noninclusion.

  • View in gallery

    New or worsened deficit documented in discharge hospital records (HR) compared with patient-reported outcome at 30 days (PRO).

  • 1

    Bruce J, , Krukowski ZH, , Al-Khairy G, , Russell EM, & Park KG: Systematic review of the definition and measurement of anastomotic leak after gastrointestinal surgery. Br J Surg 88:11571168, 2001

    • Search Google Scholar
    • Export Citation
  • 2

    Chang SM, , Parney IF, , McDermott M, , Barker FG II, , Schmidt MH, & Huang W, : Perioperative complications and neurological outcomes of first and second craniotomies among patients enrolled in the Glioma Outcome Project. J Neurosurg 98:11751181, 2003

    • Search Google Scholar
    • Export Citation
  • 3

    De Witt Hamer PC, , Robles SG, , Zwinderman AH, , Duffau H, & Berger MS: Impact of intraoperative stimulation brain mapping on glioma surgery outcome: a meta-analysis. J Clin Oncol 30:25592565, 2012

    • Search Google Scholar
    • Export Citation
  • 4

    Gulati S, , Jakola AS, , Nerland US, , Weber C, & Solheim O: The risk of getting worse: surgically acquired deficits, perioperative complications, and functional outcomes after primary resection of glioblastoma. World Neurosurg 76:572579, 2011

    • Search Google Scholar
    • Export Citation
  • 5

    Konglund A, , Rogne SG, , Lund-Johansen M, , Scheie D, , Helseth E, & Meling TR: Outcome following surgery for intracranial meningiomas in the aging. Acta Neurol Scand 127:161169, 2013

    • Search Google Scholar
    • Export Citation
  • 6

    Kotowski M, , Naggara O, , Darsaut TE, , Nolet S, , Gevry G, & Kouznetsov E, : Safety and occlusion rates of surgical treatment of unruptured intracranial aneurysms: a systematic review and meta-analysis of the literature from 1990 to 2011. J Neurol Neurosurg Psychiatry 84:4248, 2013

    • Search Google Scholar
    • Export Citation
  • 7

    Landriel Ibañez FA, , Hem S, , Ajler P, , Vecchi E, , Ciraolo C, & Baccanelli M, : A new classification of complications in neurosurgery. World Neurosurg 75:709715, 2011

    • Search Google Scholar
    • Export Citation
  • 8

    Sanai N, , Martino J, & Berger MS: Morbidity profile following aggressive resection of parietal lobe gliomas. J Neurosurg 116:11821186, 2012

    • Search Google Scholar
    • Export Citation
  • 9

    Schöller K, , Licht S, , Tonn JC, & Uhl E: Randomized controlled trials in neurosurgery—how good are we?. Acta Neurochir (Wien) 151:519527, 2009

    • Search Google Scholar
    • Export Citation
  • 10

    Solheim O, , Jakola AS, & Unsgård G: Scientific alchemy and proposed gold standards of care. World Neurosurg 82:e566e567, 2014

  • 11

    Stummer W, , Reulen HJ, , Meinel T, , Pichlmeier U, , Schumacher W, & Tonn JC, : Extent of resection and survival in glioblastoma multiforme: identification of and adjustment for bias. Neurosurgery 62:564576, 2008

    • Search Google Scholar
    • Export Citation
  • 12

    Syin D, , Woreta T, , Chang DC, , Cameron JL, , Pronovost PJ, & Makary MA: Publication bias in surgery: implications for informed consent. J Surg Res 143:8893, 2007

    • Search Google Scholar
    • Export Citation
  • 13

    Yarascavitch BA, , Chuback JE, , Almenawer SA, , Reddy K, & Bhandari M: Levels of evidence in the neurosurgical literature: more tribulations than trials. Neurosurgery 71:11311137, 2012

    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 422 173 6
PDF Downloads 269 90 3
EPUB Downloads 0 0 0