Development and external validation of a clinical prediction model for functional impairment after intracranial tumor surgery

View More View Less
  • 1 Department of Neurosurgery and Machine Intelligence in Clinical Neuroscience (MICN) Laboratory, Clinical Neuroscience Center, University Hospital Zurich, University of Zurich, Switzerland;
  • 2 Amsterdam UMC, Vrije Universiteit Amsterdam, Neurosurgery, Amsterdam Movement Sciences, Amsterdam, The Netherlands;
  • 3 Department of Neurosurgery, Fondazione IRCCS Istituto Neurologico Carlo Besta, Milan;
  • 4 Neurology, Public Health and Disability Unit, Fondazione IRCCS Istituto Neurologico Carlo Besta, Milan, Italy;
  • 5 Department of Neurosurgery, Karolinska University Hospital, Stockholm;
  • 6 Department of Clinical Neuroscience and Medicine, Karolinska Institutet, Stockholm, Sweden;
  • 7 Department of Neurosurgery, Rigshospitalet, Copenhagen, Denmark;
  • 8 Department of Neurosurgery, University Medical Center, Johannes Gutenberg University Mainz, Germany;
  • 9 Department of Neurosurgery, Medical University of Innsbruck, Austria;
  • 10 Department of Neurosurgery, Sahlgrenska University Hospital, Gothenburg;
  • 11 Institute of Neuroscience and Physiology, Sahlgrenska Academy, Gothenburg, Sweden;
  • 12 Department of Neurosurgery, University Hospital of North Norway, Tromsö;
  • 13 Department of Neurosurgery, St. Olav’s University Hospital, Trondheim, Norway;
  • 14 Department of Neurosurgery, Georg August University, University Medical Center, Göttingen, Germany;
  • 15 Department of Neurosurgery, Haaglanden Medical Center, The Hague;
  • 16 Department of Neurosurgery, Leiden University Medical Center, Leiden;
  • 17 Department of Neurology, Haaglanden Medical Center, The Hague, The Netherlands; and
  • 18 Department of Neurosurgery, Faculty of Medicine, RWTH Aachen University, Aachen, Germany
Restricted access

Purchase Now

USD  $45.00

JNS + Pediatrics - 1 year subscription bundle (Individuals Only)

USD  $505.00

JNS + Pediatrics + Spine - 1 year subscription bundle (Individuals Only)

USD  $600.00
Print or Print + Online

OBJECTIVE

Decision-making for intracranial tumor surgery requires balancing the oncological benefit against the risk for resection-related impairment. Risk estimates are commonly based on subjective experience and generalized numbers from the literature, but even experienced surgeons overestimate functional outcome after surgery. Today, there is no reliable and objective way to preoperatively predict an individual patient’s risk of experiencing any functional impairment.

METHODS

The authors developed a prediction model for functional impairment at 3 to 6 months after microsurgical resection, defined as a decrease in Karnofsky Performance Status of ≥ 10 points. Two prospective registries in Switzerland and Italy were used for development. External validation was performed in 7 cohorts from Sweden, Norway, Germany, Austria, and the Netherlands. Age, sex, prior surgery, tumor histology and maximum diameter, expected major brain vessel or cranial nerve manipulation, resection in eloquent areas and the posterior fossa, and surgical approach were recorded. Discrimination and calibration metrics were evaluated.

RESULTS

In the development (2437 patients, 48.2% male; mean age ± SD: 55 ± 15 years) and external validation (2427 patients, 42.4% male; mean age ± SD: 58 ± 13 years) cohorts, functional impairment rates were 21.5% and 28.5%, respectively. In the development cohort, area under the curve (AUC) values of 0.72 (95% CI 0.69–0.74) were observed. In the pooled external validation cohort, the AUC was 0.72 (95% CI 0.69–0.74), confirming generalizability. Calibration plots indicated fair calibration in both cohorts. The tool has been incorporated into a web-based application available at https://neurosurgery.shinyapps.io/impairment/.

CONCLUSIONS

Functional impairment after intracranial tumor surgery remains extraordinarily difficult to predict, although machine learning can help quantify risk. This externally validated prediction tool can serve as the basis for case-by-case discussions and risk-to-benefit estimation of surgical treatment in the individual patient.

ABBREVIATIONS AUC = area under the curve; EOR = extent of resection; KPS = Karnofsky Performance Status; ML = machine learning; PROM = patient-reported outcome measure.

Supplementary Materials

    • Supplementary Data (PDF 704 KB)

JNS + Pediatrics - 1 year subscription bundle (Individuals Only)

USD  $505.00

JNS + Pediatrics + Spine - 1 year subscription bundle (Individuals Only)

USD  $600.00

Contributor Notes

Correspondence Victor E. Staartjes: Machine Intelligence in Clinical Neuroscience (MICN) Laboratory, University Hospital Zurich, Switzerland. victoregon.staartjes@usz.ch.

INCLUDE WHEN CITING Published online June 12, 2020; DOI: 10.3171/2020.4.JNS20643.

Disclosures The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

  • 1

    Barker FG II, Curry WT Jr, Carter BS. Surgery for primary supratentorial brain tumors in the United States, 1988 to 2000: the effect of provider caseload and centralization of care. Neuro Oncol. 2005;7(1):4963.

    • Search Google Scholar
    • Export Citation
  • 2

    Ferroli P, Broggi M, Schiavolin S, Predicting functional impairment in brain tumor surgery: the Big Five and the Milan Complexity Scale. Neurosurg Focus. 2015;39(6):E14.

    • Search Google Scholar
    • Export Citation
  • 3

    Yordanova YN, Moritz-Gasser S, Duffau H. Awake surgery for WHO Grade II gliomas within “noneloquent” areas in the left dominant hemisphere: toward a “supratotal” resection. Clinical article. J Neurosurg. 2011;115(2):232239.

    • Search Google Scholar
    • Export Citation
  • 4

    Sanai N, Berger MS. Glioma extent of resection and its impact on patient outcome. Neurosurgery. 2008;62(4):753–764, 264–266.

  • 5

    Marko NF, Weil RJ, Schroeder JL, Extent of resection of glioblastoma revisited: personalized survival modeling facilitates more accurate survival prediction and supports a maximum-safe-resection approach to surgery. J Clin Oncol. 2014;32(8):774782.

    • Search Google Scholar
    • Export Citation
  • 6

    Stienen MN, Zhang DY, Broggi M, The influence of preoperative dependency on mortality, functional recovery and complications after microsurgical resection of intracranial tumors. J Neurooncol. 2018;139(2):441448.

    • Search Google Scholar
    • Export Citation
  • 7

    Schiavolin S, Raggi A, Scaratti C, Patients’ reported outcome measures and clinical scales in brain tumor surgery: results from a prospective cohort study. Acta Neurochir (Wien). 2018;160(5):10531061.

    • Search Google Scholar
    • Export Citation
  • 8

    Rahman M, Abbatematteo J, De Leo EK, The effects of new or worsened postoperative neurological deficits on survival of patients with glioblastoma. J Neurosurg. 2017;127(1):123131.

    • Search Google Scholar
    • Export Citation
  • 9

    Jakola AS, Gulati S, Weber C, Postoperative deterioration in health related quality of life as predictor for survival in patients with glioblastoma: a prospective study. PLoS One. 2011;6(12):e28592.

    • Search Google Scholar
    • Export Citation
  • 10

    Sagberg LM, Drewes C, Jakola AS, Solheim O. Accuracy of operating neurosurgeons’ prediction of functional levels after intracranial tumor surgery. J Neurosurg. 2017;126(4):11731180.

    • Search Google Scholar
    • Export Citation
  • 11

    Obermeyer Z, Emanuel EJ. Predicting the future—big data, machine learning, and clinical medicine. N Engl J Med. 2016;375(13):12161219.

    • Search Google Scholar
    • Export Citation
  • 12

    Galovic M, Stauber AJ, Leisi N, Development and validation of a prognostic model of swallowing recovery and enteral tube feeding after ischemic stroke. JAMA Neurol. 2019;76(5):561570.

    • Search Google Scholar
    • Export Citation
  • 13

    Khor S, Lavallee D, Cizik AM, Development and validation of a prediction model for pain and functional outcomes after lumbar spine surgery. JAMA Surg. 2018;153(7):634642.

    • Search Google Scholar
    • Export Citation
  • 14

    Senders JT, Staples PC, Karhade AV, Machine learning and neurosurgical outcome prediction: a systematic review. World Neurosurg. 2018;109:476486.e1.

    • Search Google Scholar
    • Export Citation
  • 15

    Jaja BNR, Saposnik G, Lingsma HF, Development and validation of outcome prediction models for aneurysmal subarachnoid haemorrhage: the SAHIT multinational cohort study. BMJ. 2018;360:j5745.

    • Search Google Scholar
    • Export Citation
  • 16

    Staartjes VE, Serra C, Muscas G, Utility of deep neural networks in predicting gross-total resection after transsphenoidal surgery for pituitary adenoma: a pilot study. Neurosurg Focus. 2018;45(5):E12.

    • Search Google Scholar
    • Export Citation
  • 17

    Sarnthein J, Stieglitz L, Clavien P-A, Regli L. A patient registry to improve patient safety: recording general neurosurgery complications. PLoS One. 2016;11(9):e0163154.

    • Search Google Scholar
    • Export Citation
  • 18

    Stummer W, Stepp H, Wiestler OD, Pichlmeier U. Randomized, prospective double-blinded study comparing 3 different doses of 5-aminolevulinic acid for fluorescence-guided resections of malignant gliomas. Neurosurgery. 2017;81(2):230239.

    • Search Google Scholar
    • Export Citation
  • 19

    Kubben PL, ter Meulen KJ, Schijns OE, Intraoperative MRI-guided resection of glioblastoma multiforme: a systematic review. Lancet Oncol. 2011;12(11):10621070.

    • Search Google Scholar
    • Export Citation
  • 20

    Gronningsaeter A, Kleven A, Ommedal S, SonoWand, an ultrasound-based neuronavigation system. Neurosurgery. 2000;47(6):13731380.

  • 21

    Sanai N, Mirzadeh Z, Berger MS. Functional outcome after language mapping for glioma resection. N Engl J Med. 2008;358(1):1827.

  • 22

    Nghiemphu PL, Liu W, Lee Y, Bevacizumab and chemotherapy for recurrent glioblastoma: a single-institution experience. Neurology. 2009;72(14):12171222.

    • Search Google Scholar
    • Export Citation
  • 23

    Hastie T, Tibshirani R. Generalized Additive Models. 1st ed. Chapman & Hall; 1990.

  • 24

    Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Science & Business Media; 2013.

    • Search Google Scholar
    • Export Citation
  • 25

    Hastie T. gam: generalized additive models. 2019. Accessed April 22, 2020. https://CRAN.R-project.org/package=gam

    • Export Citation
  • 26

    Kuhn M. Building predictive models in R using the caret package. J Stat Softw. 2008;28(5).

  • 27

    Batista GEAPA, Monard MC. An analysis of four missing data treatment methods for supervised learning. Appl Artif Intell. 2003;17(5–6):519533.

    • Search Google Scholar
    • Export Citation
  • 28

    Perkins NJ, Schisterman EF. The inconsistency of “optimal” cutpoints obtained using two criteria based on the receiver operating characteristic curve. Am J Epidemiol. 2006;163(7):670675.

    • Search Google Scholar
    • Export Citation
  • 29

    Janssen KJM, Moons KGM, Kalkman CJ, Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol. 2008;61(1):7686.

    • Search Google Scholar
    • Export Citation
  • 30

    Spetzler RF, Martin NA. A proposed grading system for arteriovenous malformations. J Neurosurg. 1986;65(4):476483.

  • 31

    Vasella F, Velz J, Neidert MC, Safety of resident training in the microsurgical resection of intracranial tumors: data from a prospective registry of complications and outcome. Sci Rep. 2019;9(1):954.

    • Search Google Scholar
    • Export Citation
  • 32

    Senders JT, Arnaout O, Karhade AV, Natural and artificial intelligence in neurosurgery: a systematic review. Neurosurgery. 2018;83(2):181192.

    • Search Google Scholar
    • Export Citation
  • 33

    van Rein EAJ, van der Sluijs R, Voskens FJ, Development and validation of a prediction model for prehospital triage of trauma patients. JAMA Surg. 2019;154(5):421429.

    • Search Google Scholar
    • Export Citation
  • 34

    Duffau H, Capelle L, Denvil D, Functional recovery after surgical resection of low grade gliomas in eloquent brain: hypothesis of brain compensation. J Neurol Neurosurg Psychiatry. 2003;74(7):901907.

    • Search Google Scholar
    • Export Citation
  • 35

    Chang SM, Parney IF, McDermott M, Perioperative complications and neurological outcomes of first and second craniotomies among patients enrolled in the Glioma Outcome Project. J Neurosurg. 2003;98(6):11751181.

    • Search Google Scholar
    • Export Citation
  • 36

    Reponen E, Tuominen H, Korja M. Evidence for the use of preoperative risk assessment scores in elective cranial neurosurgery: a systematic review of the literature. Anesth Analg. 2014;119(2):420432.

    • Search Google Scholar
    • Export Citation
  • 37

    Chow R, Chiu N, Bruera E, Inter-rater reliability in performance status assessment among health care professionals: a systematic review. Ann Palliat Med. 2016;5(2):8392.

    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 473 473 473
Full Text Views 66 66 66
PDF Downloads 46 46 46
EPUB Downloads 0 0 0