In-hospital mortality after traumatic brain injury surgery: a nationwide population-based comparison of mortality predictors used in artificial neural network and logistic regression models

Clinical article

Restricted access

Object

Most reports compare artificial neural network (ANN) models and logistic regression models in only a single data set, and the essential issue of internal validity (reproducibility) of the models has not been adequately addressed. This study proposes to validate the use of the ANN model for predicting in-hospital mortality after traumatic brain injury (TBI) surgery and to compare the predictive accuracy of ANN with that of the logistic regression model.

Methods

The authors of this study retrospectively analyzed 16,956 patients with TBI nationwide who were surgically treated in Taiwan between 1998 and 2009. For every 1000 pairs of ANN and logistic regression models, the area under the receiver operating characteristic curve (AUC), Hosmer-Lemeshow statistics, and accuracy rate were calculated and compared using paired t-tests. A global sensitivity analysis was also performed to assess the relative importance of input parameters in the ANN model and to rank the variables in order of importance.

Results

The ANN model outperformed the logistic regression model in terms of accuracy in 95.15% of cases, in terms of Hosmer-Lemeshow statistics in 43.68% of cases, and in terms of the AUC in 89.14% of cases. The global sensitivity analysis of in-hospital mortality also showed that the most influential (sensitive) parameters in the ANN model were surgeon volume followed by hospital volume, Charlson comorbidity index score, length of stay, sex, and age.

Conclusions

This work supports the continued use of ANNs for predictive modeling of neurosurgery outcomes. However, further studies are needed to confirm the clinical efficacy of the proposed model.

Abbreviations used in this paper:ANN = artificial neural network; AUC = area under the receiver operating characteristic curve; BNHI = Bureau of National Health Insurance; GCS = Glasgow Coma Scale; LOS = length of stay; MLP = multilayer perceptron; NPV = negative predictive value; PPV = positive predictive value; TBI = traumatic brain injury; VSR = variable sensitivity ratio.

Article Information

Drs. Lee and Lin contributed equally to this work.

Address correspondence to: Chih-Lung Lin, M.D., Ph.D., Department of Neurosurgery, Kaohsiung Medical University Hospital, 100-Shih-Chun 1st Road, Kaohsiung 80708, Taiwan, Republic of China. email: hshi@kmu.edu.tw.

Please include this information when citing this paper: published online February 1, 2013; DOI: 10.3171/2013.1.JNS121130.

© AANS, except where prohibited by US copyright law.

Headings

Figures

  • View in gallery

    Schematic diagram of the ANN model with 4 input nodes, 8 nodes in a single hidden layer, and a single output node representing in-hospital survival. HB = hidden bias; H1–8 = hidden neuron; IB = input bias; X1 = sex; X2 = age; X3 = Charlson comorbidity index; X4 = hospital volume; X5 = surgeon volume; X6 = length of stay.

References

  • 1

    Bombardier CHFann JRTemkin NREsselman PCBarber JDikmen SS: Rates of major depressive disorder and clinical outcomes following traumatic brain injury. JAMA 303:193819452010

  • 2

    Bullock MRPovlishock JT: Guidelines for the management of severe traumatic brain injury. Editor's Commentary. J Neurotrauma 24:Suppl 12 p preceding S12007

  • 3

    Cadotte DWVachhrajani SPirouzmand F: The epidemiological trends of head injury in the largest Canadian adult trauma center from 1986 to 2007. Clinical article. J Neurosurg 114:150215092011

  • 4

    Curry WTMcDermott MWCarter BSBarker FG II: Craniotomy for meningioma in the United States between 1988 and 2000: decreasing rate of mortality and the effect of provider caseload. J Neurosurg 102:9779862005

  • 5

    Das ABen-Menachem TCooper GSChak ASivak MV JrGonet JA: Prediction of outcome in acute lower-gastrointestinal haemorrhage based on an artificial neural network: internal and external validation of a predictive model. Lancet 362:126112662003

  • 6

    de Jongh MAVerhofstad MHLeenen LP: Accuracy of different survival prediction models in a trauma population. Br J Surg 97:180518132010

  • 7

    Defillo A: Traumatic brain injury. J Neurosurg 113:3994002010. (Letter)

  • 8

    Deyo RACherkin DCCiol MA: Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 45:6136191992

  • 9

    DiRusso SMSullivan THolly CCuff SNSavino J: An artificial neural network as a model for prediction of survival in trauma patients: validation for a regional trauma area. J Trauma 49:2122232000

  • 10

    Eftekhar BMohammad KArdebili HEGhodsi MKetabchi E: Comparison of artificial neural network and logistic regression models for prediction of mortality in head trauma based on initial clinical data. BMC Med Inform Decis Mak 5:32005

  • 11

    Farahvar AGerber LMChiu YLHärtl RFroelich MCarney N: Response to intracranial hypertension treatment as a predictor of death in patients with severe traumatic brain injury. Clinical article. J Neurosurg 114:147114782011. (Erratum in J Neurosurg 115:191 2011)

  • 12

    Forsström JJDalton KJ: Artificial neural networks for decision support in clinical medicine. Ann Med 27:5095171995

  • 13

    Galarneau MRWoodruff SIDye JLMohrle CRWade AL: Traumatic brain injury during Operation Iraqi Freedom: findings from the United States Navy-Marine Corps Combat Trauma Registry. J Neurosurg 108:9509572008

  • 14

    Grossman RMukherjee DChang DCPurtell MLim MBrem H: Predictors of inpatient death and complications among postoperative elderly patients with metastatic brain tumors. Ann Surg Oncol 18:5215282011

  • 15

    Hampton T: Traumatic brain injury a growing problem among troops serving in today's wars. JAMA 306:4774792011

  • 16

    Hotchkiss JRGalvan DA: Artificial neural networks and power of breathing: new approaches, expanded utility?. Crit Care Med 34:126712682006

  • 17

    Hyder AAWunderlich CAPuvanachandra PGururaj GKobusingye OC: The impact of traumatic brain injuries: a global perspective. NeuroRehabilitation 22:3413532007

  • 18

    Lang EWPitts LHDamron SLRutledge R: Outcome after severe head injury: an analysis of prediction based upon comparison of neural network versus logistic regression analysis. Neurol Res 19:2742801997

  • 19

    Livingston EHCao J: Procedure volume as a predictor of surgical outcomes. JAMA 304:95972010

  • 20

    Nakagawa TAAshwal SMathur MMysore MRBruce DConway EE Jr: Guidelines for the determination of brain death in infants and children: an update of the 1987 Task Force recommendations. Crit Care Med 39:213921552011

  • 21

    Pastorius Benziger CBernabe-Ortiz AMiranda JJBukhman G: Sex differences in health care-seeking behavior for acute coronary syndrome in a low income country, Peru. Crit Pathw Cardiol 10:991032011

  • 22

    Rughani AIDumont TMLu ZBongard JHorgan MAPenar PL: Use of an artificial neural network to predict head injury outcome. Clinical article. J Neurosurg 113:5855902010

  • 23

    Sandberg IWLo JTFancourt CLPrincipe JCKatagiri SHaykin S: Nonlinear Dynamical Systems: FeedForward Neural Network Perspectives New YorkJohn Wiley & Sons2001

  • 24

    Sarani BTemple-Lykens BKim PSonnad SBergey MPascual JL: Factors associated with mortality and brain injury after falls from the standing position. J Trauma 67:9549582009

  • 25

    Segal MEGoodman PHGoldstein RHauck WWhyte JGraham JW: The accuracy of artificial neural networks in predicting long-term outcome after traumatic brain injury. J Head Trauma Rehabil 21:2983142006

  • 26

    Servadei FCompagnone CSahuquillo J: The role of surgery in traumatic brain injury. Curr Opin Crit Care 13:1631682007

  • 27

    Shamim MSQadeer MMurtaza GEnam SAFarooqi NB: Emergency department predictors of tracheostomy in patients with isolated traumatic brain injury requiring emergency cranial decompression. Clinical article. J Neurosurg 115:100710122011

  • 28

    Shi HYLee TLee HHHo WHSun DPWang JJ: Comparison of artificial neuroal network and logistic regression models for predicting in-hospital mortality after primary liver cancer surgery. PLoS One 7:e357812012

  • 29

    Stein SCGeorgoff PMeghan SMirza KLEl Falaky OM: Relationship of aggressive monitoring and treatment to improved outcomes in severe traumatic brain injury. Clinical article. J Neurosurg 112:110511122010

  • 30

    Terrin NSchmid CHGriffith JLD'Agostino RBSelker HP: External validity of predictive models: a comparison of logistic regression, classification trees, and neural networks. J Clin Epidemiol 56:7217292003

  • 31

    Tu JV: Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J Clin Epidemiol 49:122512311996

  • 32

    Whisnant JPSacco SEO'Fallon WMFode NCSundt TM Jr: Referral bias in aneurysmal subarachnoid hemorrhage. J Neurosurg 78:7267321993

  • 33

    Zou JHan YSo SS: Overview of artificial neural networks. Methods Mol Biol 458:15232008

TrendMD

Cited By

Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 177 177 22
Full Text Views 196 196 6
PDF Downloads 145 145 4
EPUB Downloads 0 0 0

PubMed

Google Scholar