Most reports compare artificial neural network (ANN) models and logistic regression models in only a single data set, and the essential issue of internal validity (reproducibility) of the models has not been adequately addressed. This study proposes to validate the use of the ANN model for predicting in-hospital mortality after traumatic brain injury (TBI) surgery and to compare the predictive accuracy of ANN with that of the logistic regression model.
The authors of this study retrospectively analyzed 16,956 patients with TBI nationwide who were surgically treated in Taiwan between 1998 and 2009. For every 1000 pairs of ANN and logistic regression models, the area under the receiver operating characteristic curve (AUC), Hosmer-Lemeshow statistics, and accuracy rate were calculated and compared using paired t-tests. A global sensitivity analysis was also performed to assess the relative importance of input parameters in the ANN model and to rank the variables in order of importance.
The ANN model outperformed the logistic regression model in terms of accuracy in 95.15% of cases, in terms of Hosmer-Lemeshow statistics in 43.68% of cases, and in terms of the AUC in 89.14% of cases. The global sensitivity analysis of in-hospital mortality also showed that the most influential (sensitive) parameters in the ANN model were surgeon volume followed by hospital volume, Charlson comorbidity index score, length of stay, sex, and age.
This work supports the continued use of ANNs for predictive modeling of neurosurgery outcomes. However, further studies are needed to confirm the clinical efficacy of the proposed model.
Abbreviations used in this paper:ANN = artificial neural network; AUC = area under the receiver operating characteristic curve; BNHI = Bureau of National Health Insurance; GCS = Glasgow Coma Scale; LOS = length of stay; MLP = multilayer perceptron; NPV = negative predictive value; PPV = positive predictive value; TBI = traumatic brain injury; VSR = variable sensitivity ratio.
Drs. Lee and Lin contributed equally to this work.
Address correspondence to: Chih-Lung Lin, M.D., Ph.D., Department of Neurosurgery, Kaohsiung Medical University Hospital, 100-Shih-Chun 1st Road, Kaohsiung 80708, Taiwan, Republic of China. email: firstname.lastname@example.org.
Please include this information when citing this paper: published online February 1, 2013; DOI: 10.3171/2013.1.JNS121130.
DasABen-MenachemTCooperGSChakASivakMVJrGonetJA: Prediction of outcome in acute lower-gastrointestinal haemorrhage based on an artificial neural network: internal and external validation of a predictive model. Lancet362:1261–12662003
EftekharBMohammadKArdebiliHEGhodsiMKetabchiE: Comparison of artificial neural network and logistic regression models for prediction of mortality in head trauma based on initial clinical data. BMC Med Inform Decis Mak5:32005
FarahvarAGerberLMChiuYLHärtlRFroelichMCarneyN: Response to intracranial hypertension treatment as a predictor of death in patients with severe traumatic brain injury. Clinical article. J Neurosurg114:1471–14782011. (Erratum in J Neurosurg 115:191 2011)
NakagawaTAAshwalSMathurMMysoreMRBruceDConwayEEJr: Guidelines for the determination of brain death in infants and children: an update of the 1987 Task Force recommendations. Crit Care Med39:2139–21552011