Machine learning applications for the prediction of surgical site infection in neurological operations

Free access

OBJECTIVE

Surgical site infection (SSI) following a neurosurgical operation is a complication that impacts morbidity, mortality, and economics. Currently, machine learning (ML) algorithms are used for outcome prediction in various neurosurgical aspects. The implementation of ML algorithms to learn from medical data may help in obtaining prognostic information on diseases, especially SSIs. The purpose of this study was to compare the performance of various ML models for predicting surgical infection after neurosurgical operations.

METHODS

A retrospective cohort study was conducted on patients who had undergone neurosurgical operations at tertiary care hospitals between 2010 and 2017. Supervised ML algorithms, which included decision tree, naive Bayes with Laplace correction, k-nearest neighbors, and artificial neural networks, were trained and tested as binary classifiers (infection or no infection). To evaluate the ML models from the testing data set, their sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), as well as their accuracy, receiver operating characteristic curve, and area under the receiver operating characteristic curve (AUC) were analyzed.

RESULTS

Data were available for 1471 patients in the study period. The SSI rate was 4.6%, and the type of SSI was superficial, deep, and organ/space in 1.2%, 0.8%, and 2.6% of cases, respectively. Using the backward stepwise method, the authors determined that the significant predictors of SSI in the multivariable Cox regression analysis were postoperative CSF leakage/subgaleal collection (HR 4.24, p < 0.001) and postoperative fever (HR 1.67, p = 0.04). Compared with other ML algorithms, the naive Bayes had the highest performance with sensitivity at 63%, specificity at 87%, PPV at 29%, NPV at 96%, and AUC at 76%.

CONCLUSIONS

The naive Bayes algorithm is highlighted as an accurate ML method for predicting SSI after neurosurgical operations because of its reasonable accuracy. Thus, it can be used to effectively predict SSI in individual neurosurgical patients. Therefore, close monitoring and allocation of treatment strategies can be informed by ML predictions in general practice.

ABBREVIATIONS ANN = artificial neural network; ASA = American Society of Anesthesiologists; AUC = area under the receiver operating characteristic curve; CSF = cerebrospinal fluid; DT = decision tree; k-NN = k-nearest neighbors; ML = machine learning; NB = naive Bayes; NPV = negative predictive value; PPV = positive predictive value; ROC = receiver operating characteristic; SAH = subarachnoid hemorrhage; SSI = surgical site infection.

OBJECTIVE

Surgical site infection (SSI) following a neurosurgical operation is a complication that impacts morbidity, mortality, and economics. Currently, machine learning (ML) algorithms are used for outcome prediction in various neurosurgical aspects. The implementation of ML algorithms to learn from medical data may help in obtaining prognostic information on diseases, especially SSIs. The purpose of this study was to compare the performance of various ML models for predicting surgical infection after neurosurgical operations.

METHODS

A retrospective cohort study was conducted on patients who had undergone neurosurgical operations at tertiary care hospitals between 2010 and 2017. Supervised ML algorithms, which included decision tree, naive Bayes with Laplace correction, k-nearest neighbors, and artificial neural networks, were trained and tested as binary classifiers (infection or no infection). To evaluate the ML models from the testing data set, their sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), as well as their accuracy, receiver operating characteristic curve, and area under the receiver operating characteristic curve (AUC) were analyzed.

RESULTS

Data were available for 1471 patients in the study period. The SSI rate was 4.6%, and the type of SSI was superficial, deep, and organ/space in 1.2%, 0.8%, and 2.6% of cases, respectively. Using the backward stepwise method, the authors determined that the significant predictors of SSI in the multivariable Cox regression analysis were postoperative CSF leakage/subgaleal collection (HR 4.24, p < 0.001) and postoperative fever (HR 1.67, p = 0.04). Compared with other ML algorithms, the naive Bayes had the highest performance with sensitivity at 63%, specificity at 87%, PPV at 29%, NPV at 96%, and AUC at 76%.

CONCLUSIONS

The naive Bayes algorithm is highlighted as an accurate ML method for predicting SSI after neurosurgical operations because of its reasonable accuracy. Thus, it can be used to effectively predict SSI in individual neurosurgical patients. Therefore, close monitoring and allocation of treatment strategies can be informed by ML predictions in general practice.

Surgical site infection (SSI) following neurosurgical operations is a burdensome complication in the field. Such complications can impact morbidity, mortality, and economics.4,5,25 O’Keeffe et al. conducted a cost analysis of craniotomy infections, identifying an estimated cost per case of infection at £9283. The financial burden caused by craniotomy infections is often compounded by the direct costs incurred by prolonged hospitalization of the patient, diagnostic tests, treatment, and reoperation.4,18

Machine learning (ML) is used for outcome prediction in the neurosurgical field. Several ML algorithms have been developed using complex mathematical models that can learn from clinical data from, for example, neuro-oncology, neurovascular surgery, traumatic brain injury, and epilepsy.3,14,17,19,23,24,26 Memarian et al. reported that the multimodality of ML, including linear discriminant analysis, naive Bayes (NB), and support vector machines, has high accuracy in surgical outcome prediction in patients with mesial temporal lobe epilepsy.15,17 Moreover, Armañanzas et al. demonstrated that ML algorithms such as NB, logistic regression, and k-nearest neighbors (k-NN) may be powerful methods of selecting surgical candidates who have a high likelihood of remaining free from seizures in temporal lobe epilepsy.3

From literature reviews we find evidence that ML has been applied in predicting neurosurgical complications, particularly SSI. Habibi et al. compared artificial neural networks (ANNs) and traditional logistic regression models for predicting ventriculoperitoneal shunt infections in childhood hydrocephalus.12 The ANNs and logistic models predicted shunt infection with an accuracy of 83.1% and 55.7%, respectively.12 The implementation of ML algorithms to learn from medical data may help in obtaining prognostic information on diseases. It also has potential use in predicting which patients will be in the high-risk group as well as allocating treatment strategies in general practice.11 Given the burden of SSI, the purpose of this study was to compare the performance of various ML models in terms of predicting surgical infection after neurosurgical operations.

Methods

Study Design and Population

This study was performed with the approval of the Ethics Committee of the Faculty of Medicine, Songklanagarind Hospital, Prince of Songkla University. We conducted a retrospective review of patients who had undergone neurosurgical operations at Songklanagarind Hospital between 2010 and 2017 and who had at least 3 months of follow-up data after surgery. Patients without available operative data and spine surgery patients were not included in our analysis. Demographic data were collected, as were baseline clinical characteristics, operative details, American Society of Anesthesiologists (ASA) classifications, and postoperative outcomes.

The initial operation was counted as the first operation performed at Songklanagarind Hospital, whereas any subsequent operations were counted as a reoperation variable in the same patient. Operative time was defined as the time from skin incision to skin closure, which was documented in the anesthesia records by an anesthesiologist. Postoperative cerebrospinal fluid (CSF) leakage was defined as visible leaking from a surgical wound, nasal cavity (transsphenoidal operation), or subgaleal/subcutaneous collection as observed after craniotomy or spinal operations in the same admission. Postoperative ventriculostomy was defined as the ventriculostomy catheter inserted after the primary procedure but during the same operative case. Postoperative fever was defined as such when a patient developed a body temperature of 38°C or higher during the same admission.6,7 Bacteremia, urinary tract infection, and pneumonia were defined as any organism identified by culture or by evidence of infection involving a specific organ detected on an imaging test in the same admission. Previous infection was defined as such when a patient had a history of an SSI from a procedure at another hospital before having an operation at our hospital.

According to the SSI criteria defined by the Centers for Disease Control and Prevention (CDC), the primary outcomes were reviewed and routinely determined at hospital discharge, every follow-up appointment, and at outpatient clinics until December 31, 2018, using outpatient medical records. Additionally, SSIs were classified as superficial incisional SSI, deep incisional SSI, and organ/space SSI.6,7

Superficial incisional SSI is defined as purulent drainage from a superficial incision or as organisms identified by a culture. Deep incisional SSI is defined as purulent drainage from the deep soft tissue of an incision (e.g., fascial and muscle layers), an organism identified by a culture, or evidence of infection involving the deep incision that is detected on gross anatomical or histopathological exam or an imaging test.6,7 Moreover, organ/space SSIs include brain abscess, subdural or epidural infection, encephalitis, meningitis, and ventriculitis.6,7 An organ/space SSI is defined as such when purulent drainage comes from a drain that is placed into the organ/space (e.g., ventriculostomy); the main part of an abscess/other infection deeper than the fascial/muscle layers is present; organisms are identified from an aseptically obtained fluid or tissue in the organ/space by a culture; or there is evidence of infection involving the organ/space that is detected on gross anatomical or histopathological exam or on imaging.6,7

Descriptive Statistics and Survival Analysis

Clinical characteristics and therapeutic factors were first described using descriptive statistics. A survival curve was constructed from the total data using the Kaplan-Meier method. A Cox proportional-hazards regression model was used to identify the univariate and multivariate predictors of survival. In the multivariate analysis, a stepwise method with the Akaike information criterion (AIC) was used to check whether the variable deserved to be included in the model and stratification factors. A p value less than 0.05 was considered statistically significant. Statistical analysis was performed using R version 3.5.0 software (R Foundation for Statistical Computing) with the ‘survival’ package.

Machine Learning

Using random sampling without replacement, all data were divided into five subsets, as shown in Fig. 1. Therefore, 80% of each subset was used for training the ML model, while the remaining 20% was used for testing the ML model. Data preprocessing included the dichotomization of independent variables to between 0 and 1.

FIG. 1.
FIG. 1.

Workflow of training and testing the ML algorithms. Using random sampling without replacement, each sample was divided into five subsets. Therefore, 80% of each subset (white boxes) was used for training various algorithms, and 20% of the subset (gray boxes) was used to test the performance of each ML model.

Supervised ML algorithms included decision tree (DT), NB with Laplace correction, k-NN with k = 5, and ANNs. The network was trained 200 times using different random weights and tested as binary outcomes (SSI or no SSI). To validate the models from the testing data set, the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the models were determined for the predictive efficacy of each ML algorithm. Furthermore, the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC) were plotted. Additionally, AUCs revealed that values ≥ 0.9 were “excellent,” ≥ 0.80 were “good,” ≥ 0.70 were “fair,” and < 0.70 were “poor.”27 All mathematical analyses were done using RapidMiner Studio version 9.0 with an academic license. Moreover, the ROC curves and AUCs were created using the ‘plotROC’ package (R Foundation for Statistical Computing).

Results

The clinical characteristics of the 1471 eligible patients who underwent neurosurgical operations in the study period are listed in Table 1. More than half of the study population were males. The mean age was 45.07 years (SD 21.1), and 10.5% of the patients presented with diabetes mellitus. Almost all patients (95%) received the antibiotic prophylaxis, while 19.9% received preoperative dexamethasone. Additionally, hair shaving occurred in 45.4% of cases. Common diseases in the cohort included tumors, trauma, and vascular conditions, while the most frequently performed procedures were cranial operations.

TABLE 1.

Baseline characteristics of 1471 patients who underwent neurosurgical operations between 2010 and 2017

FactorNo. (%)
Sex
 Male839 (57.0)
 Female632 (43.0)
Age in yrs
 <601079 (73.4)
 ≥60392 (26.6)
Disease
 Tumor698 (47.5)
 Trauma300 (20.4)
 Vascular337 (22.9)
 Congenital49 (3.3)
 Primary infection44 (3.0)
 Other43 (2.9)
Location
 Supratentorial1365 (92.8)
 Infratentorial106 (7.2)
Underlying disease
 Diabetes mellitus154 (10.5)
 Previous infection95 (6.5)
 Coagulation46 (3.1)
Preop hair removal668 (45.4)
Dexamethasone injection293 (19.9)
Antibiotic drug prophylaxis1397 (95.0)
Type of wound
 Clean1320 (89.7)
 Clean-contaminated57 (3.9)
 Contaminated84 (5.7)
 Dirty10 (0.7)
ASA classification
 I16 (1.1)
 II288 (19.6)
 III1015 (69.0)
 IV142 (9.7)
 V10 (0.7)
Type of op
 Emergency751 (51.1)
 Elective720 (48.9)
Procedure
 Craniotomy588 (40.0)
 Craniectomy270 (18.4)
 Shunt179 (12.2)
 Burr hole94 (6.4)
 Ventriculostomy92 (6.3)
 ICP monitoring8 (0.5)
 Transsphenoidal op85 (5.8)
 Cranioplasty15 (1.0)
 Debridement for trauma6 (0.4)
 Debridement for SSI2 (0.1)
 Other125 (8.5)
Intraop ventricular exposure155 (10.5)
Finding insect in op room31 (2.1)
Postop ventriculostomy92 (6.3)
Postop CSF leakage/subgaleal collection68 (4.6)
Postop fever51 (3.5)
Bacteremia51 (3.5)
Urinary tract infection93 (6.3)
Pneumonia102 (6.9)
Reop in same admission (excluding reop due to SSI)85 (5.8)
Postop drain561 (38.1)
SSI67 (4.6)
Type of infection
 Superficial17 (1.2)
 Deep12 (0.8)
 Organ/space38 (2.6)
  Meningitis or ventriculitis30 (2.0)
  Brain abscess6 (0.4)
  Subdural empyema2 (0.1)

ICP = intracranial pressure.

The SSI rate was 4.6%, while the types of SSIs were superficial, deep, or organ/space in 1.2%, 0.8%, and 2.6% of cases, respectively. Among the organ/space SSIs, postoperative brain abscess was found in 0.4% cases, while meningitis or ventriculitis was found in 2.0% of all cases.

Survival Analysis

Time-to-event statistical analysis was possible using the Kaplan-Meier curve shown in Fig. 2A. Mean follow-up time was 313.3 weeks (SD 137.4). Also, the overall median infection-free time was not yet reached in the cohort, while the 1-, 3-, and 12-month infection-free probabilities were 97.4%, 96.4%, and 95.5%, respectively.

FIG. 2.
FIG. 2.

The Kaplan-Meier curve for infection-free neurosurgical operations. A: Kaplan-Meier curve for all cases shows that the overall median infection-free time has not been reached yet. B: Kaplan-Meier curves for infection-free probability in patients with preoperative hair removal or no hair removal. Infection-free probability was significantly lower in the patients with hair removal than in those with no hair removal (p = 0.008, log-rank test). C: Kaplan-Meier curves for postoperative fever or no fever. Infection-free probability was significantly lower in the patients with fever than in those without (p = 0.009, log-rank test). D: Kaplan-Meier curves for postoperative CSF leakage/subgaleal collection or no CSF leakage/subgaleal collection. Infection-free probability was significantly lower in patients with CSF leakage/subgaleal collection than in those without (p < 0.001, log-rank test).

According to the Cox proportional-hazards regression analysis (Table 2), the significant parameters for an increased risk of SSI included preoperative hair removal (HR 1.84, p = 0.01), postoperative fever (HR 1.79, p = 0.02), and postoperative CSF leakage/subgaleal collection (HR 4.60, p < 0.001) on univariate analysis. Using the backward stepwise method, we determined that the significant predictors of SSI in the multivariable analysis were postoperative fever (HR 1.67, p = 0.04) and postoperative CSF leakage/subgaleal collection (HR 4.24, p < 0.001).

TABLE 2.

Cox proportional-hazards regression analysis for SSI

FactorUnivariate Analysis HR (95% CI)p ValueMultivariable Analysis HR (95% CI)p Value
Sex
 MaleReference
 Female0.61 (0.35–1.03)0.06
Age in yrs
 <60Reference
 ≥601.94 (0.96–3.93)0.06
Preop KPS score
 <80Reference
 ≥801.01 (0.98–1.01)0.91
Preop GCS score
 13–15Reference
 9–120.74 (0.34–1.60)0.45
 3–81.14 (0.58–2.24)0.68
Diabetes mellitus*0.74 (0.31–1.73)0.48
Preop coagulopathy*1.34 (0.48–3.72)0.56
Preop dexamethasone*1.38 (0.79–2.40)0.24
History of previous SSI*0.57 (0.31–1.03)0.06
ASA classification
 I–IIReference
 ≥III1.09 (0.63–1.88)0.74
Tumor op*1.13 (0.69–1.86)0.62
Vascular op*0.85 (0.40–1.79)0.67
Shunt op*0.68 (0.36–1.29)0.24
Trauma op*1.14 (0.57–2.27)0.69
Primary infectious op*1.21 (0.61–2.41)0.57
Type of surgical wound
 CleanReference
 Contaminated1.36 (0.69–2.68)0.36
 Clean-contaminated0.70 (0.32–1.52)0.37
 Dirty2.40 (0.93–6.18)0.07
Region of op
 SupratentorialReference
 Infratentorial0.62 (0.23–2.38)0.62
Preop hair removal*1.84 (1.10–3.08)0.011.66 (0.99–2.78)0.054
Antibiotic prophylaxis*1.12 (0.34–3.57)0.85
Insect in op room*0.71 (0.36–1.42)0.34
Emergency op*1.33 (0.79–2.26)0.27
Op time in mins
 <240Reference
 ≥2400.98 (0.54–1.76)0.94
Postop ventriculostomy*,0.57 (0.25–1.27)0.17
Postop CSF/subgaleal collection*,§4.60 (2.08–10.17)<0.0014.24 (1.89–9.49)<0.001
Postop subgaleal drain*,0.99 (0.56–1.76)0.99
Postop fever§
 NoReferenceReference
 Fever1.79 (1.09–2.94)0.021.67 (1.01–2.76)0.04
Bacteremia*,§1.51 (0.89–2.56)0.12
Urinary tract infection*,§0.79 (0.24–2.53)0.70
Pneumonia*,§0.95 (0.37–2.38)0.95
Reop*,§,1.35 (0.80–2.26)0.25
Postop KPS score
 <80Reference
 ≥801.08 (0.67–1.72)0.74

GCS = Glasgow Coma Scale; KPS = Karnofsky Performance Scale.

Data show only “yes group” while reference groups (no group) are hidden.

Backward stepwise method.

In the same procedure.

In the same admission.

Excluded reoperation due to SSI.

For subgroup analysis of the Kaplan-Meier curve in Fig. 2B–D, infection-free probability was significantly lower in patients with hair removal than in those with no hair removal (p = 0.008, log-rank test). Infection-free probability was also significantly lower in patients with postoperative fever in the same admission than in those without (p = 0.009, log-rank test). Moreover, the patients with postoperative CSF leakage/subgaleal collection had a significantly lower infection-free probability than the patients without (p < 0.001, log-rank test).

Machine Learning Algorithms

After training the four algorithms, we used the test data set with each ML algorithm to obtain the performance data shown in Table 3. The NB algorithm had the highest performance with the largest AUC at 76%, sensitivity at 63%, specificity at 87%, PPV at 29%, NPV at 96%, and accuracy at 86%, whereas the other algorithms exhibited lower performance for predicting SSI with the cohort data. The ROC curve and AUC for all data sets are summarized graphically in Fig. 3. Overall, the data suggest that NB is the most effective methodology for predicting SSI in neurosurgical operations.

TABLE 3.

Performance data on ML algorithms

Data Set & AlgorithmSensitivitySpecificityPPVNPVAccuracyAUC
1st data set
 DT0.310.970.410.960.930.64
 NB0.560.800.140.960.790.69
 k-NN0.060.830.010.940.800.53
 ANN0.180.990.600.950.940.59
2nd data set
 DT0.310.950.350.940.900.64
 NB0.630.870.290.960.860.76
 k-NN0.091.01.00.930.930.55
 ANN0.270.980.660.940.930.63
3rd data set
 DT0.250.970.300.960.940.61
 NB0.580.970.500.970.950.71
 k-NN0.160.980.330.960.950.58
 ANN0.250.970.300.960.940.61
4th data set
 DT0.400.990.840.970.960.70
 NB0.530.900.200.970.880.72
 k-NN0.080.990.600.950.950.54
 ANN0.370.990.860.970.960.69
5th data set
 DT0.430.990.870.970.970.71
 NB0.530.930.280.970.910.74
 k-NN0.130.990.690.960.950.57
 ANN0.410.990.820.970.960.71
FIG. 3.
FIG. 3.

The ROC curve and AUC of the five data sets, with the NB algorithm having the largest AUC. First data subset (A), second data subset (B), third data subset (C), fourth data subset (D), and fifth data subset (E).

Discussion

The rate of diagnosed SSIs has a reported range of 1.1%–6.2%. In the present study, SSI was observed at 4.6%, which is similar to rates in previous studies. Moreover, we observed postoperative fever and postoperative CSF leakage/subgaleal collection in the same admission as being associated with an increased rate of SSI. Similarly, Chiang et al. reported that CSF leakage was a significant variable associated with SSI (OR 3.5, 95% CI 1.4–8.5).8 Other independent risk factors for SSI have included meningioma, longer operation time, craniotomy, dural substitute, staples in wound closure, operation performed by a specialist or senior medical officer, surgery conducted for infectious causes, ASA physical status II, and clean-contaminated wound,1,5,8–10,16,25 depending on the cohorts, although these factors were not associated with SSI in the present study.

Cranial surgery without hair removal is a modifiable risk factor that has been discussed in prior studies. Ratanalert et al. recommended not shaving hair for intracranial procedures in nonemergency neurosurgical procedures because shaving significantly increased the risk of SSI compared with not shaving. Further, Adeleye et al. mentioned that cranial surgery without hair removal is safe and does not increase the risk of surgical wound infection.2,20 In our cohort, 45.4% of cases were shaved at the discretion of the attending neurosurgeon. The patients with preoperative hair removal had a significantly increased risk of SSI on univariate analysis; however, the results were limited to interpretation of the nonrandomized data.

ML algorithms such as DT, NB, k-NN, and ANN can provide strong predictive clinical outcomes in neurosurgical practice. The DT is a ML model composed of decision rules based on optimal feature cutoff values that recursively divide independent variables into different groups in order to predict an outcome. Hostettler et al. used DT for predicting outcomes in patients with aneurysmal subarachnoid hemorrhage (SAH). The prediction accuracy of the DT models for survival on day 1 was 75.2% for Hostettler et al.,15 whereas Rau et al. reported that DT achieved an accuracy of 97.9% for the prediction of death in patients with traumatic SAH.21

NB, k-NN, and ANN are the algorithms used in ML for predicting classification. Hale et al. studied the prediction of types of meningioma using several ML algorithms, reporting that ANN had the highest performance for predicting meningioma grade (AUC = 0.8895) compared with other algorithms.14 ANNs have been adopted widely in clinical medicine; however, their use in such settings is subject to translation. Rughani et al. demonstrated that ANNs had significantly higher performance in predicting survival following traumatic brain injury compared with regression models.22 Recently, Hale et al. used ANN to predict 6-month outcomes for pediatric head injury. The results showed profound accuracy (AUC = 0.9462 ± 0.0422).13 Moreover, Shi et al. used an ANN for the prediction of in-hospital mortality after traumatic brain injury surgery. The ANN model achieved high accuracy in 95.15% of cases and a better AUC in 89.14% of cases.26 Conversely, in the present study we found that NB was an effective ML algorithm for predicting SSI, which achieved a good AUC at 78% and good accuracy at 88.5%. The results were likely affected by imbalances between the prevalence of SSI and the lack of SSI. Moreover, this study is the first to harness the utility of ML to predict SSI, meaning that continued study is needed in the future.

Finally, certain limitations in the present study should be acknowledged. Firstly, its retrospective design may have led to bias and an inability to control for confounding factors. Secondly, the imbalance of outcomes was probably the reason why performance failed to reach an excellent level. However, a future aim is to conduct a prospective study to evaluate ML performance for the increase in SSI prevalence in order to correct the imbalance problem.

Conclusions

In summary, in this study we found that NB algorithms have potential as a tool for predicting SSI after a neurosurgical operation with reasonable accuracy, meaning that they could be applied in effectively predicting serious complications. Therefore, close monitoring and allocation of treatment strategies can be informed by ML predictions in general practice.

Acknowledgments

We would like to offer special thanks to Professor Sanguansin Ratanalert for providing beneficial advice concerning manuscript preparation.

Disclosures

The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

Author Contributions

Conception and design: Tunthanathip. Acquisition of data: Tunthanathip, Taweesomboonyat. Analysis and interpretation of data: Tunthanathip. Drafting the article: Tunthanathip, Sakarunchai, Kaewborisutsakul. Critically revising the article: Tunthanathip, Oearsakul. Reviewed submitted version of manuscript: Tunthanathip, Sae-heng, Oearsakul, Sakarunchai, Taweesomboonyat. Approved the final version of the manuscript on behalf of all authors: Tunthanathip. Statistical analysis: Tunthanathip. Administrative/technical/material support: Tunthanathip, Sakarunchai, Kaewborisutsakul. Study supervision: Sae-heng, Oearsakul.

References

  • 1

    Abu Hamdeh SLytsy BRonne-Engström E: Surgical site infections in standard neurosurgery procedures—a study of incidence, impact and potential risk factors. Br J Neurosurg 28:2702752014

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 2

    Adeleye AO: Nonshaved cranial surgery in black Africans: technical report and a medium-term prospective outcome study. Neurosurg Rev 39:4494542016

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 3

    Armañanzas RAlonso-Nanclares LDefelipe-Oroquieta JKastanauskaite Ade Sola RGDefelipe J: Machine learning approach for the outcome prediction of temporal lobe epilepsy surgery. PLoS One 8:e628192013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4

    Badia JMCasey ALPetrosillo NHudson PMMitchell SACrosby C: Impact of surgical site infection on healthcare costs and patient outcomes: a systematic review in six European countries. J Hosp Infect 96:1152017

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5

    Buang SSHaspani MS: Risk factors for neurosurgical site infections after a neurosurgical procedure: a prospective observational study at Hospital Kuala Lumpur. Med J Malaysia 67:3933982012

    • Search Google Scholar
    • Export Citation
  • 6

    Centers for Disease Control and Prevention: CDC/NHSN Surveillance Definitions for Specific Types of Infections. Atlanta: Centers for Disease Control and Prevention2019 (http://www.cdc.gov/nhsn/pdfs/pscmanual/17pscNosInfDef_current.pdf) [Accessed June 10 2019]

    • Export Citation
  • 7

    Centers for Disease Control and Prevention: Surgical Site Infection (SSI) Event. Atlanta: Centers for Disease Control and Prevention2019 (https://www.cdc.gov/nhsn/pdfs/pscmanual/9pscssicurrent.pdf) [Accessed June 10 2019]

    • Export Citation
  • 8

    Chiang HYKamath ASPottinger JMGreenlee JDHoward MA IIICavanaugh JE: Risk factors and outcomes associated with surgical site infections after craniotomy or craniectomy. J Neurosurg 120:5095212014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9

    Durkin MJDicks KVBaker AWMoehring RWChen LFSexton DJ: Postoperative infection in spine surgery: does the month matter? J Neurosurg Spine 23:1281342015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10

    Erman TDemirhindi HGöçer AITuna MIldan FBoyar B: Risk factors for surgical site infections in neurosurgery patients with antibiotic prophylaxis. Surg Neurol 63:1071132005

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11

    Ferroni PZanzotto FMRiondino SScarpato NGuadagni FRoselli M: Breast cancer prognosis using a machine learning approach. Cancers (Basel) 11:E3282019

  • 12

    Habibi ZErtiaei ANikdad MSMirmohseni ASAfarideh MHeidari V: Predicting ventriculoperitoneal shunt infection in children with hydrocephalus using artificial neural network. Childs Nerv Syst 32:214321512016

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 13

    Hale ATStonko DPBrown ALim JVoce DJGannon SR: Machine-learning analysis outperforms conventional statistical models and CT classification systems in predicting 6-month outcomes in pediatric patients sustaining traumatic brain injury. Neurosurg Focus 45(5):E22018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 14

    Hale ATStonko DPWang LStrother MKChambless LB: Machine learning analyses can differentiate meningioma grade by features on magnetic resonance imaging. Neurosurg Focus 45(5):E42018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 15

    Hostettler ICMuroi CRichter JKSchmid JNeidert MCSeule M: Decision tree analysis in subarachnoid hemorrhage: prediction of outcome parameters during the course of aneurysmal subarachnoid hemorrhage using decision tree analysis. J Neurosurg 129:149915102018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 16

    López Pereira PDíaz-Agero Pérez CLópez Fresneña NLas Heras Mosteiro JPalancar Cabrera ARincón Carlavilla ÁL: ‘Epidemiology of surgical site infection in a neurosurgery department’. Br J Neurosurg 31:10152017

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 17

    Memarian NKim SDewar SEngel J JrStaba RJ: Multimodal data and machine learning for surgery outcome prediction in complicated cases of mesial temporal lobe epilepsy. Comput Biol Med 64:67782015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 18

    O’Keeffe ABLawrence TBojanic S: Oxford craniotomy infections database: a cost analysis of craniotomy infection. Br J Neurosurg 26:2652692012

  • 19

    Raj RSiironen JSkrifvars MBHernesniemi JKivisaari R: Predicting outcome in traumatic brain injury: development of a novel computerized tomography classification system (Helsinki computerized tomography score). Neurosurgery 75:6326472014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20

    Ratanalert SSaehaeng SSripairojkul BLiewchanpattana KPhuenpathom N: Nonshaved cranial neurosurgery. Surg Neurol 51:4584631999

  • 21

    Rau CSWu SCChien PCKuo PJChen YCHsieh HY: Prediction of mortality in patients with isolated traumatic subarachnoid hemorrhage using a decision tree classifier: a retrospective analysis based on a trauma registry system. Int J Environ Res Public Health 14:E14202017

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 22

    Rughani AIDumont TMLu ZBongard JHorgan MAPenar PL: Use of an artificial neural network to predict head injury outcome. J Neurosurg 113:5855902010

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 23

    Senders JTStaples PCKarhade AVZaki MMGormley WBBroekman MLD: Machine learning and neurosurgical outcome prediction: a systematic review. World Neurosurg 109:476486 486.e12018

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 24

    Senders JTZaki MMKarhade AVChang BGormley WBBroekman ML: An introduction and overview of machine learning in neurosurgical care. Acta Neurochir (Wien) 160:29382018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 25

    Sherrod BARocque BG: Morbidity associated with 30-day surgical site infection following nonshunt pediatric neurosurgery. J Neurosurg Pediatr 19:4214272017

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 26

    Shi HYHwang SLLee KTLin CL: In-hospital mortality after traumatic brain injury surgery: a nationwide population-based comparison of mortality predictors used in artificial neural network and logistic regression models. J Neurosurg 118:7467522013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 27

    Swets JA: ROC analysis applied to the evaluation of medical imaging techniques. Invest Radiol 14:1091211979

If the inline PDF is not rendering correctly, you can download the PDF file here.

Article Information

Correspondence Thara Tunthanathip: Prince of Songkla University, Hat Yai, Songkhla, Thailand. tsus4@hotmail.com.

INCLUDE WHEN CITING DOI: 10.3171/2019.5.FOCUS19241.

Disclosures The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

© AANS, except where prohibited by US copyright law.

Headings

Figures

  • View in gallery

    Workflow of training and testing the ML algorithms. Using random sampling without replacement, each sample was divided into five subsets. Therefore, 80% of each subset (white boxes) was used for training various algorithms, and 20% of the subset (gray boxes) was used to test the performance of each ML model.

  • View in gallery

    The Kaplan-Meier curve for infection-free neurosurgical operations. A: Kaplan-Meier curve for all cases shows that the overall median infection-free time has not been reached yet. B: Kaplan-Meier curves for infection-free probability in patients with preoperative hair removal or no hair removal. Infection-free probability was significantly lower in the patients with hair removal than in those with no hair removal (p = 0.008, log-rank test). C: Kaplan-Meier curves for postoperative fever or no fever. Infection-free probability was significantly lower in the patients with fever than in those without (p = 0.009, log-rank test). D: Kaplan-Meier curves for postoperative CSF leakage/subgaleal collection or no CSF leakage/subgaleal collection. Infection-free probability was significantly lower in patients with CSF leakage/subgaleal collection than in those without (p < 0.001, log-rank test).

  • View in gallery

    The ROC curve and AUC of the five data sets, with the NB algorithm having the largest AUC. First data subset (A), second data subset (B), third data subset (C), fourth data subset (D), and fifth data subset (E).

References

  • 1

    Abu Hamdeh SLytsy BRonne-Engström E: Surgical site infections in standard neurosurgery procedures—a study of incidence, impact and potential risk factors. Br J Neurosurg 28:2702752014

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 2

    Adeleye AO: Nonshaved cranial surgery in black Africans: technical report and a medium-term prospective outcome study. Neurosurg Rev 39:4494542016

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 3

    Armañanzas RAlonso-Nanclares LDefelipe-Oroquieta JKastanauskaite Ade Sola RGDefelipe J: Machine learning approach for the outcome prediction of temporal lobe epilepsy surgery. PLoS One 8:e628192013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4

    Badia JMCasey ALPetrosillo NHudson PMMitchell SACrosby C: Impact of surgical site infection on healthcare costs and patient outcomes: a systematic review in six European countries. J Hosp Infect 96:1152017

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 5

    Buang SSHaspani MS: Risk factors for neurosurgical site infections after a neurosurgical procedure: a prospective observational study at Hospital Kuala Lumpur. Med J Malaysia 67:3933982012

    • Search Google Scholar
    • Export Citation
  • 6

    Centers for Disease Control and Prevention: CDC/NHSN Surveillance Definitions for Specific Types of Infections. Atlanta: Centers for Disease Control and Prevention2019 (http://www.cdc.gov/nhsn/pdfs/pscmanual/17pscNosInfDef_current.pdf) [Accessed June 10 2019]

    • Export Citation
  • 7

    Centers for Disease Control and Prevention: Surgical Site Infection (SSI) Event. Atlanta: Centers for Disease Control and Prevention2019 (https://www.cdc.gov/nhsn/pdfs/pscmanual/9pscssicurrent.pdf) [Accessed June 10 2019]

    • Export Citation
  • 8

    Chiang HYKamath ASPottinger JMGreenlee JDHoward MA IIICavanaugh JE: Risk factors and outcomes associated with surgical site infections after craniotomy or craniectomy. J Neurosurg 120:5095212014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9

    Durkin MJDicks KVBaker AWMoehring RWChen LFSexton DJ: Postoperative infection in spine surgery: does the month matter? J Neurosurg Spine 23:1281342015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10

    Erman TDemirhindi HGöçer AITuna MIldan FBoyar B: Risk factors for surgical site infections in neurosurgery patients with antibiotic prophylaxis. Surg Neurol 63:1071132005

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11

    Ferroni PZanzotto FMRiondino SScarpato NGuadagni FRoselli M: Breast cancer prognosis using a machine learning approach. Cancers (Basel) 11:E3282019

  • 12

    Habibi ZErtiaei ANikdad MSMirmohseni ASAfarideh MHeidari V: Predicting ventriculoperitoneal shunt infection in children with hydrocephalus using artificial neural network. Childs Nerv Syst 32:214321512016

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 13

    Hale ATStonko DPBrown ALim JVoce DJGannon SR: Machine-learning analysis outperforms conventional statistical models and CT classification systems in predicting 6-month outcomes in pediatric patients sustaining traumatic brain injury. Neurosurg Focus 45(5):E22018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 14

    Hale ATStonko DPWang LStrother MKChambless LB: Machine learning analyses can differentiate meningioma grade by features on magnetic resonance imaging. Neurosurg Focus 45(5):E42018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 15

    Hostettler ICMuroi CRichter JKSchmid JNeidert MCSeule M: Decision tree analysis in subarachnoid hemorrhage: prediction of outcome parameters during the course of aneurysmal subarachnoid hemorrhage using decision tree analysis. J Neurosurg 129:149915102018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 16

    López Pereira PDíaz-Agero Pérez CLópez Fresneña NLas Heras Mosteiro JPalancar Cabrera ARincón Carlavilla ÁL: ‘Epidemiology of surgical site infection in a neurosurgery department’. Br J Neurosurg 31:10152017

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 17

    Memarian NKim SDewar SEngel J JrStaba RJ: Multimodal data and machine learning for surgery outcome prediction in complicated cases of mesial temporal lobe epilepsy. Comput Biol Med 64:67782015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 18

    O’Keeffe ABLawrence TBojanic S: Oxford craniotomy infections database: a cost analysis of craniotomy infection. Br J Neurosurg 26:2652692012

  • 19

    Raj RSiironen JSkrifvars MBHernesniemi JKivisaari R: Predicting outcome in traumatic brain injury: development of a novel computerized tomography classification system (Helsinki computerized tomography score). Neurosurgery 75:6326472014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20

    Ratanalert SSaehaeng SSripairojkul BLiewchanpattana KPhuenpathom N: Nonshaved cranial neurosurgery. Surg Neurol 51:4584631999

  • 21

    Rau CSWu SCChien PCKuo PJChen YCHsieh HY: Prediction of mortality in patients with isolated traumatic subarachnoid hemorrhage using a decision tree classifier: a retrospective analysis based on a trauma registry system. Int J Environ Res Public Health 14:E14202017

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 22

    Rughani AIDumont TMLu ZBongard JHorgan MAPenar PL: Use of an artificial neural network to predict head injury outcome. J Neurosurg 113:5855902010

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 23

    Senders JTStaples PCKarhade AVZaki MMGormley WBBroekman MLD: Machine learning and neurosurgical outcome prediction: a systematic review. World Neurosurg 109:476486 486.e12018

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 24

    Senders JTZaki MMKarhade AVChang BGormley WBBroekman ML: An introduction and overview of machine learning in neurosurgical care. Acta Neurochir (Wien) 160:29382018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 25

    Sherrod BARocque BG: Morbidity associated with 30-day surgical site infection following nonshunt pediatric neurosurgery. J Neurosurg Pediatr 19:4214272017

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 26

    Shi HYHwang SLLee KTLin CL: In-hospital mortality after traumatic brain injury surgery: a nationwide population-based comparison of mortality predictors used in artificial neural network and logistic regression models. J Neurosurg 118:7467522013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 27

    Swets JA: ROC analysis applied to the evaluation of medical imaging techniques. Invest Radiol 14:1091211979

TrendMD

Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 354 354 354
PDF Downloads 158 158 158
EPUB Downloads 0 0 0

PubMed

Google Scholar