Comparison of intracranial injury predictability between machine learning algorithms and the nomogram in pediatric traumatic brain injury

View More View Less
  • 1 Division of Neurosurgery, Department of Surgery, Faculty of Medicine, Prince of Songkla University, Hat Yai;
  • | 2 Department of Computer Science, Faculty of Science, Prince of Songkla University, Hat Yai; and
  • | 3 Department of Biotechnology, Faculty of Science and Technology, Thammasat University (Rangsit Campus), Khlong Luang, Thailand
Free access

OBJECTIVE

The overuse of head CT examinations has been much discussed, especially those for minor traumatic brain injury (TBI). In the disruptive era, machine learning (ML) is one of the prediction tools that has been used and applied in various fields of neurosurgery. The objective of this study was to compare the predictive performance between ML and a nomogram, which is the other prediction tool for intracranial injury following cranial CT in children with TBI.

METHODS

Data from 964 pediatric patients with TBI were randomly divided into a training data set (75%) for hyperparameter tuning and supervised learning from 14 clinical parameters, while the remaining data (25%) were used for validation purposes. Moreover, a nomogram was developed from the training data set with similar parameters. Therefore, models from various ML algorithms and the nomogram were built and deployed via web-based application.

RESULTS

A random forest classifier (RFC) algorithm established the best performance for predicting intracranial injury following cranial CT of the brain. The area under the receiver operating characteristic curve for the performance of RFC algorithms was 0.80, with 0.34 sensitivity, 0.95 specificity, 0.73 positive predictive value, 0.80 negative predictive value, and 0.79 accuracy.

CONCLUSIONS

The ML algorithms, particularly the RFC, indicated relatively excellent predictive performance that would have the ability to support physicians in balancing the overuse of head CT scans and reducing the treatment costs of pediatric TBI in general practice.

ABBREVIATIONS

ANN = artificial neural network; AUC = area under the receiver operating characteristic curve; EDH = epidural hematoma; GCS = Glasgow Coma Scale; LR = logistic regression; ML = machine learning; NB = naive Bayes; NPV = negative predictive value; PPV = positive predictive value; RFC = random forest classifier; SAH = subarachnoid hemorrhage; SDH = subdural hematoma; SVM = support vector machine; TBI = traumatic brain injury.

OBJECTIVE

The overuse of head CT examinations has been much discussed, especially those for minor traumatic brain injury (TBI). In the disruptive era, machine learning (ML) is one of the prediction tools that has been used and applied in various fields of neurosurgery. The objective of this study was to compare the predictive performance between ML and a nomogram, which is the other prediction tool for intracranial injury following cranial CT in children with TBI.

METHODS

Data from 964 pediatric patients with TBI were randomly divided into a training data set (75%) for hyperparameter tuning and supervised learning from 14 clinical parameters, while the remaining data (25%) were used for validation purposes. Moreover, a nomogram was developed from the training data set with similar parameters. Therefore, models from various ML algorithms and the nomogram were built and deployed via web-based application.

RESULTS

A random forest classifier (RFC) algorithm established the best performance for predicting intracranial injury following cranial CT of the brain. The area under the receiver operating characteristic curve for the performance of RFC algorithms was 0.80, with 0.34 sensitivity, 0.95 specificity, 0.73 positive predictive value, 0.80 negative predictive value, and 0.79 accuracy.

CONCLUSIONS

The ML algorithms, particularly the RFC, indicated relatively excellent predictive performance that would have the ability to support physicians in balancing the overuse of head CT scans and reducing the treatment costs of pediatric TBI in general practice.

ABBREVIATIONS

ANN = artificial neural network; AUC = area under the receiver operating characteristic curve; EDH = epidural hematoma; GCS = Glasgow Coma Scale; LR = logistic regression; ML = machine learning; NB = naive Bayes; NPV = negative predictive value; PPV = positive predictive value; RFC = random forest classifier; SAH = subarachnoid hemorrhage; SDH = subdural hematoma; SVM = support vector machine; TBI = traumatic brain injury.

Computed tomography of the brain is the radiological investigation for diagnosing traumatic brain injury (TBI), including in children. However, long-term consequences from radiation exposure, such as brain tumors or leukemia, have been a concern after cranial CT.1,2 Hence, various guidelines, such as the Pediatric Emergency Care Applied Research Network (PECARN) criteria,3 Canadian Assessment of Tomography for Childhood Head Injury (CATCH) rule,4 and Children’s Head Injury Algorithm for the Prediction of Important Clinical Events (CHALICE),5 have been established for selecting cranial CT in pediatric patients with TBI. These guidelines have led to a significantly decreased rate of CT use in children with mild TBI.6

Balancing overexamination with CT and advice to parents is necessary in real-world situations. Artificial intelligence was recently introduced as an aid in various fields, including medicine. Machine learning (ML) algorithms are one of the artificial intellegence technologies that have been used for classification and prediction in various neurosurgery specialties. Tunthanatip et al. used ML algorithms to predict surgical site infection after neurosurgical operations and found that the naive Bayes (NB) algorithm effectively predicted surgical site infection.7 Moreover, Matsuo et al. used ML algorithms to predict the outcome in patients following TBI and found that the random forest algorithm had the best predictive performance.8

Nomograms built from a model of logistic or Cox regression have been used in neurosurgery as diagnostic or prediction tools.9–11 Tunthanatip et al. developed a nomogram for predicting positive findings following CT of the brain in pediatric patients with TBI and reported an area under the receiver operating characteristic (ROC) curve (AUC) of 0.71.11 After conducting a literature review, we found a lack of evidence comparing predictive performance between ML and the nomogram with the same clinical outcome. Therefore, the objective of this study was to compare the predictive performance of ML with the nomogram for intracranial injury after cranial CT in children with TBI.

Methods

Study Design and Population

This is a retrospective cohort study of 964 children with TBI who were younger than 15 years of age who presented with TBI from January 2009 to December 2019. The study population was divided into two cohorts; 900 children composed the first cohort, which was used in a prior study by Tunthanathip et al.,11 and 64 children, who were newly admitted from January to December 2019, were added in this study. Therefore, children who died before arrival (within the first 24 hours following TBI) and those patients who did not undergo CT of the brain were excluded. Although the previous study population was included, the current study focused on comparison predictability between ML algorithms and the nomogram.

The sample size estimation was performed with diagnostic tests.12 Using the AUC reported in a prior study,8 a prevalence of positive results after cranial CT in 111 patients was needed, with a total sample size of at least 222 patients for hypothesis testing. Clinical characteristics, cranial CT findings, and treatment were reviewed from the computer-based medical records after approval from the ethics review committee. The operational definition of various variables was described for review. Pediatric Glasgow Coma Scale (GCS) scores after vital signs stabilized were used for children younger than 2 years of age.13 The severity of TBI was categorized as follows: mild TBI (GCS scores 13–15), moderate TBI (GCS scores 9–12), and severe TBI (GCS scores 3–8).14 Additionally, bradycardia and hypotension were defined by the age of the children.15

The endpoint of the present study was an intracranial injury as observed on CT of the brain; therefore, intracranial injuries were defined as injuries visible on CT, including skull fracture, basilar skull fracture, epidural hematoma (EDH), subdural hematoma (SDH), cerebral contusion, subarachnoid hemorrhage (SAH), intraventricular hemorrhage, and diffuse axonal injury.

Statistical Analysis

The descriptive statistics frequency and percentage were analyzed for categorical variables, and continuous variables were estimated using mean and standard deviation. The chi-square test was used to calculate the difference between the intracranial injury and nonintracranial injury groups after cranial CT; p < 0.05 was accepted as being statistically significant. Statistical analysis was performed using R version 3.6.2 (The R Project).

Machine Learning

After randomly splitting the full data, 720 patients were selected to build the ML model. The remaining 180 patients were combined with the 64 children who presented with TBI in 2019 and used as the unseen data set for testing the performance of ML. All clinical characteristics of the children were used for training the model, as more training variables developed the high-dimensional features and more selection of hyperparameters for the model adjustment.16–18

The supervised ML algorithms were performed for training the model with ten-fold cross-validation as follows: support vector machine (SVM), logistic regression (LR), NB, k-nearest neighbors, decision tree, random forest classifier (RFC), gradient boosting classifier, and artificial neural network (ANN). The ML was performed using Python version 3.8.7 (Python Software Foundation) with the “scikit-learn” package.

Turning of the hyperparameters of each algorithm was performed for the best parameters using the “GridSearchCV” (scikit-learn developers) package. Therefore, the training model of each algorithm with optimized hyperparameters was performed. Accuracy and the AUC curve were compared among algorithms using the training data with ten-fold cross-validation. Hence, the results from training were the model of each algorithm, which were sequentially deployed as a web-based application using Streamlit version 0.74.1 (Streamlit Inc.). Moreover, the web-based ML application was deployed via the website heraku.com.

Nomogram

For comparison proposes, a nomogram was developed using similar variables to train the model of ML algorithms. Using R with the “rms” package, a 2D graphic nomogram was built from the model of binary logistic regression with multivariable analysis. The tool was converted to a web-based nomogram using the “DynNom” package and deployed via shinyapps.io.

Performances of the Clinical Prediction Tools

Comparison performances were made between ML algorithms and the nomogram via the web-based application using an unseen data set. The criteria prediction of the positive intracranial injury used a cutoff value of 0.5 of probability for both the ML and nomogram approaches.

Consequently, a confusion matrix revealed the performance of each method for which the actual values of the outcome were known. In detail, the measures of test performance were determined as follows: sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy. Furthermore, we used the AUC to evaluate the discrimination of the model. The acceptable discrimination would be an AUC of ≥ 0.7, while good and excellent discriminations would be an AUC ≥ 0.8 and ≥ 0.9, respectively.19

Results

Clinical and Radiological Characteristics

A total of 964 pediatric patients were included (mean age 75.5 months [SD 56.04]). Overall, 83% of patients had an admission GCS scores of 13–15, and 9.4% of patients had severe TBI. Differences in baseline characteristics between children with intracranial injury and those with nonintracranial injury are displayed in Table 1. Children with intracranial injury were significantly older (p < 0.001); had been injured in road traffic accidents (p < 0.001); and had more symptoms, including loss of consciousness (p < 0.001), motor weakness (p < 0.001), scalp injury (p < 0.001), bleeding from the nose or ear (p < 0.001), hypotension (p < 0.001), bradycardia (p = 0.007), a low GCS score (p < 0.001), and a more abnormal pupillary reflex (p < 0.001).

TABLE 1.

Demographic data of pediatric patients with TBI grouped by intracranial injury and no intracranial injury

FactorNo Intracranial InjuryIntracranial Injuryp Value
Sex0.23
 M412 (61.7)195 (65.9)
 F256 (38.3)101 (34.1)
Mean age, mos81.43 (56.37)99.25 (56.84)<0.001*
Age group, mos0.59
 ≤2492 (13.8)37 (12.5)
 >24576 (86.2)259 (87.5)
Age group, mos<0.001
 ≤60290 (43.4)90 (30.4)
 >60378 (56.6)206 (69.6)
Mechanism of injury<0.001
 Ground-level fall174 (26.0)39 (13.2)
 Car crash56 (8.4)22 (7.4)
 Motorcycle accident271 (40.6)174 (58.8)
 Pedestrian68 (10.2)35 (11.8)
 Hit by an object24 (3.6)9 (3.0)
 Other15 (2.2)11 (3.7)
 Fall from height60 (9.0)6 (2.0)
Road traffic injury<0.001
 Yes395 (59.1)231 (78.0)
 No273 (40.9)65 (22.0)
Signs & symptoms
 LOC220 (32.9)135 (45.6)<0.001
 Amnesia168 (25.1)80 (27.0)0.53
 Vomiting182 (27.2)67 (22.6)0.13
 Motor weakness4 (0.6)22 (7.4)<0.001
 Scalp injury376 (56.3)211 (71.3)<0.001
 Bleeding from nose/ear2 (0.3)26 (8.8)<0.001
 Hypotension18 (2.7)23 (7.8)<0.001
 Bradycardia3 (0.4)7 (2.4)0.007
 Seizure before cranial CT29 (4.3)17 (5.7)0.34
GCS score at ED<0.001
 13–15618 (92.5)184 (62.2)
 9–1235 (5.2)36 (12.2)
 3–815 (2.2)76 (25.7)
Pupillary light reflex<0.001
 Normal reactivity, both eyes656 (98.2)249 (84.1)
 Fixed, one eye6 (0.9)21 (7.1)
 Fixed, both eyes6 (0.9)26 (8.8)

ED = emergency department; LOC = loss of consciousness. Values represent the number of patients (%) or mean (SD) unless indicated otherwise.

Independent t-test.

Table 2 details the intracranial injury following CT of the brain. The most common TBI was skull fracture, whereas SDH, contusion, and EDH were found in 41.8%, 27.3%, and 26.3% of patients, respectively. Moreover, brainstem contusion was a rare injury, only being found in 0.7% of patients.

TABLE 2.

Intracranial injury following CT of the brain

Value
No. of pts w/ intracranial injury296
Calvaria skull fracture142 (47.9)
 Linear79 (26.7)
 Simple depressed29 (9.8)
 Compound depressed25 (8.4)
 Diastatic15 (5.0)
Basilar skull fracture55 (18.6)
EDH78 (26.3)
SDH124 (41.8)
Contusion81 (27.3)
Brainstem contusion2 (0.7)
SAH60 (20.2)
Intraventricular hemorrhage16 (5.4)

Pts = patients.

Values represent the number of patients (%) unless indicated otherwise.

All clinical characteristics included in the model were in 14 parameters, although some parameters had a p value > 0.05. However, prior studies reported a significant association between these parameters and intracranial injury.20–22 Additionally, we decided to include the parameter of sex for the training model because it is one of the basic characteristics, and, in animal models, females tended to have a protective benefit for pathology after TBI.23

Machine Learning

Figure 1 shows the comparison training performances comprising median accuracy and AUC among various algorithms with supervised learning. As a result, SVM, LR, NB, and RFC had high performances (Table 3).

FIG. 1.
FIG. 1.

Boxplots of comparison performances of various algorithms using training data. A: Accuracy. B: AUC curve. DT = decision tree; GBC = gradient boosting classifier; kNN = k-nearest neighbors.

TABLE 3.

Mean accuracy and AUC of comparison training performances among various algorithms

AlgorithmMean Accuracy (SD)Mean AUC (SD)
SVM0.77 (0.17)0.86 (0.14)
LR0.76 (0.18)0.85 (0.16)
NB0.75 (0.15)0.79 (0.16)
KNN0.68 (0.18)0.63 (0.27)
DT0.70 (0.20)0.60 (0.19)
RFC0.73 (0.16)0.80 (0.16)
GBC0.72 (0.19)0.65 (0.19)
ANN0.77 (0.08)0.82 (0.15)

DT = decision tree; GBC = gradient boosting classifier; KNN = k-nearest neighbors.

Nomogram

Similarly, the 14 parameters were used for developing a nomogram (Fig. 2). To use the nomogram, a straight line is drawn upward from the 14 clinical parameters of the child to the upper points scale for scoring each parameter; thereby, the scores from each parameter can be summed. Then, another straight line is drawn downward from the scale of the total points through the outcome scale to measure the probability of a positive finding (intracranial injury) after cranial CT is performed in an individual.

FIG. 2.
FIG. 2.

A nomogram predicting intracranial injury following cranial CT. ED = emergency department.

For use in general practice, we present the web-based applications for individually predicting TBI in children; these are found at Heroku (Salesforce; https://ct-pedtbi-home.herokuapp.com) (Fig. 3) and shinyapps.io (https://thara.shinyapps.io/ct-pedtbi-nomogram/) (Fig. 4). The code can be found on GitHub (GitHub, Inc.; https://github.com/Thara-PSU/ct-pedtbi-psu).

FIG. 3.
FIG. 3.

Screenshots showing web-based applications of ML used (via a mobile phone or laptop) for predicting intracranial injury following cranial CT.

FIG. 4.
FIG. 4.

Screenshot of a web-based nomogram used for predicting intracranial injury following cranial CT.

Comparison Performances of the Clinical Prediction Tools

For testing models of ML and the nomogram, the AUC of almost all algorithms decreased except for the the RFC algorithm. RFC still had an AUC at an excellent level of 0.80 (95% CI 0.72–0.87), while performances of SVM and NB algorithms dropped to < 0.7. Also, AUC of the nomogram was acceptable at 0.74 (95% CI 0.66–0.81) (Fig. 5. Additionally, Table 4 reveals results of the test of performances and all prediction tools had high specificity and low false positives, which helps to confirm and support in negative results in children who are less likely to have intracranial injury after TBI.

FIG. 5.
FIG. 5.

ROC curves of algorithms predicting intracranial injury.

TABLE 4.

Performances of various prediction tools predicting intracranial injury using a testing data set

AlgorithmSensitivitySpecificityPPVNPVAccuracyAUC
SVM0.33 (0.21–0.44)0.96 (0.93–0.98)0.75 (0.58–0.91)0.80 (0.74–0.85)0.79 (0.74–0.84)0.65 (0.56–0.73)
LR0.48 (0.37–0.59)0.92 (0.88–0.96)0.75 (0.63–0.86)0.80 (0.74–0.85)0.79 (0.74–0.84)0.77 (0.69–0.85)
NB0.34 (0.23–0.46)0.97 (0.95–0.99)0.84 (0.70–0.98)0.80 (0.75–0.86)0.81 (0.76–0.86)0.66 (0.57–0.75)
RFC0.34 (0.23–0.46)0.95 (0.92–0.98)0.73 (0.57–0.89)0.80 (0.75–0.85)0.79 (0.74–0.84)0.80 (0.72–0.87)
ANN0.41 (0.29–0.53)0.95 (0.92–0.98)0.76 (0.62–0.90)0.82 (0.76–0.87)0.81 (0.76–0.86)0.68 (0.59–0.76)
Nomogram0.34 (0.20–0.43)0.80 (0.74–0.86)0.36 (0.23–0.49)0.76 (0.70–0.82)0.67 (0.61–0.73)0.74 (0.66–0.81)

Discussion

The results of the current study show that RFC, LR, and the nomogram have an acceptable AUC for predicting intracranial injury in pediatric TBI; however, the ML algorithms had better performance than the nomogram in all measurements. Additionally, several algorithms demonstrated the overfitting problem from dropped performances when their model was tested with unseen data. However, RFC had the best AUC for intracranial injury prediction with acceptable performance. Adil et al. applied ML to predict the individual treatment outcome in TBI and found that the AUC of the RFC reached to 0.885, while other prior studies found that the prognostic model developed from ML with a substantial level of accuracy for prediction in patients with TBI.24,25 Gravesteijn et al. explained those results from low-dimensional features for supervised learning, and validation within a new population should be performed.16

Additionally, these prediction tools differed from prior rules.4–6 Previous head CT criteria were specific to children with a GCS score of 14 or higher, while the prediction tools of the present study covered pediatric TBI with all levels of severity. As a result, characteristics of the clinical prediction tools in the present study were of low sensitivity but high accuracy. Several situations interfered with the prediction of the tools. The lower accuracy of the pediatric GCS score in preverbal children leads to an overestimated GCS score, giving the false-negative results found in the present study. Borgialli et al. compared the accuracy of the pediatric GCS score in children younger than 2 years of age with the standard GCS for those aged 2 years or older.26 They found that interrater agreement of the pediatric GCS score was lower than the standard GCS score because that scale was not familiar in practice. Moreover, no clearly cutoff value of preverbal age is appropriate to use the pediatric GCS in current practice. Prior studies from Kirkham et al. and Reilly et al. used the pediatric GCS to score children younger than 5 years of age,27,28 whereas others used a preverbal cuttoff age of 2 years.12,29

For implication in general practice, the prediction performances of ML algorithms were high specificity, PPV, and NPV. The ML algorithms, particularly the RFC algorithm, were useful in balancing overexamination with CT and reducing treatment costs in children who were less likely to develop intracranial injury after TBI.

Some limitations should be declared. First, ML algorithms need to learn from a large data source. The more pediatric patients with TBI, the more learning, which leads to better predictive ability of the models. Collaborations with multiple centers might be conducted in the future for enhancing the performance of ML algorithms or for initiating steps of new algorithm development; furthermore, those will get larger unseen data for external validation. Second, the retrospective study design may have some bias for estimating outcome; a prospective cohort study should be conducted to resolve this limitation with more reliable data. However, to our knowledge, we present the first real-time prediction tools to calculate the probability of intracranial injury in individual children with TBI. The dynamic tools are easily available and user-friendly in general practice where both ML and nomogram applications, especially in a limited-resource setting, can be used to save costs and resources when considering investigations in patients who are less likely to have TBI. Moreover, these ideas can be applied in other fields in neurosurgery, such as neurooncology, for individual prognostication or treatment decision-making in modern-day precision medicine.

Conclusions

ML algorithms, particularly the RFC, might be useful in limited-resource health settings because they support physicians in balancing the overuse of head CT, and they reduce treatment costs of pediatric TBI.

Acknowledgments

We offer our special thanks to Associate Professor Paramee Thongsuksai for their advice about manuscript preparation.

Targeted Research Gants, Faculty of Medicine, Prince of Songkla University (grant no. 62-065-10-1).

Disclosures

The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

Author Contributions

Conception and design: Tunthanathip, Phuenpathom. Acquisition of data: Tunthanathip. Analysis and interpretation of data: Tunthanathip, Duangsuwan, Wattanakitrungroj, Tongman. Drafting the article: Tunthanathip. Critically revising the article: Tunthanathip. Reviewed submitted version of manuscript: Tunthanathip, Phuenpathom. Approved the final version of the manuscript on behalf of all authors: Tunthanathip. Statistical analysis: Tunthanathip, Duangsuwan, Wattanakitrungroj, Tongman. Administrative/technical/material support: Tunthanathip, Duangsuwan, Wattanakitrungroj, Tongman.

References

  • 1

    Chodick G, Kim KP, Shwarz M, Horev G, Shalev V, Ron E. Radiation risks from pediatric computed tomography scanning. Pediatr Endocrinol Rev. 2009;7(2):2936.

    • Search Google Scholar
    • Export Citation
  • 2

    Schulze-Rath R, Hammer GP, Blettner M. Are pre- or postnatal diagnostic X-rays a risk factor for childhood cancer? A systematic review. Radiat Environ Biophys. 2008;47(3):301312.

    • Search Google Scholar
    • Export Citation
  • 3

    Niele N, van Houten M, Tromp E, van Goudoever JB, Plötz FB. Application of PECARN rules would significantly decrease CT rates in a Dutch cohort of children with minor traumatic head injuries. Eur J Pediatr. 2020;179(10):15971602.

    • Search Google Scholar
    • Export Citation
  • 4

    Kuppermann N, Holmes JF, Dayan PS, Hoyle JD Jr, Atabaki SM, Holubkov R, et al. Identification of children at very low risk of clinically-important brain injuries after head trauma: a prospective cohort study. Lancet. 2009;374(9696):11601170.

    • Search Google Scholar
    • Export Citation
  • 5

    Osmond MH, Klassen TP, Wells GA, Correll R, Jarvis A, Joubert G, et al. CATCH: a clinical decision rule for the use of computed tomography in children with minor head injury. CMAJ. 2010;182(4):341348.

    • Search Google Scholar
    • Export Citation
  • 6

    Dunning J, Daly JP, Lomas JP, Lecky F, Batchelor J, Mackway-Jones K. Derivation of the children’s head injury algorithm for the prediction of important clinical events decision rule for head injury in children. Arch Dis Child. 2006;91(11):885891.

    • Search Google Scholar
    • Export Citation
  • 7

    Tunthanathip T, Sae-Heng S, Oearsakul T, Sakarunchai I, Kaewborisutsakul A, Taweesomboonyat C. Machine learning applications for the prediction of surgical site infection in neurological operations. Neurosurg Focus. 2019;47(2):E7.

    • Search Google Scholar
    • Export Citation
  • 8

    Matsuo K, Aihara H, Nakai T, Morishita A, Tohma Y, Kohmura E. Machine learning to predict in-hospital morbidity and mortality after traumatic brain injury. J Neurotrauma. 2020;37(1):202210.

    • Search Google Scholar
    • Export Citation
  • 9

    Tunthanathip T, Ratanalert S, Sae-Heng S, Oearsakul T, Sakaruncchai I, Kaewborisutsakul A, et al. Prognostic factors and nomogram predicting survival in diffuse astrocytoma. J Neurosci Rural Pract. 2020;11(1):135143.

    • Search Google Scholar
    • Export Citation
  • 10

    Taweesomboonyat T, Kaewborisutsakul A, Tunthanathip T, Saeheng S, Oearsakul T. Necessity of in-hospital neurological observation for mild traumatic brain injury patients with negative computed tomography brain scans. J Health Sci Med Res. 2020;38:267274.

    • Search Google Scholar
    • Export Citation
  • 11

    Tunthanathip T, Duangsuwan J, Wattanakitrungroj N, Tongman S, Phuenpathom N. Clinical nomogram predicting intracranial injury in pediatric traumatic brain injury. J Pediatr Neurosci. 2020;15(4):409415.

    • Search Google Scholar
    • Export Citation
  • 12

    Negida A, Fahim NK, Negida Y. Sample size calculation guide - part 4: how to calculate the sample size for a diagnostic test accuracy study based on sensitivity, specificity, and the area under the ROC curve. Adv J Emerg Med. 2019;3(3):e33.

    • Search Google Scholar
    • Export Citation
  • 13

    James HE. Neurologic evaluation and support in the child with an acute brain insult. Pediatr Ann. 1986;15(1):1622.

  • 14

    Fulkerson DH, White IK, Rees JM, Baumanis MM, Smith JL, Ackerman LL, et al. Analysis of long-term (median 10.5 years) outcomes in children presenting with traumatic brain injury and an initial Glasgow Coma Scale score of 3 or 4. J Neurosurg Pediatr. 2015;16(4):410419.

    • Search Google Scholar
    • Export Citation
  • 15

    Tunthanathip T, Phuenpathom N. Impact of road traffic injury to pediatric traumatic brain injury in southern Thailand. J Neurosci Rural Pract. 2017;8(4):601608.

    • Search Google Scholar
    • Export Citation
  • 16

    Gravesteijn BY, Nieboer D, Ercole A, Lingsma HF, Nelson D, van Calster B, Steyerberg EW. Machine learning algorithms performed no better than regression models for prognostication in traumatic brain injury. J Clin Epidemiol. 2020;122:95107.

    • Search Google Scholar
    • Export Citation
  • 17

    Beam AL, Kohane IS. Big data and machine learning in health care. JAMA. 2018;319(13):13171318.

  • 18

    Rajkomar A, Oren E, Chen K, Dai AM, Hajaj N, Hardt M, et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Med. 2018;1:18.

    • Search Google Scholar
    • Export Citation
  • 19

    Marques A, Almeida S, Carvalho J, Cruz J, Oliveira A, Jácome C. Reliability, validity, and ability to identify fall status of the balance evaluation systems test, mini-balance evaluation systems test, and brief-balance evaluation systems test in older people living in the community. Arch Phys Med Rehabil. 2016;97(12):21662173.e1.

    • Search Google Scholar
    • Export Citation
  • 20

    Borland ML, Dalziel SR, Phillips N, Dalton S, Lyttle MD, Bressan S, et al. Vomiting with head trauma and risk of traumatic brain injury. Pediatrics. 2018;141(4):e20173123.

    • Search Google Scholar
    • Export Citation
  • 21

    Badawy MK, Dayan PS, Tunik MG, Nadel FM, Lillis KA, Miskin M, et al. Prevalence of brain injuries and recurrence of seizures in children with posttraumatic seizures. Acad Emerg Med. 2017;24(5):595605.

    • Search Google Scholar
    • Export Citation
  • 22

    Astrand R, Rosenlund C, Undén J. Scandinavian guidelines for initial management of minor and moderate head trauma in children. BMC Med. 2016;14:33.

    • Search Google Scholar
    • Export Citation
  • 23

    Rubin TG, Lipton ML. Sex differences in animal models of traumatic brain injury. J Exp Neurosci. 2019;13:1179069519844020.

  • 24

    Adil SM, Elahi C, Gramer R, et al. Predicting the individual treatment effect of neurosurgery for patients with traumatic brain injury in the low-resource setting: a machine learning approach in Uganda. J Neurotrauma. 2021;38(7):928939.

    • Search Google Scholar
    • Export Citation
  • 25

    Hernandes Rocha TA, Elahi C, Cristina da Silva N, Sakita FM, Fuller A, Mmbaga BT, et al. A traumatic brain injury prognostic model to support in-hospital triage in a low-income country: a machine learning-based approach. J Neurosurg. 2019;132(6):19611969.

    • Search Google Scholar
    • Export Citation
  • 26

    Borgialli DA, Mahajan P, Hoyle JD Jr, Powell EC, Nadel FM, Tunik MG, et al. Performance of the Pediatric Glasgow Coma Scale score in the evaluation of children with blunt head trauma. Acad Emerg Med. 2016;23(8):878884.

    • Search Google Scholar
    • Export Citation
  • 27

    Kirkham FJ, Newton CR, Whitehouse W. Paediatric coma scales. Dev Med Child Neurol. 2008;50(4):267274.

  • 28

    Reilly PL, Simpson DA, Sprod R, Thomas L. Assessing the conscious level in infants and young children: a paediatric version of the Glasgow Coma Scale. Childs Nerv Syst. 1988;4(1):3033.

    • Search Google Scholar
    • Export Citation
  • 29

    Holmes JF, Palchak MJ, MacFarlane T, Kuppermann N. Performance of the pediatric Glasgow Coma Scale in children with blunt head trauma. Acad Emerg Med. 2005;12(9):814819.

    • Search Google Scholar
    • Export Citation
  • View in gallery

    Boxplots of comparison performances of various algorithms using training data. A: Accuracy. B: AUC curve. DT = decision tree; GBC = gradient boosting classifier; kNN = k-nearest neighbors.

  • View in gallery

    A nomogram predicting intracranial injury following cranial CT. ED = emergency department.

  • View in gallery

    Screenshots showing web-based applications of ML used (via a mobile phone or laptop) for predicting intracranial injury following cranial CT.

  • View in gallery

    Screenshot of a web-based nomogram used for predicting intracranial injury following cranial CT.

  • View in gallery

    ROC curves of algorithms predicting intracranial injury.

  • 1

    Chodick G, Kim KP, Shwarz M, Horev G, Shalev V, Ron E. Radiation risks from pediatric computed tomography scanning. Pediatr Endocrinol Rev. 2009;7(2):2936.

    • Search Google Scholar
    • Export Citation
  • 2

    Schulze-Rath R, Hammer GP, Blettner M. Are pre- or postnatal diagnostic X-rays a risk factor for childhood cancer? A systematic review. Radiat Environ Biophys. 2008;47(3):301312.

    • Search Google Scholar
    • Export Citation
  • 3

    Niele N, van Houten M, Tromp E, van Goudoever JB, Plötz FB. Application of PECARN rules would significantly decrease CT rates in a Dutch cohort of children with minor traumatic head injuries. Eur J Pediatr. 2020;179(10):15971602.

    • Search Google Scholar
    • Export Citation
  • 4

    Kuppermann N, Holmes JF, Dayan PS, Hoyle JD Jr, Atabaki SM, Holubkov R, et al. Identification of children at very low risk of clinically-important brain injuries after head trauma: a prospective cohort study. Lancet. 2009;374(9696):11601170.

    • Search Google Scholar
    • Export Citation
  • 5

    Osmond MH, Klassen TP, Wells GA, Correll R, Jarvis A, Joubert G, et al. CATCH: a clinical decision rule for the use of computed tomography in children with minor head injury. CMAJ. 2010;182(4):341348.

    • Search Google Scholar
    • Export Citation
  • 6

    Dunning J, Daly JP, Lomas JP, Lecky F, Batchelor J, Mackway-Jones K. Derivation of the children’s head injury algorithm for the prediction of important clinical events decision rule for head injury in children. Arch Dis Child. 2006;91(11):885891.

    • Search Google Scholar
    • Export Citation
  • 7

    Tunthanathip T, Sae-Heng S, Oearsakul T, Sakarunchai I, Kaewborisutsakul A, Taweesomboonyat C. Machine learning applications for the prediction of surgical site infection in neurological operations. Neurosurg Focus. 2019;47(2):E7.

    • Search Google Scholar
    • Export Citation
  • 8

    Matsuo K, Aihara H, Nakai T, Morishita A, Tohma Y, Kohmura E. Machine learning to predict in-hospital morbidity and mortality after traumatic brain injury. J Neurotrauma. 2020;37(1):202210.

    • Search Google Scholar
    • Export Citation
  • 9

    Tunthanathip T, Ratanalert S, Sae-Heng S, Oearsakul T, Sakaruncchai I, Kaewborisutsakul A, et al. Prognostic factors and nomogram predicting survival in diffuse astrocytoma. J Neurosci Rural Pract. 2020;11(1):135143.

    • Search Google Scholar
    • Export Citation
  • 10

    Taweesomboonyat T, Kaewborisutsakul A, Tunthanathip T, Saeheng S, Oearsakul T. Necessity of in-hospital neurological observation for mild traumatic brain injury patients with negative computed tomography brain scans. J Health Sci Med Res. 2020;38:267274.

    • Search Google Scholar
    • Export Citation
  • 11

    Tunthanathip T, Duangsuwan J, Wattanakitrungroj N, Tongman S, Phuenpathom N. Clinical nomogram predicting intracranial injury in pediatric traumatic brain injury. J Pediatr Neurosci. 2020;15(4):409415.

    • Search Google Scholar
    • Export Citation
  • 12

    Negida A, Fahim NK, Negida Y. Sample size calculation guide - part 4: how to calculate the sample size for a diagnostic test accuracy study based on sensitivity, specificity, and the area under the ROC curve. Adv J Emerg Med. 2019;3(3):e33.

    • Search Google Scholar
    • Export Citation
  • 13

    James HE. Neurologic evaluation and support in the child with an acute brain insult. Pediatr Ann. 1986;15(1):1622.

  • 14

    Fulkerson DH, White IK, Rees JM, Baumanis MM, Smith JL, Ackerman LL, et al. Analysis of long-term (median 10.5 years) outcomes in children presenting with traumatic brain injury and an initial Glasgow Coma Scale score of 3 or 4. J Neurosurg Pediatr. 2015;16(4):410419.

    • Search Google Scholar
    • Export Citation
  • 15

    Tunthanathip T, Phuenpathom N. Impact of road traffic injury to pediatric traumatic brain injury in southern Thailand. J Neurosci Rural Pract. 2017;8(4):601608.

    • Search Google Scholar
    • Export Citation
  • 16

    Gravesteijn BY, Nieboer D, Ercole A, Lingsma HF, Nelson D, van Calster B, Steyerberg EW. Machine learning algorithms performed no better than regression models for prognostication in traumatic brain injury. J Clin Epidemiol. 2020;122:95107.

    • Search Google Scholar
    • Export Citation
  • 17

    Beam AL, Kohane IS. Big data and machine learning in health care. JAMA. 2018;319(13):13171318.

  • 18

    Rajkomar A, Oren E, Chen K, Dai AM, Hajaj N, Hardt M, et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Med. 2018;1:18.

    • Search Google Scholar
    • Export Citation
  • 19

    Marques A, Almeida S, Carvalho J, Cruz J, Oliveira A, Jácome C. Reliability, validity, and ability to identify fall status of the balance evaluation systems test, mini-balance evaluation systems test, and brief-balance evaluation systems test in older people living in the community. Arch Phys Med Rehabil. 2016;97(12):21662173.e1.

    • Search Google Scholar
    • Export Citation
  • 20

    Borland ML, Dalziel SR, Phillips N, Dalton S, Lyttle MD, Bressan S, et al. Vomiting with head trauma and risk of traumatic brain injury. Pediatrics. 2018;141(4):e20173123.

    • Search Google Scholar
    • Export Citation
  • 21

    Badawy MK, Dayan PS, Tunik MG, Nadel FM, Lillis KA, Miskin M, et al. Prevalence of brain injuries and recurrence of seizures in children with posttraumatic seizures. Acad Emerg Med. 2017;24(5):595605.

    • Search Google Scholar
    • Export Citation
  • 22

    Astrand R, Rosenlund C, Undén J. Scandinavian guidelines for initial management of minor and moderate head trauma in children. BMC Med. 2016;14:33.

    • Search Google Scholar
    • Export Citation
  • 23

    Rubin TG, Lipton ML. Sex differences in animal models of traumatic brain injury. J Exp Neurosci. 2019;13:1179069519844020.

  • 24

    Adil SM, Elahi C, Gramer R, et al. Predicting the individual treatment effect of neurosurgery for patients with traumatic brain injury in the low-resource setting: a machine learning approach in Uganda. J Neurotrauma. 2021;38(7):928939.

    • Search Google Scholar
    • Export Citation
  • 25

    Hernandes Rocha TA, Elahi C, Cristina da Silva N, Sakita FM, Fuller A, Mmbaga BT, et al. A traumatic brain injury prognostic model to support in-hospital triage in a low-income country: a machine learning-based approach. J Neurosurg. 2019;132(6):19611969.

    • Search Google Scholar
    • Export Citation
  • 26

    Borgialli DA, Mahajan P, Hoyle JD Jr, Powell EC, Nadel FM, Tunik MG, et al. Performance of the Pediatric Glasgow Coma Scale score in the evaluation of children with blunt head trauma. Acad Emerg Med. 2016;23(8):878884.

    • Search Google Scholar
    • Export Citation
  • 27

    Kirkham FJ, Newton CR, Whitehouse W. Paediatric coma scales. Dev Med Child Neurol. 2008;50(4):267274.

  • 28

    Reilly PL, Simpson DA, Sprod R, Thomas L. Assessing the conscious level in infants and young children: a paediatric version of the Glasgow Coma Scale. Childs Nerv Syst. 1988;4(1):3033.

    • Search Google Scholar
    • Export Citation
  • 29

    Holmes JF, Palchak MJ, MacFarlane T, Kuppermann N. Performance of the pediatric Glasgow Coma Scale in children with blunt head trauma. Acad Emerg Med. 2005;12(9):814819.

    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 253 253 253
PDF Downloads 227 227 227
EPUB Downloads 0 0 0