Machine learning–based preoperative predictive analytics for lumbar spinal stenosis

Free access

OBJECTIVE

Patient-reported outcome measures (PROMs) following decompression surgery for lumbar spinal stenosis (LSS) demonstrate considerable heterogeneity. Individualized prediction tools can provide valuable insights for shared decision-making. The authors aim to evaluate the feasibility of predicting short- and long-term PROMs, reoperations, and perioperative parameters by machine learning (ML) methods.

METHODS

Data were derived from a prospective registry. All patients had undergone single- or multilevel mini-open facet-sparing decompression for LSS. The prediction models were trained using various ML-based algorithms to predict the endpoints of interest. Models were selected by area under the receiver operating characteristic curve (AUC). The endpoints were dichotomized by minimum clinically important difference (MCID) and included 6-week and 12-month numeric rating scales for back pain (NRS-BP) and leg pain (NRS-LP) severity and the Oswestry Disability Index (ODI), as well as prolonged surgery (> 45 minutes), extended length of hospital stay (> 28 hours), and reoperations.

RESULTS

A total of 635 patients were included. The average age was 62 ± 10 years, and 333 patients (52%) were male. At 6 weeks, MCID was seen in 63%, 76%, and 61% of patients for ODI, NRS-LP, and NRS-BP, respectively. At internal validation, the models predicted MCID in these variables with accuracies of 69%, 76%, and 85%, and with AUCs of 0.75, 0.79, and 0.92. At 12 months, 66%, 63%, and 51% of patients reported MCID; the observed accuracies were 62%, 74%, and 66%, with AUCs of 0.68, 0.72, and 0.79. Reoperations occurred in 60 patients (9.5%), of which 27 (4.3%) occurred at the index level. Overall and index-level reoperations were predicted with 69% and 63% accuracy, respectively, and with AUCs of 0.66 and 0.61. In 15%, a length of surgery greater than 45 minutes was observed and predicted with 78% accuracy and AUC of 0.54. Only 15% of patients were admitted to the hospital for longer than 28 hours. The developed ML-based model enabled prediction of extended hospital stay with an accuracy of 77% and AUC of 0.58.

CONCLUSIONS

Preoperative prediction of a range of clinically relevant endpoints in decompression surgery for LSS using ML is feasible, and may enable enhanced informed patient consent and personalized shared decision-making. Access to individualized preoperative predictive analytics for outcome and treatment risks may represent a further step in the evolution of surgical care for patients with LSS.

ABBREVIATIONS ASA = American Society of Anesthesiologists; AUC = area under the receiver operating characteristic curve; GLM = generalized linear model; LSS = lumbar spinal stenosis; MCID = minimum clinically important difference; ML = machine learning; NPV = negative predictive value; NRS = numeric rating scale; NRS-BP = NRS for back pain; NRS-LP = NRS for leg pain; ODI = Oswestry Disability Index; PPV = positive predictive value; PROMs = patient-reported outcome measures; XGBoost = extreme gradient boosting.

OBJECTIVE

Patient-reported outcome measures (PROMs) following decompression surgery for lumbar spinal stenosis (LSS) demonstrate considerable heterogeneity. Individualized prediction tools can provide valuable insights for shared decision-making. The authors aim to evaluate the feasibility of predicting short- and long-term PROMs, reoperations, and perioperative parameters by machine learning (ML) methods.

METHODS

Data were derived from a prospective registry. All patients had undergone single- or multilevel mini-open facet-sparing decompression for LSS. The prediction models were trained using various ML-based algorithms to predict the endpoints of interest. Models were selected by area under the receiver operating characteristic curve (AUC). The endpoints were dichotomized by minimum clinically important difference (MCID) and included 6-week and 12-month numeric rating scales for back pain (NRS-BP) and leg pain (NRS-LP) severity and the Oswestry Disability Index (ODI), as well as prolonged surgery (> 45 minutes), extended length of hospital stay (> 28 hours), and reoperations.

RESULTS

A total of 635 patients were included. The average age was 62 ± 10 years, and 333 patients (52%) were male. At 6 weeks, MCID was seen in 63%, 76%, and 61% of patients for ODI, NRS-LP, and NRS-BP, respectively. At internal validation, the models predicted MCID in these variables with accuracies of 69%, 76%, and 85%, and with AUCs of 0.75, 0.79, and 0.92. At 12 months, 66%, 63%, and 51% of patients reported MCID; the observed accuracies were 62%, 74%, and 66%, with AUCs of 0.68, 0.72, and 0.79. Reoperations occurred in 60 patients (9.5%), of which 27 (4.3%) occurred at the index level. Overall and index-level reoperations were predicted with 69% and 63% accuracy, respectively, and with AUCs of 0.66 and 0.61. In 15%, a length of surgery greater than 45 minutes was observed and predicted with 78% accuracy and AUC of 0.54. Only 15% of patients were admitted to the hospital for longer than 28 hours. The developed ML-based model enabled prediction of extended hospital stay with an accuracy of 77% and AUC of 0.58.

CONCLUSIONS

Preoperative prediction of a range of clinically relevant endpoints in decompression surgery for LSS using ML is feasible, and may enable enhanced informed patient consent and personalized shared decision-making. Access to individualized preoperative predictive analytics for outcome and treatment risks may represent a further step in the evolution of surgical care for patients with LSS.

With mounting life expectancy, the incidence of symptomatic lumbar spinal stenosis (LSS) has been steadily increasing.27 Although conservative treatment is recommended and effective in the majority of cases, many studies suggest an advantage of decompression surgery over nonsurgical treatments for selected patients, and many improvements in surgical techniques have been made over the past decades.5,21,22 However, indications for and outcomes after lumbar decompression surgery remain heterogeneous, particularly in those patients who present with concomitant low-grade spondylolisthesis.12,14,15,21–24

The combination of hard-to-predict outcomes and a strong disease-linked psychological component requires in-depth preoperative counseling.4 A range of factors influencing the outcome of surgery for LSS have been identified.3,5–7,11,13,16,19,33 However, it is not always feasible to integrate many single patient factors into one communicable risk for poor outcome. Prediction models represent an interface between these important and complex factors, and produce an output that can be benchmarked and is interpretable and clinically valuable for both patients and surgeons.38 Recently, interest has shifted toward machine learning (ML) algorithms in predictive modeling.10,20,29 While some ML algorithms have been around for decades, they have only recently gained major interest for predictive analytics in medicine.31,32 This increase in the use of ML is attributable to the advent of the “big data” era, but also to the development of new algorithms and steadily improving computational power.25 Various ML techniques such as gradient-boosting machines, decision trees, random forests, and neural networks have already proven an increased predictive ability compared to more traditional statistical modeling techniques in neurosurgical patient care.1,31,34,37

The hope for these new techniques is that they will enable surgeons to accurately predict adverse events and outcomes that were previously near unpredictable, such as reoperations or a positive outcome, in order to identify those patients who will benefit most from surgical care. In addition, such tools may aid patients in making the best-informed decision for themselves. Therefore, the purpose of this study was to investigate whether comprehensive ML-based preoperative predictive analytics are feasible and robust in predicting a range of clinically relevant endpoints in patients undergoing decompression surgery for LSS.

Methods

Patient Population

In a retrospective analysis based on data from a prospective registry, we identified all patients who had undergone decompression surgery for LSS between November 2013 and November 2018 at a single Dutch spine center (Bergman Clinics). All patients underwent single- or multilevel decompression by the senior author (M.L.S.). Adult patients with ≥ 80% baseline data completion were included in this study. In patients with concomitant low-grade spondylolisthesis, the decision of whether to add a fusion procedure to decompression alone was based upon a validated decision-making protocol.36 This study was devised and reported according to the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement.8 All patients included in the registry provided written informed consent. The prospective registry was authorized by the local IRB (Medical Research Ethics Committees United), and this study was performed in accordance with the 2013 Declaration of Helsinki.

Surgical Technique

While the patient was in the knee-elbow position, a 50-mm midline incision was made. Via a muscle-splitting approach, a mini-open retractor was inserted. The interspinous ligament was cut, and the spinous process was partially resected.36 A bilateral partial hemilaminectomy was then performed. The interspinous ligament was deliberately resected to contralaterally undercut the hypertrophic ligamentum flavum and osteophytes. Whenever possible, the facet joints were left untouched to preserve biomechanical integrity. Only if necessary, their hypertrophic medial part was partially resected. The lateral recesses and foramina were further opened until the nerve roots appeared to be fully released. Discectomy was performed only in cases of significant nerve root compression by a bulging disc at the index level.

Data Collection

Clinical and radiological baseline data were obtained at the first outpatient visit to the treating surgeon. Patients underwent MRI and a full clinical workup. The 15 collected variables consisted of baseline patient-reported outcome measures (PROMs), as well as sex, age, height (cm), weight (kg), BMI, smoking status (active/inactive), regular alcohol intake (yes/no), American Society of Anesthesiologists (ASA) grade, prior surgery at the index level, single- or multilevel decompression, index levels, presence of concomitant spondylolisthesis, and the predominant presenting symptom (neurogenic claudication or [mono]radiculopathy).

Outcome Measures

Patient-Reported Outcome

For PROM measurement, patients completed a standardized questionnaire including numeric rating scales (NRSs) for back pain (NRS-BP) and leg pain (NRS-LP) severity, ranging from 0 to 10, and a validated Dutch version of the Oswestry Disability Index (ODI) to capture functional disability, ranging from 0 to 100, with higher values representing increasing severity. All PROM data were collected using a validated web-based PROM assessment tool at baseline, 6 weeks, and 12 months after surgery.30 We defined clinical success as achievement of the minimum clinically important difference (MCID) threshold set by Ostelo et al. at the 6-week or 12-month postoperative follow-up.26 Thus, an improvement from baseline of ≥ 30% represented clinical success in functional disability (ODI) or pain severity scores (NRS).

Reoperation

Any reoperations were tracked. The analysis of reoperations was stratified into reoperations at the index level and overall reoperations.

Perioperative Parameters

Length of hospital stay in hours and length of surgery in minutes were obtained. We identified patients with an extended length of stay, defined as greater than 28 hours. This cutoff was chosen according to the observed distribution in our prospective registry, and leads to an approximate 15% prevalence of extended hospital stay. Similarly, a length of surgery greater than 45 minutes was considered a prolonged surgery.

Statistical Analysis

Continuous data are given as mean ± standard deviation, and categorical data as values and percentages. Missing baseline data, which were assumed to be missing at random, were imputed using single imputation by predictive mean matching.39 Data were randomly split into two sets in a 70%/30% ratio for training and testing, respectively. On the training set, the models were trained and continually validated using bootstrapping. We trialed the following 7 ML algorithms for each variable: random forests, extreme gradient boosting (XGBoost), Bayesian generalized linear models (GLMs), boosted trees, k-nearest neighbor, simple GLMs, as well as artificial neural networks with a single hidden layer. For each outcome, after hyperparameter tuning, the model with the best performance during bootstrapping was selected based on area under the receiver operating characteristic curve (AUC), which constitutes the primary endpoint of this study. The selected final model was then evaluated on the test set for internal validation. All baseline variables were provided to all models. Whenever relevant class imbalance was present, we applied conventional random upsampling or synthetic minority oversampling (SMOTE) to the training set to ensure robustness of our models.35 At internal validation, calibration—the degree to which predicted probabilities correspond to true, observed probabilities—was assessed using the Brier score. The Brier score ranges from 0 to 1, with values closer to 0 indicating greater calibration. AUC-based individual variable importance was obtained for each of the models. All analyses were performed in R (version 3.5.1, The R Foundation for Statistical Computing). The complete code is provided in the Online-Only Content.

Results

During the study period, 635 patients who underwent lumbar decompression were identified. All patients were included in at least one of the models. The average age was 62 ± 10 years, and 333 patients (52%) were male. Detailed baseline characteristics of the study population are shown in Table 1. An overview of the endpoints observed in the cohort is provided in Table 2.

TABLE 1.

Overview of baseline patient characteristics (n = 635)

ParameterValue
Males, n (%)333 (52)
Mean age ± SD, yrs62 ± 10
Prior surgery, n (%)59 (9.3)
Active smoker, n (%)*109 (43)
Regular alcohol intake, n (%)*228 (90)
Predominant presenting symptom, n (%)
 Neurogenic claudication502 (79)
 Radiculopathy110 (17)
Concomitant spondylolisthesis, n (%)89 (14)
ASA score, n (%)
 I219 (39)
 II334 (60)
 III4 (0.7)
Mean height ± SD, cm173 ± 14
Mean weight ± SD, kg81 ± 14
Mean BMI ± SD, kg/m226 ± 3.4
Index level, n (%)
 L1–26 (1.0)
 L2–377 (12)
 L3–4276 (43)
 L4–5416 (66)
 L5–S161 (9.6)
Multilevel decompression, n (%)178 (28)
Mean baseline PROMs ± SD
 ODI40 ± 16
 NRS-LP6.7 ± 2.6
 NRS-BP5.4 ± 3.1

For these two variables data were available in 253 patients.

TABLE 2.

Summary of endpoints

EndpointNo. Available (%)Incidence (%)
6-wk PROMs*
 ODI173 (27)109 (63)
 NRS-LP196 (31)149 (76)
 NRS-BP196 (31)119 (61)
12-mo PROMs*
 ODI111 (17)73 (66)
 NRS-LP115 (18)73 (63)
 NRS-BP115 (18)59 (51)
Reoperations
 Overall635 (100)60 (9.5)
 At index level635 (100)27 (4.3)
Periop parameters
 Prolonged op, >45 mins451 (71)68 (15)
 Extended hospital stay, >28 hrs633 (99)95 (15)

For PROMs, the rate of patients achieving an MCID of at least ≥ 30% improvement from baseline is provided.

Patient-Reported Outcome

Short-Term Follow-Up

At 6 weeks postoperatively, a clinically meaningful improvement was seen in 63%, 76%, and 61% of patients for ODI, NRS-LP, and NRS-BP, respectively (Table 2). At internal validation (Table 3), the developed models were able to predict MCID in these variables with accuracies of 69%, 76%, and 85% and with AUC values of 0.75, 0.79, and 0.92, respectively (Fig. 1). This indicates fair to good discriminative ability, and the comparison of training and testing results indicates that no overfitting occurred. The Brier score indicates fair calibration of the predicted probabilities. The variables that were weighted most strongly by the models (Table 4) included baseline PROMs, concomitant spondylolisthesis, prior surgery, and smoking status.

TABLE 3.

Performance metrics of the ML-based prediction models on the training set and on unseen testing data, representing internal validation

6-wk PROMs12-mo PROMsReopsPeriop Parameters
MetricODINRS-LPNRS-BPODINRS-LPNRS-BPOverallAt Index LevelProlonged OpExtended Hospital Stay
Training (bootstrapping)
 Accuracy0.670.780.810.710.730.680.810.780.810.81
 AUC0.710.750.870.730.720.750.630.700.610.77
 Sensitivity0.740.870.890.540.830.600.350.080.860.38
 Specificity0.550.460.710.790.550.770.850.960.290.87
 PPV (precision)0.730.840.790.800.760.720.190.320.930.32
 NPV0.560.530.840.500.660.660.930.810.170.90
 F1 score*0.740.860.840.790.800.650.250.130.890.34
Testing (internal validation)
 Accuracy0.690.760.850.620.740.660.690.630.780.77
 AUC0.750.790.920.680.720.790.660.610.540.58
 Sensitivity0.800.820.860.570.900.560.320.500.850.27
 Specificity0.470.570.830.730.530.760.690.640.230.87
 PPV (precision)0.760.860.890.810.720.710.100.070.910.28
 NPV0.530.500.790.440.800.620.900.960.140.86
 F1 score*0.780.840.870.670.800.630.150.120.880.27
 Brier score0.230.160.110.200.200.180.090.050.130.13

The following algorithms were used: 1) 6-week PROMS: Bayesian GLM (ODI), neural network (NRS-LP), and boosted trees (NRS-BP); 2) 12-mo PROMs: Bayesian GLM (ODI), random forest (NRS-LP), and XGBoost machine (NRS-BP); 3) reoperations: XGBoost (Overall) and boosted trees (At Index Level); and 4) perioperative parameters: random forest (Prolonged Op) and XGBoost (Extended Hospital Stay).

The F1 score is a composite performance measure and is defined as the harmonic mean of precision and sensitivity.

Fig. 1.
Fig. 1.

AUC analysis of the ML models for prediction of 6-week (A) and 12-month (B) PROMs. Values were obtained from prediction on the testing (holdout) set, representing internal validation. Early outcome was predicted with AUC values of 0.75, 0.79, and 0.92 for functional disability, leg pain severity, and back pain severity, respectively. Similarly, AUC values of 0.68, 0.72, and 0.79 were observed for 12-month outcomes.

TABLE 4.

AUC-derived variable importance for each of the ML-based prediction models

6-wk PROMs12-mo PROMsReopsPeriop Parameters
VariableODINRS-LPNRS-BPODINRS-LPNRS-BPOverallAt Index LevelProlonged OpExtended Hospital Stay
Sex33.6929.756.133.786.12045.53*37.138.356.90
Age12.8902.1460.03*4.519.36*27.44*92.51*53.63*60.07*
BMI17.0426.16100.00*49.52*28.56*30.64*7.74*56.3417.6148.79*
Height52.11*61.60*12.1938.9917.05*17.54*48.75*75.67*59.23*23.58
Weight46.97*6.9622.1940.57*5.792.393.2197.02*4.3527.50*
Prior surgery029.3394.30*12.381.613.55015.551.605.76
Active smoker43.44*9.491.176.382.310099.69*83.82*0.71
Regular alcohol intake5.1413.0823.391.2700026.0932.7622.53
Predominant presenting symptom7.751.9019.5414.2511.850048.0504.15
Concomitant spondylolisthesis12.7446.62*39.79*3.6811.0801.986.238.230.63
ASA score19.348.020.597.832.5603.0748.8210.231.60
L1–29.750.21002.1300049.340
L2–35.5350.42*31.7519.130.090043.0552.20*2.42
L3–46.6748.52*21.6211.107.240017.8824.764.86
L4–510.5933.9712.264.541.040039.0838.083.30
L5–S18.671.068.625.413.140013.1016.980
Multilevel decompression54.8741.3531.879.505.6305.7438.4342.946.75
Baseline ODI100.00*100.00*54.71*58.03*67.54*12.96*030.1049.29100.00*
Baseline NRS-LP1.237.1731.94*100.00*31.03*8.05100.00*22.80100.00*24.84*
Baseline NRS-BP53.26*30.3818.1031.18100.00*100.00*3.78100.00*8.1819.19

Values have been scaled to importance scores between 0 and 100.

The five variables with the highest variable importance per model are marked with an asterisk.

Long-Term Follow-Up

At the 12-month follow-up, 66%, 63%, and 51% of patients reported experiencing a clinically meaningful improvement in ODI, NRS-LP, and NRS-BP, respectively (Table 2). At internal validation (Table 3), we observed accuracies of 62%, 74%, and 66%, as well as AUC values of 0.68, 0.72, and 0.79 for these long-term endpoints. Slight overfitting was observed for the 12-month ODI model, and all models demonstrated fair calibration. Baseline PROMs, as well as BMI, height, and weight, were found to have the highest variable importance (Table 4).

Reoperation

During the follow-up period, a total of 60 patients (9.5%) underwent reoperations, of which 27 (4.3%) were at the index level (Table 2). Overall, 27 patients (4.3%) underwent discectomy, 25 (3.9%) underwent additional decompression, 5 (0.8%) required fusion surgery, and 3 patients (0.5%) required reexploration. Of these reoperations, 15 (2.4%) discectomies were at the index level, as well as 7 (1.1%) decompression procedures for persistent symptoms, 2 (0.3%) fusion procedures for iatrogenic spondylolisthesis after decompression, and all 3 (0.5%) reexplorations for suspected dural defects. At internal validation (Table 3), overall and index-level reoperations were predicted with 69% and 63% accuracy, respectively, and with AUC values of 0.66 and 0.61. While slight overfitting was observed, the Brier score indicates excellent calibration of the predicted probabilities of reoperation. Age, sex, BMI, and baseline PROMs had the strongest influence on these models (Table 4).

Perioperative Parameters

Prolonged Surgery

In 15% of patients, a length of surgery greater than 45 minutes was observed (Table 2). The occurrence of a prolonged surgical procedure was predicted with an accuracy of 78%, but an AUC of 0.54. The model, based on the random forest algorithm, appears to possess a high sensitivity and positive predictive value (PPV) of 85% and 91%, respectively, but is rather unspecific (23%). This model would be fairly useful as a rule-out tool. Fair calibration was observed, and baseline leg pain severity, age, height, and high-lumbar procedures showed the highest variable importance (Table 4).

Extended Hospital Stay

Only 15% of patients were admitted to the hospital for longer than 28 hours (Table 2). The developed ML-based model enabled prediction of extended hospital stay with an accuracy of 77% and AUC of 0.58. Here, specificity was high at 87%, with a negative predictive value (NPV) of 86%, thus enabling its use as a rule-in tool. Calibration of predicted probabilities for extended hospital stay was fair, and baseline ODI and leg pain severity as well as age and BMI appeared to strongly influence the model’s decision-making process (Table 4).

Discussion

In a prospective registry of 635 patients operated on for LSS, we investigated the feasibility of preoperative identification of patients who benefit most from surgery, as well as those who are likely to require reoperation or encounter prolonged surgery or hospital stay. Effective prediction models based on ML were produced and internally validated. The models were able to predict the endpoints of interest with employable performance at internal validation from easily accessible preoperative data. ML, through nonlinear integration of preoperative variables, may enable more accurate preoperative estimation of intra-, peri-, and postoperative events, and has the potential to enable access to information on previously unpredictable endpoints, such as reoperation or unsatisfactory clinical outcome, which will be valuable to both patients and physicians.

The variability in indications, choice of surgical technique, and outcomes is notorious in elective spine surgery for LSS.21,22,27 This variability, combined with the psychological influencing factors and beliefs inherent to degenerative pathologies of the lumbar spine, makes accurate clinical predictions exceedingly hard.4 This lack of predictability for both the clinical outcome and adverse events—such as, for example, iatrogenic spondylolisthesis requiring fusion after decompression alone—means that correctly informing our patients on a case-to-case basis is complicated.3,16,36 It is often not possible to combine the large amount of known predictors and risk factors described in the literature into a single outcome prediction with which to inform our patient. Moreover, whenever a patient inquires about the odds of specific risks and benefits, often the best we can do is provide a carefully selected estimate from the literature. Subsets of patients who benefit more or less from surgery do, however, exist, and are often hard to identify. Vast clinical experience certainly enables accurate judging of a patient’s individual chances of successful outcome after surgery. However, it is not inconceivable that ML-based prediction tools (artificial intelligence) will soon overtake the performance of this “natural intelligence,” as has been the case in multiple areas of clinical neurosurgery already.31

Such prediction tools, whether based on ML or on more traditional statistical modeling methods, also have distinct advantages other than potentially increased precision. As opposed to providing the patients with a generalized treatment success rate, based on historical published data from the peer-reviewed literature, they estimate the likelihood of a given endpoint at the individual patient level. Furthermore, their output is equally and easily interpretable by patient and physician alike. The outputs of well-validated models could even be used to compare the results of surgeons or hospitals based on their individual case complexity. If used consistently, through enhanced patient selection, they have the potential to lower the proportion of patients undergoing surgery to no avail, and possibly even to avoid subsequent complications and reoperations such as iatrogenic spondylolisthesis requiring fusion. The comprehensive personalized risk profiles generated using highly accurate clinical prediction tools may even help hospitals gain insights into cost-effectiveness and ways to avoid unnecessary surgery through comparison of predicted and achieved endpoints. Knowledge on the likelihood of an extended hospital stay can guide postoperative planning, and it may even become possible to predict and prevent costly readmissions in patients discharged too early.2,18–20 Lastly, it has been suggested that clinical prediction tools could guide the preoperative optimization process by targeting and modifying specific patient factors—such as weight loss—to maximize the chances of clinical success.38

However, the outputs of clinical prediction tools may never be used as absolute “red or green lights” for surgery or for surgical success. Instead, they should complement, not replace, the vast clinical experience that neurosurgeons have, and be adopted as an adjunct to interpersonal judgment in daily clinical routine. In this way, they may enable enhanced and more individualized shared decision-making. For example, a specific patient who has suffered from mild symptomatic LSS for 3 years may be unsure as to whether he or she should undergo surgery. Tailored predictive analytics may suggest that the patient, while having a very high chance at a minimum 30% symptomatic improvement, also is prone to reoperation at the index level, and to an extended length of hospital stay. In combination with counseling by an experienced surgeon, this would allow the patient to make a somewhat more informed decision as to whether or not to undergo surgery.

Although there has been a vast increase in the use of prediction models in spine surgery, relatively little has been done in decompression surgery for LSS, and even less so using ML methods. Using conventional logistic regression, Held et al. were able to predict MCID in pain and functional disability after decompression, with AUC values of 0.62 and 0.70, respectively.17 Azimi et al. used shallow neural networks to predict treatment satisfaction in patients with LSS, with an AUC of 0.80, which was better than with their logistic regression model.1 Kalagara et al.18 were able to preoperatively predict 30-day readmissions following lumbar laminectomy using gradient boosting with an AUC of 0.69. Our results compare favorably with these works. In addition, these studies, in combination with analyses performed in other areas of spinal and cranial neurosurgery, corroborate the notion that ML models can be more precise in their predictions than more traditional statistical modeling techniques.1,18,20,25,29,32,34,37

Application of ML algorithms to predictive analytics is not without drawbacks. First, many ML algorithms—notably deep learning34,37—are often criticized for being “black box” models. That is, the user can observe what inputs are provided to the model and what the predicted output is, but the internal decision-making process of the model is unknown. Although we can visualize which variables are generally preferred by the model, as we have done using AUC-based variable importance in the current analysis, it is thus hard for the physician to explain to a patient why a particular prediction came together. In short, ML algorithms enable enhanced precision at the cost of reduced interpretability. In contrast, logistic regression provides easily interpretable odds ratios and confidence intervals. However, recent developments such as local interpretable model-agnostic explanations (LIME) are challenging this notion.28 Second, some ML architectures are prone to overfitting (Fig. 2), which is less of a problem in logistic regression, where overfitting can be easily combatted by shrinking the regression coefficients. Overfitting occurs when an algorithm adjusts too closely to the training data set, and subsequently demonstrates poor performance on new data. This occurs because the model starts memorizing specific patients in the training set, without actually learning generalizable relationships between the inputs, which would allow it to make precise forecasts on new, unseen patient data. Training in k-fold cross-validation or bootstrapping as well as internal validation enable detection and quantification of overfitting. In addition, some ML algorithms integrate specific regularization mechanisms to reduce overfitting.

Fig. 2.
Fig. 2.

Illustration of possible training data fits in predictive modeling. A 2D plane, representing two input variables, is shown. The black line represents a function trained to distinguish green and blue dots using the two input variables. A: An optimal, smooth fit (trade-off between bias and variance); the function is able to discriminate green from blue dots with only minor errors, and is likely to perform similarly well on new, unseen testing data. B: Underfitting (high bias, no variance); the function has not learned to distinguish green from blue dots even on the training data. C: Overfitting (no bias, high variance); the function has been overtrained, and is able to perfectly separate the dots in the training data. However, because it is “overfitted” to the training data, it will perform poorly on new, unseen testing data.

Limitations

Although we obtained all data from a prospective registry, selection bias cannot effectively be ruled out. Furthermore, our methodology was not prospectively decided upon. The findings of our variable importance analysis may be biased by confounders. Our prediction tool does not include measures of quality of life and objective functional impairment, nor did we have access to data on anxiety and depression, severity of stenosis, comorbidities, or neurological deficits, which are relevant data points in the management of patients with LSS. ML algorithms rely on large amounts of training data. Thus, although a comparatively large number of patients were included, and although we employed advanced data augmentation techniques, this study has to be seen as a pilot study demonstrating the feasibility of predicting these endpoints with a relatively small amount of data. Further supervised training with a larger sample, or semi-supervised training that integrates unlabeled data, is likely to improve the robustness and accuracy of the models. This would allow reducing the amount of required input variables and construction of a simpler model. Furthermore, we used MCID to dichotomize clinical outcome.26 However, the choice of other, equally viable options such as minimum detectable change or AUC-anchored methods could lead to different findings.9 Single-center data were used to derive our models. Although we have internally validated the models, this could possibly limit the generalizability of our prediction tools to different cohorts with varying patient characteristics, different clinical and surgical decision-making protocols, and surgeons with a different level of experience. Models trained on data with enough variance from a single setting, such as ours, can often be applied with considerable success at other centers. Nonetheless, external validation of existing models is highly desirable. In the future, however, individual centers may also opt to train their own, center-specific models.34 Our study shows that this is feasible.

Conclusions

Using data from a prospective registry, we trained and internally validated preoperative predictive analytics for patient-reported outcome, reoperations, prolonged surgery, and extended length of hospital stay. Prediction of clinically relevant endpoints in decompression surgery for LSS using machine learning is feasible, and may enable enhanced informed patient consent and personalized shared decision-making. Access to individualized preoperative predictive analytics for outcome and treatment risks may represent a further step in the evolution of surgical care for patients with LSS.

Disclosures

The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

Author Contributions

Conception and design: Staartjes, Schröder. Acquisition of data: all authors. Analysis and interpretation of data: Staartjes, Siccoli, Schröder. Drafting the article: Staartjes, Siccoli, Schröder. Critically revising the article: Staartjes, de Wispelaere, Schröder. Reviewed submitted version of manuscript: Staartjes, de Wispelaere, Schröder. Approved the final version of the manuscript on behalf of all authors: Staartjes. Statistical analysis: Staartjes, Siccoli. Administrative/technical/material support: Staartjes, de Wispelaere, Schröder. Study supervision: Schröder.

Supplemental Information

Online-Only Content

Supplemental material is available online.

R code for the ML modeling process. The code was executed in R (version 3.5.1, The R Foundation for Statistical Computing) on a machine running Windows 10 (Microsoft Corp.).

References

  • 1

    Azimi PBenzel ECShahzadi SAzhari SMohammadi HR: Use of artificial neural networks to predict surgical satisfaction in patients with lumbar spinal canal stenosis: clinical article. J Neurosurg Spine 20:3003052014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 2

    Bernatz JTAnderson PA: Thirty-day readmission rates in spine surgery: systematic review and meta-analysis. Neurosurg Focus 39(4):E72015

  • 3

    Blumenthal CCurran JBenzel ECPotter RMagge SNHarrington JF Jr: Radiographic predictors of delayed instability following decompression without fusion for degenerative grade I lumbar spondylolisthesis. J Neurosurg Spine 18:3403462013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4

    Burgstaller JMWertli MMSteurer JKessels AGHHeld UGramke HF: the influence of pre- and postoperative fear avoidance beliefs on postoperative pain and disability in patients with lumbar spinal stenosis: analysis of the Lumbar Spinal Outcome Study (LSOS) data. Spine (Phila Pa 1976) 42:E425E4322017

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 5

    Chan AKBisson EFBydon MGlassman SDFoley KTPotts EA: Laminectomy alone versus fusion for grade 1 lumbar spondylolisthesis in 426 patients from the prospective Quality Outcomes Database. J Neurosurg Spine 30:2342412018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6

    Choma TJMroz TEGoldstein CLArnold PShamji MF: Emerging techniques in degenerative thoracolumbar surgery. Neurosurgery 80(3S):S55S602017

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7

    Cobo Soriano JSendino Revuelta MFabregate Fuente MCimarra Díaz IMartínez Ureña PDeglané Meneses R: Predictors of outcome after decompressive lumbar surgery and instrumented posterolateral fusion. Eur Spine J 19:184118482010

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 8

    Collins GSReitsma JBAltman DGMoons KGM: Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 350:g75942015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9

    Copay AGGlassman SDSubach BRBerven SSchuler TCCarreon LY: Minimum clinically important difference in lumbar spine surgery patients: a choice of methods using the Oswestry Disability Index, Medical Outcomes Study Questionnaire Short Form 36, and pain scales. Spine J 8:9689742008

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10

    Dietz NSharma MAlhourani AUgiliweneza BWang DNuño MA: Variability in the utility of predictive models in predicting patient-reported outcomes following spine surgery for degenerative conditions: a systematic review. Neurosurg Focus 45(5):E102018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11

    Elsayed GMcClugage SGErwood MSDavis MCDupépé EBSzerlip P: Association between payer status and patient-reported outcomes in adult patients with lumbar spinal stenosis treated with decompression surgery. J Neurosurg Spine 30:1982102018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 12

    Försth PÓlafsson GCarlsson TFrost ABorgström FFritzell P: A randomized, controlled trial of fusion surgery for lumbar spinal stenosis. N Engl J Med 374:141314232016

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 13

    Ghogawala ZBenzel ECAmin-Hanjani SBarker FG IIHarrington JFMagge SN: Prospective outcomes evaluation after decompression with or without instrumented fusion for lumbar stenosis and degenerative grade I spondylolisthesis. J Neurosurg Spine 1:2672722004

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 14

    Ghogawala ZDziura JButler WEDai FTerrin NMagge SN: Laminectomy plus fusion versus laminectomy alone for lumbar spondylolisthesis. N Engl J Med 374:142414342016

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 15

    Ghogawala ZShaffrey CIAsher ALHeary RFLogvinenko TMalhotra NR: The efficacy of lumbar discectomy and single-level fusion for spondylolisthesis: results from the NeuroPoint-SD registry: clinical article. J Neurosurg Spine 19:5555632013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 16

    Guha DHeary RFShamji MF: Iatrogenic spondylolisthesis following laminectomy for degenerative lumbar stenosis: systematic review and current concepts. Neurosurg Focus 39(4):E92015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 17

    Held UBurgstaller JMWertli MMPichierri GWinklhofer SBrunner F: Prognostic function to estimate the probability of meaningful clinical improvement after surgery—results of a prospective multicenter observational cohort study on patients with lumbar spinal stenosis. PLoS One 13:e02071262018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 18

    Kalagara SEltorai AEMDurand WMDePasse JMDaniels AH: Machine learning modeling for predicting hospital readmission following lumbar laminectomy. J Neurosurg Spine 1:192018

    • Search Google Scholar
    • Export Citation
  • 19

    Kanaan SFYeh HWWaitman RLBurton DCArnold PMSharma NK: Predicting discharge placement and health care needs after lumbar spine laminectomy. J Allied Health 43:88972014

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20

    Karhade AVOgink PThio QBroekman MCha TGormley WB: Development of machine learning algorithms for prediction of discharge disposition after elective inpatient surgery for lumbar degenerative disc disorders. Neurosurg Focus 45(5):E62018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 21

    Lønne GFritzell PHägg ONordvall DGerdhem PLagerbäck T: Lumbar spinal stenosis: comparison of surgical practice variation and clinical outcome in three national spine registries. Spine J 19:41492019

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 22

    Lønne GSchoenfeld AJCha TDNygaard ØPZwart JAHSolberg T: Variation in selection criteria and approaches to surgery for lumbar spinal stenosis among patients treated in Boston and Norway. Clin Neurol Neurosurg 156:77822017

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 23

    Machado GCFerreira PHYoo RIHarris IAPinheiro MBKoes BW: Surgical options for lumbar spinal stenosis. Cochrane Database Syst Rev 11:CD0124212016

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 24

    Mummaneni PVWhitmore RGCurran JNZiewacz JEWadhwa RShaffrey CI: Cost-effectiveness of lumbar discectomy and single-level fusion for spondylolisthesis: experience with the NeuroPoint-SD registry. Neurosurg Focus 36(6):E32014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 25

    Obermeyer ZEmanuel EJ: Predicting the future—big data, machine learning, and clinical medicine. N Engl J Med 375:121612192016

  • 26

    Ostelo RWJGDeyo RAStratford PWaddell GCroft PVon Korff M: Interpreting change scores for pain and functional status in low back pain: towards international consensus regarding minimal important change. Spine (Phila Pa 1976) 33:90942008

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 27

    Raad MDonaldson CJEl Dafrawy MHSciubba DMRiley LHNeuman BJ: Trends in isolated lumbar spinal stenosis surgery among working US adults aged 40–64 years2010–2014. J Neurosurg Spine 29:1691752018

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 28

    Ribeiro MTSingh SGuestrin C: “Why should I trust you?”: Explaining the predictions of any classifier. arXiv:1602.04938 2016

  • 29

    Scheer JKOh TSmith JSShaffrey CIDaniels AHSciubba DM: Development of a validated computer-based preoperative predictive model for pseudarthrosis with 91% accuracy in 336 adult spinal deformity patients. Neurosurg Focus 45(5):E112018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 30

    Schröder MLde Wispelaere MPStaartjes VE: Are patient-reported outcome measures biased by method of follow-up? Evaluating paper-based and digital follow-up after lumbar fusion surgery. Spine J 19:65702019

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 31

    Senders JTArnaout OKarhade AVDasenbrock HHGormley WBBroekman ML: Natural and artificial intelligence in neurosurgery: a systematic review. Neurosurgery 83:1811922018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 32

    Senders JTZaki MMKarhade AVChang BGormley WBBroekman ML: An introduction and overview of machine learning in neurosurgical care. Acta Neurochir (Wien) 160:29382018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 33

    Sigmundsson FGJönsson BStrömqvist B: Outcome of decompression with and without fusion in spinal stenosis with degenerative spondylolisthesis in relation to preoperative pain pattern: a register study of 1,624 patients. Spine J 15:6386462015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 34

    Staartjes VEde Wispelaere MPVandertop WPSchröder ML: Deep learning-based preoperative predictive analytics for patient-reported outcomes following lumbar diskectomy: feasibility of center-specific modeling. Spine J [epub ahead of print] 2018

    • Search Google Scholar
    • Export Citation
  • 35

    Staartjes VESchröder ML: Class imbalance in machine learning for neurosurgical outcome prediction: are our models valid? J Neurosurg Spine 29:6116122018 (Letter)

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 36

    Staartjes VESchröder ML: Effectiveness of a decision-making protocol for the surgical treatment of lumbar stenosis with grade 1 degenerative spondylolisthesis. World Neurosurg 110:e355e3612018

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 37

    Staartjes VESerra CMuscas GMaldaner NAkeret Kvan Niftrik CHB: Utility of deep neural networks in predicting gross-total resection after transsphenoidal surgery for pituitary adenoma: a pilot study. Neurosurg Focus 45(5):E122018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 38

    Steinmetz MPMroz T: Value of adding predictive clinical decision tools to spine surgery. JAMA Surg 153:6432018

  • 39

    van Buuren SGroothuis-Oudshoorn CGM: mice: multivariate imputation by chained equations in R. J Stat Softw 45(3):1672011

If the inline PDF is not rendering correctly, you can download the PDF file here.

Article Information

Correspondence Victor E. Staartjes: c/o Bergman Clinics, Naarden, The Netherlands. victor.staartjes@gmail.com.

INCLUDE WHEN CITING DOI: 10.3171/2019.2.FOCUS18723.

Disclosures The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

© AANS, except where prohibited by US copyright law.

Headings

Figures

  • View in gallery

    AUC analysis of the ML models for prediction of 6-week (A) and 12-month (B) PROMs. Values were obtained from prediction on the testing (holdout) set, representing internal validation. Early outcome was predicted with AUC values of 0.75, 0.79, and 0.92 for functional disability, leg pain severity, and back pain severity, respectively. Similarly, AUC values of 0.68, 0.72, and 0.79 were observed for 12-month outcomes.

  • View in gallery

    Illustration of possible training data fits in predictive modeling. A 2D plane, representing two input variables, is shown. The black line represents a function trained to distinguish green and blue dots using the two input variables. A: An optimal, smooth fit (trade-off between bias and variance); the function is able to discriminate green from blue dots with only minor errors, and is likely to perform similarly well on new, unseen testing data. B: Underfitting (high bias, no variance); the function has not learned to distinguish green from blue dots even on the training data. C: Overfitting (no bias, high variance); the function has been overtrained, and is able to perfectly separate the dots in the training data. However, because it is “overfitted” to the training data, it will perform poorly on new, unseen testing data.

References

  • 1

    Azimi PBenzel ECShahzadi SAzhari SMohammadi HR: Use of artificial neural networks to predict surgical satisfaction in patients with lumbar spinal canal stenosis: clinical article. J Neurosurg Spine 20:3003052014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 2

    Bernatz JTAnderson PA: Thirty-day readmission rates in spine surgery: systematic review and meta-analysis. Neurosurg Focus 39(4):E72015

  • 3

    Blumenthal CCurran JBenzel ECPotter RMagge SNHarrington JF Jr: Radiographic predictors of delayed instability following decompression without fusion for degenerative grade I lumbar spondylolisthesis. J Neurosurg Spine 18:3403462013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 4

    Burgstaller JMWertli MMSteurer JKessels AGHHeld UGramke HF: the influence of pre- and postoperative fear avoidance beliefs on postoperative pain and disability in patients with lumbar spinal stenosis: analysis of the Lumbar Spinal Outcome Study (LSOS) data. Spine (Phila Pa 1976) 42:E425E4322017

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 5

    Chan AKBisson EFBydon MGlassman SDFoley KTPotts EA: Laminectomy alone versus fusion for grade 1 lumbar spondylolisthesis in 426 patients from the prospective Quality Outcomes Database. J Neurosurg Spine 30:2342412018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 6

    Choma TJMroz TEGoldstein CLArnold PShamji MF: Emerging techniques in degenerative thoracolumbar surgery. Neurosurgery 80(3S):S55S602017

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 7

    Cobo Soriano JSendino Revuelta MFabregate Fuente MCimarra Díaz IMartínez Ureña PDeglané Meneses R: Predictors of outcome after decompressive lumbar surgery and instrumented posterolateral fusion. Eur Spine J 19:184118482010

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 8

    Collins GSReitsma JBAltman DGMoons KGM: Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 350:g75942015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 9

    Copay AGGlassman SDSubach BRBerven SSchuler TCCarreon LY: Minimum clinically important difference in lumbar spine surgery patients: a choice of methods using the Oswestry Disability Index, Medical Outcomes Study Questionnaire Short Form 36, and pain scales. Spine J 8:9689742008

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 10

    Dietz NSharma MAlhourani AUgiliweneza BWang DNuño MA: Variability in the utility of predictive models in predicting patient-reported outcomes following spine surgery for degenerative conditions: a systematic review. Neurosurg Focus 45(5):E102018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 11

    Elsayed GMcClugage SGErwood MSDavis MCDupépé EBSzerlip P: Association between payer status and patient-reported outcomes in adult patients with lumbar spinal stenosis treated with decompression surgery. J Neurosurg Spine 30:1982102018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 12

    Försth PÓlafsson GCarlsson TFrost ABorgström FFritzell P: A randomized, controlled trial of fusion surgery for lumbar spinal stenosis. N Engl J Med 374:141314232016

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 13

    Ghogawala ZBenzel ECAmin-Hanjani SBarker FG IIHarrington JFMagge SN: Prospective outcomes evaluation after decompression with or without instrumented fusion for lumbar stenosis and degenerative grade I spondylolisthesis. J Neurosurg Spine 1:2672722004

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 14

    Ghogawala ZDziura JButler WEDai FTerrin NMagge SN: Laminectomy plus fusion versus laminectomy alone for lumbar spondylolisthesis. N Engl J Med 374:142414342016

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 15

    Ghogawala ZShaffrey CIAsher ALHeary RFLogvinenko TMalhotra NR: The efficacy of lumbar discectomy and single-level fusion for spondylolisthesis: results from the NeuroPoint-SD registry: clinical article. J Neurosurg Spine 19:5555632013

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 16

    Guha DHeary RFShamji MF: Iatrogenic spondylolisthesis following laminectomy for degenerative lumbar stenosis: systematic review and current concepts. Neurosurg Focus 39(4):E92015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 17

    Held UBurgstaller JMWertli MMPichierri GWinklhofer SBrunner F: Prognostic function to estimate the probability of meaningful clinical improvement after surgery—results of a prospective multicenter observational cohort study on patients with lumbar spinal stenosis. PLoS One 13:e02071262018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 18

    Kalagara SEltorai AEMDurand WMDePasse JMDaniels AH: Machine learning modeling for predicting hospital readmission following lumbar laminectomy. J Neurosurg Spine 1:192018

    • Search Google Scholar
    • Export Citation
  • 19

    Kanaan SFYeh HWWaitman RLBurton DCArnold PMSharma NK: Predicting discharge placement and health care needs after lumbar spine laminectomy. J Allied Health 43:88972014

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 20

    Karhade AVOgink PThio QBroekman MCha TGormley WB: Development of machine learning algorithms for prediction of discharge disposition after elective inpatient surgery for lumbar degenerative disc disorders. Neurosurg Focus 45(5):E62018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 21

    Lønne GFritzell PHägg ONordvall DGerdhem PLagerbäck T: Lumbar spinal stenosis: comparison of surgical practice variation and clinical outcome in three national spine registries. Spine J 19:41492019

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 22

    Lønne GSchoenfeld AJCha TDNygaard ØPZwart JAHSolberg T: Variation in selection criteria and approaches to surgery for lumbar spinal stenosis among patients treated in Boston and Norway. Clin Neurol Neurosurg 156:77822017

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 23

    Machado GCFerreira PHYoo RIHarris IAPinheiro MBKoes BW: Surgical options for lumbar spinal stenosis. Cochrane Database Syst Rev 11:CD0124212016

    • PubMed
    • Search Google Scholar
    • Export Citation
  • 24

    Mummaneni PVWhitmore RGCurran JNZiewacz JEWadhwa RShaffrey CI: Cost-effectiveness of lumbar discectomy and single-level fusion for spondylolisthesis: experience with the NeuroPoint-SD registry. Neurosurg Focus 36(6):E32014

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 25

    Obermeyer ZEmanuel EJ: Predicting the future—big data, machine learning, and clinical medicine. N Engl J Med 375:121612192016

  • 26

    Ostelo RWJGDeyo RAStratford PWaddell GCroft PVon Korff M: Interpreting change scores for pain and functional status in low back pain: towards international consensus regarding minimal important change. Spine (Phila Pa 1976) 33:90942008

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 27

    Raad MDonaldson CJEl Dafrawy MHSciubba DMRiley LHNeuman BJ: Trends in isolated lumbar spinal stenosis surgery among working US adults aged 40–64 years2010–2014. J Neurosurg Spine 29:1691752018

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 28

    Ribeiro MTSingh SGuestrin C: “Why should I trust you?”: Explaining the predictions of any classifier. arXiv:1602.04938 2016

  • 29

    Scheer JKOh TSmith JSShaffrey CIDaniels AHSciubba DM: Development of a validated computer-based preoperative predictive model for pseudarthrosis with 91% accuracy in 336 adult spinal deformity patients. Neurosurg Focus 45(5):E112018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 30

    Schröder MLde Wispelaere MPStaartjes VE: Are patient-reported outcome measures biased by method of follow-up? Evaluating paper-based and digital follow-up after lumbar fusion surgery. Spine J 19:65702019

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 31

    Senders JTArnaout OKarhade AVDasenbrock HHGormley WBBroekman ML: Natural and artificial intelligence in neurosurgery: a systematic review. Neurosurgery 83:1811922018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 32

    Senders JTZaki MMKarhade AVChang BGormley WBBroekman ML: An introduction and overview of machine learning in neurosurgical care. Acta Neurochir (Wien) 160:29382018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 33

    Sigmundsson FGJönsson BStrömqvist B: Outcome of decompression with and without fusion in spinal stenosis with degenerative spondylolisthesis in relation to preoperative pain pattern: a register study of 1,624 patients. Spine J 15:6386462015

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 34

    Staartjes VEde Wispelaere MPVandertop WPSchröder ML: Deep learning-based preoperative predictive analytics for patient-reported outcomes following lumbar diskectomy: feasibility of center-specific modeling. Spine J [epub ahead of print] 2018

    • Search Google Scholar
    • Export Citation
  • 35

    Staartjes VESchröder ML: Class imbalance in machine learning for neurosurgical outcome prediction: are our models valid? J Neurosurg Spine 29:6116122018 (Letter)

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 36

    Staartjes VESchröder ML: Effectiveness of a decision-making protocol for the surgical treatment of lumbar stenosis with grade 1 degenerative spondylolisthesis. World Neurosurg 110:e355e3612018

    • Crossref
    • Search Google Scholar
    • Export Citation
  • 37

    Staartjes VESerra CMuscas GMaldaner NAkeret Kvan Niftrik CHB: Utility of deep neural networks in predicting gross-total resection after transsphenoidal surgery for pituitary adenoma: a pilot study. Neurosurg Focus 45(5):E122018

    • Crossref
    • PubMed
    • Search Google Scholar
    • Export Citation
  • 38

    Steinmetz MPMroz T: Value of adding predictive clinical decision tools to spine surgery. JAMA Surg 153:6432018

  • 39

    van Buuren SGroothuis-Oudshoorn CGM: mice: multivariate imputation by chained equations in R. J Stat Softw 45(3):1672011

TrendMD

Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 434 434 107
PDF Downloads 340 340 91
EPUB Downloads 0 0 0

PubMed

Google Scholar