Use of artificial neural networks to predict surgical satisfaction in patients with lumbar spinal canal stenosis

Clinical article

Full access

Object

The purpose of this study was to develop an artificial neural network (ANN) model for predicting 2-year surgical satisfaction, and to compare the new model with traditional predictive tools in patients with lumbar spinal canal stenosis.

Methods

The 2 prediction models included an ANN and a logistic regression (LR) model. The patient age, sex, duration of symptoms, walking distance, visual analog scale scores of leg pain or numbness, the Japanese Orthopaedic Association score, the Neurogenic Claudication Outcome Score, and the stenosis ratio values were determined as the input variables for the ANN and LR models that were developed. Patient surgical satisfaction was recorded using a standardized measure. The ANNs were fed patient data to predict 2-year surgical satisfaction based on several input variables. Sensitivity analysis was applied to the ANN model to identify the important variables. The receiver operating characteristic–area under curve (ROC-AUC), Hosmer-Lemeshow statistics, and accuracy rate were calculated for evaluating the 2 models.

Results

A total of 168 patients (59 male, 109 female; mean age 59.8 ± 11.6 years) were divided into training (n = 84), testing (n = 42), and validation (n = 42) data sets. Postsurgical satisfaction was 88.7% at 2-year follow-up. The stenosis ratio was the important variable selected by the ANN. The ANN model displayed a better accuracy rate in 96.9% of patients, a better Hosmer-Lemeshow statistic in 42.4% of patients, and a better ROC-AUC in 80% of patients, compared with the LR model.

Conclusions

The findings show that an ANN can predict 2-year surgical satisfaction for use in clinical application and is more accurate compared with an LR model.

Abbreviations used in this paper:ANN = artificial neural network; JOA = Japanese Orthopaedic Association; LBP = low-back pain; LR = logistic regression; LSCS = lumbar spinal canal stenosis; MLP = multilayer perceptron; NCOS = Neurogenic Claudication Outcome Score; ROC-AUC = receiver operating characteristic–area under curve; SR = stenosis ratio; VAS = visual analog scale.

Abstract

Object

The purpose of this study was to develop an artificial neural network (ANN) model for predicting 2-year surgical satisfaction, and to compare the new model with traditional predictive tools in patients with lumbar spinal canal stenosis.

Methods

The 2 prediction models included an ANN and a logistic regression (LR) model. The patient age, sex, duration of symptoms, walking distance, visual analog scale scores of leg pain or numbness, the Japanese Orthopaedic Association score, the Neurogenic Claudication Outcome Score, and the stenosis ratio values were determined as the input variables for the ANN and LR models that were developed. Patient surgical satisfaction was recorded using a standardized measure. The ANNs were fed patient data to predict 2-year surgical satisfaction based on several input variables. Sensitivity analysis was applied to the ANN model to identify the important variables. The receiver operating characteristic–area under curve (ROC-AUC), Hosmer-Lemeshow statistics, and accuracy rate were calculated for evaluating the 2 models.

Results

A total of 168 patients (59 male, 109 female; mean age 59.8 ± 11.6 years) were divided into training (n = 84), testing (n = 42), and validation (n = 42) data sets. Postsurgical satisfaction was 88.7% at 2-year follow-up. The stenosis ratio was the important variable selected by the ANN. The ANN model displayed a better accuracy rate in 96.9% of patients, a better Hosmer-Lemeshow statistic in 42.4% of patients, and a better ROC-AUC in 80% of patients, compared with the LR model.

Conclusions

The findings show that an ANN can predict 2-year surgical satisfaction for use in clinical application and is more accurate compared with an LR model.

Nearly everyone experiences low-back pain (LBP) at some point in his or her life. A common cause of LBP is lumbar spinal canal stenosis (LSCS).4 The term LSCS implies that a portion of the spinal canal has narrowed.8 The symptoms of LSCS include pain, numbness, or weakness in the legs, groin, hips, buttocks, and lower back. Symptoms usually worsen when walking or standing (claudication).8 The ability to predict the surgical success rate is important in choosing the appropriate management of LSCS in these patients.

Medical informatics has been applied to develop prediction models to assess, diagnose, and treat these patients, such as logistic regression (LR) and artificial neural networks (ANNs) or neural networks (in short). One of the models, LR, is a traditional predictive tool. The other, ANN, is a computational model based on the functioning of biological neural networks that can be used as nonlinear statistical data modeling tools with which the complex relationships between inputs and outputs (observed data) are modeled or patterns are found. The ANNs try to simulate the learning process of human beings; in other words they learn as humans learn, through observing samples. Actually they are made of a group of interconnected nodes (artificial neurons) that interact with each other based on predefined computational rules. Based on these rules, passing sample data (pairs of observed input/output data) through ANNs makes them modify their structure in such a way that they will be able to estimate the input/output relationship pattern of the systems under study. The resultant network at the end of this learning process can then be used to estimate or predict outputs for new inputs. This capability makes ANNs powerful tools for applications such as pattern recognition, medical diagnosis, financial applications, data mining, email spam filtering, and so on.

Different neuron arrangements and learning methods lead to different ANN types. The most common type is called a multilayer perceptron (MLP), which consists of 3 layers: 1) input layer; 2) hidden layer; and 3) output layer (Fig. 1). With learning processes, sample data are fed into the input layer and the network output is compared with the true expected output. The difference between the network's actual output and expected output equates to an error that is used to modify the interconnection of neurons that are weighted based on a specific mathematical method. This method is called back-propagation. It enables the network to optimally emulate the model of the system under study. The resultant trained network can be used to predict or estimate output for new inputs. For medical applications, due to the limited number of treatment options available, these prediction models can potentially improve diagnostic accuracy, treatment decisions, and efficiency.7,9

Fig. 1.
Fig. 1.

Schematic diagram of the ANN.

Relationships between prognostic factors and surgical satisfaction have not been previously investigated using ANNs. In this study we sought to develop an ANN model based on the age, visual analog scale (VAS) score for leg pain or numbness, walking distance, stenosis ratio (SR), the Japanese Orthopaedic Association (JOA) score, and the Neurogenic Claudication Outcome Score (NCOS). We also sought to determine whether ANNs perform better at predicting 2-year surgical satisfaction compared with LR in patients with LSCS.

Methods

Patients and Data Collection

This retrospective study population included 168 patients who had undergone surgery for LSCS between May 2007 and January 2012. The diagnosis of LSCS was established using clinical symptoms, neurological examinations, and imaging studies including plain radiography, MRI, and CT studies of the lumbar spine. All patients had the typical symptoms of LSCS, such as neurogenic intermittent claudication and leg pain and/or numbness. Diagnosis was confirmed by more than 1 spine surgeon in all patients. The level(s) of stenosis were explored on the MRI or CT studies. There were no restrictions on patient choice with regard to level(s) of LSCS, age, or other characteristics. The exclusion criteria were prior lumbar spine surgery and spinal anomalies.

Demographic data, including age, sex, and body weight were recorded. The walking distance (in meters) and duration of symptoms (in months) were recorded. A 100-mm VAS was used to measure both leg pain and numbness.10 The JOA score for assessing LBP was also calculated (total 29 points);1 a high JOA score indicates a better clinical outcome. The NCOS was determined (total 100 points); a high NCOS shows a better clinical outcome.2 Patient surgical satisfaction was recorded using standardized questionnaires as shown in Table 1.12

TABLE 1:

Satisfaction measures in patients with LSCS*

  1. Overall, how successful has your operation been?

    • Very successful, complete relief

    • Fairly successful, a good deal of relief

    • Not very successful, only a little relief

    • Failure, no relief

    • Worse than before

  2. If you had a friend with the same trouble you had, would you recommend the operation? Yes/No

“Satisfaction” requires “a” or “b” and “Yes” to the above questions.

Additional Measure

The cross-sectional area at the most stenotic level and at the pedicle level uninvolved by stenosis was measured using the Hamanishi technique (Fig. 2).5 The calculations were done by 2 independent surgeons, and these investigators were blinded to each other's findings.

Fig. 2.
Fig. 2.

Chart showing Hamanishi technique to measure cross-sectional area on axial MRI slices.

The SR was also calculated. In 1999, Lurencin and colleagues6 described the SR. It was used to determine the severity of stenosis. This ratio is the cross-sectional area of the canal at the axial MRI slice showing the greatest neurological compression at disc level over the cross-sectional area at the pedicle level above.

The ANN Model

The ANN was used and was based on the standard method and created using the Statistical Package for the Social Sciences (SPSS) software program. Our chosen model was an MLP ANN that is the most common type. An MLP ANN consists of a series of nodes arranged with 3 layers: 1) an input layer; 2) a hidden layer; and 3) an output layer. The MLP ANN used input (age, VAS score for leg pain or numbness, SR, walking distance, JOA score, and NCOS) and output data (surgical satisfaction) (samples for training set) to define (learn) the interrelationships among the data. Patients were partitioned by a 2:1:1 ratio to generate training, testing, and validation. Once the MLP ANN had been trained, it could then predict results from new sets of input data.3,11

Logistic Regression

Traditional statistical analysis of variable significance was performed with standard LR on the same data set from which the ANN was assessed.

Statistical Analysis

All statistical analyses were performed using PASW Statistics 18 (version 18; SPSS, Inc., 2009). For each individual parameter and for comparisons of the ANN and LR models, the receiver operating characteristic–area under curve (ROC-AUC) was generated and used to calculate specificities, positive predictive value, and negative predictive value for the 2 models and parameters at 95% sensitivity. Discrimination capability was assessed by calculating the AUC from the ROC analysis. For each pair of ANN and LR models (trained and tested on the same data sets), Hosmer-Lemeshow statistics, ROC-AUC, and accuracy rate were calculated and compared using t-tests for the validation group (42 patients).

Ethics Statement

The research was approved by the ethics committee of Shahid-Beheshti University of Medical Sciences, Tehran, Iran.

Results

Demographic data for the patients with LSCS and their scores on the JOA test, the NCOS, and the SR are shown in Table 2. A total of 168 (59 male, 109 female) patients were included (mean age 59.8 ± 11.6 years, range 17–84 years) and were divided into training (n = 84), testing (n = 42), and validation (n = 42) data sets. Interrelationships between predictor variables (input nodes), hidden variables (3 of them in 1 hidden layer), and surgical satisfaction/dissatisfaction (output nodes) are illustrated in Fig. 3. Postsurgical satisfaction was 149 of 168 patients (88.7%) at 2-year follow-up.

TABLE 2:

Demographic data and preoperative status in 168 patients with lumbar spinal stenosis*

CharacteristicValueRange
age (yrs)59.8 ± 11.617–84
sex (% male)35%
body weight (kg)76.8 ± 11.154–101
duration of Sxs (mos)46.0 ± 23.02–62
walking distance (m)321.4 ± 2180–600
VAS score
 leg pain (mm)37.9 ± 27.610–100
 leg numbness (mm)57.6 ± 22.830–100
SR0.46 ± 0.190.18–0.86
JOA score12.9 ± 6.72–21
NCOS27.2 ± 12.10–38

Values are expressed as the mean ± SD unless otherwise noted. Sxs = symptoms.

Fig. 3.
Fig. 3.

Diagram showing ANN output, with insets for each layer—output figure was generated by PASW Statistics 18 (version 18; SPSS, Inc., 2009). Input layer (left-hand group): bias = input layer bias; VAS.N = VAS score of leg numbness; distance = walking distance; VAS.P = VAS score of leg pain. Hidden (H) layer (center group): H (1:1), H (1:2), and H (1:3); bias = hidden layer bias. Output layer (right-hand group): satisfied = Y denotes satisfaction; satisfied = N denotes dissatisfaction at 2-year follow-up.

The results of comparisons between the ANN and LR models and individual parameters are shown in Tables 3 and 4. The sensitivity analysis identified that SR is the important variable selected by the ANN model. Compared with the LR models, the ANN models had a better accuracy rate in 96.9% of patients, a better Hosmer-Lemeshow statistic in 42.4% of patients, and a better ROC-AUC in 80% of patients as well as better sensitivity.

TABLE 3:

Comparison of the ROC-AUC and predictive values of ANN and LR models, and individual parameters to predict surgical satisfaction in 168 patients*

Parameter/ModelROC-AUC (%)p ValueSpecificity (%)PPV (%)NPV (%)
ANN0.810.002416989
LR0.770.002346382
SR0.750.02266175
NCOS0.630.25215659
JOA0.610.34195556
VAS.P0.540.46165554
VAS.N0.530.78145352
walking distance0.510.8155248
age0.480.8635218

NPV = negative predictive value; PPV = positive predictive value; VAS.N = VAS score for leg numbness; VAS.P = VAS score for leg pain.

Asymptotic significance on ROC-AUC analysis.

Specificity and predictive values at 95% sensitivity.

TABLE 4:

Comparison of ANN and LR models for predicting surgical satisfaction in the 42 patients in the validation group*

ParameterANN (95% CI)LR (95% CI)p Value
accuracy rate (%)96.9 (95.1–98.2)88.4 (85.7–90.1)<0.001
ROC-AUC0.80 (0.87–0.81)0.76 (0.72–0.79)<0.001
H-L statistics42.4 (36.6–48.7)55.6 (50.2–58.5)<0.001

H-L = Hosmer-Lemeshow.

Discussion

Predicting surgical satisfaction is most important during follow-up in patients with LSCS. Individual parameters such as VAS score for leg pain or numbness, SR, walking distance, JOA score, and NCOS score can provide a measure of prognostic utility; however, this study shows that the combination of these parameters in this ANN model could be used to make more precise predictions. The full structural details of the resulting ANN, which does not require further training, can be saved and used as a software estimator or predictor.

To date, there have been no studies that have analyzed 2-year surgical satisfaction in patients with LSCS based on the ANN model. In this study, in comparison with the l LR model, the ANN model was more accurate in predicting surgical satisfaction (p < 0.001). With regard to accuracy rate (96.9%), it seems that the ANN model may be helpful to clinicians for surgical decision making and for self-assessment in patients with LSCS.

Maximizing the accuracy of the ANN for optimal clinical efficacy is an achievable end point that will be tracked in a follow-up study in which more patients and additional parameters will be used for the input layer of the ANN. Nevertheless, the ANN model introduced in this study is an acceptable test for predicting 2-year surgical satisfaction in patients with LSCS.

In this study, the important variable selected by the ANN was SR. Therefore, the SR parameter can be considered to be an effective variable to predict surgical satisfaction. However, further studies are needed to determine cutoff points of the SR for decision making in the management of this disease.

In this study, validation data sets (n = 42) were performed based on ANNs by using the standardized questionnaire introduced by Weiner et al.12 There are various instruments for measuring performance status or functionality in patients with LBP. The Oswestry Disability Index, the Roland Morris Disability Questionnaire, the NCOS, the JOA score, the Swiss Spinal Stenosis, the Core Outcome Measures Index, the STarT Back Screening Tool, the JOA Back Pain Evaluation Questionnaire, and the Short Form-36 scores are among well-known instruments for measuring functionality in patients with LSCS, and have been used to measure functionality in these patients. However, no existing standard cutoff points exist for the data acquired by these instruments for the assessment of surgical satisfaction in these patients. If standard cutoff points are subsequently made available, such instruments may be used.

A limitation of this study is that we were unable to identify all variables affecting surgical satisfaction in patients with LSCS to enter in the input layer in the ANN model. However, further studies of this model may consider the effect of a more detailed database that contains complications and clinical analysis findings, as well as more detailed results data, and may provide a more complete model. Second, this study is limited by its retrospective nature; we could not evaluate outcome measures. The third limitation is the comparatively small number of patients, especially in validation group. Finally, studies are needed to investigate the differences between long- and short-term follow-up of surgical satisfaction in an ANN model of these patients.

Conclusions

The study findings show that 2-year surgical satisfaction can be readily predictable by using the ANNs developed for use in clinical environments and that this model is more accurate compared with an LR model.

Acknowledgment

We thank the staff of the Neurosurgery Unit at Imam-Hossain Hospital, Tehran, Iran.

Disclosure

The authors declare that they have no competing interests.

Author contributions to the study and manuscript preparation include the following. Conception and design: Azimi. Acquisition of data: Azimi. Analysis and interpretation of data: Azimi. Drafting the article: Azimi. Critically revising the article: all authors. Reviewed submitted version of manuscript: all authors. Approved the final version of the manuscript on behalf of all authors: Azimi. Study supervision: Benzel.

This article contains some figures that are displayed in color online but in black-and-white in the print edition.

References

  • 1.

    Azimi PMohammadi HRMontazeri A: An outcome measure of functionality and pain in patients with lumbar disc herniation: a validation study of the Japanese Orthopedic Association (JOA) score. J Orthop Sci 17:3413452012

  • 2.

    Azimi PMohammadi HRMontazeri A: An outcome measure of functionality in patients with lumber spinal stenosis: a validation study of the Iranian version of Neurogenic Claudication Outcome Score (NCOS). BMC Neurol 12:1012012

  • 3.

    Cross SSHarrison RFKennedy RL: Introduction to neural networks. Lancet 346:107510791995

  • 4.

    Gevirtz C: Update on treatment of lumbar spinal stenosis. Part 1: defining the problem, diagnosis, and appropriate imaging. Topics in Pain Management 25:152010

  • 5.

    Hamanishi CMatukura NFujita MTomihara MTanaka S: Cross-sectional area of the stenotic lumbar dural tube measured from the transverse views of magnetic resonance imaging. J Spinal Disord 7:3883931994

  • 6.

    Laurencin CTLipson SJSenatus PBotchwey EJones TRKoris M: The stenosis ratio: a new tool for the diagnosis of degenerative spinal stenosis. Int J Surg Investig 1:1271311999

  • 7.

    Li YCLiu LChiu WTJian WS: Neural network modeling for surgical decisions on traumatic brain injury patients. Int J Med Inform 57:192000

  • 8.

    Markman JDGaud KG: Lumbar spinal stenosis in older adults: current understanding and future directions. Clin Geriatr Med 24:369388viii2008

  • 9.

    Patel JLGoyal RK: Applications of artificial neural networks in medical science. Curr Clin Pharmacol 2:2172262007

  • 10.

    Price DDMcGrath PARafii ABuckingham B: The validation of visual analogue scales as ratio scale measures for chronic and experimental pain. Pain 17:45561983

  • 11.

    Rughani AIDumont TMLu ZBongard JHorgan MAPenar PL: Use of an artificial neural network to predict head injury outcome. Clinical article. J Neurosurg 113:5855902010

  • 12.

    Weiner BKPatel NMWalker MA: Outcomes of decompression for lumbar spinal canal stenosis based upon preoperative radiographic severity. J Orthop Surg 2:32007

If the inline PDF is not rendering correctly, you can download the PDF file here.

Article Information

Address correspondence to: Parisa Azimi, M.D., Functional Neurosurgery Research Center of Shohada Tajrish Hospital, Shahid Beheshti University of Medical Sciences, Sharadari St., Tajrish Square, Tehran 1989934148, Iran. email: parisa.azimi@gmail.com.

Please include this information when citing this paper: published online January 17, 2014; DOI: 10.3171/2013.12.SPINE13674.

© AANS, except where prohibited by US copyright law.

Headings

Figures

  • View in gallery

    Schematic diagram of the ANN.

  • View in gallery

    Chart showing Hamanishi technique to measure cross-sectional area on axial MRI slices.

  • View in gallery

    Diagram showing ANN output, with insets for each layer—output figure was generated by PASW Statistics 18 (version 18; SPSS, Inc., 2009). Input layer (left-hand group): bias = input layer bias; VAS.N = VAS score of leg numbness; distance = walking distance; VAS.P = VAS score of leg pain. Hidden (H) layer (center group): H (1:1), H (1:2), and H (1:3); bias = hidden layer bias. Output layer (right-hand group): satisfied = Y denotes satisfaction; satisfied = N denotes dissatisfaction at 2-year follow-up.

References

1.

Azimi PMohammadi HRMontazeri A: An outcome measure of functionality and pain in patients with lumbar disc herniation: a validation study of the Japanese Orthopedic Association (JOA) score. J Orthop Sci 17:3413452012

2.

Azimi PMohammadi HRMontazeri A: An outcome measure of functionality in patients with lumber spinal stenosis: a validation study of the Iranian version of Neurogenic Claudication Outcome Score (NCOS). BMC Neurol 12:1012012

3.

Cross SSHarrison RFKennedy RL: Introduction to neural networks. Lancet 346:107510791995

4.

Gevirtz C: Update on treatment of lumbar spinal stenosis. Part 1: defining the problem, diagnosis, and appropriate imaging. Topics in Pain Management 25:152010

5.

Hamanishi CMatukura NFujita MTomihara MTanaka S: Cross-sectional area of the stenotic lumbar dural tube measured from the transverse views of magnetic resonance imaging. J Spinal Disord 7:3883931994

6.

Laurencin CTLipson SJSenatus PBotchwey EJones TRKoris M: The stenosis ratio: a new tool for the diagnosis of degenerative spinal stenosis. Int J Surg Investig 1:1271311999

7.

Li YCLiu LChiu WTJian WS: Neural network modeling for surgical decisions on traumatic brain injury patients. Int J Med Inform 57:192000

8.

Markman JDGaud KG: Lumbar spinal stenosis in older adults: current understanding and future directions. Clin Geriatr Med 24:369388viii2008

9.

Patel JLGoyal RK: Applications of artificial neural networks in medical science. Curr Clin Pharmacol 2:2172262007

10.

Price DDMcGrath PARafii ABuckingham B: The validation of visual analogue scales as ratio scale measures for chronic and experimental pain. Pain 17:45561983

11.

Rughani AIDumont TMLu ZBongard JHorgan MAPenar PL: Use of an artificial neural network to predict head injury outcome. Clinical article. J Neurosurg 113:5855902010

12.

Weiner BKPatel NMWalker MA: Outcomes of decompression for lumbar spinal canal stenosis based upon preoperative radiographic severity. J Orthop Surg 2:32007

TrendMD

Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 149 149 47
PDF Downloads 113 113 17
EPUB Downloads 0 0 0

PubMed

Google Scholar