Inadequacy of 3-month Oswestry Disability Index outcome for assessing individual longer-term patient experience after lumbar spine surgery

Full access

OBJECTIVE

Prospective longitudinal outcomes registries are at the center of evidence-driven health care reform. Obtaining real-world outcomes data at 12 months can be costly and challenging. In the present study, the authors analyzed whether 3-month outcome measurements sufficiently represent 12-month outcomes for patients with degenerative lumbar disease undergoing surgery.

METHODS

Data from 3073 patients undergoing elective spine surgery for degenerative lumbar disease were entered into a prospective multicenter registry (N2QOD). Baseline, 3-month, and 12-month follow-up Oswestry Disability Index (ODI) scores were recorded. The absolute differences between actual 12- and 3-month ODI scores was evaluated. Additionally, the authors analyzed the absolute difference between actual 12-month ODI scores and a model-predicted 12-month ODI score (the model used patients' baseline characteristics and actual 3-month scores). The minimal clinically important difference (MCID) for ODI of 12.8 points and the substantial clinical benefit (SCB) for ODI of 18.8 points were used based on the previously published values. The concordance rate of achieving MCID and SCB for ODI at 3-and 12-months was computed.

RESULTS

The 3-month ODI scores differed from 12-month scores by an absolute difference of 11.9 ± 10.8, and predictive modeling estimations of 12-month ODI scores differed from actual 12-month scores by a mean (± SD) of 10.7 ± 9.0 points (p = 0.001). Sixty-four percent of patients (n = 1982) achieved an MCID for ODI at 3 months in comparison with 67% of patients (n = 2088) by 12 months; 51% (n = 1731) and 61% (n = 1860) of patients achieved SCB for ODI at 3 months and 12 months, respectively. Almost 20% of patients had ODI scores that varied at least 20 points (the point span of an ODI functional category) between actual 3- and 12-month values. In the aggregate analysis of achieving MCID, 77% of patients were concordant and 23% were discordant in achieving or not achieving MCID at 3 and 12 months. The discordance rates of achieving or not achieving MCID for ODI were in the range of 19% to 27% for all diagnoses and treatments (decompression with and without fusion). The positive and negative predictive value of 3-months ODI to predict 12-month ODI was 86% and 60% for MCID and 82% and 67% for SCB.

CONCLUSIONS

Based on their findings, the authors conclude the following: 1) Predictive methods for functional outcome based on early patient experience (i.e., baseline and/or 3-month data) should be used to help evaluate the effectiveness of procedures in patient populations, rather than serving as a proxy for long-term individual patient experience. 2) Prospective longitudinal registries need to span at least 12 months to determine the effectiveness of spine care at the individual patient and practitioner level.

ABBREVIATIONSMCID = minimal clinically important difference; N2QOD = National Neurosurgery Quality and Outcome Database; ODI = Oswestry Disability Index; PRO patient-reported outcome; SCB = substantial clinical benefit.

OBJECTIVE

Prospective longitudinal outcomes registries are at the center of evidence-driven health care reform. Obtaining real-world outcomes data at 12 months can be costly and challenging. In the present study, the authors analyzed whether 3-month outcome measurements sufficiently represent 12-month outcomes for patients with degenerative lumbar disease undergoing surgery.

METHODS

Data from 3073 patients undergoing elective spine surgery for degenerative lumbar disease were entered into a prospective multicenter registry (N2QOD). Baseline, 3-month, and 12-month follow-up Oswestry Disability Index (ODI) scores were recorded. The absolute differences between actual 12- and 3-month ODI scores was evaluated. Additionally, the authors analyzed the absolute difference between actual 12-month ODI scores and a model-predicted 12-month ODI score (the model used patients' baseline characteristics and actual 3-month scores). The minimal clinically important difference (MCID) for ODI of 12.8 points and the substantial clinical benefit (SCB) for ODI of 18.8 points were used based on the previously published values. The concordance rate of achieving MCID and SCB for ODI at 3-and 12-months was computed.

RESULTS

The 3-month ODI scores differed from 12-month scores by an absolute difference of 11.9 ± 10.8, and predictive modeling estimations of 12-month ODI scores differed from actual 12-month scores by a mean (± SD) of 10.7 ± 9.0 points (p = 0.001). Sixty-four percent of patients (n = 1982) achieved an MCID for ODI at 3 months in comparison with 67% of patients (n = 2088) by 12 months; 51% (n = 1731) and 61% (n = 1860) of patients achieved SCB for ODI at 3 months and 12 months, respectively. Almost 20% of patients had ODI scores that varied at least 20 points (the point span of an ODI functional category) between actual 3- and 12-month values. In the aggregate analysis of achieving MCID, 77% of patients were concordant and 23% were discordant in achieving or not achieving MCID at 3 and 12 months. The discordance rates of achieving or not achieving MCID for ODI were in the range of 19% to 27% for all diagnoses and treatments (decompression with and without fusion). The positive and negative predictive value of 3-months ODI to predict 12-month ODI was 86% and 60% for MCID and 82% and 67% for SCB.

CONCLUSIONS

Based on their findings, the authors conclude the following: 1) Predictive methods for functional outcome based on early patient experience (i.e., baseline and/or 3-month data) should be used to help evaluate the effectiveness of procedures in patient populations, rather than serving as a proxy for long-term individual patient experience. 2) Prospective longitudinal registries need to span at least 12 months to determine the effectiveness of spine care at the individual patient and practitioner level.

Current growth in medical costs is unsustainable, and many health care observers believe that health care quality is inconsistent. In this environment, “value-based” reforms are being adopted by most stakeholders to help achieve sustainability of the current US health care system.6,12,30,33–35,37 Safety, effectiveness, patient-experience, and cost of care are the central components of the health care value equation. The value of medical services is generally described at both population and individual levels. The former allows for broad statements about the relative benefit of specific therapeutic interventions; the latter allows for assessments of individual practitioner, group or hospital performance, and individual patient experience.

Most health care stakeholders have publically promoted prospective registries that measure both the safety and effectiveness of care as the preferred method by which to identify health care waste and low-value care.6,36 Robust data collection standards create large sample sizes and allow appropriate risk adjustment, which enhances our understanding of the real-world effectiveness of care.3,4,43,47,49 Participation in high-quality patient care registries, however, can be logistically and financially challenging.21,36,44,45

Perhaps nowhere are the challenges associated with robust clinical registry programs more apparent than in the collection of patient-reported outcomes (PROs). PRO metrics are rapidly becoming vital components of comprehensive, prospective registry data collection systems.15,28,50 In particular, the longitudinal acquisition of PROs has been deemed essential for assessing the sustainability of treatment effects in asset-intense clinical fields such as spine surgery.3,5,7,43,47,49 Furthermore, PROs may be more reflective of underlying health status than physician reporting.27 The use of PROs, however, introduces complexities related to cost, loss to follow-up, and responder burden in obtaining data at multiple end points.9,29 In light of the expense and difficulty associated with obtaining real-world PRO data, it has been suggested that short-term outcomes data might, in certain circumstances, serve as an adequate surrogate for long-term data.25,32,39,43 Given the logistical challenges and financial restraints of collecting long-term patient-reported data at a scalable level, it is vital to understand how well short-term outcomes represent long-term outcomes for both patient populations and individual patients.

In this analysis, we used a large national spine care outcomes database to help determine whether 3-month Oswestry Disability Index (ODI) scores can be used as a reliable proxy for long-term (i.e., 12-month) follow-up status in the identification of effective versus noneffective care for degenerative lumbar disease after surgery. Specifically, we asked if 3-month ODI scores adequately determine 12-month ODI scores regardless of pathology at the individual patient level. We set out to evaluate this question by doing the following: 1) examining the absolute difference between 3- and 12-month ODI scores, 2) adding 3-month ODI scores as an additional variable to baseline preoperative patient characteristics in a multivariate 12-month predictive model, and 3) determining if positive or negative responses to surgery (as measured by minimal clinically important difference [MCID] and substantial clinical benefit [SCB]) at 3 months reliably predicts patient clinical status at 12 months.

Methods

Data from patients undergoing elective spine surgery for degenerative lumbar disease were entered into the prospective multicenter National Neurosurgery Quality and Outcome Database (N2QOD) registry over a 2-year period. The N2QOD is a prospective observational registry designed to establish risk-adjusted expected morbidity rates and 1-year outcomes for the most common lumbar surgical procedures performed by spine surgeons (see Appendix for a list of sites).7 Patient demographics, history, clinical features, surgery details, postoperative complications, readmissions, and PROs were longitudinally collected. Baseline, 3-month, and 12-month postoperative pain, disability, and quality of life were assessed via phone interview by an independent data coordinator not involved with clinical care. Validated questionnaires are used to collect data on the following outcome measures: 1) pain—numeric rating scale (NRS) for back and leg pain;2 2) disease-specific physical disability—Oswestry Disability Index (ODI);19,23 and 3) health-related quality of life–the EQ-5D questionnaire.8,18 For the purposes of this analysis, we enrolled all lumbar surgery cases into the N2QOD database from August 2012 to October 2013 with complete 3-months and 1-year follow-up information spanning from August 2013 to October 2014. We chose to compare the 3- and 12-month outcomes for the back pain–related disability metric ODI, as several studies have shown that ODI is the most relevant and important determinant of pain and disability outcomes following lumbar spine surgery.17,19,28,42

Clinical improvement for disability was defined using the concepts of MCID and SCB. MCID represents a critical threshold clinical change compared with the baseline that is considered to be meaningful (noticeable) improvement to the patient. SCB identifies the change in the outcome that is not only clinically perceptible, but also considered a significant improvement from the patients' perspective.26,53 Previous analyses of these clinical improvement measurement tools have compared the change in score for a PRO following the intervention to another externally validated measure of outcomes or anchors, such as perceived improvement (health transition index of the SF-36)14 or satisfaction (North American Spine Society satisfaction questionnaire)40 following the procedure in question.10,13,26,31,41 As reported in the literature, for MCID threshold calculation, the health transition index responses of “much better” or “somewhat better” are classified as responders, whereas “about the same,” “unchanged,” and “worse” are categorized as nonresponders; the satisfaction index responses of “surgery met their expectation” and “I did not improve as much as I had hoped but I would undergo same surgery for same results” are classified as responders and other responses as nonresponders. For SCB threshold calculation, health transition index and satisfaction index responses of “much better” and “mostly satisfied” versus “about the same” and “unsure” are classified as responder versus nonresponders, respectively. The present study cohort included patients who underwent various lumbar surgeries. As such, the MCID threshold for the ODI reported by Copay et al.13 of 12.8 points, applicable to all lumbar surgery for degenerative spine pathology, was used. The SCB threshold for ODI of 18.8 points for lumbar arthrodesis surgery, reported by Glassman et al., was also used in our study.26,41

Statistical Analysis

Median, quartiles, mean and standard deviation for continuous variables, and frequency for discrete data were calculated for patient demographics. The change in the ODI score from baseline to 3 months and baseline to 12 months was calculated.

Multivariable proportional odds ordinal regression analyses were used to build a predictive model. All the patient characteristics, including age, sex, race, smoking, employment status, revision surgery, number of levels, ASA (American Society of Anesthesiologists) grade, body mass index, arthritis, diabetes, estimated blood loss, 90-day morbidity, lumbar fusion, anterior or posterior approach, and preoperative ODI scores along with 3-month ODI scores, were included to build the model. The 12-month ODI score was used as the outcome variable. The predictive model was used to estimate the 12-month ODI scores. To examine whether 3-month ODI scores can adequately predict 12-month scores, we calculated the absolute differences between 12- and 3-month ODI scores; we also calculated the absolute differences between 12-month and model-predicted 12-month ODI scores. A paired t-test was used to compare the absolute difference in the disability scores at 3-month and 12-month follow-up times.

The concordance rates of achieving or not achieving MCID and SCB for ODI scores at 3 and 12 months were calculated. Cases in which MCID and SCB for ODI scores at 3 months were either achieved or not achieved and in which that status was maintained at 12 months were defined as concordant. Cases in which MCID or SCB status changed between 3 and 12 months were defined as discordant. McNemar's test was used to compare the concordance and discordance rate of achieving or not achieving MCID for ODI at 3 and 12 months. A separate multivariable logistic regression analysis was conducted to analyze the factors associated with discordance in achieving or not achieving MCID for ODI. All the analyses were tested at 0.05 level of significance, and the analysis was performed using the SPSS version 20 (IBM, Chicago, Inc) and R 3.1.2 (www.R-project.org).

Results

Patient Demographics

A total of 3073 patients undergoing elective lumbar surgery for stenosis, lumbar disc herniation, revision same-level disc herniation, or spondylolisthesis were included in the study. Mean age ± SD of patients in the cohort was 60.20 ± 13.5 years, and 1557 (51%) were male. Table 1 summarizes the patient demographic characteristics. Principal diagnoses in descending order of frequency were lumbar stenosis (37.3%), lumbar disc herniation (33.6%), lumbar spondylolisthesis (22.7%), and recurrent disc herniation (6.4%). Thirty-seven percent (1138) of patients underwent lumbar fusion surgery and 10.3% (318) patients had prior lumbar surgery.

TABLE 1.

Summary of demographic characteristics

CharacteristicTotal No. of PatientsNo. (% or mean) or Q1, Q2, Q3*
Age306949.8, 60.9, 69.0
Sex3073
  Female1515 (49)
  Male1557 (51)
BMI305925.8, 29.3, 33.8
Hispanic3049
  No2988 (98)
  Yes61 (2)
Race3073
  White2787 (91)
  Black202 (7)
  Other84 (3)
Prior surgery3069
  Primary2751 (90)
  Revision318 (10)
Smoker3025
  No1582 (52)
  Former541 (18)
  Current902 (30)
Diabetes3073
  No2528 (82)
  Yes545 (18)
CAD3072
  No2663 (87)
  Yes409 (13)
Osteoporosis3073
  No2884 (94)
  Yes189 (6)
Anxiety3073
  No2600 (85)
  Yes473 (15)
Depression3072
  No2447 (80)
  Yes625 (20)
Dominant symptom3072
  Leg pain1159 (38)
  Back pain661 (22)
  Back & leg pain equal1252 (41)
Motor deficit3063
  No2160 (71)
  Yes903 (29)
ASA score3065
  1180 (6)
  21673 (55)
  3/41212 (39)
Duration of symptoms3045
  <3 mos319 (10)
  >3 mos2726 (90)
Diagnosis3073
  Stenosis1148 (37)
  Herniated disc1032 (34)
  Recurrent herniated disc196 (6)
  Spondylolisthesis697 (23)
No. of Levels2939
  11139 (39)
  21239 (42)
  3460 (16)
  4101 (3)
Arthrodesis3065
  No1927 (63)
  Yes1138 (37)
Interbody graft3062
  No2129 (70)
  Yes933 (30)
Insurance3066
  Medicare1848 (60)
  Private/others1218 (40)
Baseline PRO scores3073
  ODI38, 50, 60 (49 ± 17)
  EQ-5D42, 77, 88 (66 ± 26)
  NRS–back pain5.0, 7.0, 9.0 (6.4 ± 2.9)
  NRS–leg pain5.0, 8.0, 9.0 (6.8 ± 2.7)

ASA = American Society of Anesthesiologists; BMI = body mass index; CAD = coronary artery disease; NRS = numeric rating scale.

Q1 = lower quartile, Q2 = median quartile, and Q3 = upper quartile. The lower quartile represents the cut-off point at lowest 25% of data, the median quartile represents the median, and the upper quartile represents the cut-off point where 75% of the data are smaller than Q3 and 25% larger than Q3.

Outcomes

Mean Difference Between Absolute 3-Month ODI Versus 12-Month ODI (Disability) Scores

The mean baseline ODI scores were 49.6 ± 16.6; 3-month scores were 27.3 ± 19.6, and 12-month scores were 24.3 ± 20.6. A significant improvement in ODI scores was found at the 3-month and 12-month follow-up compared with the baseline: 48.4 ± 16.4 vs 27.5 ± 19.5 (p < 0.0001) and 48.4 ± 16.4 vs 25.2 ± 20.3 (p < 0.0001) for stenosis; 48.7 ± 16.9 vs 23.5 ± 19.3 (p < 0.0001) and 48.7 ± 16.9 vs 21.0 ± 20.2 (p < 0.0001) for disc herniation; 55.9 ± 14.7 vs 37.4 ± 19.7 (p < 0.0001) and 55.9 ± 14.7 vs 37.9 ± 21.2 (p < 0.0001) for recurrent disc herniation; and 49.5 ± 16.3 vs 29.3 ± 18 (p < 0.0001) and 49.5 ± 16.3 vs 24.1 ± 20.2 (p < 0.0001) for spondylolisthesis. The 3-month ODI scores differed from 12-month scores for all diagnoses by an absolute mean difference of 11.9 ± 10.8 points. Table 2 summarizes the difference between the actual 3-month and the actual 12-month ODI scores.

TABLE 2.

Summary of absolute difference between 12- and 3-months ODI scores and the absolute difference between the model-predicted 12-month scores and actual 12-month scores*

VariableDifference Btwn 3-mo Actual Score & 12-mo Actual ScoreDifference Btwn 12-mo Actual Score & 12-mo Model-Predicted Scorep Value
Q1Q2Q3Mean ± SDQ1Q2Q3Mean ± SD
All diagnoses4.08.916.211.9 ± 10.84.18.314.610.7 ± 9.0<0.001
Stenosis4.09.118.012.5 ± 11.54.48.915.311.3 ± 9.3<0.001
Disc3.98.014.110.7 ± 10.23.97.513.19.7 ± 8.4<0.001
Recurrent4.09.116.412.2 ± 11.44.18.814.511.2 ± 9.90.016
Spondylolisthesis4.410.018.012.1 ± 10.04.18.415.510.6 ± 8.6<0.001

The predictive model was built after adjusting for all of the baseline patient and disease characteristics in addition to adding in the actual 3-month ODI score. The model-predicted score that accounts for baseline PRO scores, patient characteristics, and 3-month actual scores not surprisingly better predicts the actual 12-month scores.

Q1= lower quartile, Q2 = median quartile, and Q3 = upper quartile. Lower quartile represents cut-off point at lowest 25% of data, the median quartile represents the median, and the upper quartile represents the cut-off point where 75% of the data are smaller than Q3 and 25% larger than Q3.

Augmented Predictive Model

In an attempt to improve our extrapolative capability with respect to the direction and extent of patient responses to treatment, 3-month ODI values were introduced as an additional input to baseline patient characteristics in our 12-month ODI multivariable predictive model. The model-predicted 12-month ODI scores differed from actual 12-month ODI by 10.7 ± 9.0 points. Although the model-estimated 12-month ODI scores were statistically significantly better than the 3-month ODI scores alone in assessing actual 12-month ODI scores for all diagnoses (p = 0.001) and for individual diagnoses (p = 0.001 to 0.016), these methods yielded estimates that were effectively similar from a clinical perspective.

The data in Table 2 demonstrates that the differences for predicted and actual scores are, on average, approximately 10 points on a 0–100 scale for ODI. However, in 25% of patients, both the augmented predictive model and absolute 3-month scores yielded estimates that varied from actual 12-month scores in excess of 14.6 and 16.2 points, respectively. Almost 1 in 5 (19.4%) patients had scores that varied at least 20 points (the span of an ODI functional category) between actual 3- and 12-month ODI scores; 7% of patients had score that varied by 30 or more points. The augmented predictive model yielded slightly better results: 13.6% and 4.1% of patients had predicted scores that varied 20 (or greater) and 30 (or greater) ODI points, respectively, from the actual 12-month scores. Figure 1 is a scatter plot representing individual patients' 3-and 12-month ODI scores. The correlation coefficient is 0.7 (indicating moderate general agreement); variability from 3- to 12-month scores is represented by points (individual patients) at varying distances from the diagonal line of equivalence.

FIG. 1.
FIG. 1.

Scatter plot demonstrating the correlation between 3-month ODI scores and 12-month ODI scores. The correlation coefficient is 0.70. Each dot represents an individual patient, the closer the dots are to the reference line the better 3-month score correlates with 12-month score. The dots that are away from the reference line indicate outlier individuals with poor agreement between the 3-month and 12-month score.

The multivariable prediction model correctly predicted the direction of change for ODI scores from baseline to 12 months for 89.3% of the patients. Similarly, the direction of change from baseline to 3-month ODI scores correctly predicted the continued direction of change to 12 months for 85.5% of patients.

Concordance Between 3-Month and 12-Month Scores

Seventy-one percent of all patients examined (2188 of 3073) achieved an MCID for ODI at 12 months and 61% (1860) achieved an SCB for ODI at 12 months. Sixty-eight percent of patients (2097) achieved an MCID for ODI at 3 months and 51% (1731) achieved an SCB for ODI at 3 months. Eighty-six percent (1795 of 2097) maintained the MCID threshold at 12 months and 14% subsequently worsened and lost the MCID threshold at 12 months. Additionally, 40% of (393 of 976) patients who failed to achieve an MCID threshold at 3 months went on to improve and achieve an MCID at 12 months.

In aggregate analysis, 77% (2378 of 3073) of all patients examined were concordant in achieving (or not achieving) the MCID threshold at 3 months and 12 months postoperatively, while 23% of patients were discordant. Similarly, 75% of patients were concordant in achieving an SCB at 3 and 12 months, while 25% (695 of 3073) were discordant. To reiterate, discordance is defined as either loss or gain of MCID/SCB between 3 and 12 months. Comparable patterns of discordance were observed in all diagnoses examined. The discordance rate of achieving or not achieving an MCID and SCB for ODI at 3 months and 12 months, respectively, for each diagnosis was as follows: lumbar stenosis, 24% and 26%; disc herniation, 19% and 21%; recurrent disc herniation, 27% and 22%; and spondylolisthesis, 26% and 27%. The discordance rates of achieving or not achieving MCID for ODI were in the range of 25% and 20%, respectively, in patients undergoing decompression with and without fusion.

Table 3 summarizes the number of patients achieving or not achieving an MCID for ODI at 3 and 12 months stratified by diagnosis. Table 4 summarizes the number of patients achieving or not achieving an SCB for ODI at 3 and 12 months stratified by diagnosis. Figure 2 provides a scatter plot comparing baseline and 12-month disability scores (ODI), underscoring a substantial extent of variation in achieving an MCID and SCB for ODI score at 3 months and 12 months.

TABLE 3.

MCID/SCB for ODI rates at 12 months postoperatively for patients in whom an MCID/SCB was achieved at 3 months*

DiagnosisMCID/SCB Achieved at 3 MosMCID/SCB Maintained at 12 Mos (concordant)MCID/SCB Not Maintained at 12 Mos (discordant)
MCID
  All diagnoses68% (2097/3073)86% (1795)14% (302)
    Fusion65% (745/1146)84% (626)16% (119)
    Nonfusion70% (1352/1927)86% (1169)14% (183)
  Lumbar stenosis66% (763/1148)84% (638)16% (125)
    Fusion66% (208/316)86% (178)14% (30)
    Nonfusion67% (555/832)83% (460)17% (95)
  Disc herniation73% (755/1032)89% (671)11% (84)
    Fusion58% (38/65)92% (35)8% (3)
    Nonfusion74% (717/967)89% (636)11% (81)
  Recurrent disc herniation62% (122/196)74% (90)26% (32)
    Fusion64% (102/159)72% (73)28% (29)
    Nonfusion54% (20/37)85% (17)5% (3)
  Spondylolisthesis66% (457/697)87% (396)13% (61)
    Fusion66% (397/606)86% (340)14% (57)
    Nonfusion66% (60/91)93% (56)7% (4)
SCB
  All diagnoses56% (1731/3073)82% (1419)18% (312)
    Fusion52% (595/1146)81% (480)19% (115)
    Nonfusion59% (1136/1927)83% (939)17% (197)
  Lumbar stenosis54% (621/1148)79% (491)21% (130)
    Fusion53% (169/316)82% (139)18% (30)
    Nonfusion54% (452/832)78% (352)22% (100)
  Disc herniation63% (647/1032)85% (552)15% (95)
    Fusion51% (33/65)76% (25)24% (8)
    Nonfusion63% (614/967)86% (527)14% (87)
  Recurrent disc herniation55% (107/196)82% (88)18% (20)
    Fusion51% (81/159)68% (55)32% (26)
    Nonfusion43% (16/37)75% (12)25% (4)
  Spondylolisthesis53% (366/697)84% (309)16% (57)
    Fusion51% (312/606)84% (261)16% (51)
    Nonfusion59% (54/91)89% (48)11% (6)

Values are reported as the percentage (number) of patients.

TABLE 4.

MCID/SCB for ODI at 12 months postoperatively for patients in whom an MCID/SCB was not achieved at 3 months*

DiagnosisMCID/SCB Not Achieved at 3 MosMCID/SCB Still Not Achieved at 12 Mos (concordant)MCID/SCB Achieved at 12 Mos (discordant)
MCID
  All diagnoses32% (976/3073)60% (583)40% (393)
    Fusion35% (401/1146)57% (227)43% (174)
    Nonfusion30% (575/1927)62% (356)38% (219)
  Lumbar stenosis34% (385/1148)62% (239)38% (146)
    Fusion34% (108/316)66% (71)34% (37)
    Nonfusion33% (277/832)61% (168)39% (109)
  Disc herniation27% (277/1032)61% (169)39% (108)
    Fusion42% (27/65)52% (14)48% (13)
    Nonfusion26% (250/967)62% (155)38% (95)
  Recurrent disc herniation38% (74/196)72% (53)28% (21)
    Fusion36% (57/159)67% (38)33% (19)
    Nonfusion46% (17/37)88% (15)12% (2)
  Spondylolisthesis34% (240/697)49% (118)51% (122)
    Fusion34% (209/606)50% (104)50% (105)
    Nonfusion34% (31/91)58% (18)42% (13)
SCB
  All diagnoses44% (1342/3073)67% (901)33% (441)
    Fusion48% (551/1146)65% (356)35% (195)
    Nonfusion41% (791/1927)69% (545)31% (246)
  Lumbar stenosis46% (527/1148)68% (359)32% (168)
    Fusion47% (147/316)69% (101)31% (46)
    Nonfusion46% (380/832)68% (258)32% (122)
  Disc herniation37% (385/1032)69% (265)31% (120)
    Fusion49% (32/65)72% (23)28% (9)
    Nonfusion37% (353/967)69% (242)31% (111)
  Recurrent disc herniation45% (88/196)72% (64)28% (25)
    Fusion49% (78)81% (63)19% (15)
    Nonfusion57% (21/37)19% (4)81% (17)
  Spondylolisthesis47% (331/697)60% (197)40% (134)
    Fusion49% (294/606)57% (169)43% (294)
    Nonfusion41% (37/91)76% (28)24% (9)

Values are reported as the percentage (number) of patients.

FIG. 2.
FIG. 2.

Scatter plot comparing baseline and 12-month ODI disability scores. A substantial amount of variation can be seen in achieving MCID (left) and SCB (right) for ODI scores at 3 months and 12 months after elective surgery for lumbar degenerative disease. The reference line represents the MCID threshold (12.8) and SCB threshold (18.8). All the patients' with dots to the right of the reference line (blue and green dots, left; pink and blue dots, right) represent those who achieved clinical benefit at 12 months. Figure is available in color online only.

Predictive Value of Patient Subgroups

In a separate multivariable logistic regression analysis, controlling for all the baseline patient characteristics, we sought to determine if nondiagnostic subgroups existed for which 3-month scores more or less reliably predicted 12-month scores. With respect to this possibility, higher baseline ODI scores were associated with better concordance between 3- and 12-month values (p = 0.0002). For every 10-unit increment increase in baseline ODI score, the odds of discordance decreased by 13.3% (95% CI 5.0%–20.9%). In contrast, preoperative depression was associated with a higher likelihood of discordance between 3- and 12-month scores (OR 1.7, CI 1.1–2.6, p = 0.012).

In another attempt to evaluate the ability of patient subgroup experiences at 3 months to predict 12-month outcomes, we considered achievement of MCID/SCB thresholds a positive outcome and not achieving MCID/SCB thresholds a negative outcome. Achievement of a MCID/SCB at 3 months has a positive predictive value for continued achievement at 12 months of 86% (MCID) and 82% (SCB). If patients did not achieve an MCID or SCB by 3 months, the negative predictive value for continued nonresponder status at 12 months was observed to be only 60% and 67% for MCID and SCB, respectively. Table 5 demonstrates the positive and negative predictive values of the 3-month disability scores in estimating 12-month outcomes.

TABLE 5.

Positive and negative predictive values of the 3-month ODI scores in estimating 12-month ODI scores

Achieved MCID at 12 Mos (%)Total (%)Achieved SCB at 12 Mos (%)Total (%)
Achieved MCID at 3 MosYesNoAchieved SCB at 3 MosYesNo
Yes1795 (58)302 (10)2097 (68)Yes1419 (46)312 (10)1731 (56)
No393 (13)583 (19)976 (32)No441 (15)901 (29)1342 (44)
Total2188 (71)885 (29)3073Total1860 (61)1213 (39)3073
PPV (1795/2097) = 86%

NPV (583/976) = 60%
PPV (1419/1731) = 82%

NPV (901/1342) = 67%

NPV = negative predictive value; PPV = positive predicative value.

Achieving MCID/SCB at 12 months was used as a reference test.

Discussion

Enhancing health care value requires accurate assessment and effective quality improvement at both the population level and individual patient level. The infrastructure needed to longitudinally track quality of care can be costly and time consuming. Therefore, it is understandable that policymakers and clinicians want to accurately identify high-quality care in the most cost- and time-efficient manner. In this regard, the capabilities of high-quality registries such as N2QOD are being closely observed by payers and policymakers. The implications for all stakeholders will be tremendous, and balancing the need for manageable data acquisition costs with the imperative of accurate clinical guidance is critical. Thus, we set out, in the present analysis, to determine if 3-month outcomes adequately predict 12-month outcomes at the population level and the individual patient level.

Differences Between Predictive Model and Actual 12-Month Scores in Populations Versus Individual Patients

Analysis of our data indicates that patients undergoing elective surgery for degenerative spinal conditions experienced significant improvement in disability at 3 months; that improvement was generally maintained at 12 months. Both the predictive model and absolute 3-month scores differed from actual scores, on average, by approximately 10 points on a 0–100 scale for ODI. Furthermore, the predictive model and absolute 3-month scores reliably predicted the direction of change (approximately 90% and 85% of the time, respectively). Finally, achieving detectable or a significant clinical benefit at 3 months had a positive predictive value over 80% for maintaining the same status at 12 months. Based on these findings, we conclude that our predictive model and 3-month ODI scores are reasonable predictors of the direction and average magnitude of change for patient populations. In that regard, the predictive model and 3-month outcome scores have potential value in a health care policy context, such as in the assessment of the comparative effectiveness of various therapeutic interventions.

However, the data also demonstrates that neither our augmented predictive model (baseline patient characteristic combined with 3-month ODI scores) nor the actual 3-month ODI scores alone could accurately predict 12-month scores for individuals. Figure 1 graphically illustrates clinical outcome variability between 3 and 12 months at the individual patient level (as evidenced by the large number of individuals with ODI scores at a distance from the diagonal line of equivalence). This variability is further supported by the observation that ODI values for approximately 20% of patients differed at least 20 points (the point span of an ODI functional category) between actual 3- and 12-month scores. Furthermore, about 25% of patients undergoing elective spine surgery will experience significant changes in their disability status (i.e., it will get better or worse) between the 3- and 12-month follow-up. This latter observation was consistent regardless of diagnosis or surgical procedure.

From these data, we conclude that our augmented predictive model and 3-month ODI scores are inadequate predictors of the direction and average magnitude of change for individual patients. More precisely, these methods are not well suited to accurately assess the experience of individual patients, nor can they adequately assess the performance of individual providers. They may, however, have significant value in preoperative patient counseling assuming predictions are represented as probabilities and not as a single expected outcome.

Interestingly, discordance was minimized in patients with higher baseline ODI scores. This finding suggests that more severely disabled patients either improve or do not improve by 3 months, and then they maintain that status at longer-term follow-up. In contrast, patients with depression were more likely to have discordant status. In the context of previously published data demonstrating that depression and anxiety strongly influence Health Related Quality of Life outcomes,1,2,16 discordance in 3- and 12-month scores in this subgroup might be associated with fluctuation in patients’ underlying comorbidity. Finally, if patients had a negative 3-month outcome (i.e., they did not achieve MCID or SCB), the predictive value of that outcome for continued nonresponder status at 12 months was only in the range of 60%–67%. This suggests there was greater uncertainty in estimating the direction of 12-month outcomes when patients did not achieve clinical benefit at 3 months.

If one is asking the question, “Is this treatment effective?” then our data support the concept that 3-month outcomes reasonably predict 12-month outcomes, at the population level. However, most patient care registries are not being used for the assessment of relative effectiveness (i.e., replacing randomized controlled trials for comparative effectiveness research) and are rather being directed to evaluations of individual practitioner/group and hospital performance, along with evaluations of individual patient experience.

These data suggest that the actual 12-month PRO scores are indispensable to adequately measure clinical outcome following lumbar spine surgery. Specifically, 3-month ODI does not serve as an adequate metric for assessing individual patient experience after spine surgery. Therefore, despite the obvious practical benefits of solely relying on short-term outcomes assessments,51 providers and payers focusing on the safety and effectiveness of spinal surgery need to be cautious in concluding that early responses to therapy accurately predict long-term outcomes, particularly in light of our observations that substantial proportions of patients will both improve and decline following initial assessments.

All health care stakeholders have an interest in accurately defining, measuring, and promoting value-based, high-quality spine care. In this regard, limitations of current data collection methodologies should not be used as justification to accept inadequate proxies for accurate and reliable outcome data. Rather, our collective focus should be on the creation of novel data collection strategies, HIT (health information technology) interoperability, easier-to-administer outcome metrics, and incentives for patients to participate in the reporting of information related to their individual care experiences.

Study Limitations

There are certain limitations of using the ODI questionnaire that need to be considered. Previous authors have questioned the reliability of several sections of the ODI questionnaire including “Heavy weight lifting” and “Sex life.”20,22,46 However, given the proven overall reliability, validity, ease of administration, high sensitivity and specificity for low-back pain, the ODI is the most commonly used and well-accepted measure of disability for lumbar spine disease. In the present analysis, the decision to compare the 3- and 12-month outcomes using the ODI was based on the premise that the ODI is the most responsive and valid measure of functional outcomes following lumbar spine surgery.17,19,28,42 Although we did not expect patients to report similar responses regarding quality of life (that is, on the EQ-5D), a strong correlation between health-related quality of life (EQ-5D) and ODI has been previously demonstrated.38

Interest in MCID and SCB calculations has arisen from the important realization that statistically significant differences in treatment effects often fail to translate into clinically significant differences from a patient's perspective.11,14,26,41,53 However, despite the practical and intuitive appeal of these measures, the use of MCID and SCB as a proxy for clinical benefit has faced some scrutiny in the spine outcomes literature.24,40,48,52 Their major criticism relates to the lack of objective external criteria to compute the thresholds. In addition, specific SCB thresholds for patients undergoing microdiscectomy and decompression with and without fusion have not been reported in the literature; in our study the SCB threshold reported for lumbar decompression and fusion was used for all the patients.26 Nevertheless, the MCID and SCB are often used to evaluate the relative effectiveness of treatment, but the statistical and clinical utility of these benchmarks is controversial. Clearly, additional studies are needed to validate the threshold values for MCID and SCB and perhaps evaluate more sophisticated metrics to quantify the meaningful clinical benefit.

Conclusions

There are 3 primary purposes for clinical registries: 1) comparing the effectiveness of treatments for a given pathology, 2) assessing individual provider care quality, and 3) evaluating the experience of patients through the continuum of care. In this analysis, 3-month and augmented model–predicted ODI scores were, on average, within 10 points on a 0–100 scale of actual 12-month scores. This places an analysis of the aggregate data within the boundaries of a single ODI category, therefore representing differences that are qualitatively similar. However, on the individual patient level, there can be much greater variation between actual and predicted scores. Furthermore, many patients failing to benefit from surgery by 3-months subsequently do so by 12-months and vice versa. Based on the presents findings, we conclude the following: 1) Predictive methods for functional outcome based on early patient experience (i.e., baseline and/or 3-month data) should be used to help evaluate the effectiveness of procedures in patient populations, rather than serving as a proxy for long-term individual patient experience. 2) Prospective longitudinal registries need to span at least 12 months to determine the effectiveness of spine care at the individual patient and practitioner level.

Acknowledgments

A portion of this work was supported through a grant from the Neurosurgery Research and Education Foundation (NREF).

Appendix

The following institutions comprise the N2QOD Vanguard Sites.

Department of Neurosurgery, Brain and Spine Center, Holland, Michigan (A Kremer); Department of Neurosurgery, University of Tennessee Health Sciences Center, Semmes-Murphey Neuro logic & Spine Institute, Memphis, Tennessee (JM Sorenson); De partment of Neurosurgery, BayCare Clinic Neurological, Green Bay, Wisconsin (WE Griffitt); Department of Neurosurgery, North Shore University Health System, Skokie, Illinois (NY Stadlan); Department of Surgery, East Texas Medical Center, Tyler Neurosurgical, Tyler, Texas (TW Grahm); Department of Neurosurgery, University of Utah, Salt Lake City, Utah (MH Schmidt); Department of Orthopedic Surgery, University of Louisville and the Norton Leatherman Spine Center, Louisville, Kentucky (S Glassman); Springfield Neurologic and Spine Institute, Springfield, Missouri (TB Briggs); North Jersey Brain & Spine Institute, Oradell, New Jersey (RD Vingan); Department of Neurological Surgery, University of California, San Francisco, California (P Mummaneni); Department of Neuro surgery, Atlantic Neurosurgical Specialists, Morristown, New Jersey (J Knightly); Department of Neurological Surgery, Carolina Neurosurgery and Spine Associates and Neurological Institute, Caro linas Healthcare System, Charlotte, North Carolina (AL Asher, MJ McGirt); Department of Orthopedics Surgery and Neurosurgery, Vanderbilt Spine Center, Vanderbilt University Medical Center, Nashville, Tennessee (CJ Devin, JS Cheng); and Division of Neurological Surgery, Barrow Neurological Institute, St. Joseph’s Hospital and Medical Center, Phoenix, Arizona (N Theodore).

References

  • 1

    Abtahi AMBrodke DSLawrence BDZhang CSpiker WR: Association between patient-reported measures of psychological distress and patient satisfaction scores after spine surgery. J Bone Joint Surg Am 97:8248282015

  • 2

    Adogwa OVerla TThompson PPenumaka AKudyba KJohnson K: Affective disorders influence clinical outcomes after revision lumbar surgery in elderly patients with symptomatic adjacent-segment disease, recurrent stenosis, or pseudarthrosis: clinical article. J Neurosurg Spine 21:1531592014

  • 3

    Aebi MGrob D: SSE Spine Tango: a European Spine Registry promoted by the Spine Society of Europe (SSE). Eur Spine J 13:6616622004

  • 4

    Agrawal BMBrooks NPResnick DK: Wisconsin Spine Outcome Study-Pilot: preliminary data. Neurosurg Focus 33:1E152012

  • 5

    Arts DGDe Keizer NFScheffer GJ: Defining and improving data quality in medical registries: a literature review, case study, and generic framework. J Am Med Inform Assoc 9:6006112002

  • 6

    Asher ALDevin CJMroz TFehlings MParker SLMcGirt MJ: Clinical registries and evidence-based care pathways: raising the bar for meaningful measurement and delivery of value-based care. Spine (Phila Pa 1976) 39:22 Suppl 1S136S1382014

  • 7

    Asher ALSperoff TDittus RSParker SLDavies JMSelden N: The National Neurosurgery Quality and Outcomes Database (N2QOD): a collaborative North American outcomes registry to advance value-based spine care. Spine (Phila Pa 1976) 39:22 Suppl 1S106S1162014

  • 8

    Badia XDiaz-Prieto AGorriz MTHerdman MTorrado HFarrero E: Using the EuroQol-5D to measure changes in quality of life 12 months after discharge from an intensive care unit. Intensive Care Med 27:190119072001

  • 9

    Bresnahan BWRundell SD: Including patient-reported outcomes and patient-reported resource-use questionnaires in studies. Acad Radiol 21:112911372014

  • 10

    Carreon LYGlassman SDCampbell MJAnderson PA: Neck Disability Index, Short Form-36 physical component summary, and pain scales for neck and arm pain: the minimum clinically important difference and substantial clinical benefit after cervical spine fusion. Spine J 10:4694742010

  • 11

    Carreon LYSanders JODiab MSucato DJSturm PFGlassman SD: The minimum clinically important difference in Scoliosis Research Society-22 appearance, activity, and pain domains after surgical correction of adolescent idiopathic scoliosis. Spine (Phila Pa 1976) 35:207920832010

  • 12

    Cesta T: Case management insider. The new value-based purchasing efficiency measure: are you ready?. Hosp Case Manag 22:1671702014

  • 13

    Copay AGGlassman SDSubach BRBerven SSchuler TCCarreon LY: Minimum clinically important difference in lumbar spine surgery patients: a choice of methods using the Oswestry Disability Index, Medical Outcomes Study questionnaire Short Form 36, and pain scales. Spine J 8:9689742008

  • 14

    Copay AGSubach BRGlassman SDPolly DW JrSchuler TC: Understanding the minimum clinically important difference: a review of concepts and methods. Spine J 7:5415462007

  • 15

    D’Arcy LPRich EC: From comparative effectiveness research to patient-centered outcomes research: policy history and future directions. Neurosurg Focus 33:1E72012

  • 16

    Daubs MDNorvell DCMcGuire RMolinari RHermsmeyer JTFourney DR: Fusion versus nonoperative care for chronic low back pain: do psychological factors affect outcomes?. Spine (Phila Pa 1976) 36:21 SupplS96S1092011

  • 17

    DeVine JNorvell DCEcker EFourney DRVaccaro AWang J: Evaluating the correlation and responsiveness of patient-reported pain with function and quality-of-life outcomes after spine surgery. Spine (Phila Pa 1976) 36:21 SupplS69S742011

  • 18

    EuroQol Group: EuroQol—a new facility for the measurement of health-related quality of life. Health Policy 16:1992081990

  • 19

    Fairbank JCCouper JDavies JBO’Brien JP: The Oswestry Low Back Pain Disability Questionnaire. Physiotherapy 66:2712731980

  • 20

    Fairbank JCPynsent PB: The Oswestry Disability Index. Spine (Phila Pa 1976) 25:294029522000

  • 21

    Franklin PDHarrold LAyers DC: Incorporating patient-reported outcomes in total joint arthroplasty registries: challenges and opportunities. Clin Orthop Relat Res 471:348234882013

  • 22

    Fritz JMIrrgang JJ: A comparison of a modified Oswestry Low Back Pain Disability Questionnaire and the Quebec Back Pain Disability Scale. Phys Ther 81:7767882001

  • 23

    Gallagher EJLiebman MBijur PE: Prospective validation of clinically important changes in pain severity measured on a visual analog scale. Ann Emerg Med 38:6336382001

  • 24

    Gatchel RJMayer TG: Testing minimal clinically important difference: consensus or conundrum?. Spine J 10:3213272010

  • 25

    Ghogawala ZShaffrey CIAsher ALHeary RFLogvinenko TMalhotra NR: The efficacy of lumbar discectomy and single-level fusion for spondylolisthesis: results from the NeuroPoint-SD registry: clinical article. J Neurosurg Spine 19:5555632013

  • 26

    Glassman SDCopay AGBerven SHPolly DWSubach BRCarreon LY: Defining substantial clinical benefit following lumbar spine arthrodesis. J Bone Joint Surg Am 90:183918472008

  • 27

    Gliklich REDreyer NA: Registries for Evaluating Patient Outcomes: A User’s Guide Rockville, MDAgency for Healthcare Research and Quality2007

  • 28

    Godil SSParker SLZuckerman SLMendenhall SKGlassman SDMcGirt MJ: Accurately measuring the quality and effectiveness of lumbar surgery in registry efforts: determining the most valid and responsive instruments. Spine J 14:288528912014

  • 29

    Harris PATaylor RThielke RPayne JGonzalez NConde JG: Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42:3773812009

  • 30

    Huntoon KMMcCluney CJScannell CAWiley EABruno RAndrews A: Healthcare reform and the next generation: United States medical student attitudes toward the Patient Protection and Affordable Care Act. PLoS One 6:e235572011

  • 31

    Jaeschke RSinger JGuyatt GH: Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials 10:4074151989

  • 32

    Jarvik JGComstock BABresnahan BWNedeljkovic SSNerenz DRBauer Z: Study protocol: the Back Pain Outcomes using Longitudinal Data (BOLD) registry. BMC Musculoskelet Disord 13:642012

  • 33

    Larkin DJSwanson RCFuller SCortese DA: The Affordable Care Act: a case study for understanding and applying complexity concepts to health care reform. J Eval Clin Pract [epub ahead of print]2014

  • 34

    Lohr KNSchroeder SA: A strategy for quality assurance in Medicare. N Engl J Med 322:7077121990

  • 35

    May EL: Financial planning in a value-based world. Healthc Exec 29:10121416182014

  • 36

    McGirt MJParker SLAsher ALNorvell DSherry NDevin CJ: Role of prospective registries in defining the value and effectiveness of spine care. Spine (Phila Pa 1976) 39:22 Suppl 1S117S1282014

  • 37

    Centers for Medicare and Medicaid Services (CMS) HHS: Medicare program; hospital inpatient prospective payment systems for acute care hospitals and the long-term care hospital prospective payment system and fiscal year 2015 rates; quality reporting requirements for specific providers; reasonable compensation equivalents for physician services in excluded hospitals and certain teaching hospitals; provider administrative appeals and judicial review; enforcement provisions for organ transplant centers; and electronic health record (EHR) incentive program. Final rule. Fed Regist 79:49853505362014

  • 38

    Mueller BCarreon LYGlassman SD: Comparison of the EuroQOL-5D with the Oswestry Disability Index, back and leg pain scores in patients with degenerative lumbar spine pathology. Spine (Phila Pa 1976) 38:7577612013

  • 39

    Mummaneni PVWhitmore RGCurran JNZiewacz JEWadhwa RShaffrey CI: Cost-effectiveness of lumbar discectomy and single-level fusion for spondylolisthesis: experience with the NeuroPoint-SD registry. Neurosurg Focus 36:6E32014

  • 40

    Norman GRStratford PRegehr G: Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol 50:8698791997

  • 41

    Parker SLAdogwa OPaul ARAnderson WNAaronson OCheng JS: Utility of minimum clinically important difference in assessing pain, disability, and health state after transforaminal lumbar interbody fusion for degenerative lumbar spondylolisthesis. J Neurosurg Spine 14:5986042011

  • 42

    Ruiz FKBohl DDWebb MLRusso GSGrauer JN: Oswestry Disability Index is a better indicator of lumbar motion than the Visual Analogue Scale. Spine J 14:186018652014

  • 43

    Schluessmann EDiel PAghayev EZweig TMoulin PRöder C: SWISSspine: a nationwide registry for health technology assessment of lumbar disc prostheses. Eur Spine J 18:8518612009

  • 44

    Schoenfeld AJHarris MBLiu HBirkmeyer JD: Variations in Medicare payments for episodes of spine surgery. Spine J 14:279327982014

  • 45

    Snyder CFJensen RESegal JBWu AW: Patient-reported outcomes (PROs): putting the patient perspective in patient-centered outcomes research. Med Care 51:8 Suppl 3S73S792013

  • 46

    Spiegel MALafage RLafage VRyan DMarascalchi BTrimba Y: Developing the Total Disability Index Based on an analysis of the interrelationships and limitations of ODI and NDI. Spine (Phila Pa 1976) 41:74812016

  • 47

    Strömqvist BFritzell PHägg OJönsson B: The Swedish Spine Register: development, design and utility. Eur Spine J 18:Suppl 32943042009

  • 48

    Theodore BR: Methodological problems associated with the present conceptualization of the minimum clinically important difference and substantial clinical benefit. Spine J 10:5075092010

  • 49

    Utku SBaysal HZileli M: Spine surgery database: a Turkish registry for spinal disorders. Turk Neurosurg 20:2232302010

  • 50

    Washington AELipstein SH: The Patient-Centered Outcomes Research Institute—promoting better information, decisions, and health. N Engl J Med 365:e312011

  • 51

    Whitmore RGCurran JNAli ZSMummaneni PVShaffrey CIHeary RF: Predictive value of 3-month lumbar discectomy outcomes in the NeuroPoint-SD Registry. J Neurosurg Spine 23:4594662015

  • 52

    Wright AHannon JHegedus EJKavchak AE: Clinimetrics corner: a closer look at the minimal clinically important difference (MCID). J Manual Manip Ther 20:1601662012

  • 53

    Zannikos SLee LSmith HE: Minimum clinically important difference and substantial clinical benefit: Does one size fit all diagnoses and patients?. Semin Spine Surg 26:8112014

Disclosures

Dr. Foley reports ownership in BioD and Discgenics, a consulting relationship with Medtronic, ownership of stock in Medtronic, Nuvasive, and SpineWave, and holds patents with Medtronic and Nuvasive. Dr. Glassman reports employment with Norton Healthcare, holding patents with Medtronic, and receiving non–study-related research support from Norton Healthcare and Nuvasive. Dr. Nian reports a consulting relationship with Medtronic. Dr. Mummaneni reports a consulting relationship with DePuy Spine, stock ownership in Spinicity/ISD, receiving royalties from Thieme, Taylor and Francis, and Springer, receiving grants from NREF and AO Spine, and honoraria from AO Spine.

Author Contributions

Conception and design: Asher, Devin, Speroff, Glassman, McGirt. Acquisition of data: Chotai, Nian. Analysis and interpretation of data: Chotai, Devin, Speroff, Harrell, Nian. Drafting the article: Asher, Chotai, Devin, McGirt. Critically revising the article: Asher, Chotai, Devin, Speroff, Harrell, Dittus, Mummaneni, Knightly, Glassman, Bydon, McGirt. Reviewed submitted version of manuscript: all authors. Statistical analysis: Chotai, Harrell, Nian. Administrative/technical/material support: Dittus. Study supervision: Asher, Devin, Speroff, Mummaneni, Knightly, Archer, Foley, McGirt.

If the inline PDF is not rendering correctly, you can download the PDF file here.

Article Information

INCLUDE WHEN CITING Published online March 18, 2016; DOI: 10.3171/2015.11.SPINE15872.

Correspondence Anthony L. Asher, Department of Neurological Surgery, Carolina Neurosurgery and Spine Associates, 225 Baldwin Rd., Charlotte, NC 28204. email: tony.asher@cnsa.com.

© AANS, except where prohibited by US copyright law.

Headings

Figures

  • View in gallery

    Scatter plot demonstrating the correlation between 3-month ODI scores and 12-month ODI scores. The correlation coefficient is 0.70. Each dot represents an individual patient, the closer the dots are to the reference line the better 3-month score correlates with 12-month score. The dots that are away from the reference line indicate outlier individuals with poor agreement between the 3-month and 12-month score.

  • View in gallery

    Scatter plot comparing baseline and 12-month ODI disability scores. A substantial amount of variation can be seen in achieving MCID (left) and SCB (right) for ODI scores at 3 months and 12 months after elective surgery for lumbar degenerative disease. The reference line represents the MCID threshold (12.8) and SCB threshold (18.8). All the patients' with dots to the right of the reference line (blue and green dots, left; pink and blue dots, right) represent those who achieved clinical benefit at 12 months. Figure is available in color online only.

References

  • 1

    Abtahi AMBrodke DSLawrence BDZhang CSpiker WR: Association between patient-reported measures of psychological distress and patient satisfaction scores after spine surgery. J Bone Joint Surg Am 97:8248282015

  • 2

    Adogwa OVerla TThompson PPenumaka AKudyba KJohnson K: Affective disorders influence clinical outcomes after revision lumbar surgery in elderly patients with symptomatic adjacent-segment disease, recurrent stenosis, or pseudarthrosis: clinical article. J Neurosurg Spine 21:1531592014

  • 3

    Aebi MGrob D: SSE Spine Tango: a European Spine Registry promoted by the Spine Society of Europe (SSE). Eur Spine J 13:6616622004

  • 4

    Agrawal BMBrooks NPResnick DK: Wisconsin Spine Outcome Study-Pilot: preliminary data. Neurosurg Focus 33:1E152012

  • 5

    Arts DGDe Keizer NFScheffer GJ: Defining and improving data quality in medical registries: a literature review, case study, and generic framework. J Am Med Inform Assoc 9:6006112002

  • 6

    Asher ALDevin CJMroz TFehlings MParker SLMcGirt MJ: Clinical registries and evidence-based care pathways: raising the bar for meaningful measurement and delivery of value-based care. Spine (Phila Pa 1976) 39:22 Suppl 1S136S1382014

  • 7

    Asher ALSperoff TDittus RSParker SLDavies JMSelden N: The National Neurosurgery Quality and Outcomes Database (N2QOD): a collaborative North American outcomes registry to advance value-based spine care. Spine (Phila Pa 1976) 39:22 Suppl 1S106S1162014

  • 8

    Badia XDiaz-Prieto AGorriz MTHerdman MTorrado HFarrero E: Using the EuroQol-5D to measure changes in quality of life 12 months after discharge from an intensive care unit. Intensive Care Med 27:190119072001

  • 9

    Bresnahan BWRundell SD: Including patient-reported outcomes and patient-reported resource-use questionnaires in studies. Acad Radiol 21:112911372014

  • 10

    Carreon LYGlassman SDCampbell MJAnderson PA: Neck Disability Index, Short Form-36 physical component summary, and pain scales for neck and arm pain: the minimum clinically important difference and substantial clinical benefit after cervical spine fusion. Spine J 10:4694742010

  • 11

    Carreon LYSanders JODiab MSucato DJSturm PFGlassman SD: The minimum clinically important difference in Scoliosis Research Society-22 appearance, activity, and pain domains after surgical correction of adolescent idiopathic scoliosis. Spine (Phila Pa 1976) 35:207920832010

  • 12

    Cesta T: Case management insider. The new value-based purchasing efficiency measure: are you ready?. Hosp Case Manag 22:1671702014

  • 13

    Copay AGGlassman SDSubach BRBerven SSchuler TCCarreon LY: Minimum clinically important difference in lumbar spine surgery patients: a choice of methods using the Oswestry Disability Index, Medical Outcomes Study questionnaire Short Form 36, and pain scales. Spine J 8:9689742008

  • 14

    Copay AGSubach BRGlassman SDPolly DW JrSchuler TC: Understanding the minimum clinically important difference: a review of concepts and methods. Spine J 7:5415462007

  • 15

    D’Arcy LPRich EC: From comparative effectiveness research to patient-centered outcomes research: policy history and future directions. Neurosurg Focus 33:1E72012

  • 16

    Daubs MDNorvell DCMcGuire RMolinari RHermsmeyer JTFourney DR: Fusion versus nonoperative care for chronic low back pain: do psychological factors affect outcomes?. Spine (Phila Pa 1976) 36:21 SupplS96S1092011

  • 17

    DeVine JNorvell DCEcker EFourney DRVaccaro AWang J: Evaluating the correlation and responsiveness of patient-reported pain with function and quality-of-life outcomes after spine surgery. Spine (Phila Pa 1976) 36:21 SupplS69S742011

  • 18

    EuroQol Group: EuroQol—a new facility for the measurement of health-related quality of life. Health Policy 16:1992081990

  • 19

    Fairbank JCCouper JDavies JBO’Brien JP: The Oswestry Low Back Pain Disability Questionnaire. Physiotherapy 66:2712731980

  • 20

    Fairbank JCPynsent PB: The Oswestry Disability Index. Spine (Phila Pa 1976) 25:294029522000

  • 21

    Franklin PDHarrold LAyers DC: Incorporating patient-reported outcomes in total joint arthroplasty registries: challenges and opportunities. Clin Orthop Relat Res 471:348234882013

  • 22

    Fritz JMIrrgang JJ: A comparison of a modified Oswestry Low Back Pain Disability Questionnaire and the Quebec Back Pain Disability Scale. Phys Ther 81:7767882001

  • 23

    Gallagher EJLiebman MBijur PE: Prospective validation of clinically important changes in pain severity measured on a visual analog scale. Ann Emerg Med 38:6336382001

  • 24

    Gatchel RJMayer TG: Testing minimal clinically important difference: consensus or conundrum?. Spine J 10:3213272010

  • 25

    Ghogawala ZShaffrey CIAsher ALHeary RFLogvinenko TMalhotra NR: The efficacy of lumbar discectomy and single-level fusion for spondylolisthesis: results from the NeuroPoint-SD registry: clinical article. J Neurosurg Spine 19:5555632013

  • 26

    Glassman SDCopay AGBerven SHPolly DWSubach BRCarreon LY: Defining substantial clinical benefit following lumbar spine arthrodesis. J Bone Joint Surg Am 90:183918472008

  • 27

    Gliklich REDreyer NA: Registries for Evaluating Patient Outcomes: A User’s Guide Rockville, MDAgency for Healthcare Research and Quality2007

  • 28

    Godil SSParker SLZuckerman SLMendenhall SKGlassman SDMcGirt MJ: Accurately measuring the quality and effectiveness of lumbar surgery in registry efforts: determining the most valid and responsive instruments. Spine J 14:288528912014

  • 29

    Harris PATaylor RThielke RPayne JGonzalez NConde JG: Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42:3773812009

  • 30

    Huntoon KMMcCluney CJScannell CAWiley EABruno RAndrews A: Healthcare reform and the next generation: United States medical student attitudes toward the Patient Protection and Affordable Care Act. PLoS One 6:e235572011

  • 31

    Jaeschke RSinger JGuyatt GH: Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials 10:4074151989

  • 32

    Jarvik JGComstock BABresnahan BWNedeljkovic SSNerenz DRBauer Z: Study protocol: the Back Pain Outcomes using Longitudinal Data (BOLD) registry. BMC Musculoskelet Disord 13:642012

  • 33

    Larkin DJSwanson RCFuller SCortese DA: The Affordable Care Act: a case study for understanding and applying complexity concepts to health care reform. J Eval Clin Pract [epub ahead of print]2014

  • 34

    Lohr KNSchroeder SA: A strategy for quality assurance in Medicare. N Engl J Med 322:7077121990

  • 35

    May EL: Financial planning in a value-based world. Healthc Exec 29:10121416182014

  • 36

    McGirt MJParker SLAsher ALNorvell DSherry NDevin CJ: Role of prospective registries in defining the value and effectiveness of spine care. Spine (Phila Pa 1976) 39:22 Suppl 1S117S1282014

  • 37

    Centers for Medicare and Medicaid Services (CMS) HHS: Medicare program; hospital inpatient prospective payment systems for acute care hospitals and the long-term care hospital prospective payment system and fiscal year 2015 rates; quality reporting requirements for specific providers; reasonable compensation equivalents for physician services in excluded hospitals and certain teaching hospitals; provider administrative appeals and judicial review; enforcement provisions for organ transplant centers; and electronic health record (EHR) incentive program. Final rule. Fed Regist 79:49853505362014

  • 38

    Mueller BCarreon LYGlassman SD: Comparison of the EuroQOL-5D with the Oswestry Disability Index, back and leg pain scores in patients with degenerative lumbar spine pathology. Spine (Phila Pa 1976) 38:7577612013

  • 39

    Mummaneni PVWhitmore RGCurran JNZiewacz JEWadhwa RShaffrey CI: Cost-effectiveness of lumbar discectomy and single-level fusion for spondylolisthesis: experience with the NeuroPoint-SD registry. Neurosurg Focus 36:6E32014

  • 40

    Norman GRStratford PRegehr G: Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol 50:8698791997

  • 41

    Parker SLAdogwa OPaul ARAnderson WNAaronson OCheng JS: Utility of minimum clinically important difference in assessing pain, disability, and health state after transforaminal lumbar interbody fusion for degenerative lumbar spondylolisthesis. J Neurosurg Spine 14:5986042011

  • 42

    Ruiz FKBohl DDWebb MLRusso GSGrauer JN: Oswestry Disability Index is a better indicator of lumbar motion than the Visual Analogue Scale. Spine J 14:186018652014

  • 43

    Schluessmann EDiel PAghayev EZweig TMoulin PRöder C: SWISSspine: a nationwide registry for health technology assessment of lumbar disc prostheses. Eur Spine J 18:8518612009

  • 44

    Schoenfeld AJHarris MBLiu HBirkmeyer JD: Variations in Medicare payments for episodes of spine surgery. Spine J 14:279327982014

  • 45

    Snyder CFJensen RESegal JBWu AW: Patient-reported outcomes (PROs): putting the patient perspective in patient-centered outcomes research. Med Care 51:8 Suppl 3S73S792013

  • 46

    Spiegel MALafage RLafage VRyan DMarascalchi BTrimba Y: Developing the Total Disability Index Based on an analysis of the interrelationships and limitations of ODI and NDI. Spine (Phila Pa 1976) 41:74812016

  • 47

    Strömqvist BFritzell PHägg OJönsson B: The Swedish Spine Register: development, design and utility. Eur Spine J 18:Suppl 32943042009

  • 48

    Theodore BR: Methodological problems associated with the present conceptualization of the minimum clinically important difference and substantial clinical benefit. Spine J 10:5075092010

  • 49

    Utku SBaysal HZileli M: Spine surgery database: a Turkish registry for spinal disorders. Turk Neurosurg 20:2232302010

  • 50

    Washington AELipstein SH: The Patient-Centered Outcomes Research Institute—promoting better information, decisions, and health. N Engl J Med 365:e312011

  • 51

    Whitmore RGCurran JNAli ZSMummaneni PVShaffrey CIHeary RF: Predictive value of 3-month lumbar discectomy outcomes in the NeuroPoint-SD Registry. J Neurosurg Spine 23:4594662015

  • 52

    Wright AHannon JHegedus EJKavchak AE: Clinimetrics corner: a closer look at the minimal clinically important difference (MCID). J Manual Manip Ther 20:1601662012

  • 53

    Zannikos SLee LSmith HE: Minimum clinically important difference and substantial clinical benefit: Does one size fit all diagnoses and patients?. Semin Spine Surg 26:8112014

TrendMD

Cited By

Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 311 311 23
PDF Downloads 218 218 13
EPUB Downloads 0 0 0

PubMed

Google Scholar