How well do subjective Hospital Compare metrics reflect objective outcomes in spine surgery?

Full access

OBJECTIVE

The accuracy of public reporting in health care is an issue of debate. The authors investigated the association of patient satisfaction measures from a public reporting platform with objective outcomes for patients undergoing spine surgery.

METHODS

The authors performed a cohort study involving patients undergoing elective spine surgery from 2009 to 2013 who were registered in the New York Statewide Planning and Research Cooperative System database. This cohort was merged with publicly available data from the Centers for Medicare and Medicaid Services (CMS) Hospital Compare website. A mixed-effects regression analysis, controlling for clustering at the hospital level, was used to investigate the association of patient satisfaction metrics with outcomes.

RESULTS

During the study period, 160,235 patients underwent spine surgery. Using a mixed-effects multivariable regression analysis, the authors demonstrated that undergoing elective spine surgery in hospitals with a higher percentage of patient-assigned high satisfaction scores was not associated with a decreased rate of discharge to rehabilitation (OR 0.77, 95% CI 0.57–1.06), mortality (OR 0.96, 95% CI 0.90–1.01), or hospitalization charges (β 0.04, 95% CI −0.16 to 0.23). However, it was associated with decreased length of stay (LOS; β −0.19, 95% CI −0.33 to −0.05). Similar associations were identified for hospitals with a higher percentage of patients who claimed they would recommend these institutions to others.

CONCLUSIONS

Merging a comprehensive all-payer cohort of spine surgery patients in New York state with data from the CMS Hospital Compare website, the authors were not able to demonstrate an association of improved performance in patient satisfaction measures with decreased mortality, rate of discharge to rehabilitation, and hospitalization charges. Increased patient satisfaction was associated with decreased LOS.

ABBREVIATIONSCAD = coronary artery disease; CHF = congestive heart failure; CMS = Centers for Medicare and Medicaid Services; CRF = chronic renal failure; DM = diabetes mellitus; ICD-9-CM = International Classification of Disease, Ninth Revision, Clinical Modification; IQR = interquartile range; LOS = length of stay; N2QOD = National Neurosurgical Quality and Outcomes Database; PVD = peripheral vascular disease; SPARCS = Statewide Planning and Research Cooperative System; TIA = transient ischemic attack.

OBJECTIVE

The accuracy of public reporting in health care is an issue of debate. The authors investigated the association of patient satisfaction measures from a public reporting platform with objective outcomes for patients undergoing spine surgery.

METHODS

The authors performed a cohort study involving patients undergoing elective spine surgery from 2009 to 2013 who were registered in the New York Statewide Planning and Research Cooperative System database. This cohort was merged with publicly available data from the Centers for Medicare and Medicaid Services (CMS) Hospital Compare website. A mixed-effects regression analysis, controlling for clustering at the hospital level, was used to investigate the association of patient satisfaction metrics with outcomes.

RESULTS

During the study period, 160,235 patients underwent spine surgery. Using a mixed-effects multivariable regression analysis, the authors demonstrated that undergoing elective spine surgery in hospitals with a higher percentage of patient-assigned high satisfaction scores was not associated with a decreased rate of discharge to rehabilitation (OR 0.77, 95% CI 0.57–1.06), mortality (OR 0.96, 95% CI 0.90–1.01), or hospitalization charges (β 0.04, 95% CI −0.16 to 0.23). However, it was associated with decreased length of stay (LOS; β −0.19, 95% CI −0.33 to −0.05). Similar associations were identified for hospitals with a higher percentage of patients who claimed they would recommend these institutions to others.

CONCLUSIONS

Merging a comprehensive all-payer cohort of spine surgery patients in New York state with data from the CMS Hospital Compare website, the authors were not able to demonstrate an association of improved performance in patient satisfaction measures with decreased mortality, rate of discharge to rehabilitation, and hospitalization charges. Increased patient satisfaction was associated with decreased LOS.

The enactment13 of the Medicare Access and Children's Health Insurance Program Reauthorization Act into law brings quality measurement and reporting into the forefront of health care in the US.2,3,23 This legislation highlights the development of several public reporting avenues for quality metrics, with the Centers for Medicare and Medicaid Services (CMS) Hospital Compare website being the most prominent. The Hospital Compare website aims to empower informed health care choices by patients. To that end, it contains several subjective metrics focusing on provider-patient relationships, communication, care coordination and efficiency, as well as various objective measures (such as mortality) for myocardial infarction, congestive heart failure (CHF), and pneumonia. Currently, and despite the lack of subspecialty-specific metrics for other areas of medicine (e.g., spine surgery), patients are encouraged to make decisions based on such generic data.

However, the correlation of such subjective metrics with hospital performance in objective outcomes remains an issue of debate.4,10 Critics of such ranking systems9 claim that these websites are misleading given the limited consideration of important measures such as mortality, length of stay (LOS), or hospitalization charges for most areas of medicine.1 In addition, the lack of risk adjustment and accounting for the comorbidity burden of the patient population treated by each hospital fuels the debate on the appropriateness of these comparison metrics.10 There has been no prior study attempting to validate these claims in a comprehensive group of subspecialty patients.

We used the New York Statewide Planning and Research Cooperative System (SPARCS) (http://www.health.ny.gov/statistics/sparcs/) to study the association of subjective Hospital Compare metrics with mortality, discharge to rehabilitation, LOS, and hospitalization charges for patients undergoing spine surgery. A mixed-effects multivariable regression model, controlling for clustering at the hospital level, was used to control for confounding.

Methods

New York SPARCS

All patients undergoing spine surgery who were registered in the SPARCS database between 2009 and 2013 were included in the analysis. For these years, SPARCS contains patient-level details for every hospital discharge, ambulatory surgery, and emergency department admission in New York State as coded from admission and billing records.

Hospital Compare Website

Hospital Compare (www.hospitalcompare.hhs.gov) is a public reporting program operated by the CMS that reports process-of-care, patient satisfaction, and outcome measures for more than 4000 Medicare-certified hospitals in the US. Data between 2009 and 2013 were included in the analysis. The website is designed such that patients can make side-by-side comparisons of hospitals that are geographically close to one another with the stated purpose to “help patients make the right decisions about their healthcare.”

We merged SPARCS and Hospital Compare data using the hospital identities. We followed an automated methodology to ensure that the hospitals in SPARCS matched exactly with hospitals of the Hospital Compare database. Subsequently the hospitals were manually confirmed for appropriate matching to ensure that the correct satisfaction scores were assigned to appropriate patients. The 2 coauthors performed the manual confirmation. Only 5 hospitals in SPARCS were not matched after this process, and were excluded from further analysis. Patients from a total of 165 hospitals were included in the analysis.

Cohort Definition

To establish the cohort of patients, we used International Classification of Disease, Ninth Revision, Clinical Modification (ICD-9-CM) codes (03.0, 03.01, 03.02, 03.09, 03.1, 03.2, 03.21, 03.29, 03.09, 03.4, 03.5, 03.51, 03.52, 03.53, 03.59, 03.6, 80.50, 80.51, 80.52, 80.53, 80.54, 80.59, 81.00, 81.01, 81.02, 81.03, 81.04, 81.05, 81.06, 81.07, 81.08, 81.30, 81.31, 81.32, 81.33, 81.34, 81.35, 81.36, 81.37, 81.38, 81.39, 81.62, 81.63, 81.64, 84.51, 84.59, 84.60, 84.61, 84.62, 84.63, 84.64, 84.65, 84.66, 84.67, 84.68, 84.69, 84.80, 84.81, 84.82, 84.83, 84.84, 84.85) to identify patients in the database who underwent spine surgery (Table S1) between 2009 and 2013. Patient transfers were excluded.

Outcome Variables

The primary outcome variable was the rate of discharge to a rehabilitation facility after the initial hospitalization for spine surgery. Secondary outcomes were mortality during the initial hospitalization, LOS during the initial hospitalization, and total hospitalization charges.

Exposure Variables

The primary exposure variable was whether the hospital had a high percentage of patients assigning the institution a satisfaction score of 9 or 10 (on a scale of 1–10). The secondary exposure variable was whether the hospital had a high percentage of patients who would recommend the hospital to others. Both were binary variables. The cutoffs to determine whether the hospital had a high percentage of the above satisfaction indices were the respective median satisfaction scores across all hospitals in the database.

We chose to include these metrics because the rest of the patient satisfaction and process measures were collinear with these two (e.g., if patients scored a hospital highly on “provider communication” they were also likely to recommend it), and the outcomes measures were not specific to spine surgery but referred to myocardial infarction, pneumonia, and CHF.

Covariates (Table S2) used for risk adjustment included age, sex, race (African American, Hispanic, Asian, Caucasian, other), and insurance (private, Medicare, Medicaid, uninsured, other). The comorbidities used for risk adjustment were based on the individual comorbidities used for the calculation of the Charlson Comorbidity Score and were depression, diabetes mellitus (DM), smoking, chronic lung disease, hypertension, hypercholesterolemia, peripheral vascular disease (PVD), CHF, coronary artery disease (CAD), history of ischemic stroke, history of transient ischemic attack (TIA), alcohol abuse, obesity, chronic renal failure (CRF), and coagulopathy. Only variables that were defined as “present on admission” were considered part of the patient's preadmission comorbidity profile.

Statistical Analysis

The association of Hospital Compare subjective metrics with our outcome measures was examined in a multivariable setting. A logistic regression was used for categorical outcomes (mortality, discharge to rehabilitation), and a linear regression for linear outcomes (LOS, hospitalization charges). The values of LOS and hospitalization charges were positively skewed, and therefore a logarithmic transformation was used for the models. The covariates used for risk adjustment in these models were age, sex, race, insurance, and all the comorbidities mentioned previously. Mixed-effects methods were employed, with hospital facility used as a random-effects variable to account for clustering at the hospital level. All analyses were repeated after stratification for subgroups of procedures performed. The direction of the observed associations did not change and these associations are reported further in Results.

Regression diagnostics were used for all models. All results are based on 2-sided tests, and the level of statistical significance was set at 0.05. Only cases without missing values were included in the analysis. This study, based on 160,235 patients, has sufficient power (90%) at a 5% Type I error rate to detect differences in mortality, as small as 12.7%. Statistical analyses were performed using the 64-bit version of R (version 3.1.0, R Foundation for Statistical Computing) and SPSS (version 22, IBM).

Results

Patient Characteristics

In the selected study period there were 160,235 patients undergoing spine surgery (mean age 55.0 years, 49.5% females) who were registered in SPARCS. The characteristics of our cohort at baseline can be seen in Table 1.

TABLE 1.

Patient characteristics*

VariableAll PatientsHospital Satisfaction Scores
< Median> Median
Sample size160,23583,88476,351
Mean age in yrs (SD)54.99 (14.72)54.36 (14.53)55.69 (14.91)
Sex
  F79,233 (49.45)41,759 (49.78)37,474 (49.08)
  M81,002 (50.55)42,125 (50.22)38,877 (50.92)
Insurance status
  Medicare42,315 (26.45)20,634 (24.65)21,681 (28.43)
  Medicaid6113 (3.82)3989 (4.77)2124 (2.79)
  Private insurance82,555 (51.61)42,036 (50.22)40,519 (53.14)
  Self-pay3701 (2.31)2239 (2.67)1462 (1.92)
  Other25,283 (15.81)14,813 (17.70)10,470 (13.73)
Race
  Caucasian114,918 (71.91)58,759 (70.22)56,159 (73.76)
  African American15,444 (9.66)10,148 (12.13)5296 (6.96)
  Hispanic11,568 (7.24)7675 (9.17)3893 (5.11)
  Asian2824 (1.77)1224 (1.46)1600 (2.10)
  Other15,063 (9.43)5876 (7.02)9187 (12.07)
Procedure
  Laminectomy24,884 (15.53)11,880 (14.16)13,004 (17.03)
  Discectomy23,456 (14.64)11,114 (13.25)12,342 (16.16)
  Spinal fusion111,895 (69.83)60,890 (72.59)51,005 (66.80)
Comorbidities
  DM25,231 (15.75)14,198 (16.93)11,033 (14.45)
  Tobacco abuse23,577 (14.71)14,042 (16.74)9535 (12.49)
  Obesity18,081 (11.28)10,069 (12.00)8012 (10.49)
  History of TIA31 (0.02)16 (0.02)15 (0.02)
  History of ischemic stroke75 (0.05)53 (0.06)22 (0.03)
  CAD15,595 (9.73)8218 (9.80)7377 (9.66)
  COPD24,699 (15.41)14,003 (16.69)10,696 (14.01)
  CHF3199 (2.00)1892 (2.26)1307 (1.71)
  Coagulopathy1701 (1.06)925 (1.10)776 (1.02)
  CRF4064 (2.54)2253 (2.69)1811 (2.37)
  Hypertension71,735 (44.77)38,215 (45.56)33,520 (43.90)
  Hypercholesterolemia47,806 (29.83)24,121 (28.76)23,685 (31.02)
  Alcohol2440 (1.52)1546 (1.84)894 (1.17)
  PVD3387 (2.11)1856 (2.21)1531 (2.01)
  Depression18,596 (11.61)9735 (11.61)8861 (11.61)

COPD = chronic obstructive pulmonary disease.

All data given as number of patients (%) unless otherwise indicated.

Patients of hospitals where > 68% of patients (median national rate) gave their hospital a 9 or 10 rating.

Discharge to Rehabilitation

Overall, 52,087 (32.5%) were discharged to rehabilitation after spine surgery in our cohort (Table 2). Patients treated in hospitals with a higher percentage of patient-assigned high satisfaction scores were associated with an increased rate of discharge to rehabilitation (OR 1.35, 95% CI, 1.32–1.38) in the unadjusted analysis. However, no association was identified (OR 0.77, 95% CI 0.57–1.06) in a mixed-effects logistic regression model (Table 3). Likewise, patients treated in hospitals with a higher percentage of patients who claimed they would recommend these institutions were associated with an increased rate of discharge to rehabilitation (OR 1.45, 95% CI 1.43–1.50) in the unadjusted analysis. However, no association was identified (OR 0.87, 95% CI 0.64–1.19) in a mixed-effects logistic regression model (Table 4). These associations remained after stratification for subgroups of spinal procedures performed (Tables 3 and 4).

TABLE 2.

Outcomes

OutcomeAll PatientsHospital Satisfaction Scores
< Median> Median*
Mortality (%)575 (0.36)328 (0.47)164 (0.21)
Unfavorable discharge (%)52,087 (32.47)24,609 (29.60)27,051 (35.56)
Median LOS in days (IQR)3 (1–5)3 (2–5)3 (1–5)
Median total inpatient charges in $ (IQR)44,495 (24,809–79,403)44,182 (25,319–78,145)44,403 (24,153–78,987)

Patients of hospitals where > 68% of patients (median national rate) gave their hospital a 9 or 10 rating.

TABLE 3.

Association of outcomes with subjective Hospital Compare metrics (treatment in hospitals with a higher percentage of patient-assigned high satisfaction scores)*

OutcomeLaminectomyDiscectomySpinal FusionTotal
OR or β (95% CI)p ValueOR or β (95% CI)p ValueOR or β (95% CI)p ValueOR or β (95% CI)p Value
Inpatient mortality0.90 (0.78–1.04)0.1500.98 (0.85–1.15)0.8810.96 (0.90–1.03)0.2740.96 (0.90–1.01)0.129
Discharge to rehabilitation0.84 (0.64–1.09)0.1870.86 (0.66–1.12)0.2550.82 (0.56–1.19)0.2870.77 (0.57–1.06)0.105
LOS (days)−0.30 (−0.45 to −0.14)<0.0001−0.27 (−0.38 to −0.15)<0.0001−0.23 (−0.41 to −0.06)0.007−0.19 (−0.33 to −0.05)0.007
Hospitalization charges−0.13 (−0.36 to 0.10)0.268−0.05 (−0.22 to 0.12)0.5640.02 (−0.17 to 0.21)0.8390.04 (−0.16 to 0.23)0.707

All analyses used mixed effects, which included “hospital” as a random-effect variable.

Analyses based on logistic regression, given as OR (95% CI).

Analyses based on linear regression with γ modeling; point estimates are β coefficients, given as β (95% CI).

TABLE 4.

Association of outcomes with subjective hospital compare metrics (treatment in hospitals with a higher percentage of patients who claimed they would recommend these institutions)*

OutcomeLaminectomyDiscectomySpinal FusionTotal
OR or β (95% CI)p ValueOR or β (95% CI)p ValueOR or β (95% CI)p ValueOR or β (95% CI)p Value
Inpatient mortality0.90 (0.78–1.04)0.1600.98 (0.84–1.15)0.8190.96 (0.89–1.03)0.2060.95 (0.90–1.01)0.094
Discharge to rehabilitation0.90 (0.69–1.17)0.4330.97 (0.74–1.25)0.7860.94 (0.64–1.40)0.7600.87 (0.64–1.19)0.394
LOS (days)−0.27 (−0.43 to −0.11)0.001−0.25 (−0.36 to −0.13)<0.0001−0.16 (−0.34 to 0.01)0.074−0.22 (−0.39 to −0.11)<0.0001
Hospitalization charges−0.01 (−0.24 to 0.22)0.9260.037 (−0.13 to 0.21)0.6730.14 (−0.05 to 0.33)0.1650.15 (−0.05 to 0.34)0.144

All analyses used mixed effects, which included “hospital” as a random-effect variable.

Analyses based on logistic regression, given as OR (95% CI).

Analyses based on linear regression with γ modeling; point estimates are β coefficients, given as β (95% CI).

Inpatient Mortality

Overall, 575 (0.36%) inpatient deaths were recorded after spine surgery in our cohort (Table 2). Patients treated in hospitals with a higher percentage of patient-assigned high satisfaction scores were associated with decreased mortality (OR 0.45, 95% CI 0.37–0.54) in the unadjusted analysis. However, no association was identified (OR 0.96, 95% CI 0.90–1.01) in a mixed-effects logistic regression model (Table 3). Likewise, patients treated in hospitals with a higher percentage of patients who claimed they would recommend these institutions were associated with decreased mortality (OR 0.42, 95% CI 0.35–0.51) in the unadjusted analysis. However, no association was identified (OR 0.95, 95% CI 0.90–1.01) in a mixed-effects logistic regression model (Table 4). These associations remained after stratification for subgroups of spinal procedures performed (Tables 3 and 4).

Length of Stay

The median LOS was 3 days (interquartile range [IQR] 1–5 days) after spine surgery in our cohort (Table 2). Patients treated in hospitals with a higher percentage of patient-assigned high satisfaction scores were associated with a shorter LOS (β −0.15, 95% CI −0.16 to −0.14) in the unadjusted analysis. This association persisted (Table 3) in a mixed-effects linear regression model (β −0.19, 95% CI −0.33 to −0.05). Likewise, patients treated in hospitals with a higher percentage of patients who claimed they would recommend these institutions were associated with shorter LOS (β −0.12, 95% CI −0.13 to −0.11) in the unadjusted analysis. This association persisted (β −0.22, 95% CI −0.39 to −0.11) in a mixed-effects linear regression model (Table 4). These associations remained after stratification for subgroups of spinal procedures performed (Tables 3 and 4).

Hospitalization Charges

The median hospitalization charges were $44,495 (IQR $24,809–$79,403) after spine surgery in our cohort (Table 2). Patients treated in hospitals with a higher percentage of patient-assigned high satisfaction scores were not associated with decreased hospitalization charges (β 0.01, 95% CI −0.003 to 0.013) in the unadjusted analysis. This association persisted (Table 3) in a mixed-effects linear regression model (β 0.04, 95% CI −0.16 to 0.23). Likewise, patients treated in hospitals with a higher percentage of patients who claimed they would recommend these institutions were associated with increased hospitalization charges (β 0.13, 95% CI 0.12–0.14) in the unadjusted analysis. However, no association was identified (β 0.15, 95% CI −0.05 to 0.34) in a mixed-effects linear regression model (Table 4). These associations remained after stratification for subgroups of spinal procedures performed (Tables 3 and 4).

Discussion

Using a comprehensive all-payer cohort of patients undergoing spine surgery, we did not identify an association of subjective Hospital Compare ratings with decreased mortality, rate of discharge to rehabilitation, and hospitalization charges for New York State hospitals. Increased patient satisfaction was associated with decreased LOS. Our results were robust when considering several advanced observational techniques to account for confounders. Quality measurement and reporting are an integral part of evolving health care reform.3 CMS is using Hospital and Physician Compare as the main avenues to satisfy reporting requirements and enable consumers to be active participants in their care. Despite some criticism for these websites,17 reliable public reporting is expected to raise quality of care, lower average cost, and improve population health.6,9,23 However, there are concerns that the subjective and nonspecific nature of the reported metrics might be misleading for patient choices, especially for subspecialties such as spine surgery. In addition, previous studies in various surgical principles have failed to demonstrate a convincing correlation of patient satisfaction with clinical outcomes.8,11,22

Prior observational studies have reported conflicting results on the validity of publicly reported Hospital Compare measures. Shafi et al.,19 in a study of the National Trauma Data Bank, demonstrated that Hospital Compare metrics, reflecting hospital performance on general medical conditions, were not associated with the outcomes of trauma patients. In their conclusions, they advocated for the development of trauma-specific measures. In contrast, Werner and Bradlow24 demonstrated that better performance in disease-specific Hospital Compare metrics for pneumonia, myocardial infarction, and CHF was associated with modestly improved mortality rates, specific to these conditions. In addition, Stulberg et al.,21 using a commercial database, identified an association of performance on infection process measures with decreased rates of surgical site infections. Rosenkrantz and Doshi16 demonstrated that hospitals ranked highly by US News and World Report were associated with lower rates of overutilization of imaging studies based on publicly reported Hospital Compare measures. In another study, Safavi et al.18 investigated the distribution of the values of surgical quality measures and demonstrated significantly positively skewed distributions. They concluded that these sets of measures make it difficult to distinguish among hospitals that patients are likely to choose from. Paddock et al.15 came to similar conclusions by using comparative rankings provided by the Hospital Compare website.

Despite these widely varied results, a recent survey-based study demonstrated a correlation of Hospital Compare metrics with the policy decisions made by hospital leadership.7,10 Executives from 380 hospitals indicated that reported measures exert a strong influence over local planning and improvement efforts:10 87.1% of hospitals incorporated performance on publicly reported measures into their annual goals, whereas 90.2% reported regularly reviewing the results with their board of trustees, and 94.3% with senior clinical and administrative leaders.10 They indicated that patient experience measures, which are undoubtedly the most subjective, are the ones shaping hospital reputations more significantly. However, surveyed leaders expressed concerns about the clinical meaningfulness, unintended consequences, and methods of public reporting.10

Using such patient-experience measures, our analysis demonstrated no association of hospital performance on subjective patient satisfaction metrics with most outcomes of spine surgery. This underscores the significant pitfalls of the current reporting platforms. Although Hospital Compare can be used as a basis for further improvement and development of future websites, it is not ready yet for widespread use, especially for subspecialties such as spine surgery. It currently lacks spine-specific measures, and we found no association of generic measures with hard outcomes, such as mortality. Policy makers should take this into account when designing penalties for hospitals with inferior satisfaction scores. Improved performance on these measures may be a marker of institutions that are dedicated to improving patient satisfaction through promoting a “culture of excellence,” which might lead to shorter hospitalizations, as we observed in our cohort. However, this might not mean necessarily better care for some subspecialties. Such subjective metrics do not currently allow the side-by-side comparison of different institutions in regard to the quality of care.

For this goal of Hospital Compare to be met, spine-specific, patient-centered quality-oriented outcomes should be monitored and reported. Recent legislative reform13 allows a more active role for registries in this public reporting process, avoiding the long and costly task of developing metrics through the National Quality Forum.14 In this direction, the NeuroPoint Alliance (http://www.neuropoint.org/NPA%20N2QOD.html) has created the first lumbar spine–specific Qualified Clinical Data Registry Reporting (https://www.medicare.gov/physiciancompare/), based on metrics developed as part of the National Neurosurgical Quality and Outcomes Database (N2QOD).12 These spine-specific measures, in the setting of rigorous risk-adjustment, will allow the head-to-head comparison of different hospitals and serve the national goals of quality reporting.

Overall, quality measurement and reporting is still in its infancy.9,25 Different quality initiatives use various metrics to create ratings, some of which are based on numerical data and some on solicited opinion. Methodologies are widely varied and results are presented in different ways.25 Most importantly, rating systems often disagree with each other,1 highlighting the complexity and multidimensionality of the enterprise.5,20 The small market of surgical subspecialties makes it unlikely that specific metrics will be developed, unless such efforts are initiated by specialty societies with a vested interest in quality improvement. The reality, though, is that hospital rankings (including Hospital Compare) are publicly available and physicians often have to answer patient questions on them. Our results highlight that these rankings should be interpreted with extreme caution, especially for subspecialties such as spine surgery.

Our study has several limitations common to administrative databases. Residual confounding could account for some of the observed associations. However, we used advanced observational techniques to control for con-founders. In addition, coding inaccuracies will undoubtedly occur and can affect our estimates. However, coding for procedures is expected to be accurate because they are a revenue generator and are under scrutiny by payers. Although SPARCS includes all hospitals from all New York State, the generalization of this analysis to the US population at large is uncertain. Additionally, we were lacking posthospitalization and long-term data on our patients. Quality metrics (i.e., 36-Item Short Form Health Survey score) are also not available through this database, and therefore we cannot compare our patients on these outcomes. Initiatives such as the N2QOD will provide such opportunities in the future. Finally, causality cannot be definitively established based on observational data, despite the use of advanced techniques. Only future prospective studies can firmly establish such relationships.

Conclusions

The correlation of subjective Hospital Compare metrics with hospital performance in hard outcomes remains an issue of debate. Merging a comprehensive all-payer cohort of spine surgery patients in New York State with data from the CMS Hospital Compare website, we were not able to demonstrate an association of improved performance in patient satisfaction measures with decreased mortality, rate of discharge to rehabilitation, and hospitalization charges. Increased patient satisfaction was associated with decreased LOS. Our results were robust when considering several advanced observational techniques to account for confounders.

References

  • 1

    Austin JMJha AKRomano PSSinger SJVogus TJWachter RM: National hospital ratings systems share few common scores and may generate confusion instead of clarity. Health Aff (Millwood) 34:4234302015

    • Search Google Scholar
    • Export Citation
  • 2

    Bekelis KGoodney RPDzebisashvili NGoodman DCBronner KK: Variation in the Care of Surgical Conditions: Cerebral Aneurysms Lebanon, NHDartmouth Institute for Health Policy and Clinical Practice2014

    • Search Google Scholar
    • Export Citation
  • 3

    Birkmeyer NJBirkmeyer JD: Strategies for improving surgical quality—should payers reward excellence or effort?. N Engl J Med 354:8648702006

    • Search Google Scholar
    • Export Citation
  • 4

    Cassel CKConway PHDelbanco SFJha AKSaunders RSLee TH: Getting more performance from performance measurement. N Engl J Med 371:214521472014

    • Search Google Scholar
    • Export Citation
  • 5

    Donabedian A: Evaluating the quality of medical care. 1966. Milbank Q 83:6917292005

  • 6

    Fung CHLim YWMattke SDamberg CShekelle PG: Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 148:1111232008

    • Search Google Scholar
    • Export Citation
  • 7

    Goff SLLagu TPekow PSHannon NSHinchey KLJackowitz TA: A qualitative analysis of hospital leaders' opinions about publicly reported measures of health care quality. Jt Comm J Qual Patient Saf 41:1691762015

    • Search Google Scholar
    • Export Citation
  • 8

    Gupta DPatel KLis CG: Self-rated health supersedes patient satisfaction with service quality as a predictor of survival in prostate cancer. Health Qual Life Outcomes 13:1372015

    • Search Google Scholar
    • Export Citation
  • 9

    Harder BComarow A: Hospital quality reporting by US News & World Report: why, how, and what's ahead. JAMA 313:190319042015

  • 10

    Lindenauer PKLagu TRoss JSPekow PSShatz AHannon N: Attitudes of hospital leaders toward publicly reported measures of health care quality. JAMA Intern Med 174:190419112014

    • Search Google Scholar
    • Export Citation
  • 11

    Lis CGPatel KGupta D: The relationship between patient satisfaction with service quality and survival in non–small cell lung cancer—is self-rated health a potential confounder?. PLoS One 10:e01346172015

    • Search Google Scholar
    • Export Citation
  • 12

    McGirt MJSperoff TDittus RSHarrell FEJ JrAsher AL: The National Neurosurgery Quality and Outcomes Database (N2QOD): general overview and pilot-year project description. Neurosurg Focus 34:1E62013

    • Search Google Scholar
    • Export Citation
  • 13

    Medicare Access and CHIP Reauthorization Act of 2015 Pub L No. 114-10 129 Stat 87

  • 14

    National Quality Forum: Consensus Development Process (http://www.qualityforum.org/Measuring_Performance/Consensus_Development_Process.aspx)[Accessed January 20 2016]

    • Search Google Scholar
    • Export Citation
  • 15

    Paddock SMAdams JLHoces de la Guardia F: Better-than-average and worse-than-average hospitals may not significantly differ from average hospitals: an analysis of Medicare Hospital Compare ratings. BMJ Qual Saf 24:1281342015

    • Search Google Scholar
    • Export Citation
  • 16

    Rosenkrantz ABDoshi A: Characterizing the performance of the nation's hospitals in the Hospital Outpatient Quality Reporting Program's imaging efficiency measures. J Am Coll Radiol 12:1661732015

    • Search Google Scholar
    • Export Citation
  • 17

    Ryan AMNallamothu BKDimick JB: Medicare's public reporting initiative on hospital quality had modest or no impact on mortality from three key conditions. Health Aff (Millwood) 31:5855922012

    • Search Google Scholar
    • Export Citation
  • 18

    Safavi KCDai FGilbertsen TASchonberger RB: Variation in surgical quality measure adherence within hospital referral regions: do publicly reported surgical quality measures distinguish among hospitals that patients are likely to compare?. Health Serv Res 49:110811202014

    • Search Google Scholar
    • Export Citation
  • 19

    Shafi SParks JAhn CGentilello LMNathens ABHemmila MR: Centers for Medicare and Medicaid services quality indicators do not correlate with risk-adjusted mortality at trauma centers. J Trauma 68:7717772010

    • Search Google Scholar
    • Export Citation
  • 20

    Sheps MC: Approaches to the quality of hospital care. Public Health Rep 70:8778861955

  • 21

    Stulberg JJDelaney CPNeuhauser DVAron DCFu PKoroukian SM: Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA 303:247924852010

    • Search Google Scholar
    • Export Citation
  • 22

    Susarla SMGanske IHelliwell LMorris DEriksson EChun YS: Comparison of clinical outcomes and patient satisfaction in immediate single-stage versus two-stage implant-based breast reconstruction. Plast Reconstr Surg 135:1e8e2015

    • Search Google Scholar
    • Export Citation
  • 23

    Werner RMBradlow ET: Public reporting on hospital process improvements is linked to better patient outcomes. Health Aff (Millwood) 29:131913242010

    • Search Google Scholar
    • Export Citation
  • 24

    Werner RMBradlow ET: Relationship between Medicare's hospital compare performance measures and mortality rates. JAMA 296:269427022006

    • Search Google Scholar
    • Export Citation
  • 25

    Zuger A: Hospital ratings: a guide for the perplexed. JAMA 313:191119122015

Disclosures

The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.

Author Contributions

Conception and design: Bekelis. Acquisition of data: Bekelis. Analysis and interpretation of data: both authors. Drafting the article: Bekelis. Critically revising the article: both authors. Reviewed submitted version of manuscript: both authors. Statistical analysis: Bekelis. Administrative/technical/material support: Missios. Study supervision: Bekelis.

Supplemental Information

Online-Only Content

Supplemental material is available with the online version of the article.

If the inline PDF is not rendering correctly, you can download the PDF file here.

Article Information

INCLUDE WHEN CITING Published online March 18, 2016; DOI: 10.3171/2016.1.SPINE151155.

Correspondence Kimon Bekelis, Section of Neurosurgery, Dartmouth-Hitchcock Medical Center, 1 Medical Center Dr., Lebanon, NH 03756. email: kbekelis@gmail.com.

© AANS, except where prohibited by US copyright law.

Headings

References

  • 1

    Austin JMJha AKRomano PSSinger SJVogus TJWachter RM: National hospital ratings systems share few common scores and may generate confusion instead of clarity. Health Aff (Millwood) 34:4234302015

    • Search Google Scholar
    • Export Citation
  • 2

    Bekelis KGoodney RPDzebisashvili NGoodman DCBronner KK: Variation in the Care of Surgical Conditions: Cerebral Aneurysms Lebanon, NHDartmouth Institute for Health Policy and Clinical Practice2014

    • Search Google Scholar
    • Export Citation
  • 3

    Birkmeyer NJBirkmeyer JD: Strategies for improving surgical quality—should payers reward excellence or effort?. N Engl J Med 354:8648702006

    • Search Google Scholar
    • Export Citation
  • 4

    Cassel CKConway PHDelbanco SFJha AKSaunders RSLee TH: Getting more performance from performance measurement. N Engl J Med 371:214521472014

    • Search Google Scholar
    • Export Citation
  • 5

    Donabedian A: Evaluating the quality of medical care. 1966. Milbank Q 83:6917292005

  • 6

    Fung CHLim YWMattke SDamberg CShekelle PG: Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 148:1111232008

    • Search Google Scholar
    • Export Citation
  • 7

    Goff SLLagu TPekow PSHannon NSHinchey KLJackowitz TA: A qualitative analysis of hospital leaders' opinions about publicly reported measures of health care quality. Jt Comm J Qual Patient Saf 41:1691762015

    • Search Google Scholar
    • Export Citation
  • 8

    Gupta DPatel KLis CG: Self-rated health supersedes patient satisfaction with service quality as a predictor of survival in prostate cancer. Health Qual Life Outcomes 13:1372015

    • Search Google Scholar
    • Export Citation
  • 9

    Harder BComarow A: Hospital quality reporting by US News & World Report: why, how, and what's ahead. JAMA 313:190319042015

  • 10

    Lindenauer PKLagu TRoss JSPekow PSShatz AHannon N: Attitudes of hospital leaders toward publicly reported measures of health care quality. JAMA Intern Med 174:190419112014

    • Search Google Scholar
    • Export Citation
  • 11

    Lis CGPatel KGupta D: The relationship between patient satisfaction with service quality and survival in non–small cell lung cancer—is self-rated health a potential confounder?. PLoS One 10:e01346172015

    • Search Google Scholar
    • Export Citation
  • 12

    McGirt MJSperoff TDittus RSHarrell FEJ JrAsher AL: The National Neurosurgery Quality and Outcomes Database (N2QOD): general overview and pilot-year project description. Neurosurg Focus 34:1E62013

    • Search Google Scholar
    • Export Citation
  • 13

    Medicare Access and CHIP Reauthorization Act of 2015 Pub L No. 114-10 129 Stat 87

  • 14

    National Quality Forum: Consensus Development Process (http://www.qualityforum.org/Measuring_Performance/Consensus_Development_Process.aspx)[Accessed January 20 2016]

    • Search Google Scholar
    • Export Citation
  • 15

    Paddock SMAdams JLHoces de la Guardia F: Better-than-average and worse-than-average hospitals may not significantly differ from average hospitals: an analysis of Medicare Hospital Compare ratings. BMJ Qual Saf 24:1281342015

    • Search Google Scholar
    • Export Citation
  • 16

    Rosenkrantz ABDoshi A: Characterizing the performance of the nation's hospitals in the Hospital Outpatient Quality Reporting Program's imaging efficiency measures. J Am Coll Radiol 12:1661732015

    • Search Google Scholar
    • Export Citation
  • 17

    Ryan AMNallamothu BKDimick JB: Medicare's public reporting initiative on hospital quality had modest or no impact on mortality from three key conditions. Health Aff (Millwood) 31:5855922012

    • Search Google Scholar
    • Export Citation
  • 18

    Safavi KCDai FGilbertsen TASchonberger RB: Variation in surgical quality measure adherence within hospital referral regions: do publicly reported surgical quality measures distinguish among hospitals that patients are likely to compare?. Health Serv Res 49:110811202014

    • Search Google Scholar
    • Export Citation
  • 19

    Shafi SParks JAhn CGentilello LMNathens ABHemmila MR: Centers for Medicare and Medicaid services quality indicators do not correlate with risk-adjusted mortality at trauma centers. J Trauma 68:7717772010

    • Search Google Scholar
    • Export Citation
  • 20

    Sheps MC: Approaches to the quality of hospital care. Public Health Rep 70:8778861955

  • 21

    Stulberg JJDelaney CPNeuhauser DVAron DCFu PKoroukian SM: Adherence to surgical care improvement project measures and the association with postoperative infections. JAMA 303:247924852010

    • Search Google Scholar
    • Export Citation
  • 22

    Susarla SMGanske IHelliwell LMorris DEriksson EChun YS: Comparison of clinical outcomes and patient satisfaction in immediate single-stage versus two-stage implant-based breast reconstruction. Plast Reconstr Surg 135:1e8e2015

    • Search Google Scholar
    • Export Citation
  • 23

    Werner RMBradlow ET: Public reporting on hospital process improvements is linked to better patient outcomes. Health Aff (Millwood) 29:131913242010

    • Search Google Scholar
    • Export Citation
  • 24

    Werner RMBradlow ET: Relationship between Medicare's hospital compare performance measures and mortality rates. JAMA 296:269427022006

    • Search Google Scholar
    • Export Citation
  • 25

    Zuger A: Hospital ratings: a guide for the perplexed. JAMA 313:191119122015

TrendMD

Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 211 211 7
PDF Downloads 161 161 5
EPUB Downloads 0 0 0

PubMed

Google Scholar