Predicting nonroutine discharge after elective spine surgery: external validation of machine learning algorithms

Presented at the 2019 AANS/CNS Joint Section on Disorders of the Spine and Peripheral Nerves

Restricted access

OBJECTIVE

Nonroutine discharge after elective spine surgery increases healthcare costs, negatively impacts patient satisfaction, and exposes patients to additional hospital-acquired complications. Therefore, prediction of nonroutine discharge in this population may improve clinical management. The authors previously developed a machine learning algorithm from national data that predicts risk of nonhome discharge for patients undergoing surgery for lumbar disc disorders. In this paper the authors externally validate their algorithm in an independent institutional population of neurosurgical spine patients.

METHODS

Medical records from elective inpatient surgery for lumbar disc herniation or degeneration in the Transitional Care Program at Brigham and Women’s Hospital (2013–2015) were retrospectively reviewed. Variables included age, sex, BMI, American Society of Anesthesiologists (ASA) class, preoperative functional status, number of fusion levels, comorbidities, preoperative laboratory values, and discharge disposition. Nonroutine discharge was defined as postoperative discharge to any setting other than home. The discrimination (c-statistic), calibration, and positive and negative predictive values (PPVs and NPVs) of the algorithm were assessed in the institutional sample.

RESULTS

Overall, 144 patients underwent elective inpatient surgery for lumbar disc disorders with a nonroutine discharge rate of 6.9% (n = 10). The median patient age was 50 years and 45.1% of patients were female. Most patients were ASA class II (66.0%), had 1 or 2 levels fused (80.6%), and had no diabetes (91.7%). The median hematocrit level was 41.2%. The neural network algorithm generalized well to the institutional data, with a c-statistic (area under the receiver operating characteristic curve) of 0.89, calibration slope of 1.09, and calibration intercept of −0.08. At a threshold of 0.25, the PPV was 0.50 and the NPV was 0.97.

CONCLUSIONS

This institutional external validation of a previously developed machine learning algorithm suggests a reliable method for identifying patients with lumbar disc disorder at risk for nonroutine discharge. Performance in the institutional cohort was comparable to performance in the derivation cohort and represents an improved predictive value over clinician intuition. This finding substantiates initial use of this algorithm in clinical practice. This tool may be used by multidisciplinary teams of case managers and spine surgeons to strategically invest additional time and resources into postoperative plans for this population.

ABBREVIATIONS ACS = American College of Surgeons; ASA = American Society of Anesthesiologists; AUC = area under the ROC curve; CCI = Charlson Comorbidity Index; IQR = interquartile range; NPV = negative predictive value; NSQIP = National Surgical Quality Improvement Program; PPV = positive predictive value; RAT = Risk Assessment Tool; ROC = receiver operating characteristic; TCP = Transitional Care Program.
Article Information

Contributor Notes

Correspondence William B. Gormley: Brigham and Women’s Hospital, Harvard Medical School, Boston, MA. wgormley@bwh.harvard.edu.INCLUDE WHEN CITING Published online July 26, 2019; DOI: 10.3171/2019.5.SPINE1987.Disclosures The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper.
Headings
References
  • 1

    Backay ABystrzycki ASmit VKeogh MO’Reilly GMitra B: Accuracy of rapid disposition by emergency clinicians. Aust Health Rev 41:1331382017

    • Search Google Scholar
    • Export Citation
  • 2

    Berkowitz REFang ZHelfand BKIJones RNSchreiber RPaasche-Orlow MK: Project ReEngineered Discharge (RED) lowers hospital readmissions of patients discharged from a skilled nursing facility. J Am Med Dir Assoc 14:7367402013

    • Search Google Scholar
    • Export Citation
  • 3

    Best MJBuller LTFalakassa JVecchione D: Risk factors for nonroutine discharge in patients undergoing spinal fusion for intervertebral disc disorders. Iowa Orthop J 35:1471552015

    • Search Google Scholar
    • Export Citation
  • 4

    Bilimoria KYLiu YParuch JLZhou LKmiecik TEKo CY: Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons. J Am Coll Surg 217:83342.e1 32013

    • Search Google Scholar
    • Export Citation
  • 5

    Brier GW: Verification of forecasts expressed in terms of probability. Mon Weather Rev 78:131950

  • 6

    Cohen JBMyckatyn TMBrandt K: The importance of patient satisfaction: A blessing, a curse, or simply irrelevant? Plast Reconstr Surg 139:2572612017

    • Search Google Scholar
    • Export Citation
  • 7

    Collins GSReitsma JBAltman DGMoons KG: Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. BMC Med 13:12015

    • Search Google Scholar
    • Export Citation
  • 8

    Cook CCoronado RABettger JPGraham JE: The association of discharge destination with 30-day rehospitalization rates among older adults receiving lumbar spinal fusion surgery. Musculoskelet Sci Pract 34:77822018

    • Search Google Scholar
    • Export Citation
  • 9

    Cutler DMGhosh K: The potential for cost savings through bundled episode payments. N Engl J Med 366:107510772012

  • 10

    Gonçalves-Bradley DCLannin NAClemson LMCameron IDShepperd S: Discharge planning from hospital. Cochrane Database Syst Rev 1:CD0003132016

    • Search Google Scholar
    • Export Citation
  • 11

    Hansen LOGreenwald JLBudnitz THowell EHalasyamani LMaynard G: Project BOOST: effectiveness of a multihospital effort to reduce rehospitalization. J Hosp Med 8:4214272013

    • Search Google Scholar
    • Export Citation
  • 12

    Jack BWChetty VKAnthony DGreenwald JLSanchez GMJohnson AE: A reengineered hospital discharge program to decrease rehospitalization: a randomized trial. Ann Intern Med 150:1781872009

    • Search Google Scholar
    • Export Citation
  • 13

    Jones CEHollis RHWahl TSOriel BSItani KMMorris MS: Transitional care interventions and hospital readmissions in surgical populations: a systematic review. Am J Surg 212:3273352016

    • Search Google Scholar
    • Export Citation
  • 14

    Kanaan SFYeh HWWaitman RLBurton DCArnold PMSharma NK: Predicting discharge placement and health care needs after lumbar spine laminectomy. J Allied Health 43:88972014

    • Search Google Scholar
    • Export Citation
  • 15

    Karhade AVLarsen AMGCote DJDubois HMSmith TR: National databases for neurosurgical outcomes research: options, strengths, and limitations. Neurosurgery 83:3333442018

    • Search Google Scholar
    • Export Citation
  • 16

    Karhade AVOgink PThio QBroekman MCha TGormley WB: Development of machine learning algorithms for prediction of discharge disposition after elective inpatient surgery for lumbar degenerative disc disorders. Neurosurg Focus 45(5):E62018

    • Search Google Scholar
    • Export Citation
  • 17

    Kasparek MFBoettner FRienmueller AWeber MFunovics PTKrepler P: Predicting medical complications in spine surgery: evaluation of a novel online risk calculator. Eur Spine J 27:244924562018

    • Search Google Scholar
    • Export Citation
  • 18

    Liu JGormley NDasenbrock HHAglio LSSmith TRGormley WB: Cost-benefit analysis of transitional care in neurosurgery. Neurosurgery [epub ahead of print] 2018

    • Export Citation
  • 19

    McGirt MJSivaganesan AAsher ALDevin CJ: Prediction model for outcome after low-back surgery: individualized likelihood of complication, hospital readmission, return to work, and 12-month improvement in functional disability. Neurosurg Focus 39(6):E132015

    • Search Google Scholar
    • Export Citation
  • 20

    Murphy MEMaloney PRMcCutcheon BARinaldo LShepherd DKerezoudis P: Predictors of discharge to a nonhome facility in patients undergoing lumbar decompression without fusion for degenerative spine disease. Neurosurgery 81:6386492017

    • Search Google Scholar
    • Export Citation
  • 21

    Piazza MSharma NOsiemo BMcClintock SMissimer EGardiner D: Initial assessment of the risk assessment and the prediction tool in a heterogeneous neurosurgical patient population. Neurosurgery 85:50572019

    • Search Google Scholar
    • Export Citation
  • 22

    Porter ME: A strategy for health care reform—toward a value-based system. N Engl J Med 361:1091122009

  • 23

    Porter ME: What is value in health care? N Engl J Med 363:247724812010

  • 24

    Robertson FCLogsdon JLDasenbrock HHYan SCRaftery SMSmith TR: Transitional care services: a quality and safety process improvement program in neurosurgery. J Neurosurg 128:157015772018

    • Search Google Scholar
    • Export Citation
  • 25

    Seicean ASeicean SNeuhauser DBenzel ECWeil RJ: The influence of race on short-term outcomes after laminectomy and/or fusion spine surgery. Spine (Phila Pa 1976) 42:34412017

    • Search Google Scholar
    • Export Citation
  • 26

    Sheridan SLDonahue KEBrenner AT: Beginning with high value care in mind: A scoping review and toolkit to support the content, delivery, measurement, and sustainment of high value care. Patient Educ Couns 102:2382522019

    • Search Google Scholar
    • Export Citation
  • 27

    Steyerberg EW: Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer2008

    • Export Citation
  • 28

    Sullivan BMing DBoggan JCSchulteis RDThomas SChoi J: An evaluation of physician predictions of discharge on a general medicine service. J Hosp Med 10:8088102015

    • Search Google Scholar
    • Export Citation
  • 29

    Tsai TCJoynt KEOrav EJGawande AAJha AK: Variation in surgical-readmission rates and quality of hospital care. N Engl J Med 369:113411422013

    • Search Google Scholar
    • Export Citation
  • 30

    Ugiliweneza BKong MNosova KHuang KTBabu RLad SP: Spinal surgery: variations in health care costs and implications for episode-based bundled payments. Spine (Phila Pa 1976) 39:123512422014

    • Search Google Scholar
    • Export Citation
  • 31

    Vasquez RAChotai SFreeman THKay HFCheng JSMcGirt MJ: Impact of discharge disposition on 30-day readmissions following elective spine surgery. Neurosurgery 81:7727782017

    • Search Google Scholar
    • Export Citation
  • 32

    Vaziri SWilson JAbbatematteo JKubilis PChakraborty SKshitij K: Predictive performance of the American College of Surgeons universal risk calculator in neurosurgical patients. J Neurosurg 128:9429472018

    • Search Google Scholar
    • Export Citation
  • 33

    Veeravagu ALi ASwinney CTian LMoraff AAzad TD: Predicting complication risk in spine surgery: a prospective analysis of a novel risk assessment tool. J Neurosurg Spine 27:81912017

    • Search Google Scholar
    • Export Citation
  • 34

    Wang XHu YZhao BSu Y: Predictive validity of the ACS-NSQIP surgical risk calculator in geriatric patients undergoing lumbar surgery. Medicine (Baltimore) 96:e84162017

    • Search Google Scholar
    • Export Citation
TrendMD
Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 43 43 19
Full Text Views 24 24 8
PDF Downloads 27 27 13
EPUB Downloads 0 0 0
PubMed
Google Scholar