Editorial: Does it pass the sniff test? Mining the NSQIP-P database for neurosurgical diseases

View More View Less
  • Department of Neurosurgery, Division of Pediatric Neurosurgery, University of Utah, Salt Lake City, Utah
Full access

If the inline PDF is not rendering correctly, you can download the PDF file here.

The National Surgical Quality Improvement Program Pediatric database (NSQIP-P), a centralized pediatric quality improvement tool compiled and maintained by the American College of Surgeons, features data from 60,000 de-identified pediatric cases treated in more than 50 medical centers in the United States. At the end of each week, trained medical reviewers identify cases, collect data from medical records pertaining to 150 nonspecialty-specific variables, and track 30-day outcomes. The promise and appeal of using the NSQIP-P database is obvious, and its use in evaluating patient outcomes is now being described. For instance, in one study, NSQIP-P information was used to analyze outcomes for CSF shunt surgery.3

In this issue, Cherian and colleagues describe their efforts to mine the NSQIP-P database for information related to myelomeningocele (MMC) closure.2 They present a detailed analysis of postoperative complications, return to the operating room, or unplanned readmissions within 30 days of postnatal myelomeningocele closure in 114 patients. Three general types of information were examined: epidemiological (patient age, gestational age, sex, race and ethnicity, weight, American Society of Anesthesiologists physical status); intraoperative (operative time in minutes, use of a subspecialty team for wound closure, simultaneous shunt placement); and 30-day outcomes (various NSQIP-P–defined complications, return to the operating room, unplanned admissions).

Key findings from the study include a 27% incidence of NSQIP-P–defined complications (42 complications in 31 patients), with postoperative wound complications being the most common (27 cases). Forty patients had an additional operative procedure within 30 days. Age at MMC repair was significantly correlated with postoperative complications, with immediate closure leading to more problems (OR 3.74). Of the 24 closures done on Day 0 of life, twice as many were larger than 5 cm in diameter (15% vs 27%). Twelve cases required an unplanned readmission within 30 days, with the most common reasons being hydrocephalus and the need for wound revision. A birth weight less than 2500 g was significantly associated with unplanned readmission (OR 4.1, 95% CI 1.07–16). Only 24 patients (21%) underwent ventriculoperitoneal shunt placement within 30 days of MMC closure, with 8 patients undergoing shunt placement at the time of MMC closure.

Several of these results are relatively surprising, especially the low rate of shunt placement 30 days after MMC closure. How do we reconcile these findings with other high-quality published data, such as the Management of Myelomeningocele Study (MOMS) trial? In that trial, an 84% shunting rate at 12 months was observed in infants with postnatal MMC closure.1 Cherian et al. suggest that a longer time for follow-up in the MOMS trial, as well as perhaps the more recent attitude of permissive ventriculomegaly in MMC patients, may explain these differences, but a substantial gap exists. To be fair, this issue is complex and dynamic, with attitudes toward modern treatment options in flux. Clearly, this issue will require further study.

Similarly, it is hard to make sense of the finding that the day of postnatal MMC closure should have a lot of bearing on wound complications. This factor has not been studied, but typically, other issues, such as the size of the MMC sac and how much extra tissue is available, dictate the quality of the closure, not the timing. These types of factors are not accounted for in this study. The authors found that even using a second subspecialty team (i.e., plastic surgery) to help with wound closure was not a significant factor in reducing wound complications. It may be that using Current Procedural Terminology code (work-related) methodology for coding case complexity is not accurate in this regard, as we cannot assume that MMC defect size is a proxy for wound difficulty. For instance, small defects at the hip level can be difficult to close, whereas large defects in the lumbar region with lots of extra tissue may be less challenging. Factors that most pediatric neurosurgeons would probably consider important when closing an MMC, such as dural closure technique, the use of a dural graft, the handling of the placode, and the use of tissue glue, were also not accounted for and not examined.

Another point to consider is that return to the operating room for treatment of hydrocephalus may not really be an MMC closure–related complication but instead is merely a manifestation of the natural history of MMC. A revised analysis without counting these procedures as “complications” would be very interesting. Likewise, the relatively small number of patients studied (114) potentially limits the power of the analysis. This point could be addressed more directly in the discussion.

The previous points are meant to highlight the fact that this study needs to be put into proper perspective. Perhaps its main message is to highlight the potential shortcomings of the NSQIP-P database for complex, disease-specific problems. The authors rightfully acknowledge this in their discussion, and they should be congratulated for their thoughtful analysis.

As for using the data derived from this study for benchmarking purposes or quality improvement initiatives, one should be very careful. This study's ultimate usefulness probably lies in pointing out the importance of disease-specific data collection rather than in using it as a standard of comparison. As it stands now, for future reference this study would have to be cited with an asterisk.

References

  • 1

    Adzick NS, , Thom EA, , Spong CY, , Brock JW III, , Burrows PK, & Johnson MP, : A randomized trial of prenatal versus postnatal repair of myelomeningocele. N Engl J Med 364:9931004, 2011

    • Search Google Scholar
    • Export Citation
  • 2

    Cherian J, , Staggers KA, , Pan IW, , Lopresti M, , Jea A, & Lam S: Thirty-day outcomes after postnatal myelomeningocele repair: a National Surgical Quality Improvement Program Pediatric database anaysis. J Neurosurg Pediatr [epub ahead of print June 3, 2016. DOI: 10.3171/2016.1.PEDS15674]

    • Search Google Scholar
    • Export Citation
  • 3

    Piatt JH Jr: Thirty-day outcomes of cerebrospinal fluid shunt surgery: data from the National Surgical Quality Improvement Program-Pediatrics. J Neurosurg Pediatr 14:179183, 2014

    • Search Google Scholar
    • Export Citation

Disclosures

The author reports no conflict of interest.

View More View Less
  • 1 Department of Neurosurgery, Texas Children's Hospital, and
  • 2 Baylor College of Medicine, Houston, Texas
Keywords:

Response

Thank you for the insightful editorial and the opportunity for conversation. We appreciate Dr. Brockmeyer's positive encouragement about our analysis and discussion. We agree that this study highlights the importance of disease-specific data collection as well as the potential shortcomings of the NSQIP-P database.

Our study examines data that are reported in NSQIP-P, which are available, not only to clinicians and researchers, but also to administrators and policy makers. Increasingly, researchers, hospital management, third-party payers, and governing bodies seek data and outcome measures by which to rate and assess health care performance. The utility and application of such endeavors has yet to be defined.

The American College of Surgeons advocates for use of the NSQIP-P with the goal of collecting perioperative clinical metrics by which to drive quality improvement initiatives. Unlike hospital administrative discharge data, trained NSQIP-P surgical clinical reviewers collect data by medical chart review. Indeed, 30-day outcomes have been proposed by general surgeons as quality indicators.2 However, we point out in our study that this time frame and these definitions are not particularly applicable to pediatric neurosurgery.

As Dr. Brockmeyer highlighted and as we emphasized in our discussion, the NSQIP-P database as it stands lacks attention to clinical details vital to the understanding of the complex conditions of pediatric neurosurgery patients.

For instance, pediatric neurosurgeons immediately recognize that return to the operating room within 30 days for CSF diversion is not necessarily a complication of MMC closure but rather the natural history of the disease itself. We conducted an alternative analysis without counting hydrocephalus surgeries as NSQIP-P complications; the significant findings of the study remained unchanged. Nevertheless, this divergence in definition of what constitutes planned or unplanned return to the operating room in NSQIP-P needs to be addressed in the near future, especially from the perspective of disease-specific management.

In terms of surgical treatment of hydrocephalus, 31% of the 114 MMC patients underwent ventriculoperitoneal shunt placement or endoscopic third ventriculostomy treatment by 1 month after MMC closure in the NSQIP-P data. This number at 1 month is lower than the published 84% shunt rate at 12 months reported in the MOMS trial.1,3 The timing of intervention on hydrocephalus after MMC closure has yet to be defined and studied in the literature. A preliminary retrospective review of our institution's experience reveals that shunt rates are indeed appreciably lower at 1 month than at 12 months, though this complex and interesting problem deserves dedicated multiinstitutional study in the future. This question remains unanswered and clearly cannot be addressed with an NSQIP-P study.

As delineated in our discussion of limitations, factors that pediatric neurosurgeons consider important in MMC closure, especially details related to anatomy and surgical technique, are not collected in NSQIP-P. We agree with Dr. Brockmeyer about the need to stress this shortcoming, as lack of specialty-specific and disease-specific data limits clinically relevant investigation and interpretation. Without strong pediatric neurosurgery representation in and reorganization of the NSQIP-P program, these details will continue to remain uncollected.

Most importantly, this study shows NSQIP-P limitations and advocates for disease-specific data collection. This message marks only the first step. While pediatric neurosurgeons may consider eschewing the reliability or usefulness of these data, NSQIP-P participation continues at an increasing number of pediatric centers in the United States, at considerable financial and manpower expense to each institution.

Outside parties acting from nonneurosurgical perspectives are unable to discern these limitations from non-specialty-specific databases. Collectively, our specialty may choose to offer other viable alternatives.

The field of pediatric neurosurgery needs to take notice and take ownership. It is critical that our specialty remains at the forefront of interpreting the significance and quality of our own work.

References

  • 1

    Adzick NS, , Thom EA, , Spong CY, , Brock JW III, , Burrows PK, & Johnson MP, : A randomized trial of prenatal versus postnatal repair of myelomeningocele. N Engl J Med 364:9931004, 2011

    • Search Google Scholar
    • Export Citation
  • 2

    Birkmeyer JD, , Hamby LS, , Birkmeyer CM, , Decke MV, , Karon MN, & Dow RW: Is unplanned return to the operating room a useful quality indicator in general surgery?. Arch Surg 136:405411, 2001

    • Search Google Scholar
    • Export Citation
  • 3

    Tulipan N, , Wellons JC III, , Thom EA, , Gupta N, , Sutton LN, & Burrows PK, : Prenatal surgery for myelomeningocele and the need for cerebrospinal fluid shunt placement. J Neurosurg Pediatr 16:613620, 2015

    • Search Google Scholar
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

Contributor Notes

ACCOMPANYING ARTICLE DOI: 10.3171/2016.1.PEDS15674.

INCLUDE WHEN CITING Published online June 3, 2016; DOI: 10.3171/2016.2.PEDS1639.

  • 1

    Adzick NS, , Thom EA, , Spong CY, , Brock JW III, , Burrows PK, & Johnson MP, : A randomized trial of prenatal versus postnatal repair of myelomeningocele. N Engl J Med 364:9931004, 2011

    • Search Google Scholar
    • Export Citation
  • 2

    Cherian J, , Staggers KA, , Pan IW, , Lopresti M, , Jea A, & Lam S: Thirty-day outcomes after postnatal myelomeningocele repair: a National Surgical Quality Improvement Program Pediatric database anaysis. J Neurosurg Pediatr [epub ahead of print June 3, 2016. DOI: 10.3171/2016.1.PEDS15674]

    • Search Google Scholar
    • Export Citation
  • 3

    Piatt JH Jr: Thirty-day outcomes of cerebrospinal fluid shunt surgery: data from the National Surgical Quality Improvement Program-Pediatrics. J Neurosurg Pediatr 14:179183, 2014

    • Search Google Scholar
    • Export Citation
  • 1

    Adzick NS, , Thom EA, , Spong CY, , Brock JW III, , Burrows PK, & Johnson MP, : A randomized trial of prenatal versus postnatal repair of myelomeningocele. N Engl J Med 364:9931004, 2011

    • Search Google Scholar
    • Export Citation
  • 2

    Birkmeyer JD, , Hamby LS, , Birkmeyer CM, , Decke MV, , Karon MN, & Dow RW: Is unplanned return to the operating room a useful quality indicator in general surgery?. Arch Surg 136:405411, 2001

    • Search Google Scholar
    • Export Citation
  • 3

    Tulipan N, , Wellons JC III, , Thom EA, , Gupta N, , Sutton LN, & Burrows PK, : Prenatal surgery for myelomeningocele and the need for cerebrospinal fluid shunt placement. J Neurosurg Pediatr 16:613620, 2015

    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 427 232 5
PDF Downloads 202 46 0
EPUB Downloads 0 0 0