The National Neurosurgery Quality and Outcomes Database (N2QOD): general overview and pilot-year project description

Free access

Given the unsustainable costs of US health care, universal agreement exists among payers, regulatory agencies, and other health care stakeholders that reform must include substantial improvements in the quality, effectiveness, and value of health care delivery. The Institute of Medicine and the American Recovery and Reinvestment Act of 2009 have called for the establishment of prospective registries to capture patient-centered data from real-world practice as a high priority to guide evidence-based reform. As a result, the American Association of Neurological Surgeons launched the National Neurosurgery Quality and Outcomes Database (N2QOD) and began enrolling patients in March 2012 into its initial pilot project: a web-based lumbar spine module. As a nationwide, prospective longitudinal registry utilizing patient reported outcome instruments, the N2QOD lumbar spine surgery pilot aims to systematically measure and aggregate surgical safety and 1-year postoperative outcome data from approximately 30 neurosurgical practices across the US with the primary aim of demonstrating the feasibility and validity of standardized 1-year outcome measurement from everyday real-world practice. At the end of the pilot year, 1) risk-adjusted modeling will be developed for the safety, quality, and effectiveness of lumbar surgical care (morbidity, readmission, improvements in pain, disability, quality of life, and return to work); 2) data integrity and validation will be demonstrated via internal quality control analyses and auditing, and 3) the feasibility of obtaining a high level of follow-up (~80%) of nationwide 1-year outcome measurement will be established. N2QOD will use only prospective clinical data, will avoid the use of administrative data proxies, and will rely on neurosurgically relevant risk factors for risk adjustment. Once national benchmarks of quality and effectiveness are accurately established and validated utilizing practice-based data extractors in the pilot year, N2QOD aims to introduce non–full-time employee (FTE)–dependent methodologies such as electronic medical record auto-extraction. N2QOD's non–FTE-dependent methodologies can then be validated against practice-based data extractor–derived measures of safety and effectiveness with the aim of more rapid expansion into the majority of US practice groups. The general overview, methods, and registry design of the N2QOD pilot year (lumbar module) are presented here.

Abbreviations used in this paper:AANS = American Association of Neurological Surgeons; CFR = Code of Federal Regulations; EMR = electronic medical record; EQ-5D = EuroQol Group–5D; FTE = full-time employee; HIPAA = Health Insurance Portability and Accountability Act of 1996; NPA = NeuroPoint Alliance, Inc.; NRS = numeric rating scale; N2QOD = National Neurosurgery Quality and Outcomes Database; ODI = Oswestry Disability Index; PHI = protected health information; QALY = quality-adjusted life-year; REDCap = Research Electronic Data Capture; VIMPH = Vanderbilt Institute for Medicine and Public Health.

Given the unsustainable costs of US health care, universal agreement exists among payers, regulatory agencies, and other health care stakeholders that reform must include substantial improvements in the quality, effectiveness, and value of health care delivery. The Institute of Medicine and the American Recovery and Reinvestment Act of 2009 have called for the establishment of prospective registries to capture patient-centered data from real-world practice as a high priority to guide evidence-based reform. As a result, the American Association of Neurological Surgeons launched the National Neurosurgery Quality and Outcomes Database (N2QOD) and began enrolling patients in March 2012 into its initial pilot project: a web-based lumbar spine module. As a nationwide, prospective longitudinal registry utilizing patient reported outcome instruments, the N2QOD lumbar spine surgery pilot aims to systematically measure and aggregate surgical safety and 1-year postoperative outcome data from approximately 30 neurosurgical practices across the US with the primary aim of demonstrating the feasibility and validity of standardized 1-year outcome measurement from everyday real-world practice. At the end of the pilot year, 1) risk-adjusted modeling will be developed for the safety, quality, and effectiveness of lumbar surgical care (morbidity, readmission, improvements in pain, disability, quality of life, and return to work); 2) data integrity and validation will be demonstrated via internal quality control analyses and auditing, and 3) the feasibility of obtaining a high level of follow-up (~80%) of nationwide 1-year outcome measurement will be established. N2QOD will use only prospective clinical data, will avoid the use of administrative data proxies, and will rely on neurosurgically relevant risk factors for risk adjustment. Once national benchmarks of quality and effectiveness are accurately established and validated utilizing practice-based data extractors in the pilot year, N2QOD aims to introduce non–full-time employee (FTE)–dependent methodologies such as electronic medical record auto-extraction. N2QOD's non–FTE-dependent methodologies can then be validated against practice-based data extractor–derived measures of safety and effectiveness with the aim of more rapid expansion into the majority of US practice groups. The general overview, methods, and registry design of the N2QOD pilot year (lumbar module) are presented here.

Health care in the US is at a defining crossroads. The recently passed Patient Protection and Affordable Care Act, signed into law March 23, 2010, has rapidly and dramatically shifted the focus of health care reform toward a critical analysis of quality in health care delivery.11 Universal agreement exists among payers, regulatory agencies, and other health care stakeholders that “value measures” taking into account both quality and cost variations should be used to modify physician reimbursement and drive reform. While significant resources are now being invested toward implementing the infrastructure for “value-based purchasing,” the methods by which to determine value and quality are being debated. Effectiveness of care (outcome measurement) is the numerator in the value equation. Health care costs are the denominator. At the heart of this evidence-driven health care reform process is the need to accurately record patient-centered data on safety and effectiveness of real-world care. Without access to data on effectiveness of care in everyday practice, health care reform will reward cheaper care rather than more valuable care. As these methods evolve, there exists an opportunity for physicians to impact the form and method by which the quality and value of health care is defined.

In March 2012, the AANS launched the National Neurosurgery Quality and Outcomes Database (N2QOD) (http://www.neuropoint.org/NPA%20N2QOD.html).2 The lumbar module was released as the initial 12-month pilot project to determine the feasibility of a nationwide, prospective longitudinal neurosurgical outcomes registry and assess the utility of its data. While the N2QOD lumbar spine pilot project and the subsequent cervical spine module focus on quality and effectiveness of spine surgery, the architecture of cranial modules is actively being designed. The quality of spine surgery manifests not only in 30-day morbidity, but equally in patient-reported outcomes and effectiveness. Simply surviving a lumbar laminectomy without a complication is not a sufficiently high marker of high-quality surgical spine care if the patient's back and leg pain remain unimproved. For appropriate risk-adjustment of surgical outcomes, a multitude of covariate risk factors in addition to the safety, effectiveness, and cost of neurosurgical interventions must be assessed across a wide range of practice settings, geographical locations, and pathology and patient subtypes. It is only by coordinating a nationwide quality and effectiveness registry that the volume and heterogeneity of care for such risk-adjusted benchmarking can be effectively achieved. This paper provides an overview of the aims and methods of the N2QOD pilot year lumbar module.

The National Neurosurgery Quality and Outcomes Database

Primary Aims of N2QOD

While nationwide aggregate data on the safety and effectiveness of spine care can be used to answer many questions relevant to evidence-based practice, policy, and comparative effectiveness, the primary aims of N2QOD are:

  1. To provide surgeons a risk-adjusted quality-reporting platform clinically relevant to neurosurgical disorders. Neurosurgery and spine providers, along with the treatments they deliver, will most certainly face inaccurate profiling during the health care reform process if entities introduce their own quality measurement systems into health care, utilizing poor risk adjustment and inaccurate proxies for outcome. The N2QOD aims to provide surgeons a quality measurement and feedback system that utilizes meaningful patient-centered data that are clinically relevant, with risk adjustment producing performance that does not penalize providers for treating high-risk diseases and patients. Through this system, providers will have the opportunity to learn which diseases, which patient groups, and which treatments are most effectively treated with surgery and identify areas for improving the quality of neurosurgical and spine care.

  2. To define nationwide benchmarks of 30-day surgical morbidity, perioperative mortality, return to work, and 1-year improvement in pain, disability, and quality of life after common surgical procedures. National risk-adjusted benchmarks for both the safety and the effectiveness of common neurosurgery and spine surgery procedures will be defined, allowing surgeons for the first time in US neurosurgery and spine care to accurately know whether their surgical safety and outcomes, for their unique patient population, lie within the expected range of national standards. By capturing 1-year patient-centered outcome data in real-world care throughout all regions and practice settings in US health care, the relative effectiveness of surgical procedures can be firmly defined for specific disease states and patient subgroups. These data can drive a truly evidence-based policy discussion across all payer groups, government agencies, and in the public forum. N2QOD is an opportunity to prove the effectiveness of neurosurgical and spine treatments.

  3. To provide practice groups their site-specific data, establishing the quality and effectiveness of their care for purposes of continual improvement and enhancing transparency and accountability of care. Each practice group will be able to use their measures of quality, safety, and effectiveness data to assess and continually improve care as well as meet their own unique challenges with hospital and payer negotiations and contracting. Comparisons of site-specific data will help identify best practices across the country that can be spread to other sites. Empowering providers with the evidence to prove the value of their care will greatly enhance neurosurgery and spine providers' ability to ensure their place in the local health care marketplace.

  4. To enhance shared and informed decision making. Creating a feedback loop of evidence on the expected safety (risks) and effectiveness (benefits) of surgery allows for improvements in decision making with respect to the quality of information on which decisions are made and the sharing of the decision-making process between the patient and the surgeon. Patients can be told what they may expect in terms of likelihood of surgical complications as well as the extent of expected improvement in quality of life, return to work, and pain improvement based on evidence most relevant to their care—data from their specific practice setting adjusted for their specific risk factors. Robust modeling of the national aggregate data can inform expectations for outcomes specific for each patient's personal characteristics. Providing feedback on risk-adjusted reports of expected safety and outcome specific for the patient's profile allows for both personalized medicine and enhanced informed and shared decision making. Through this process, N2QOD keeps the patient at the center of practice reform.

Primary Aims of the Lumbar Module Pilot Year

The aforementioned aims represent the primary goals of the greater N2QOD program. The lumbar module pilot has the following feasibility aims within the first 12 months:

  1. To demonstrate the feasibility of standardized 12-month outcomes data collection from approximately 30 practice groups across the US. The challenge of the pilot year is to determine whether neurosurgical practice groups of all types and from all regions of the US can be coordinated to a) collect and enter quality and outcomes data in a standardized fashion and b) collect 3- and 12-month outcomes with more than 80% follow-up. Because proper disease-based inclusion and exclusion and proper methods of case sampling are imperative to generate representative patient cohorts within the registry, the feasibility of applying standardized methodology to case enrollment and outcomes collection at every site will be assessed. One-year patient interview and outcome measurement follow-up rates will be assessed and analyzed for registry process and methods refinement.

  2. To develop quality assurance and data integrity methodology to assure accuracy within N2QOD. The strength of evidence generated by N2QOD is only as good as the accuracy of the data entered into the database. The goal of the pilot year is to populate a registry with less than 2% missing data and more than 98% data accuracy with auditing. The N2QOD web-based tools, methodology, and process will be critically analyzed at the conclusion of the pilot year to determine the extent of data entry feasibility, integrity, and validity.

Overview of Process and Methods

N2QOD is designed as a prospective longitudinal registry aimed to measure neurosurgical and spine surgical care as it exists in the real-world health care setting. Several processes have been incorporated to maximize feasibility of patient enrollment and data collection without altering or biasing health care delivery, to minimize confounding or bias in patient selection and measurement, and to build in extensive quality control and data integrity confirmation processes to ensure that the data entered into N2QOD accurately represent the care delivered both nationwide and at each site. The resources needed to participate in any of the disease-based N2QOD modules are 1) an EMR system, 2) a web-based data entry portal (arranged with a subscription to N2QOD), and 3) a data extractor (50% effort of a full-time employee). At many sites, the data extractor is an FTE whose position is funded by the hospital system or a nurse within the practice group.

The AANS, through the NeuroPoint Alliance, Inc. (NPA, http://www.neuropoint.org), provides oversight and continued development of N2QOD through a committee structure that is diversified in representation and expertise (Fig. 1). The AANS partnered with the Institute for Medicine and Public Health at Vanderbilt University (VIMPH) and the Department of Biostatistics of the Vanderbilt School of Medicine to provide third-party oversight, guidance, registry coordination, and analysis support. This collaboration provides N2QOD with a balance between clinical and scientific expertise as well as allowing increased transparency and unbiased oversight of its leadership.

Fig. 1.
Fig. 1.

Oversight and leadership of N2QOD. The unique challenges as they relate to the scientific advancement, operational execution, and business development of N2QOD are addressed within 2 committees (Scientific, Operations, and Business Committees), which report to the NPA board of directors (BOD) through the N2QOD Executive Committee (composed of the 2 chairs of each committee, VIMPH repesentatives, and the sitting president of the AANS).

Prior to launching N2QOD, each site must designate a surgeon representative, an administrative representative, and a data extractor. Each representative is issued a secure login access to the N2QOD web-based community forum through Google Apps as well as secure access to the web-based REDCap data entry portal.8 The N2QOD web-based community forum allows for open posts of questions and answers between site data extractors, as well as frequent postings relevant to the N2QOD community. Prior to patient accrual, data extractors are required to undergo training on data entry and N2QOD standard operating procedures. N2QOD is built on the infrastructure and data entry portal of REDCap. REDCap (Research Electronic Data Capture) is a secure web application for building and managing online surveys and databases. It provides an intuitive user interface that streamlines project development and improves data entry through real-time validation rules (with automated data type and range checks) and was developed specifically around HIPAA security guidelines. In addition to being used at Vanderbilt University, where it was developed, REDCap has been disseminated for local use at more than 270 other academic and nonprofit institutions in 49 countries on 6 continents (http://project-redcap.org).

Patient enrollment into the registry is a standardized process across all sites. The first 6 surgical cases meeting inclusion criteria each week are enrolled. The included diagnoses will vary for each disease-based module (for example, lumbar spine, cervical spine, cerebrovascular) and are based on strict clinical and imaging criteria. The 5 diagnoses that comprise the lumbar module in the pilot year are listed in Table 1. To allow for unbiased representative sampling, dates for each enrollment week are standardized across all sites and span a rolling 6-day cycle. Hence, the 1st day of each 6-day week falls on each weekday with equal frequency. This prevents a disproportionate volume of enrollment on any one day of the week or from any one surgeon's operative day. Data extractors review the operative case log the week prior to enrollment, determine diagnosis-based inclusion eligibility, and then contact the patient via telephone or in the clinic to obtain baseline measures of pain, disability, quality of life, and occupational variables. In addition, data extractors access the clinical chart to obtain baseline data for disease, imaging, and patient variables (Table 2). If data extractors are unsure of whether the available clinical and imaging data support inclusion or exclusion, consultation feedback from VIMPH and/or mentors who are knowledgeable and experienced extractors from other N2QOD centers matched to beginning data extractors is provided within 24 hours of a query to achieve a high quality of screening. The variables obtained at baseline for the lumbar spine module are listed in Table 3. All data are entered into N2QOD using the REDCap web-based portal with “point and click” data entry (Fig. 2). Validated patient-reported outcome instruments for pain, disability, and quality of life are used through a partnership with their proprietary owners, MAPI Research Trust and the EuroQol Group. The N2QOD data are securely stored for analysis and report generation by VIMPH (Fig. 3). Each site has the ability to extract their own data at any time from N2QOD in accordance with their institution's privacy and confidentiality regulations. This allows sites to access their site-specific data for uses that may be uniquely valuable for their own needs. The N2QOD electronic registry automatically notifies data extractors of 3-month and 1-year postoperative dates for follow-up data collection. Variables obtained for the lumbar module at 3 and 12 months are listed in Table 4. Data extractors contact patients during their clinic visit or via telephone 3 months ± 2 weeks, and 12 ± 1 months after surgery to obtain patient-reported outcome measurements. Outcome measures collected in the N2QOD lumbar surgery pilot include occupation capacity (return to work), back and leg pain (NRS), low-back specific disability (ODI), and QALY via the preference-based heath state measure EQ-5D (EuroQol Group).6,7,9,10 For practices administering these outcome instruments in clinic as their standard of care in the medical record, outcome data can be extracted from the EMR chart.

TABLE 1:

The 5 lumbar disease states included in the pilot-year lumbar spine module*

Disease StateDescription
I. symptomatic lumbar disc herniationDegenerative lumbar disc herniation that results in nerve root compression causing pain, weakness, or numbness in the distribution of the affected nerve root. Presence of disc herniation may be identified by MRI or CT.
II. symptomatic recurrent lumbar disc herniationDegenerative lumbar disc herniation at the same level & side of a prior surgical discectomy (nonfusion) that results in nerve root compression that causes pain, weakness, or numbness in the distribution of the affected nerve root. Presence of disc herniation may be identified by MRI or CT.
III. lumbar spondylolisthesisGrade I lumbar spondylolisthesis occurring from a congenital deformity of the pars interarticularis or from a degenerative process associated with spinal canal &/or foraminal stenosis that results in mechanical back pain &/or radiating leg pain or neurogenic claudication in the distribution of the affected nerve roots. Lumbar spondylolisthesis can be identified by either MRI or CT w/ an anterior or posterior slip of an adjacent vertebral body by no more than 25%.
IV. lumbar stenosisDegenerative narrowing of the central, lateral recess, or foraminal lumbar spinal canal w/o listhesis or dynamic instability that results in nerve root compression causing pain, weakness, numbness, or neurogenic claudication in the distribution of the affected nerve roots. Lumbar stenosis may be identified by MRI or CT.
V. lumbar adjacent segment diseaseDegenerative stenosis, listhesis, or mechanical instability of the motion segment adjacent to a prior fusion that results in mechanical back pain &/or nerve root compression that causes pain, weakness, numbness, or neurogenic claudication in the distribution of the affected nerve roots. Lumbar adjacent segment disease can be identified by MRI or CT w/ or w/o dynamic plain films.

* The first 6 patients per cycle undergoing surgical treatment for 1 of these 5 disease are enrolled without need for written informed consent. Patients are enrolled entirely on the basis of diagnoses, regardless of the specific type of surgery utilized.

TABLE 2:

Timeline for data collection*

Timeline of Patient Data CollectionWeek Prior to Op3 mos Postop ± 2 wks12 mos Postop ± 1 mo
review charts of consecutive cases on OR schedule for inclusionX
enter baseline chart data (patient enrollment & history)X
contact patient via phone/clinic to obtain baseline outcome measuresX
collect surgical & 90-day morbidity variables from chartX
contact patient via phone/clinic to obtain 3-mo outcome measuresX
contact patient via phone/clinic to obtain 12-mo outcome measuresX

* Data extractors screen the OR schedule the week before surgery and review the EMR to determine whether patients meet inclusion based on diagnosis criteria (radiology and clinic notes). Baseline patient, disease, and imaging data are obtained from the chart, while baseline pain, disability, occupational, and quality-of-life data are collected by telephone or in the preoperative clinic visit. Three months after surgery, treatment and 90-day perioperative morbidity data are collected via chart review, while outcome data are collected via patient interview via telephone or in clinic. Outcome data are collected again 12 months after surgery via telephone or clinic interview. Abbreviation: OR = operating room.

TABLE 3:

Lumbar spine module (pilot-year) baseline variables obtained from EMRs and patient interviews

Patient Enrollment Form (chart abstraction)
1. medical record no. (hospital MRN, optional)
2. Social Security no. (optional)
3. name & phone no. (optional)
4. date of birth
5. date of surgery
6. gender
7. height & weight
8. insurance payer
9. surgeon
10. hospital where surgery performed
Patient History Form (chart abstraction)
1. principal spine diagnosis (inclusion criteria)
2. major surgery w/in past yr
3. all patient comorbidities
4. predominant symptoms
5. motor deficits
6. duration of symptoms
7. prior surgery at same or adjacent level
8. disc collapse on MRI or CT scan (level of surgery only)
9. Modic endplate changes on MRI (level of surgery only)
10. listhesis (level of surgery only)
Patient Baseline Questions (patient interview)
1. ethnicity/race
2. level of education
3. Workers Compensation claim
4. liability or disability insurance claim
5. spinal injury caused by a motor vehicle or work related
6. occupational status, work details, intent to return to work
7. back & leg pain NRS (range 1–10)
8. ODI (range 0%–100%)
9. EQ-5D preference-based health state, QALY (range −0.1 to 1.0)
Fig. 2.
Fig. 2.

Example of web-based portal data entry using REDCap. A: Entry of perioperative surgical morbidity data. The lumbar module is designed as a “smart” portal that only prompts for details when relevant. Standardized data entry options allow for more data integrity. B: Entry of patient-reported outcome data from a patient interview to generate QALY assessment. The EQ-5D is used by N2QOD with permission from the EuroQol Group.

Fig. 3.
Fig. 3.

Schematic of N2QOD data collection and reporting. Data are entered in standardized format into N2QOD using the REDCap web-based portal. VIMPH performs data analysis and modeling and prepares site-specific reports of risk-adjusted safety and effectiveness. Quality control is a 2-way process between sites and VIMPH study coordinators. VIMPH provides weekly missing data reports and performs data validation auditing, while site coordinators provide feedback to VIMPH on the needs of process improvement via weekly coordinator teleconferences and a monthly practice-based learning network.

TABLE 4:

Lumbar spine module (pilot year) 3- and 12-month postoperative variables obtained from EMRs and patient interviews*

3-Month Data Entry
Surgery Form (chart abstraction)
 1. ASA grade
 2. surgical approach (anterior, posterior, 2-stage)
 3. decompression details & levels
 4. arthrodesis details & levels
 5. posterior instrumentation & details
 6. anterior instrumentation & details
 7. estimated blood loss
 8. date of surgery
 9. duration of surgery
90-Day Morbidity Form (chart abstraction)
 1. dates of admission & discharge
 2. place discharged to
 3. re-admit to hospital w/in 90 days of surgery
 4. return to OR w/in 90 days of surgery
 5. patient died w/in 90 days of surgery
 6. complications w/in 90-days of discharge from hospital
Patient Outcome & Satisfaction Form (patient interview)
 1. NASS patient satisfaction index
 2. return to work after surgery & details
 3. outside hospital readmission/reoperation
 4. back & leg pain NRS (range 1–10)
 5. ODI (range 0%–100%)
 6. EQ-5D preference-based health state, QALY (range −0.1 to 1.0)
12-Month Data Entry
Patient Outcome Form (patient interview)
 1. return to work after surgery & details
 2. same- or adjacent-level reoperation
 3. back & leg pain NRS (range 1–10)
 4. ODI (range 0%–100%)
 5. EQ-5D preference-based health state, QALY (range −0.1 to 1.0)

* ASA = American Society of Anesthesiologists; NASS = North American Spine Society.

Surgical Quality and Outcome Reporting

An important aim of the N2QOD registry is to provide subscribing practice groups with risk-adjusted annual reports on the safety and effectiveness of their surgical care (quality reports) along with national norms. Once national aggregate data, which are expected to represent tens of thousands of cases annually, have accrued substantial statistical precision, risk-adjusted models will be generated for each major endpoint of safety and effectiveness, identifying real-world variables that independently influence outcome. This will allow for robust risk adjustment and provide expected outcomes specific for each site's unique patient populations and health care setting variables. More specifically, each site will have reports of expected morbidity, length of hospital stay, and expected improvement in pain, disability, quality of life, return to work, and patient satisfaction based on the risk factor profile of their patient population. Primary endpoints of analysis for the lumbar module will include 1) perioperative measures (blood loss, length of stay, need for inpatient rehabilitation or skilled nursing, and 90-day morbidity, readmission, and reoperation data), 2) occupational outcome (return to work and quantified occupational capacity), 3) patient satisfaction, and 4) pain (NRS: 1–10), functional disability (ODI: 0%–100%), and quality of life (EQ-5D QALY: −0.11 to 1.0). Reports will provide annual and time-trended incidences of complications and 3-month and 12-month improvements in patient outcomes. Practices and surgeons will not be ranked, but rather they will be able to see where they lie within the variance of national norms for each quality endpoint, allowing for disease- or treatment-specific targeted improvement. These reports will only be seen by that unique site and all reports will be securely protected along with patient data at VIMPH. Nevertheless, all practice groups will have the data to demonstrate the effectiveness of their care and learn with evidence where they can improve. Reports on quality performance will be generated for multiple uses. Performance reports tailored to the following perspectives and purposes are currently planned: 1) the provider perspective for use in quality improvement and practice based learning, 2) the payer/hospital/market perspective for use in external marketing and negotiation, and 3) the patient perspective for use in the clinic for evidence-based shared decision making. It is the ultimate goal of N2QOD that these comprehensive practice performance reports add value to multiple areas of health care delivery not only for participating sites but also for the public.

Future Evolution of Methods

N2QOD is designed for continued evolution of data collection methodology. As experience and feedback grow from field experience with N2QOD, refinement of data collection technology is expected. The initial pilot year relies on a data extractor from every participating practice group. While the pilot process is aimed to minimize burden on any practice group's standard of care and cost, the most substantial investment for participating surgeon groups remains on average 50% of the time of a full-time employee. N2QOD plans to evolve into a registry system not dependent on a data extractor. Because accurate nationwide or regional incidence rates for surgical morbidity and outcomes are currently unknown, immediate use of EMR automated data extraction or automated data collection tools would generate outcome and morbidity incidence rates of unclear accuracy and with no gold standard to which to compare. Any automated data collection tools will need validation before policy and practice decisions can be derived from them. Once national benchmarks of quality and effectiveness are accurately established and validated utilizing the initial FTE-dependent methodology, N2QOD aims to introduce non-FTE dependent methodologies such as EMR automated data extraction using techniques from natural language processing. N2QOD's non–FTE-dependent methodologies can then be validated against FTE-dependent standards of safety and effectiveness measurements with the aim of more rapid expansion into the majority of US practice groups.

Basis for Registry Design

In the emerging environment of registry design, several approaches to patient entry have been described. In its pilot year, N2QOD will only be including surgically treated patients with very clear and standardized medically refractory spine diseases. The N2QOD disease-based entry criteria allows inclusion of surgical patients based entirely on their spinal pathology, regardless of their type of surgical treatment, allowing real-world comparative effectiveness evidence to be generated. Although N2QOD will not include patients undergoing medical treatment for spine symptoms in the first year, it remains a diagnosis-based inclusion registry for surgical disease. For example, evidence will be generated on the comparative effectiveness of facetectomy and fusion versus laminectomy and/or discectomy alone for recurrent disc herniation. Questions can be answered from real-word care: Does laminectomy and fusion versus laminectomy alone improve outcomes for lumbar stenosis with or without various radiographic degenerative disc disease findings, and if so, in which patient subtypes? What is the comparative effectiveness of minimally invasive approaches compared with traditional surgical treatments in the real world? Which patient subtypes are unlikely to benefit from surgical treatment and might therefore represent poor candidates for surgery despite failure of medical management? Patient entry into N2QOD is designed to capture patients with well-defined diseases downstream from when they present in primary care clinics with “symptoms” and poorly defined disease. Because spine surgery remains an option only for patients whose conditions are refractory to comprehensive medical management, it is in these cases that the safety and effectiveness of surgical care is relevant.

Whether to include medically treated patients as well as surgically treated patients in N2QOD has been discussed since 2009, when the idea of N2QOD was first formally discussed. It was concluded that the first priority was to assess outcomes in surgically relevant disease states and cases. Back pain is a symptom, not a diagnosis. Patients need to be more accurately categorized with a much more granular diagnosis classification scheme. Surgeons do not treat back pain per se but operate on only a small subset of patients with conditions amenable to surgical intervention. Even then, surgery is pursued in patients with chronic spine conditions only after failure of nonsurgical management. Patients differ by condition severity, natural history, response to nonsurgical and surgical treatment, and preference. Thus, 2 patients with an identical diagnosis (MRI findings and symptoms) may have completely different natural histories, treatment responses, and ultimately very different treatment needs. In real-world care, almost by definition, patients who continue to pursue medical treatment for “back symptoms” in primary care clinics 12 months after initial presentation are medical responders and do not represent an appropriate comparison group for surgical cohorts. Once surgical diseases and surgical patients are well defined and measured in N2QOD, methods to accurately identify similar and appropriate disease and patient subgroups within the “sea” of medical spine patients can be subsequently incorporated for comparison.

In addition, the primary goal of the N2QOD database is to measure quality of care, not just in outcomes but in the processes of care that produces these outcomes. Thus, as in the Society for Thoracic Surgery (STS) database, it is important to measure what surgeons do.

Lastly, there is enough detail and standardization of various clinical and imaging variables in N2QOD that matched patient subgroup analysis can be performed with medical registries with granularity similar to that of N2QOD. A surgical registry cannot accurately define its comparative medical patient population (a very small subset of medically managed patients) until it has firmly characterized its surgical cohorts as they exist in real-world care. N2QOD will begin to record data pertaining to the neurosurgical population in US health care and categorize this population; as it evolves it will allow meaningful comparisons to tightly defined real-world medical cohorts.

Ensuring High Quality and Valid Evidence

In an evolving landscape of health care quality measurement, there are several characteristics of N2QOD that separate it from other quality measurement efforts in health care. Many of these are discussed below.

Patient-Centered Clinical Measurement (High-Quality Data, Avoiding Proxies)

Nearly all high-quality reporting systems currently in use rely on administrative (nonclinical) data or structural or process surrogate measures of quality rather than actual patient-center outcomes or they focus solely on endpoints such as morbidity, mortality, length of stay, or hospital readmission. Furthermore, many are retrospective and lack standardized disease variables or classification schemes. In contrast, N2QOD is a prospective database that relies entirely on clinical data entered only by practice groups with clinical expertise in neurosurgical and spine disease. Standardized and tightly defined diagnosis criteria allow for more accurate and well-defined disease states. A multitude of surgical treatments are recorded, allowing a representative assessment of real-world therapies and surgical techniques. Only endpoints that are clinically meaningful to neurosurgical and spine diseases are included, and surrogate outcome measures are never substituted. Neurosurgery-specific and spine-specific risk factors that influence morbidity and outcome are collected as variables throughout the 1-year registry process for each patient and are specifically tailored for each disease state. Lastly, a high rate of capture of 1-year outcome data is a priority. To facilitate completeness of data, the N2QOD project currently provides participants with quality assurance reports on missing data and patient tracking and is preparing for a multitude of data collection options. The use of other tools for future data collection—such as mobile device applications, automated calling services, or open patient-community web portals—is currently being discussed.

Minimizing Bias and Confounding (Real-World Practice Setting, Avoiding Research Consent)

The advantage of prospective registries over randomized controlled trials lies not only in their timeliness, feasibility, and cost-effectiveness with respect to patient enrollment and data collection, but in their true representativeness of real-world care. Efficacy in artificially controlled research settings may not be generalizable to community health care delivery. In many real-world practices, patient populations, surgeon practice patterns and quality, and health care delivery resources are not accurately represented by the settings of randomized controlled trials, and hence, efficacy in the research setting does not translate to effectiveness in real-world care. N2QOD takes advantage of the key principles of well-designed prospective registry science to generate evidence that is maximally representative of real-world care by minimizing bias and confounding at almost every methodological level.

First, N2QOD is designated as a quality improvement rather than research enterprise and therefore allows collection of protected health information (PHI) data without the need for written informed consent. The Department of Health and Human Services, in response to mandates in HIPAA, set forth the Privacy and Security Rules (45CFR §160) and the Common Rule, which refers to Subpart A of 45 CFR §46.3–5 These regulatory rules guide the practice of all federally funded agencies and most private agencies with regard to oversight of ethical research standards, HIPAA mandates, and use of PHI. Recent determinations from the Office for Human Research Protection and Agency for Healthcare Research and Quality helped clarify the means by which N2QOD complies with the Common and Privacy and Security Rules to meet the criteria for quality improvement versus research. These governing documents state that quality registry data can be used to summarize health care provider performance or for any use which meets the definition of health care operations under the Privacy Rule (that is, conducting quality assessment and improvement activities, including outcomes evaluation and development of clinical guidelines provided that obtaining generalizable knowledge [research] is not the primary purpose of any studies resulting from such activities) (45CFR §164.501). Furthermore, N2QOD's electronic data storage system meets all criteria of PHI data protection in HIPAA's Privacy and Security Rules. A dramatic example of how the requirement for informed consent can significantly decrease the number of patients available for registry analysis and introduce selection bias in data collection was analyzed by Armstrong et al.1 Analysis of the University of Michigan Acute Coronary Syndrome Registry before and after HIPAA showed that participation rates decreased from 96% before HIPAA to 34% after HIPAA. Additionally, patients who gave consent were not representative of the larger treatment group. Capturing real-world care ensures that N2QOD's enrolled patient cohorts are not confounded subgroups of real-world patient populations, but truly representative.

Second, patient enrollment methods are also designed to produce accurate representation of practice groups' patients. Patient enrollment is standardized across all sites, disallowing any exclusion of patients otherwise meeting diagnosis criteria. Surgical schedule logs are archived to demonstrate transparency and auditability of patient selection processes. A rolling 6-day enrollment cycle (as described above) minimizes time-based and surgeon-based biasing. Patient selection and all endpoint measurement is taken out of the hands of the surgeon and is performed by an unbiased data extractor not involved with the patient's care. Standardized and representative sampling by a nonclinical care FTE coupled with lack of written informed consent allows N2QOD to measure care accurately as it occurs in its unaltered form, proving a true and representative measure of real-world health care delivery safety and effectiveness.

Quality Control and Validation

Quality control and auditing of processes and data are imperative to ensure that valid conclusions can be drawn from the evidence obtained in any measurement process. Regardless of patient volume or long-term follow-up rates, a high degree of missing data will limit the value of any analytical reports. Hence, N2QOD has invested significant resources into the ongoing real-time quality control of methods compliance and data integrity. Prior to completing and marking a baseline, 3-month, or 12-month case report form as complete, each site's data extractor receives a weekly missing data report from VIMPH. This process provides feedback and opportunity for review, which allows individual data extractors to fill in erroneous entries and to confirm when data are truly unavailable. The VIMPH project coordination office provides telephone or video conference consultation to sites to clarify any patient inclusion or process questions. A weekly teleconference is held between the VIMPH project coordination center and the sites' data extractors for process improvement and feedback. As sites demonstrate the ability to accurately capture 6 cases per week with process compliance and minimal missing data, enrollment volume is allowed to escalate to meet each site's specific needs. Sites are also provided a quarterly report demonstrating their overall compliance and missing data rates that may persist despite the weekly quality control feedback process. Sites falling outside of acceptable compliance are provided additional training webinars and process improvement consultation via the VIMPH project coordination center. Sites are expected to have missing data rates of less than 2%. At the national aggregate data level, quarterly missing data reports and process compliance reports are reviewed with the purpose of discussing evolution of N2QOD methods, variables, and reports. N2QOD is a learning registry system that is designed to use internal feasibility data to evolve and improve at the macro level as well as the site level.

N2QOD has also established a priori standardized methods of site audits, which are a central component of N2QOD. The registry is only as valuable as the integrity of the data entered into it. To ensure data accuracy and compliance with methods that are designed to minimize bias, several processes and data points are audited. Ten percent of practice groups undergo an on-site audit. Surgical schedule logs are reviewed to ensure that appropriate case selection criteria were met, minimizing the risk of “cherry picking” or inadvertently biased practices. Accuracy of diagnoses and treatment variables are confirmed via medical record audit. Major perioperative safety endpoints with respect to morbidity and readmission will also be confirmed as accurate. Although patient-reported outcome assessment will not be repeated to ensure retest validity, the process of outcome data collection will be audited. Convincing outside stakeholders of the validity of N2QOD will depend on quality assurance and site auditing validation analysis. N2QOD methods have been designed with this aim in mind.

N2QOD Pilot: Initial Feasibility

In March 2012, the AANS, through the NPA, launched the N2QOD registry. The first patient was enrolled that same month. In August 2012, 5 months after the lumbar pilot module launch, 21 sites were enrolling patients, and 24 sites had contracted with N2QOD. All sites expressing interest were allowed to join in a first-come, first-served basis. Enrollment remains open to all interested sites. Sites continue to be phased in with 2-month training cycles.

As of August, 5 months after the launch of N2QOD, 1357 patients were enrolled into the lumbar module from 21 sites, representing 120 neurosurgeons and 30 hospital systems. The rate of missing baseline clinical variables was less than 3%. Eight hundred patients had crossed their 3-month postoperative point with a 90% follow-up rate for 3-month outcome assessment. With 14 sites engaged in N2QOD contract processing in August 2012, the 25- to 30-site pilot goal is expected to be achieved. While the 12-month data capture rate will not be available until the spring of 2013, initial results of minimal missing data and high 3-month follow-up rates suggest that a prospective, nationwide, outcome-based registry will be feasible. Complete analysis of the 1st-year lumbar surgery pilot will be required in order to draw firm conclusions.

Conclusions

Neurosurgery and spine surgery providers have a choice: either define and continually improve the quality, effectiveness, safety and value of care, or have others do so. N2QOD is an opportunity for neurosurgeons and spine surgeons to define the metrics that will guide their delivery of care and systematically improve the process and outcomes of care. N2QOD differs from other current national registries in that 1) it relies solely on meaningful patient-centered clinical data and does not use administrative data or proxies of safety and outcome or a priori data points, 2) analysis design is driven by neurosurgical expertise, and 3) N2QOD has incorporated a robust quality assurance and data validation process to ensure that the end product of neurosurgery's national registry is trusted by all stakeholders in US health care. The N2QOD lumbar pilot year aims to 1) demonstrate the feasibility of coordinated nationwide 1-year outcomes collection from everyday practice with minimal missing data and a high degree of data integrity, and 2) define risk-adjusted benchmarks for surgical morbidity, return to work, and improvement in pain, disability, and quality of life following lumbar spine surgery. The greater N2QOD program aims to 1) evolve into a robust quality reporting platform for multiple neurosurgical disease states, 2) determine the value and effectiveness of real-world neurosurgical care in order to help guide evidence-based health care reform, and 3) provide practice groups the evidence they need locally for practice-based learning and quality improvement, increased informed shared patient decision making, and local health care marketplace negotiations. N2QOD is an opportunity to ensure that effective and valuable neurosurgery and spine care continues to be available to patients in the years to come.

Disclosure

The authors report no direct funding for preparation of this manuscript, but it describes the results of N2QOD, which has been set up with the support of AANS.

Author contributions to the study and manuscript preparation include the following. Conception and design: all authors. Analysis and interpretation of data: all authors. Drafting the article: all authors. Critically revising the article: McGirt, Speroff, Asher. Reviewed submitted version of manuscript: McGirt. Approved the final version of the manuscript on behalf of all authors: McGirt. Statistical analysis: McGirt, Speroff, Dittus, Harrell. Study supervision: McGirt, Speroff, Dittus.

References

  • 1

    Armstrong DKline-Rogers EJani SMGoldman EBFang JMukherjee D: Potential impact of the HIPAA privacy rule on data collection in a registry of patients with acute coronary syndrome. Arch Intern Med 165:112511292005

    • Search Google Scholar
    • Export Citation
  • 2

    Asher ALMcGirt MJGhogawala Z: The national Neurosurgery Quality and Outcomes Database (N2QOD) Through NeuroPoint Alliance. CNSQ 13:26282012

    • Search Google Scholar
    • Export Citation
  • 3

    Code of Federal Regulations: PART 46—Protection Of Human Subjects Washington, DCNational Archives and Records Administration(http://www.ecfr.gov/cgi-bin/text-idx?c=ecfr&SID=3e9644ce778582647d13ce9b5be62499&rgn=div5&view=text&node=45:1.0.1.1.25&idno=45) [Accessed October 16 2012]

    • Search Google Scholar
    • Export Citation
  • 4

    Code of Federal Regulations: PART 160—General Administrative Requirements Washington, DCNational Archives and Records Administration(http://www.ecfr.gov/cgi-bin/text-idx?c=ecfr&sid=f4ce45b46b27ab78d9a0f92eb09c86a8&rgn=div5&view=text&node=45:1.0.1.3.76&idno=45) [Accessed October 16 2012]

    • Search Google Scholar
    • Export Citation
  • 5

    Code of Federal Regulations: PART 164—Security And Privacy Washington, DCNational Archives and Records Administration(http://www.ecfr.gov/cgi-bin/text-idx?c=ecfr&sid=c36430f533ff7b69620177d423da0b76&rgn=div5&view=text&node=45:1.0.1.3.79&idno=45) [Accessed October 16 2012]

    • Search Google Scholar
    • Export Citation
  • 6

    EuroQol Group: EuroQol—a new facility for the measurement of health-related quality of life. Health Policy 16:1992081990

  • 7

    Fairbank JCPynsent PB: The Oswestry Disability Index. Spine (Phila Pa 1976) 25:294029522000

  • 8

    Harris PATaylor RThielke RPayne JGonzalez NConde JG: Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42:3773812009

    • Search Google Scholar
    • Export Citation
  • 9

    Jensen MPTurner JARomano JMFisher LD: Comparative reliability and validity of chronic pain intensity measures. Pain 83:1571621999

    • Search Google Scholar
    • Export Citation
  • 10

    Johnson JACoons SJErgo ASzava-Kovats G: Valuation of EuroQOL (EQ-5D) health states in an adult US sample. Pharmacoeconomics 13:4214331998

    • Search Google Scholar
    • Export Citation
  • 11

    Sorrel AL: Patient Protection and Affordable Care Act update. J Med Assoc Ga 101:10122012

If the inline PDF is not rendering correctly, you can download the PDF file here.

Article Information

Address correspondence to: Matthew J. McGirt, M.D., 4347 Village at Vanderbilt, Nashville, Tennessee 37232-8618. email: matt.mcgirt@vanderbilt.edu.

Please include this information when citing this paper: DOI: 10.3171/2012.10.FOCUS12297.

© AANS, except where prohibited by US copyright law.

Headings

Figures

  • View in gallery

    Oversight and leadership of N2QOD. The unique challenges as they relate to the scientific advancement, operational execution, and business development of N2QOD are addressed within 2 committees (Scientific, Operations, and Business Committees), which report to the NPA board of directors (BOD) through the N2QOD Executive Committee (composed of the 2 chairs of each committee, VIMPH repesentatives, and the sitting president of the AANS).

  • View in gallery

    Example of web-based portal data entry using REDCap. A: Entry of perioperative surgical morbidity data. The lumbar module is designed as a “smart” portal that only prompts for details when relevant. Standardized data entry options allow for more data integrity. B: Entry of patient-reported outcome data from a patient interview to generate QALY assessment. The EQ-5D is used by N2QOD with permission from the EuroQol Group.

  • View in gallery

    Schematic of N2QOD data collection and reporting. Data are entered in standardized format into N2QOD using the REDCap web-based portal. VIMPH performs data analysis and modeling and prepares site-specific reports of risk-adjusted safety and effectiveness. Quality control is a 2-way process between sites and VIMPH study coordinators. VIMPH provides weekly missing data reports and performs data validation auditing, while site coordinators provide feedback to VIMPH on the needs of process improvement via weekly coordinator teleconferences and a monthly practice-based learning network.

References

  • 1

    Armstrong DKline-Rogers EJani SMGoldman EBFang JMukherjee D: Potential impact of the HIPAA privacy rule on data collection in a registry of patients with acute coronary syndrome. Arch Intern Med 165:112511292005

    • Search Google Scholar
    • Export Citation
  • 2

    Asher ALMcGirt MJGhogawala Z: The national Neurosurgery Quality and Outcomes Database (N2QOD) Through NeuroPoint Alliance. CNSQ 13:26282012

    • Search Google Scholar
    • Export Citation
  • 3

    Code of Federal Regulations: PART 46—Protection Of Human Subjects Washington, DCNational Archives and Records Administration(http://www.ecfr.gov/cgi-bin/text-idx?c=ecfr&SID=3e9644ce778582647d13ce9b5be62499&rgn=div5&view=text&node=45:1.0.1.1.25&idno=45) [Accessed October 16 2012]

    • Search Google Scholar
    • Export Citation
  • 4

    Code of Federal Regulations: PART 160—General Administrative Requirements Washington, DCNational Archives and Records Administration(http://www.ecfr.gov/cgi-bin/text-idx?c=ecfr&sid=f4ce45b46b27ab78d9a0f92eb09c86a8&rgn=div5&view=text&node=45:1.0.1.3.76&idno=45) [Accessed October 16 2012]

    • Search Google Scholar
    • Export Citation
  • 5

    Code of Federal Regulations: PART 164—Security And Privacy Washington, DCNational Archives and Records Administration(http://www.ecfr.gov/cgi-bin/text-idx?c=ecfr&sid=c36430f533ff7b69620177d423da0b76&rgn=div5&view=text&node=45:1.0.1.3.79&idno=45) [Accessed October 16 2012]

    • Search Google Scholar
    • Export Citation
  • 6

    EuroQol Group: EuroQol—a new facility for the measurement of health-related quality of life. Health Policy 16:1992081990

  • 7

    Fairbank JCPynsent PB: The Oswestry Disability Index. Spine (Phila Pa 1976) 25:294029522000

  • 8

    Harris PATaylor RThielke RPayne JGonzalez NConde JG: Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42:3773812009

    • Search Google Scholar
    • Export Citation
  • 9

    Jensen MPTurner JARomano JMFisher LD: Comparative reliability and validity of chronic pain intensity measures. Pain 83:1571621999

    • Search Google Scholar
    • Export Citation
  • 10

    Johnson JACoons SJErgo ASzava-Kovats G: Valuation of EuroQOL (EQ-5D) health states in an adult US sample. Pharmacoeconomics 13:4214331998

    • Search Google Scholar
    • Export Citation
  • 11

    Sorrel AL: Patient Protection and Affordable Care Act update. J Med Assoc Ga 101:10122012

TrendMD

Cited By

Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 607 607 55
PDF Downloads 467 467 23
EPUB Downloads 0 0 0

PubMed

Google Scholar