Development and validation of prediction scores for nosocomial infections, reoperations, and adverse events in the daily clinical setting of neurosurgical patients with cerebral and spinal tumors

View More View Less
  • 1 Department of Neurosurgery and
  • 2 Institute of Medical Informatics, University Hospital Münster, Germany
Free access

OBJECTIVE

Various quality indicators are currently under investigation, aiming at measuring the quality of care in neurosurgery; however, the discipline currently lacks practical scoring systems for accurately assessing risk. The aim of this study was to develop three accurate, easy-to-use risk scoring systems for nosocomial infections, reoperations, and adverse events for patients with cerebral and spinal tumors.

METHODS

The authors developed a semiautomatic registry with administrative and clinical data and included all patients with spinal or cerebral tumors treated between September 2017 and May 2019. Patients were further divided into development and validation cohorts. Multivariable logistic regression models were used to develop risk scores by assigning points based on β coefficients, and internal validation of the scores was performed.

RESULTS

In total, 1000 patients were included. An unplanned 30-day reoperation was observed in 6.8% of patients. Nosocomial infections were documented in 7.4% of cases and any adverse event in 14.5%. The risk scores comprise variables such as emergency admission, nursing care level, ECOG performance status, and inflammatory markers on admission. Three scoring systems, NoInfECT for predicting the incidence of nosocomial infections (low risk, 1.8%; intermediate risk, 8.1%; and high risk, 26.0% [p < 0.001]), LEUCut for 30-day unplanned reoperations (low risk, 2.2%; intermediate risk, 6.8%; and high risk, 13.5% [p < 0.001]), and LINC for any adverse events (low risk, 7.6%; intermediate risk, 15.7%; and high risk, 49.5% [p < 0.001]), showed satisfactory discrimination between the different outcome groups in receiver operating characteristic curve analysis (AUC ≥ 0.7).

CONCLUSIONS

The proposed risk scores allow efficient prediction of the likelihood of adverse events, to compare quality of care between different providers, and further provide guidance to surgeons on how to allocate preoperative care.

ABBREVIATIONS ACCI = age-adjusted CCI; AUC = area under the ROC curve; CCI = Charlson Comorbidity Index; CRP = C-reactive protein; ECOG = European Cooperative Oncology Group; INR = international normalized ratio; LEUCut = Leukocytosis, ECOG on admission, Urgency of surgery and Cutting-suture time of index surgery; LINC = Leukocytosis, length of stay in the ICU, Nursing care level, and CRP on admission; LOS = length of stay; NoInfECT = Nursing care level, length of stay on the ICU, Emergency admission, CRP on admission, and recurrenT diagnosis; ROC = receiver operating characteristic.

OBJECTIVE

Various quality indicators are currently under investigation, aiming at measuring the quality of care in neurosurgery; however, the discipline currently lacks practical scoring systems for accurately assessing risk. The aim of this study was to develop three accurate, easy-to-use risk scoring systems for nosocomial infections, reoperations, and adverse events for patients with cerebral and spinal tumors.

METHODS

The authors developed a semiautomatic registry with administrative and clinical data and included all patients with spinal or cerebral tumors treated between September 2017 and May 2019. Patients were further divided into development and validation cohorts. Multivariable logistic regression models were used to develop risk scores by assigning points based on β coefficients, and internal validation of the scores was performed.

RESULTS

In total, 1000 patients were included. An unplanned 30-day reoperation was observed in 6.8% of patients. Nosocomial infections were documented in 7.4% of cases and any adverse event in 14.5%. The risk scores comprise variables such as emergency admission, nursing care level, ECOG performance status, and inflammatory markers on admission. Three scoring systems, NoInfECT for predicting the incidence of nosocomial infections (low risk, 1.8%; intermediate risk, 8.1%; and high risk, 26.0% [p < 0.001]), LEUCut for 30-day unplanned reoperations (low risk, 2.2%; intermediate risk, 6.8%; and high risk, 13.5% [p < 0.001]), and LINC for any adverse events (low risk, 7.6%; intermediate risk, 15.7%; and high risk, 49.5% [p < 0.001]), showed satisfactory discrimination between the different outcome groups in receiver operating characteristic curve analysis (AUC ≥ 0.7).

CONCLUSIONS

The proposed risk scores allow efficient prediction of the likelihood of adverse events, to compare quality of care between different providers, and further provide guidance to surgeons on how to allocate preoperative care.

ABBREVIATIONS ACCI = age-adjusted CCI; AUC = area under the ROC curve; CCI = Charlson Comorbidity Index; CRP = C-reactive protein; ECOG = European Cooperative Oncology Group; INR = international normalized ratio; LEUCut = Leukocytosis, ECOG on admission, Urgency of surgery and Cutting-suture time of index surgery; LINC = Leukocytosis, length of stay in the ICU, Nursing care level, and CRP on admission; LOS = length of stay; NoInfECT = Nursing care level, length of stay on the ICU, Emergency admission, CRP on admission, and recurrenT diagnosis; ROC = receiver operating characteristic.

In Brief

The aim of this study was to develop easy-to-use risk scoring systems for nosocomial infections, reoperations, and adverse events for patients with cerebral and spinal tumors. The proposed risk scores allow efficient prediction of the likelihood of adverse events, to compare quality of care between different providers and further provide guidance to surgeons on how to allocate preoperative care.

Measuring the quality of delivered care has gained increasing attention in all medical fields over the past decade. Healthcare administrators have begun to consider quality aspects for remuneration purposes, an aspect that has also been implemented in the field of neurosurgery.1 For this purpose, various performance and quality indicators are under current investigation, aiming at measuring the quality of care. Among them are the 30-day reoperation rate, the 30-day readmission rate, and nosocomial infection and mortality rates.2–7

Not only should quality indicators be used for measuring the quality of care, but also they could be of potential use for benchmarking purposes to compare the outcome and quality of different neurosurgical departments. As departments treat patients with different complexity levels and varying baseline medical comorbidities, methods of adequate risk adjustment are indispensable.

Efforts to develop models to predict the likelihood of adverse outcome have been made at the healthcare provider level. For this purpose, administrative data from large databases are often used, as they appear to be attractive per se, since the data are evaluable without additional financial outlay and are readily available and retrievable.8,9 However, administrative data have been shown to not reflect the endogenous patient risk.10

Still, there is only little information about risk factors and methods for adequate risk adjustment to compare different healthcare providers. Building on the results of the first prospective study on outcome predictors in neurosurgery by Reponen et al., we aimed at developing a high-quality clinical data registry with prospective data entry.11 This data registry combined administrative and clinical data to design novel risk prediction models for predicting different quality outcome measures in neurosurgical patients with cerebral and spinal tumors.

Methods

All adult patients who were diagnosed with a primary or recurrent, benign or malignant spinal or brain tumor and hospitalized between September 2017 and May 2019 at the Department of Neurosurgery, University Hospital Münster, were prospectively included in this study. Patients who had previously undergone treatment of the tumor in a different neurosurgical department and admitted to our department for a surgical complication were excluded from the analysis.

A semiautomatic database was designed in collaboration with the Institute of Medical Informatics. The database automatically retrieved baseline and administrative data from the digital medical records of each patient such as age, sex, data on admission, duration and timing of surgery, and laboratory results on admission (leukocytes, platelets, C-reactive protein [CRP], and prothrombin time/international normalized ratio [INR]). Blood results were further stratified into clinically relevant stages: leukopenia (< 3910 leukocytes/μl), leukocytosis (> 10,900 leukocytes/μl), CRP > 1 mg/L, and INR > 1.2 as defined by the laboratory reference ranges. Continuous variables such as age and cutting-suture time (i.e., time from incision to suture) were converted into categorial variables, using the median as the cutoff value. For the development of risk scores, it is considered useful to transfer numeric variables into categorial variables to discriminate between two values and assign the patients to the various risk groups. BMI was stratified according the WHO obesity classification. The European Cooperative Oncology Group (ECOG) performance status was stratified into 0 and ≥ 1 to differentiate asymptomatic patients from patients who show symptoms of their underlying disease. Nursing care level in the German context refers to persons with a permanent significant limitation in everyday life competence and having a need for assistance in the field of basic nursing and household assistance, and it is comparable to the US nursing home level of care. For our purpose, two categories were defined, discriminating patients with a permanent significant limitation in everyday life competence (any nursing care level) from those without.

After craniotomy for brain tumor resection, patients routinely stay 1 day in the ICU at our department. We therefore stratified length of stay (LOS) in the ICU into ≤ 1 and ≥ 2 days. Information regarding neuropathological results, secondary diagnoses, surgical outcome, and adverse events occurring during the inpatient stay was manually obtained by clinicians from medical records and added to the database. Neurosurgical diagnosis was subdivided into different groups, such as glioma, meningioma, metastasis, and lymphoma.

Secondary diagnoses, already present on admission, were classified into the 19 items of the Charlson Comorbidity Index (CCI)12 and supplemented by other metrics associated with a possible impact on surgical outcome, such as decubitus ulcers, drug abuse, epilepsy, coagulopathy, previous radio- or chemotherapy, paralysis, nicotine abuse, oral anticoagulation, use of antiplatelet drugs, and stroke. The age-adjusted CCI (ACCI) was calculated as described before.12,13

Regarding selection of variables, we included variables that have been proved to impact related outcome measures (e.g., CCI12) that are part of existing data registries (e.g., National Neurosurgical Audit Program [NNAP]14 and National Neurosurgery Quality and Outcomes Database [N2QOD],15 currently known as the Quality Outcomes Database [QOD]) or variables that we believed could potentially impact the outcome measure based on our clinical experience (e.g., cutting-suture time and type of surgery).

The primary outcome measures of this study were 30-day unplanned reoperation rates, 30-day unplanned readmission rates, 30-day mortality rates, and nosocomial and surgical site infection rates within 30 days, as defined before.5 In addition, the variable “any adverse event” was calculated, describing the occurrence of one of these mentioned outcomes. Readmissions to hospitals other than the index center were not recorded. Inclusion and analysis were case-based and not patient-specific. The study was approved by the local ethics committee, and patient consent was obtained in each case.

Statistical Analysis

IBM SPSS Statistics version 25.0 software (IBM Corp.) was used for statistical analysis. All cases were randomly assigned into a development group (n = 700) or a validation group (n = 300). Data were described using standard statistics with absolute and relative frequencies for categorical variables and mean, median, and range for continuous variables, as appropriate. The chi-square test and univariate logistic regression modeling were applied to identify differences regarding the proportion of the variables, as appropriate. Variables that had an effect on primary outcome measures were identified within the development group. Variables that were statistically significant in univariate analyses were entered into a multivariable logistic regression model. Odds ratios (ORs) were obtained with corresponding 95% confidence intervals (CIs). In order to develop prognostic scores, we used the β coefficients of the prognostic relevant variables and assigned points proportional to the coefficient. The population was divided into three categories: patients at low, intermediate, and high risk for the occurrence of any of the outcome measures. For internal validation of the score, the validation group was used. To assess the risk prediction model, receiver operating characteristic (ROC) curves of both groups were conducted, with an area under the curve (AUC) ≥ 0.7 reflecting reasonable discrimination results. A probability value < 0.05 was considered statistically significant throughout the entire analyses. All reported p values are two-sided. Cases with missing information about one variable were excluded from the corresponding statistical analyses but not from the entire study. For the analysis of reoperation, only cases with surgery during index admission were considered. Statistical models were conceptualized in collaboration with statisticians. This study is reported with respect to the STROBE Statement.16

Results

In total, 1000 cases of patients with both benign and malignant cerebral and spinal tumors were included in the study. Most patients were diagnosed with glioma (n = 365, 36.5%). Baseline characteristics as well as details on the development and validation cohort are summarized in Table 1. The development group contained more patients aged between 18 and 56 years (p = 0.03). There were no further statistically relevant differences between both cohorts.

TABLE 1.

Baseline characteristics of all cases stratified into development and validation cohorts

CategoryAll CasesDevelopment GroupValidation Groupp Value
Median, range56.0 (18–91)55.0 (18–91)57.0 (18–85)0.84
Age, yrs18–56515 (51.5)376 (53.7)139 (46.3)0.03
>56485 (48.5)324 (46.3)161 (53.7)
SexMale488 (48.8)346 (49.4)142 (47.3)0.54
Female512 (51.2)354 (50.6)158 (52.7)
Mean, range25.72 (15.8–63.7)25.93 (15.8–54.1)25.56 (17.2–63.7)0.25
BMI, kg/m2<18.521 (2.3)15 (2.3)6 (2.2)
18.5–24.9388 (41.9)271 (41.7)117 (42.4)0.98
>24.9517 (55.8)364 (56.0)153 (55.4)
Glioma365 (36.5)253 (36.1)112 (37.3)
Cranial meningioma154 (15.4)101 (14.4)53 (17.7)
Cerebral metastasis141 (14.1)101 (14.4)40 (13.3)
Sellar tumor100 (10.0)77 (11.0)23 (7.7)
Vestibular schwannoma37 (3.7)19 (2.7)18 (6.0)
Cerebral lymphoma16 (1.6)12 (1.7)4 (1.3)
Neurosurgical diagnosisOther benign brain tumors35 (3.5)25 (3.6)10 (3.3)0.29
Other malignant brain tumors18 (1.8)13 (1.9)5 (1.7)
Spinal meningioma60 (6.0)45 (6.4)15 (5.0)
Spinal metastasis31 (3.1)25 (3.6)6 (2.0)
Spinal lymphoma9 (0.9)5 (0.7)4 (1.3)
Other benign spinal tumors21 (2.1)14 (2.0)7 (2.3)
Other malignant spinal tumors13 (1.3)10 (1.4)3 (1.0)
Tumor dignityBenign391 (39.1)275 (39.3)116 (38.7)0.85
Malignant609 (60.9)425 (60.7)184 (61.3)
Primary vs recurrent tumorPrimary596 (59.6)407 (58.1)189 (63.0)0.15
Recurrent404 (40.4)293 (41.9)111 (37.0)
ACCI≤2479 (47.9)344 (49.1)135 (45.0)0.23
>2521 (52.1)356 (50.9)165 (55.0)
Alcohol abuse16 (1.6)12 (1.7)4 (1.3)0.66
Tobacco abuse99 (9.9)70 (10.0)29 (9.7)0.87
Arterial hypertension311 (31.1)211 (30.1)100 (66.7)0.32
Secondary diagnoses on admissionAnemia445 (44.5)309 (44.1)136 (45.3)0.73
Chemotherapy w/in 6 wks57 (5.7)44 (6.3)13 (4.3)0.22
Stroke25 (2.5)13 (1.9)12 (4.0)0.05
Epilepsy187 (18.7)132 (18.9)55 (18.3)0.85
Pulmonary embolism11 (1.1)8 (1.1)3 (1.0)0.84
No. of secondary diagnoses on admission≤1878 (87.8)608 (86.9)270 (90.0)0.16
>1122 (12.2)92 (13.1)30 (10.0)
Leukopenia, leukocytes/μl<391049 (4.9)36 (5.1)13 (4.3)0.59
Leukocytosis, leukocytes/μl>10,900199 (19.9)142 (20.3)57 (19.0)0.64
Thrombocytopenia, leukocytes/μl≤100,00011 (1.1)7 (1.0)4 (1.3)0.64
CRP, mg/L≤1838 (83.8)583 (83.3)255 (85.0)0.5
>1162 (16.2)117 (16.7)45 (15.0)
INR≤1.231 (3.1)25 (3.6)6 (2.0)0.19
>1.2969 (96.9)675 (96.4)294 (98.9)
Nursing care levelNone913 (94.3)639 (94.8)274 (93.2)0.32
Any55 (5.7)35 (5.2)20 (6.8)
I93 (10.5)65 (9.3)28 (9.3)
ASA classII579 (65.6)407 (58.1)172 (57.3)0.96
III206 (23.2)143 (20.4)63 (21.0)
IV5 (0.6)4 (0.6)1 (0.3)
3–1211 (1.1)10 (1.4)1 (0.3)
GCS score13 or 1436 (3.6)24 (3.4)12 (4.0)0.29
15953 (95.3)666 (95.1)287 (95.7)
ECOG PS0652 (62.5)462 (66.0)190 (63.3)0.42
≥1348 (34.8)238 (34.0)110 (36.7)
Home752 (75.2)524 (74.9)228 (76.0)
Other hospital203 (20.3)143 (20.4)60 (20.0)
Discharge locationRehabilitation clinic3 (0.3)2 (0.3)1 (0.3)0.92
Nursing home2 (0.2)1 (0.1)1 (0.3)
Other department w/in hospital40 (4.0)30 (4.3)10 (3.3)
Urgency of admissionElective885 (88.5)614 (87.7)271 (90.3)0.23
Emergency115 (11.5)86 (12.3)29 (9.7)
Urgency of surgeryElective840 (93.8)586 (93.6)254 (94.1)0.79
Emergency56 (6.2)40 (6.4)16 (5.9)
0104 (10.4)74 (10.6)30 (10.0)
No. of surgeries1827 (82.7)580 (82.9)247 (82.3)0.80
≥269 (6.9)46 (6.6)23 (7.7)
Cutting-suture time, mins0–200437 (48.8)312 (49.8)125 (46.3)0.33
>200459 (51.2)314 (50.2)145 (53.7)
Biopsy vs resectionResection797 (89.0)559 (89.3)238 (88.1)0.62
Biopsy99 (11.0)67 (10.7)32 (11.9)
ICU LOS, days≤1904 (90.4)641 (91.6)263 (87.7)0.06
≥296 (9.6)59 (8.4)37 (12.3)

ASA = American Society of Anesthesiologists; GCS = Glasgow Coma Scale; PS = performance status.

Values represent the number of cases (%) unless stated otherwise. Percentages are based on the number of cases with existing data for that variable. Boldface type indicates statistical significance.

Outcome Measures

The 30-day unplanned readmission rate was 4.5% (n = 45). An unplanned neurosurgical reoperation within 30 days after index surgery was observed in 61 cases (6.8%). Surgical site infections were documented in 1.8% (n = 18). The nosocomial infection rate was 7.4% (n = 74). The overall mortality rate after 30 days was 1.1% (n = 11). The median LOS was 7.17 days (range 1–73 days). There were no statistically significant differences between the outcome measures and the underlying tumor entity. Any adverse event occurred in 14.5% (n = 145). In addition, the outcome measures did not show any significant differences between the development and validation cohorts.

Risk Factors for Outcome Measures and Adverse Events

Leukocytosis present on admission was a risk factor for an unplanned reoperation within 30 days (OR 2.85, 95% CI 1.32–6.16; p < 0.001) and was also associated with any adverse event (OR 1.83, 95% CI 1.02–3.28; p < 0.001). Patients treated for a recurrent tumor were at higher risk for developing a surgical site infection (OR 29.39, 95% CI 1.85–465.68; p = 0.02). Further independent risk predictors from the multivariate analysis are summarized in Table 2.

TABLE 2.

Multivariate logistic regression model predicting 30-day unplanned reoperations, unplanned readmissions, and mortality; surgical site infections; nosocomial infections; and any adverse event

95% CI
CategoryORLower LimitUpper Limitp Value
Nosocomial infection*
 Nursing care levelNoneRef<0.001
Any6.941.6329.58
 ICU LOS, days≤1Ref<0.001
≥212.424.2136.62
 Urgency of admissionElectiveRef0.03
Emergency3.361.149.90
 CRP, mg/L≤1Ref<0.001
>13.341.417.92
 Primary vs recurrent tumorPrimaryRef<0.001
Recurrent3.401.547.53
30-day unplanned reoperation
 Leukocytosis, leukocytes/μl≤10,900Ref<0.001
>10,9002.851.326.16
 ECOG PS0Ref0.02
≥14.281.2115.16
 Urgency of surgeryElectiveRef<0.001
Emergency5.401.6317.90
 Cutting-suture time, mins0–200Ref<0.001
>2003.441.527.78
Any adverse event
 Leukocytosis, leukocytes/μl≤10,900Ref<0.001
>10,9001.831.023.28
 ICU LOS, days≤1Ref<0.001
≥211.425.7022.88
 Nursing care levelNoRef<0.001
Any5.141.9113.85
 CRP, mg/L≤1Ref<0.001
>12.531.354.74
30-day unplanned readmission§
 Age, yrs0–562.571.045.860.041
>56Ref
 GliomaNoRef0.03
Yes2.341.075.13
30-day mortality
 Urgency of admissionElectiveRef<0.001
Emergency15.092.04111.89
Surgical site infection**
 Primary vs recurrent tumorPrimaryRef0.02
Recurrent29.391.85465.68
 Cranial meningiomaNoRef0.03
Yes19.751.28305.77

The following is a list of variables that were included in each model but were not statistically significant.

ACCI, age, anemia, BMI, ECOG performance status, leukocytosis, new neurological deficit, number of secondary diagnoses, number of surgeries, pulmonary embolism, thrombocytopenia, urgency of admission, and urgency of surgery.

American Society of Anesthesiologists class, Glasgow Coma Scale score, number of secondary diagnoses, and nursing care level.

ACCI, anemia, degree of care, ECOG performance status, urgency of admission, and urgency of surgery.

Age, anemia, chemotherapy within 6 weeks, glioma, nursing care level, and primary versus recurrent tumor.

ACCI, biopsy versus resection, CRP, ECOG performance status, Glasgow Coma Scale score, ICU LOS, leukocytosis, location before admission, number of secondary diagnoses, nursing care level, and thrombocytopenia.

Anemia, INR, number of surgeries, sex, stroke, urgency of admission, urgency of surgery, and vestibular schwannoma.

Model Development and Estimation of Risk Scores

We developed easy-to-use prognostic scores for stratifying 30-day reoperation, nosocomial infection, and any adverse events (Table 3). We also attempted to develop models for the other outcome measures, surgical site infection, 30-day mortality, and 30-day readmission. However, the model performance was low when using ROC analysis.

TABLE 3.

Calculation of the scores to predict the occurrence of nosocomial infections, 30-day unplanned reoperations, and any adverse event

Categoryβ Coefficientp ValueScore
NoInfECT score
 Nursing care levelNoneRef0.009+20
Any1.96
 ICU LOS, days≤1Ref<0.001+25
≥22.52
 Urgency of admissionElectiveRef0.028+12
Emergency1.21
 CRP, mg/L≤1Ref0.006+12
>11.21
 Primary vs recurrent tumorPrimaryRef0.002+12
Recurrent1.23
 Total (max score 81)
LEUCut score
 Leukocytosis, leukocytes/μl≤10,900Ref0.008+10
>10,9001.05
 ECOG PS0Ref0.024+15
≥11.45
 Urgency of admissionElectiveRef0.028+12
Emergency1.21
 Cutting-suture time, mins0–200Ref0.003+12
>2001.24
 Total (max score 49)
LINC score*
 Leukocytosis, leukocytes/μl≤10,900Ref0.044+6
>10,9000.60
 ICU LOS, days≤1Ref<0.001+24
≥22.44
 Nursing care levelNoneRef0.001+16
Any1.64
 CRP, mg/L≤1Ref0.004+9
>10.93
 Total (max score 55)

Any adverse event is characterized by the occurrence of at least 1 of the following: 30-day unplanned reoperation, unplanned readmission, and/or mortality; surgical site infection; and/or nosocomial infection.

Nosocomial Infections

The risk score NoInfECT for nosocomial infections comprised Nursing care level, length of stay on the ICU, Emergency admission, CRP on admission, and recurrenT diagnosis. The model performance was good, revealing an AUC of 0.8 for the development cohort and 0.7 for the validation cohort. Patients were stratified according to the sum of the score into low-, intermediate-, and high-risk groups for manifestation of nosocomial infections (Table 4). The incidence of nosocomial infections varied significantly between the different risk groups (1.8% in the low-risk, 8.1% in the intermediate-risk, and 26% in the high-risk groups; p < 0.001) (Fig. 1A). Nosocomial infections included urinary tract infection, pneumonia, meningitis, and surgical site infection.

TABLE 4.

Stratification of risk groups based on the score for the occurrence of nosocomial infection, 30-day unplanned reoperation, and any adverse event

Risk GroupScoreNo. of Cases (%)No. of Events (%)OR95% CIp Value
Low0394 (40.7)7 (1.8)
NoInfECT scoreIntermediate1–25470 (48.6)38 (8.1)4.862.15–11.02<0.001
High>25104 (10.7)27 (26)19.398.15–46.11<0.001
Low0–10276 (30.8)6 (2.2)
LEUCut scoreIntermediate11–25428 (47.8)29 (6.8)3.271.34–7.980.009
High>25192 (21.4)26 (13.5)7.052.84–17.48<0.001
Low0580 (59.9)44 (7.6)
LINC scoreIntermediate1–22287 (29.7)45 (15.7)2.271.46–3.53<0.001
High>23101 (10.4)50 (49.5)11.947.27–19.62<0.001
FIG. 1.
FIG. 1.

Percentage of patients with nosocomial infections (A), 30-day unplanned reoperations (B), and any adverse event (C) as a function of the different scores, stratified into low-, intermediate-, and high-risk groups. ***p < 0.001.

Thirty-Day Unplanned Reoperation

Components of the LEUCut risk score predicting 30-day unplanned reoperations are Leukocytosis, ECOG on admission, Urgency of surgery and Cutting-suture time of index surgery (Table 3). Validation of the score yielded an AUC of 0.7 for both the development and validation groups, indicating a fair discrimination. Classifying patients into different risk groups revealed an incidence for reoperation of 2.2% in the low-risk, 6.8% in the intermediate-risk, and 13.5% in the high-risk groups (p < 0.001) (Table 4 and Fig. 1B). Reasons for unplanned reoperations included complications due to implants, hydrocephalus, CSF leakage, secondary hemorrhage, decrease in vigilance, tumor progression, and second-look or repeated stereotaxy due to unclear results.

Any Adverse Event

We further developed a score to predict the occurrence of any adverse event. The score comprises Leukocytosis, length of stay in the ICU, Nursing care level, and CRP on admission (LINC) (Table 3). The AUCs were 0.73 and 0.71 in the development and validation cohorts, respectively. Again, patients were classified into low-risk (incidence 7.6%), intermediate-risk (15.7%), and high-risk (49.5%) groups (Table 4 and Fig. 1C).

Discussion

Comparing outcome and quality of different neurosurgical providers for benchmarking purposes is only feasible if it is based on evidence of accurate risk stratification models retrieved from large cohort studies. Otherwise, hospitals with higher-risk patients, although performing comparatively well, are mistakenly perceived as less successful in terms of surgical performance. As grading surgeons and hospitals according to their success rates in neurosurgical procedures has become common practice in countries such as the United Kingdom14 and the US,17 the importance of high-quality clinical data registries for the European sector is indisputable.

Risk scoring systems have proved to be helpful, easy-to-use methods in different medical specialties when predicting the probability of undesirable adverse events depending on individual patient characteristics.18–20 Research regarding quality indicators in neurosurgery has widely been carried out; however, the discipline currently lacks practical scoring systems for accurately quantifying and predicting risk.6 While indices for mortality and 30-day unplanned readmission rates have been discussed before,21,22 we developed novel risk predication scores for the occurrence of 30-day unplanned reoperations, nosocomial infections, and any adverse events in neurosurgical patients undergoing surgery for cranial and spinal tumors.

On the basis of the semiautomatic clinical registry database, three scores were developed to equip clinicians with practical tools that allow for instant calculation of a risk, estimating the occurrence of nosocomial infections (NoInfECT score), 30-day unplanned reoperations (LEUCut score), and any adverse event (LINC score).

Each scoring system showed AUC results between 0.7 and 0.8 in ROC analysis, indicating satisfactory predictability of the occurrence of the specific event. The scores were subsequently internally validated in a separate group of patients from the same center.

All three outcome measures are based on both static patient baseline characteristics (e.g., ECOG performance status and nursing care level) and modifiable and unsteady factors that can be helpful in allocating appropriate preoperative care (e.g., high inflammatory markers). As our analysis showed that inflammatory markers on admission (leukocytosis and CRP) have a strong impact on outcome measures, aiming to decrease inflammatory activity prior to surgery might be beneficial in terms of prophylactic means. Adequate investigation of the focus of infection and precise treatment should be performed prior to surgical treatment, which might require postponing the date of surgery, if justifiable.23 In this case, individual findings will be discussed by the team, taking into consideration the underlying neurosurgical disease, the general health condition of the patient, and the severity of the infection. In addition, LOS in the ICU appears to be important for outcome prediction. After cranial surgery, our patients routinely stay for 1 night after surgery in the ICU. We focus on early mobilization and follow strict hygienic standards to impact that variable.

Another decisive advantage of our scoring systems is their high practicability. Other scores are criticized due to their impracticability in the clinical setting. As an example, the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) Surgical Risk Calculator requires manual data entry of 22 variables.24 Out of the high number of analyzed variables, not more than 5 factors were statistically significant in multivariable analysis for each of the developed outcome measures in our study and included in the scores. In the daily clinical setting, depending on the score, 4 or 5 variables are needed to predict outcome. Furthermore, all variables are of categorical property and therefore easy to classify for the clinician. Difficulties of different coding strategies are mitigated by succinct categories. With respect to already existing scores, such as the NSQIP (22 variables)24 and the Surgical Risk Preoperative Assessment System (SURPAS) (8 variables),25 we believe that our scoring systems will be comparatively easy to use in the daily routine.

To underline the applicability of our score, we present a small case illustration. A 75-year-old patient with level 1 nursing care after a hip fracture a year previously and resulting impairment of mobility was admitted after a seizure. MRI revealed a right frontal 2-cm contrast-enhancing lesion, consistent with a high-grade glioma. On admission, the patient was neurologically unremarkable. Laboratory results revealed elevated CRP and leukocytosis. Using the LINC score for any adverse event, the patient’s score reached 31 points (assuming a 1-day stay in the ICU), and he was assigned to the high-risk group. Chest radiographs revealed pneumonia, and antibiotic treatment was initiated; after 3 days, infectious parameters normalized and the score decreased to 16 points (intermediate-risk group). Thus, by treating the pneumonia prior to surgery, we lowered the patient’s risk of experiencing any adverse event by 33.8 percentage points.

A widely used score for the prediction of 30-day mortality and unplanned 30-day readmission is the LACE index.26 Originally developed for internal medicine, the LACE index relies on variables less likely to play a crucial role for neurooncological patients. This score includes emergency visits within 6 months before index admission as one of the 4 input variables. Nevertheless, patients with spinal or cerebral tumors often show an acute onset of symptoms like seizures or motor deficits. Only in rare cases can an emergency admission in the near past be tracked. A recent study validated the score in a neurosurgical collective and demonstrated good correlation between high scoring results and mortality and readmission.21 We could not extract a sufficient prediction model for these two outcome measures from our data in a pure neurooncological setting. In contrast, our scores were specifically designed for patients with spinal and cerebral tumors.

Another score, the SURPAS, mentioned above, is based on 8 variables and integrated into electronic health records (EHRs).25 Similarly, this score only relies on administrative data and does not draw on clinical registries.

Although Reponen et al. suggested combining the Helsinki ASA (a local modification of the American Society of Anesthesiologists Physical Classification System) score with age and CRP for risk prediction in neurosurgery,11 for our cohort these variables were statistically significant in the univariate analysis, while CRP was the only variable out of three that was significant in the multivariable logistic regression model.

Compared with other neurosurgical providers and data from the literature, outcome measures derived from our department are favorable. The unplanned reoperation rate is comparable with other providers who report rates between 3.1% and 8.9%.27–29

Similarly, nosocomial infection rates (7.4%) and surgical site infection rates (1.8%) were lower when compared with other published results.30,31 The occurrence of any adverse event (14.5%) appears to be low; however, it is challenging to appropriately compare, as it is highly dependent on which events are included in this cumulative variable.

A major strength of our study is the incorporation of both clinical and administrative data into our registry for the development of risk adjustment models. Previous studies addressing this issue have generally relied solely on administrative data.10,32,33

Comprehensive registries of administrative data are common practice in neurosurgery to track patient characteristics and allow for risk adjustment, such as the NNAP in the United Kingdom and the N2QOD in the US.14,17 Both databases were launched with the aim to track quality of care and identify risk adjustment tools for nationwide comparisons of neurosurgical providers. Many studies on quality indicators in neurosurgery are based on comprehensive patient data derived from these databases.8,17,27,34,35 Most studies do not distinguish between comorbidities and patient characteristics that are present on admission and those occurring during the in-hospital stay. Recent studies demonstrated the problem of misleading results when solely relying on administrative data.36 Furthermore, quality and scope of coding of administrative data differ between different healthcare providers, which makes direct comparisons challenging.37

Although the reliance on readily available data appears to be favorable, we believe that a combined clinical and administrative data registry leads to the most accurate results when analyzing and comparing outcome measures. Our semiautomatic database combined clinical and administrative data to include more than 100 variables that addressed both baseline characteristics on admission and complications that occurred during inpatient stay. Whereas baseline characteristics were retrieved automatically from electronic medical records, reasons for reoperations, complications during surgery, or details on tumor histology were inserted manually.

Comorbidities derived from the ACCI showed no significant impact on our outcome measures. The score allocates each comorbidity category an associated weight.38,39 As the score was developed more than 30 years ago and potentially does not accurately represent present circumstances and conditions with possible effects on neurosurgical outcome, additional comorbidities not covered by the ACCI, such as stroke or epilepsy, were included in our database.

Hospital LOS is discussed as a possible outcome measure in the scientific literature.40–42 However, we do not consider LOS as a potential risk measure, as the prediction of expected hospital stay is highly dependent on tumor entity, surgical procedure, and underlying healthcare system.43,44 Furthermore, scheduling the date of surgery might be affected by urgency of surgery, the necessity of preoperative examinations, and additional imaging or bed occupancy of the department, so that some surgeries might be performed several days after admission and LOS might be prolonged without indicating worse surgical quality. In many neurooncological cases, the selected procedure is at the discretion of the surgeon. As a consequence, complete resections, which may result in superior long-term results for the patient compared with obtaining a biopsy, are followed by longer regeneration times at the hospital without indicating worse quality of the delivered care.44

Limitations

The study is limited by the fact that data were derived from patients admitted to a single university-affiliated neurosurgical center. The results might not be generalizable to other neurosurgical providers because of different local practices and management guidelines. Further studies are required for external validation of our scores. In addition, the actual readmission rate could be higher than recorded due to the fact that some patients may have sought readmission to a different hospital that was not captured in this study. The common population bias of predictive models could also apply to our scoring systems. Although the study might be biased toward the patient population it was initially developed for, the patient cohort was highly heterogeneous. As all adult patients with different spinal and cerebral tumors were included, the group represents a common patient cohort of neurooncological centers in Europe and Germany. To validate the risk scoring systems on more homogeneous patient groups, focused analyses based on different tumor classes (glioma, meningioma, vestibular schwannoma, pituitary adenoma, and spinal tumor) are currently being carried out and will be subject to further studies.

The observation period was from September 2017 to May 2019. A long data collection period might be subject to concern that clinical procedures may have changed. However, all relevant international and national guidelines for the treatment of these patients have not substantially changed over the past 2 years.

Conclusions

We developed easy-to-use and practical scoring systems for the prediction of common adverse events in neurooncological daily practice based on a registry with administrative and clinical data. The scores not only allow for effectively predicting the likelihood of nosocomial infections, reoperations, and adverse events, but also demonstrate an appropriate benchmark for quality of care and enable adequate risk adjustment to compare different neurosurgical providers. Thereby, the scores and their variables provide guidance to surgeons on how to allocate preoperative care.

Acknowledgments

Dr. Schipmann acknowledges the Medical Faculty of the Westphalian Wilhelms–University of Münster for their support.

Disclosures

Prof. Stummer: consultant and lecture activities from medac (Wedel, Germany), Carl Zeiss Meditech (Oberkochen, Germany), and NxDc (Lexington, Kentucky). Dr. Schwake: honoraria from MagForce AG.

Author Contributions

Conception and design: Schipmann, Holling, Stummer. Acquisition of data: Schipmann, Lohmann. Analysis and interpretation of data: Schipmann, Lohmann, Warneke, Schwake, Suero Molina. Drafting the article: Schipmann, Lohmann, Stummer. Critically revising the article: Warneke, Schwake, Holling. Reviewed submitted version of manuscript: Schipmann, Suero Molina. Approved the final version of the manuscript on behalf of all authors: Schipmann. Statistical analysis: Schipmann. Administrative/technical/material support: Schipmann, Brix, Varghese, Warneke, Stummer. Study supervision: Schipmann, Stummer.

References

  • 1

    Kim DH , Dagi TF , Bean JR . Neurosurgical practice in transition: a review . Neurosurgery . 2017 ;80 (4S ):S4 S9 .

  • 2

    McLaughlin N , Jin P , Martin NA . Assessing early unplanned reoperations in neurosurgery: opportunities for quality improvement . J Neurosurg . 2015 ;123 (1 ):198 205 .

    • Search Google Scholar
    • Export Citation
  • 3

    Moghavem N , Morrison D , Ratliff JK , Hernandez-Boussard T . Cranial neurosurgical 30-day readmissions by clinical indication . J Neurosurg . 2015 ;123 (1 ):189 197 .

    • Search Google Scholar
    • Export Citation
  • 4

    Schipmann S , Akalin E , Doods J , When the infection hits the wound: matched case-control study in a neurosurgical patient collective including systematic literature review and risk factors analysis . World Neurosurg . 2016 ;95 :178 189 .

    • Search Google Scholar
    • Export Citation
  • 5

    Schipmann S , Brix T , Varghese J , Adverse events in brain tumor surgery: incidence, type, and impact on current quality metrics . Acta Neurochir (Wien) . 2019 ;161 (2 ):287 306 .

    • Search Google Scholar
    • Export Citation
  • 6

    Schipmann S , Schwake M , Suero Molina E , Quality indicators in cranial neurosurgery: Which are presently substantiated? A systematic review . World Neurosurg . 2017 ;104 :104 112 .

    • Search Google Scholar
    • Export Citation
  • 7

    Shah MN , Stoev IT , Sanford DE , Are readmission rates on a neurosurgical service indicators of quality of care? J Neurosurg . 2013 ;119 (4 ):1043 1049 .

    • Search Google Scholar
    • Export Citation
  • 8

    Dasenbrock HH , Yan SC , Smith TR , Readmission after craniotomy for tumor: a National Surgical Quality Improvement Program analysis . Neurosurgery . 2017 ;80 (4 ):551 562 .

    • Search Google Scholar
    • Export Citation
  • 9

    Dashti SR , Baharvahdat H , Spetzler RF , Operative intracranial infection following craniotomy . Neurosurg Focus . 2008 ;24 (6 ):E10 .

    • Search Google Scholar
    • Export Citation
  • 10

    Schipmann S , Varghese J , Brix T , Establishing risk-adjusted quality indicators in surgery using administrative data—an example from neurosurgery . Acta Neurochir (Wien) . 2019 ;161 (6 ):1057 1065 .

    • Search Google Scholar
    • Export Citation
  • 11

    Reponen E , Korja M , Niemi T , Preoperative identification of neurosurgery patients with a high risk of in-hospital complications: a prospective cohort of 418 consecutive elective craniotomy patients . J Neurosurg . 2015 ;123 (3 ):594 604 .

    • Search Google Scholar
    • Export Citation
  • 12

    Charlson ME , Pompei P , Ales KL , MacKenzie CR . A new method of classifying prognostic comorbidity in longitudinal studies: development and validation . J Chronic Dis . 1987 ;40 (5 ):373 383 .

    • Search Google Scholar
    • Export Citation
  • 13

    Charlson M , Szatrowski TP , Peterson J , Gold J . Validation of a combined comorbidity index . J Clin Epidemiol . 1994 ;47 (11 ):1245 1251 .

    • Search Google Scholar
    • Export Citation
  • 14

    The Society of British Neurological Surgeons . Neurosurgical National Audit Programme , 2018 . https://www.sbns.org.uk/index.php/audit/. Accessed February 5, 2020 .

    • Search Google Scholar
    • Export Citation
  • 15

    McGirt MJ , Godil SS , Asher AL , Quality analysis of anterior cervical discectomy and fusion in the outpatient versus inpatient setting: analysis of 7288 patients from the NSQIP database . Neurosurg Focus . 2015 ;39 (6 ):E9 .

    • Search Google Scholar
    • Export Citation
  • 16

    von Elm E , Altman DG , Egger M , The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies . Int J Surg . 2014 ;12 (12 ):1495 1499 .

    • Search Google Scholar
    • Export Citation
  • 17

    McGirt MJ , Speroff T , Dittus RS , The National Neurosurgery Quality and Outcomes Database (N2QOD): general overview and pilot-year project description . Neurosurg Focus . 2013 ;34 (1 ):E6 .

    • Search Google Scholar
    • Export Citation
  • 18

    Chi NF , Kuan YC , Huang YH , Development and validation of risk score to estimate 1-year late poststroke epilepsy risk in ischemic stroke patients . Clin Epidemiol . 2018 ;10 :1001 1011 .

    • Search Google Scholar
    • Export Citation
  • 19

    Han K , Yun JS , Park YM , Development and validation of a risk prediction model for severe hypoglycemia in adult patients with type 2 diabetes: a nationwide population-based cohort study . Clin Epidemiol . 2018 ;10 :1545 1559 .

    • Search Google Scholar
    • Export Citation
  • 20

    Tisdale JE , Jaynes HA , Kingery JR , Development and validation of a risk score to predict QT interval prolongation in hospitalized patients . Circ Cardiovasc Qual Outcomes . 2013 ;6 (4 ):479 487 .

    • Search Google Scholar
    • Export Citation
  • 21

    Linzey JR , Nadel JL , Wilkinson DA , Validation of the LACE Index (Length of stay, Acuity of admission, Comorbidities, Emergency department use) in the adult neurosurgical patient population . Neurosurgery . 2020 ;86 (1 ):E33 E37 .

    • Search Google Scholar
    • Export Citation
  • 22

    Tomlinson SB , Piper K , Kimmell KT , Vates GE . Preoperative frailty score for 30-day morbidity and mortality after cranial neurosurgery . World Neurosurg . 2017 ;107 :959 965 .

    • Search Google Scholar
    • Export Citation
  • 23

    De Magistris L , Paquette B , Orry D , Preoperative inflammation increases the risk of infection after elective colorectal surgery: results from a prospective cohort . Int J Colorectal Dis . 2016 ;31 (9 ):1611 1617 .

    • Search Google Scholar
    • Export Citation
  • 24

    Bilimoria KY , Liu Y , Paruch JL , Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons . J Am Coll Surg . 2013 ;217 (5 ):833 842 .e1–3 .

    • Search Google Scholar
    • Export Citation
  • 25

    Gibula DR , Singh AB , Bronsert MR , Accurate preoperative prediction of unplanned 30-day postoperative readmission using 8 predictor variables . Surgery . 2019 ;166 (5 ):812 819 .

    • Search Google Scholar
    • Export Citation
  • 26

    van Walraven C , Dhalla IA , Bell C , Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community . CMAJ . 2010 ;182 (6 ):551 557 .

    • Search Google Scholar
    • Export Citation
  • 27

    Dasenbrock HH , Yan SC , Chavakula V , Unplanned reoperation after craniotomy for tumor: a National Surgical Quality Improvement Program analysis . Neurosurgery . 2017 ;81 (5 ):761 771 .

    • Search Google Scholar
    • Export Citation
  • 28

    Kwinta BM , Krzyżewski RM , Kliś KM , Emergency reoperations in cranial neurosurgery . World Neurosurg . 2017 ;105 :749 754 .

  • 29

    Senders JT , Muskens IS , Cote DJ , Thirty-day outcomes after craniotomy for primary malignant brain tumors: a National Surgical Quality Improvement Program analysis . Neurosurgery . 2018 ;83 (6 ):1249 1259 .

    • Search Google Scholar
    • Export Citation
  • 30

    Davies BM , Jones A , Patel HC . Surgical-site infection surveillance in cranial neurosurgery . Br J Neurosurg . 2016 ;30 (1 ):35 37 .

    • Search Google Scholar
    • Export Citation
  • 31

    Jiménez-Martínez E , Cuervo G , Hornero A , Risk factors for surgical site infection after craniotomy: a prospective cohort study . Antimicrob Resist Infect Control . 2019 ;8 :69 .

    • Search Google Scholar
    • Export Citation
  • 32

    Amin BY , Tu TH , Schairer WW , Pitfalls of calculating hospital readmission rates based on nonvalidated administrative data sets: presented at the 2012 Joint Spine Section Meeting: clinical article . J Neurosurg Spine . 2013 ;18 (2 ):134 138 .

    • Search Google Scholar
    • Export Citation
  • 33

    Iezzoni LI . Assessing quality using administrative data . Ann Intern Med . 1997 ;127 (8 Pt 2 ):666 674 .

  • 34

    Dasenbrock HH , Angriman F , Smith TR , Readmission after aneurysmal subarachnoid hemorrhage: a nationwide readmission database analysis . Stroke . 2017 ;48 (9 ):2383 2390 .

    • Search Google Scholar
    • Export Citation
  • 35

    Karhade AV , Vasudeva VS , Dasenbrock HH , Thirty-day readmission and reoperation after surgery for spinal tumors: a National Surgical Quality Improvement Program analysis . Neurosurg Focus . 2016 ;41 (2 ):E5 .

    • Search Google Scholar
    • Export Citation
  • 36

    Ghali WA , Quan H , Brant R . Risk adjustment using administrative data: impact of a diagnosis-type indicator . J Gen Intern Med . 2001 ;16 (8 ):519 524 .

    • Search Google Scholar
    • Export Citation
  • 37

    Nimptsch U . Disease-specific trends of comorbidity coding and implications for risk adjustment in hospital administrative data . Health Serv Res . 2016 ;51 (3 ):981 1001 .

    • Search Google Scholar
    • Export Citation
  • 38

    Charlson M , Wells MT , Ullman R , The Charlson comorbidity index can be used prospectively to identify patients who will incur high future costs . PLoS One . 2014 ;9 (12 ):e112479 .

    • Search Google Scholar
    • Export Citation
  • 39

    Quan H , Li B , Couris CM , Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries . Am J Epidemiol . 2011 ;173 (6 ):676 682 .

    • Search Google Scholar
    • Export Citation
  • 40

    Dasenbrock HH , Liu KX , Devine CA , Length of hospital stay after craniotomy for tumor: a National Surgical Quality Improvement Program analysis . Neurosurg Focus . 2015 ;39 (6 ):E12 .

    • Search Google Scholar
    • Export Citation
  • 41

    Stecker MM , Stecker M , Falotico J . Predictive model of length of stay and discharge destination in neuroscience admissions . Surg Neurol Int . 2017 ;8 :17 .

    • Search Google Scholar
    • Export Citation
  • 42

    Volpato S , Bazzano S , Fontana A , Multidimensional Prognostic Index predicts mortality and length of stay during hospitalization in the older patients: a multicenter prospective study . J Gerontol A Biol Sci Med Sci . 2015 ;70 (3 ):325 331 .

    • Search Google Scholar
    • Export Citation
  • 43

    Lu M , Sajobi T , Lucyk K , Systematic review of risk adjustment models of hospital length of stay (LOS) . Med Care . 2015 ;53 (4 ):355 365 .

    • Search Google Scholar
    • Export Citation
  • 44

    Schipmann S , Varghese J , Brix T , Establishing risk-adjusted quality indicators in surgery using administrative data—an example from neurosurgery . Acta Neurochir (Wien) . 2019 ;161 (6 ):1057 1065 .

    • Search Google Scholar
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

Contributor Notes

Correspondence Stephanie Schipmann: University Hospital Münster, Germany. stephanie.schipmann@ukmuenster.de.

INCLUDE WHEN CITING Published online March 20, 2020; DOI: 10.3171/2020.1.JNS193186.

Disclosures Prof. Stummer: consultant and lecture activities from medac (Wedel, Germany), Carl Zeiss Meditech (Oberkochen, Germany), and NxDc (Lexington, Kentucky). Dr. Schwake: honoraria from MagForce AG.

  • View in gallery

    Percentage of patients with nosocomial infections (A), 30-day unplanned reoperations (B), and any adverse event (C) as a function of the different scores, stratified into low-, intermediate-, and high-risk groups. ***p < 0.001.

  • 1

    Kim DH , Dagi TF , Bean JR . Neurosurgical practice in transition: a review . Neurosurgery . 2017 ;80 (4S ):S4 S9 .

  • 2

    McLaughlin N , Jin P , Martin NA . Assessing early unplanned reoperations in neurosurgery: opportunities for quality improvement . J Neurosurg . 2015 ;123 (1 ):198 205 .

    • Search Google Scholar
    • Export Citation
  • 3

    Moghavem N , Morrison D , Ratliff JK , Hernandez-Boussard T . Cranial neurosurgical 30-day readmissions by clinical indication . J Neurosurg . 2015 ;123 (1 ):189 197 .

    • Search Google Scholar
    • Export Citation
  • 4

    Schipmann S , Akalin E , Doods J , When the infection hits the wound: matched case-control study in a neurosurgical patient collective including systematic literature review and risk factors analysis . World Neurosurg . 2016 ;95 :178 189 .

    • Search Google Scholar
    • Export Citation
  • 5

    Schipmann S , Brix T , Varghese J , Adverse events in brain tumor surgery: incidence, type, and impact on current quality metrics . Acta Neurochir (Wien) . 2019 ;161 (2 ):287 306 .

    • Search Google Scholar
    • Export Citation
  • 6

    Schipmann S , Schwake M , Suero Molina E , Quality indicators in cranial neurosurgery: Which are presently substantiated? A systematic review . World Neurosurg . 2017 ;104 :104 112 .

    • Search Google Scholar
    • Export Citation
  • 7

    Shah MN , Stoev IT , Sanford DE , Are readmission rates on a neurosurgical service indicators of quality of care? J Neurosurg . 2013 ;119 (4 ):1043 1049 .

    • Search Google Scholar
    • Export Citation
  • 8

    Dasenbrock HH , Yan SC , Smith TR , Readmission after craniotomy for tumor: a National Surgical Quality Improvement Program analysis . Neurosurgery . 2017 ;80 (4 ):551 562 .

    • Search Google Scholar
    • Export Citation
  • 9

    Dashti SR , Baharvahdat H , Spetzler RF , Operative intracranial infection following craniotomy . Neurosurg Focus . 2008 ;24 (6 ):E10 .

    • Search Google Scholar
    • Export Citation
  • 10

    Schipmann S , Varghese J , Brix T , Establishing risk-adjusted quality indicators in surgery using administrative data—an example from neurosurgery . Acta Neurochir (Wien) . 2019 ;161 (6 ):1057 1065 .

    • Search Google Scholar
    • Export Citation
  • 11

    Reponen E , Korja M , Niemi T , Preoperative identification of neurosurgery patients with a high risk of in-hospital complications: a prospective cohort of 418 consecutive elective craniotomy patients . J Neurosurg . 2015 ;123 (3 ):594 604 .

    • Search Google Scholar
    • Export Citation
  • 12

    Charlson ME , Pompei P , Ales KL , MacKenzie CR . A new method of classifying prognostic comorbidity in longitudinal studies: development and validation . J Chronic Dis . 1987 ;40 (5 ):373 383 .

    • Search Google Scholar
    • Export Citation
  • 13

    Charlson M , Szatrowski TP , Peterson J , Gold J . Validation of a combined comorbidity index . J Clin Epidemiol . 1994 ;47 (11 ):1245 1251 .

    • Search Google Scholar
    • Export Citation
  • 14

    The Society of British Neurological Surgeons . Neurosurgical National Audit Programme , 2018 . https://www.sbns.org.uk/index.php/audit/. Accessed February 5, 2020 .

    • Search Google Scholar
    • Export Citation
  • 15

    McGirt MJ , Godil SS , Asher AL , Quality analysis of anterior cervical discectomy and fusion in the outpatient versus inpatient setting: analysis of 7288 patients from the NSQIP database . Neurosurg Focus . 2015 ;39 (6 ):E9 .

    • Search Google Scholar
    • Export Citation
  • 16

    von Elm E , Altman DG , Egger M , The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies . Int J Surg . 2014 ;12 (12 ):1495 1499 .

    • Search Google Scholar
    • Export Citation
  • 17

    McGirt MJ , Speroff T , Dittus RS , The National Neurosurgery Quality and Outcomes Database (N2QOD): general overview and pilot-year project description . Neurosurg Focus . 2013 ;34 (1 ):E6 .

    • Search Google Scholar
    • Export Citation
  • 18

    Chi NF , Kuan YC , Huang YH , Development and validation of risk score to estimate 1-year late poststroke epilepsy risk in ischemic stroke patients . Clin Epidemiol . 2018 ;10 :1001 1011 .

    • Search Google Scholar
    • Export Citation
  • 19

    Han K , Yun JS , Park YM , Development and validation of a risk prediction model for severe hypoglycemia in adult patients with type 2 diabetes: a nationwide population-based cohort study . Clin Epidemiol . 2018 ;10 :1545 1559 .

    • Search Google Scholar
    • Export Citation
  • 20

    Tisdale JE , Jaynes HA , Kingery JR , Development and validation of a risk score to predict QT interval prolongation in hospitalized patients . Circ Cardiovasc Qual Outcomes . 2013 ;6 (4 ):479 487 .

    • Search Google Scholar
    • Export Citation
  • 21

    Linzey JR , Nadel JL , Wilkinson DA , Validation of the LACE Index (Length of stay, Acuity of admission, Comorbidities, Emergency department use) in the adult neurosurgical patient population . Neurosurgery . 2020 ;86 (1 ):E33 E37 .

    • Search Google Scholar
    • Export Citation
  • 22

    Tomlinson SB , Piper K , Kimmell KT , Vates GE . Preoperative frailty score for 30-day morbidity and mortality after cranial neurosurgery . World Neurosurg . 2017 ;107 :959 965 .

    • Search Google Scholar
    • Export Citation
  • 23

    De Magistris L , Paquette B , Orry D , Preoperative inflammation increases the risk of infection after elective colorectal surgery: results from a prospective cohort . Int J Colorectal Dis . 2016 ;31 (9 ):1611 1617 .

    • Search Google Scholar
    • Export Citation
  • 24

    Bilimoria KY , Liu Y , Paruch JL , Development and evaluation of the universal ACS NSQIP surgical risk calculator: a decision aid and informed consent tool for patients and surgeons . J Am Coll Surg . 2013 ;217 (5 ):833 842 .e1–3 .

    • Search Google Scholar
    • Export Citation
  • 25

    Gibula DR , Singh AB , Bronsert MR , Accurate preoperative prediction of unplanned 30-day postoperative readmission using 8 predictor variables . Surgery . 2019 ;166 (5 ):812 819 .

    • Search Google Scholar
    • Export Citation
  • 26

    van Walraven C , Dhalla IA , Bell C , Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community . CMAJ . 2010 ;182 (6 ):551 557 .

    • Search Google Scholar
    • Export Citation
  • 27

    Dasenbrock HH , Yan SC , Chavakula V , Unplanned reoperation after craniotomy for tumor: a National Surgical Quality Improvement Program analysis . Neurosurgery . 2017 ;81 (5 ):761 771 .

    • Search Google Scholar
    • Export Citation
  • 28

    Kwinta BM , Krzyżewski RM , Kliś KM , Emergency reoperations in cranial neurosurgery . World Neurosurg . 2017 ;105 :749 754 .

  • 29

    Senders JT , Muskens IS , Cote DJ , Thirty-day outcomes after craniotomy for primary malignant brain tumors: a National Surgical Quality Improvement Program analysis . Neurosurgery . 2018 ;83 (6 ):1249 1259 .

    • Search Google Scholar
    • Export Citation
  • 30

    Davies BM , Jones A , Patel HC . Surgical-site infection surveillance in cranial neurosurgery . Br J Neurosurg . 2016 ;30 (1 ):35 37 .

    • Search Google Scholar
    • Export Citation
  • 31

    Jiménez-Martínez E , Cuervo G , Hornero A , Risk factors for surgical site infection after craniotomy: a prospective cohort study . Antimicrob Resist Infect Control . 2019 ;8 :69 .

    • Search Google Scholar
    • Export Citation
  • 32

    Amin BY , Tu TH , Schairer WW , Pitfalls of calculating hospital readmission rates based on nonvalidated administrative data sets: presented at the 2012 Joint Spine Section Meeting: clinical article . J Neurosurg Spine . 2013 ;18 (2 ):134 138 .

    • Search Google Scholar
    • Export Citation
  • 33

    Iezzoni LI . Assessing quality using administrative data . Ann Intern Med . 1997 ;127 (8 Pt 2 ):666 674 .

  • 34

    Dasenbrock HH , Angriman F , Smith TR , Readmission after aneurysmal subarachnoid hemorrhage: a nationwide readmission database analysis . Stroke . 2017 ;48 (9 ):2383 2390 .

    • Search Google Scholar
    • Export Citation
  • 35

    Karhade AV , Vasudeva VS , Dasenbrock HH , Thirty-day readmission and reoperation after surgery for spinal tumors: a National Surgical Quality Improvement Program analysis . Neurosurg Focus . 2016 ;41 (2 ):E5 .

    • Search Google Scholar
    • Export Citation
  • 36

    Ghali WA , Quan H , Brant R . Risk adjustment using administrative data: impact of a diagnosis-type indicator . J Gen Intern Med . 2001 ;16 (8 ):519 524 .

    • Search Google Scholar
    • Export Citation
  • 37

    Nimptsch U . Disease-specific trends of comorbidity coding and implications for risk adjustment in hospital administrative data . Health Serv Res . 2016 ;51 (3 ):981 1001 .

    • Search Google Scholar
    • Export Citation
  • 38

    Charlson M , Wells MT , Ullman R , The Charlson comorbidity index can be used prospectively to identify patients who will incur high future costs . PLoS One . 2014 ;9 (12 ):e112479 .

    • Search Google Scholar
    • Export Citation
  • 39

    Quan H , Li B , Couris CM , Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries . Am J Epidemiol . 2011 ;173 (6 ):676 682 .

    • Search Google Scholar
    • Export Citation
  • 40

    Dasenbrock HH , Liu KX , Devine CA , Length of hospital stay after craniotomy for tumor: a National Surgical Quality Improvement Program analysis . Neurosurg Focus . 2015 ;39 (6 ):E12 .

    • Search Google Scholar
    • Export Citation
  • 41

    Stecker MM , Stecker M , Falotico J . Predictive model of length of stay and discharge destination in neuroscience admissions . Surg Neurol Int . 2017 ;8 :17 .

    • Search Google Scholar
    • Export Citation
  • 42

    Volpato S , Bazzano S , Fontana A , Multidimensional Prognostic Index predicts mortality and length of stay during hospitalization in the older patients: a multicenter prospective study . J Gerontol A Biol Sci Med Sci . 2015 ;70 (3 ):325 331 .

    • Search Google Scholar
    • Export Citation
  • 43

    Lu M , Sajobi T , Lucyk K , Systematic review of risk adjustment models of hospital length of stay (LOS) . Med Care . 2015 ;53 (4 ):355 365 .

    • Search Google Scholar
    • Export Citation
  • 44

    Schipmann S , Varghese J , Brix T , Establishing risk-adjusted quality indicators in surgery using administrative data—an example from neurosurgery . Acta Neurochir (Wien) . 2019 ;161 (6 ):1057 1065 .

    • Search Google Scholar
    • Export Citation

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 73 73 73
PDF Downloads 43 43 43
EPUB Downloads 0 0 0