Editorial: SEER insights

Full access

If the inline PDF is not rendering correctly, you can download the PDF file here.

In their article in the Journal of Neurosurgery, Sonabend et al. queried the Surveillance Epidemiology and End Results (SEER) database to better understand the epidemiology and get a glimpse into the efficacy of various therapies for CNS hemangiopericytomas.5 These are relatively uncommon lesions with heterogeneous presentations and perhaps even greater heterogeneity in treatment strategies. Current therapeutic paradigms, however, are based on limited clinical experience and relatively small single-institution series. There are still no prospective clinical trials to guide us in the treatment of this disease. The authors use multivariate analyses to identify factors associated with overall survival. They report that young age, supratentorial location, and gross-total resection (GTR) in combination with adjuvant radiation treatment are associated with longer overall survival. Interestingly, the benefit of GTR is not identified in the absence of radiation therapy.

The SEER database allows for the study of a nationally selected cohort of patients, providing large patient numbers, and potentially limiting bias that results when evaluating patients from a single institution. It thus allows for the study of rare pathologies of which sufficient numbers cannot be collected from single or even multiple institutions. However, its use does have several limitations. First, while the use of SEER data may limit the impact of bias from individual institutions, it suffers from unique biases associated with national databases. For example, the SEER data are not from a random sample of cancer patients in the US, as data are collected from various selected collection sites. The SEER database represents only about 26% of Americans treated for cancer, making its data a relatively small sample of cancer cases nationwide rather than a comprehensive overview. As a result, case data may be geographically imbalanced. Second, various clinical and tumor variables cannot be gathered from the database, including patient performance status, duration of symptoms, and imaging characteristics. Third, tumor progression cannot be determined, although the use of additional radiotherapy may be a surrogate for progression. Fourth, treatment characteristics, such as the modality and dose of radiotherapy and the use of alternative therapies, cannot be discerned. Additionally, future therapies such as resection due to tumor progression after initial biopsy and radiation therapy are often omitted. Fifth, patient comorbidities are not well documented and therefore may affect mortality in the various patient groups. Sixth, as there is no central pathology review of cases in the SEER database, misdiagnosed cases may be present. Finally, as with all national databases, miscoding of the various patient, tumor, and treatment characteristics may be present.

The use of such national databases is becoming more common. Databases that focus on disease states such as cancer, patient populations such as Medicare recipients, or the US Department of Veterans Affairs can be interesting sources of pooled information. Articles that use such data also teach us how we might better construct single-institution or multiinstitution studies. These databases might include fewer patients, but collect the data we really want to know, allow more intensive auditing of data, and provide more robust findings.

With statistical methods such as the log-rank test and Cox proportional hazards model, an assessment can be made of the association between clinical factors and survival outcome within the SEER database. It is important to remember that, as always, the power of such analyses is a function of the number of events or deaths and not the number of patients in the database. For this study, we estimate approximately 23 deaths among the 227 patients based on a 5-year survival estimate of 83% and a median follow-up of 34 months. With such a small number of deaths, the power of tests to detect clinically important effects is limited. For example, a log-rank test comparing the survival of the 54 patients who underwent GTR alone to that experienced by all other patients has 80% power to detect a hazard ratio of 0.22 or 4.38 (α = 0.05, 2-tailed). In other words, there is reasonable power to detect a 78% reduction in the rate of dying with only a GTR relative to all other options combined. A pairwise comparison such as a comparison of GTR alone to biopsy alone (reference group) has less power to detect this hazard ratio given that the total number of deaths involved with the comparison is reduced. The lack of statistical significance reported in the manuscript is inconclusive as it does not reasonably rule out the possibility of a clinically important effect that represents a smaller reduction in the rate of dying.

Statistical models assessed the effect of each individual factor on survival (that is, univariate analyses) as well as the joint effect of multiple factors (that is, multivariate analyses). Model bias increases in multivariate Cox modeling as the number of events per candidate variables decreases. The statistical literature recommends a minimum of 5–20 events per candidate variable to avoid concern about the validity of the model.1,2,4,6 Eleven predictors were included in the reported multivariate model based on this data set that contains only 23 events.

The Cox model also assumes that the hazard ratio associated with each predictor is invariant over time, referred to as the proportional hazards assumption. In their paper Sonabend et al. include assessment of that assumption for each model predictor using a goodness-of-fit test based on Schoenfeld residuals. Kleinbaum and Klein provided an excellent introduction to this approach to assessing the proportional hazards assumption as well as alternative approaches including the examination of log-log survival plots and the inclusion of time-dependent covariables into the Cox model.3 Given the small number of events within this data set, there is limited ability to detect a lack of proportionality.

A 1-way ANOVA was used within the context of a Cox model to examine the effect of treatment on survival. Outcome within each combination of resection and radiation therapy was compared with the reference group of biopsy, and the survival for patients with GTR plus radiation therapy was barely significantly better than that observed for patients with biopsy only. A 2-way interaction might have been able to tease out the separate effects of surgery (biopsy/subtotal resection/GTR) and radiation therapy (yes/no), and to determine if the effect of radiation therapy is consistent across the various levels of surgical extent (that is, statistical interaction). In this case, an examination of the hazard ratios for the various combinations of surgical extent and radiation suggests that the omission of radiation increases the hazard of death 2–4 fold regardless of surgical extent (Table 1).

TABLE 1:

Hazard ratio for treatment combinations relative to biopsy alone (without radiation)*

Treatmentw/ XRTw/o XRTRatio of HR w/o XRT & HR w/ XRT
biopsy0.241.04.17
subtotal resection0.390.731.87
GTR0.310.531.71

XRT = radiotherapy.

Sonabend et al. should be commended for their article, which provides additional insight into outcomes and treatment modalities for patients with hemangiopericytomas.5 Further follow-up from the SEER database will be beneficial over time as the median follow-up was 34 months, a relatively short time period for a disease with a median survival of greater than 1–15 years. This may also help determine the effects of treatment on distant metastasis of hemangiopericytomas. Most importantly, this study highlights many of the difficulties of determining outcomes and the optimal treatment strategies for patients with uncommon CNS disorders, and it underscores the necessity of future multicenter clinical trials that address the optimal therapies for these patients.

Disclosure

The authors report no conflict of interest.

References

  • 1

    Concato JPeduzzi PHolford TRFeinstein AR: Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. J Clin Epidemiol 48:149515011995

    • Search Google Scholar
    • Export Citation
  • 2

    Harrell FE JrLee KLMark DB: Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15:3613871996

    • Search Google Scholar
    • Export Citation
  • 3

    Kleinbaum DGKlein M: Survival Analysis: A Self-Learning Text ed 3New YorkSpringer2012

  • 4

    Peduzzi PConcato JFeinstein ARHolford TR: Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol 48:150315101995

    • Search Google Scholar
    • Export Citation
  • 5

    Sonabend AMZacharia BEGoldstein HBruce SSHershman DNeugut AI: The role for adjuvant radiotherapy in the treatment of hemangiopericytoma: a Surveillance, Epidemiology, and End Results analysis. Clinical article. J Neurosurg [epub ahead of print November 29 2013. DOI: 10.3171/2013.10.JNS13113]

    • Search Google Scholar
    • Export Citation
  • 6

    Vittinghoff EMcCulloch CE: Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol 165:7107182007

    • Search Google Scholar
    • Export Citation

Response

We would like to thank Sampson et al., for their thoughtful critique of our paper and insight into the merits and limitations of the SEER database. The rarity of CNS hemangiopericytomas and the lack of any true Class I data limit blanket conclusions regarding the optimal management of these lesions. To this end we performed an analysis of epidemiology and outcomes from the SEER database for these tumors.

Sampson and colleagues commented on the relative merits of observational studies that utilize, as in our case, population-based administrative data sets such as SEER versus the more typical retrospective case series from single institutions. Both, of course, have their roles in providing important information in the absence of randomized trials or in the setting of rare tumors. A classic single-institution study with a record review allows, as they point out, the collection of certain details, such as imaging characteristics or duration of symptoms prior to diagnosis that cannot be ascertained from a large database. Details of treatment are also generally available from chart review studies.

However, database studies such as SEER provide multiple major advantages that may well outweigh their deficits. The numbers available, especially for rare tumors such as CNS hemangiopericytomas, are of a different order of magnitude. The databases are population based, meaning that there is no bias in terms of hospital referral (most single-institution studies tend to come from local specialized centers for a disease and hence represent a highly selected series of cases with a highly biased mode of treatment). We disagree with Sampson and colleagues when they imply that the SEER database is significantly selected because it collects 26% of cases in the US. Studies have shown that the sample collected is reasonably representative of the overall US population.2

Utilizing the SEER database, we found that the combination of GTR and radiation therapy was associated with an overall survival benefit. Due to the relatively short follow-up period (shorter than the median survival), there were few deaths within the time period and thus few events per variable in the proportional hazards model. Sampson et al. are correct in pointing out that such a shortcoming can increase bias and limit the power to detect differences that might exist. A notable example of a finding that did not reach significance in these data is the previously described survival benefit associated with GTR for CNS hemangiopericytoma.3

Clearly, the selection of covariates for inclusion in the model also affects the number of events per variable. We felt that the other variables (tumor location, tumor size, and age) included in the full model were necessary to address potential alternative explanations.

Sampson and colleagues raise the issue that goodness-of-fit testing of the Schoenfeld residuals may have a limited ability to detect lack of proportionality due to the small number of events. Sampson and colleagues estimated 23 events over the duration of this study, but the actual number of deaths was 38. In general, the small number of events is an issue inherent to the study of such a rare disease, even in a national database. However, alternative approaches from goodness-of-fit testing of the Schoenfeld residuals for testing this assumption also have drawbacks. Inclusion of time-dependent covariables entails significance testing, and thus carries limits similar to those of the goodness-of-fit approach. Kleinbaum and Klein note that log-log survival plots are more subjective than goodness-of-fit testing.1

A complementary analysis performed by Sampson et al. examined the statistical interaction between the extent of resection and radiation therapy derived from the SEER data we presented. When examined in this manner the results suggest that the addition of radiation therapy improves survival regardless of the extent of resection. This is consistent with our Kaplan-Meier analysis which demonstrated a significant increase in survival among patients receiving radiation therapy compared with patients not receiving radiation therapy when all patients were analyzed together regardless of resection status (log-rank test, p < 0.05). These findings further support the possibility of radiation therapy being associated with prolonged survival for patients with CNS hemangiopericytoma.

Questions remain: should radiation therapy be routinely offered to patients with CNS hemangiopericytomas? What level of evidence might justify the potential morbidity of radiation in this setting? In the absence of any Class I data to support the use of radiation therapy, close to 50% of the patients with CNS hemangiopericytoma in our SEER cohort were subjected to radiation treatment. The fields of neurosurgery and neuro-oncology are studded with clinical scenarios in which management decisions for uncommon diseases lack evidence-based support, or rely on scarce anecdotal case reports. Analysis of large-scale databases provides an opportunity for obtaining valuable preliminary answers to such difficult clinical questions and informs the development of patient-centered multiinstitutional clinical registries.

Whereas any given uncommon disease does not represent a significant problem from the public health perspective, the sum of all “orphan” pathologies and clinical scenarios that cannot be subject to a randomized controlled trial, constitutes a major challenge for practicing neurosurgeons and neuro-oncologists. Thus, there is a valuable role for complementing traditional clinical research with retrospective, and hopefully in the near future, prospective nationwide databases.

References

  • 1

    Kleinbaum DKlein M: Survival Analysis: A Self-Learning Text ed 3New YorkSpringer2012

  • 2

    Merrill RMCapocaccia RFeuer EJMariotto A: Cancer prevalence estimates based on tumour registry data in the Surveillance, Epidemiology, and End Results (SEER) Program. Int J Epidemiol 29:1972072000

    • Search Google Scholar
    • Export Citation
  • 3

    Rutkowski MJSughrue MEKane AJAranda DMills SABarani IJ: Predictors of mortality following treatment of intracranial hemangiopericytoma. Clinical article. J Neurosurg 113:3333392010

    • Search Google Scholar
    • Export Citation

If the inline PDF is not rendering correctly, you can download the PDF file here.

Article Information

Please include this information when citing this paper: published online November 29, 2013; DOI: 10.3171/2013.6.JNS13993.

© AANS, except where prohibited by US copyright law.

Headings

References

  • 1

    Concato JPeduzzi PHolford TRFeinstein AR: Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. J Clin Epidemiol 48:149515011995

    • Search Google Scholar
    • Export Citation
  • 2

    Harrell FE JrLee KLMark DB: Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15:3613871996

    • Search Google Scholar
    • Export Citation
  • 3

    Kleinbaum DGKlein M: Survival Analysis: A Self-Learning Text ed 3New YorkSpringer2012

  • 4

    Peduzzi PConcato JFeinstein ARHolford TR: Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. J Clin Epidemiol 48:150315101995

    • Search Google Scholar
    • Export Citation
  • 5

    Sonabend AMZacharia BEGoldstein HBruce SSHershman DNeugut AI: The role for adjuvant radiotherapy in the treatment of hemangiopericytoma: a Surveillance, Epidemiology, and End Results analysis. Clinical article. J Neurosurg [epub ahead of print November 29 2013. DOI: 10.3171/2013.10.JNS13113]

    • Search Google Scholar
    • Export Citation
  • 6

    Vittinghoff EMcCulloch CE: Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol 165:7107182007

    • Search Google Scholar
    • Export Citation
  • 1

    Kleinbaum DKlein M: Survival Analysis: A Self-Learning Text ed 3New YorkSpringer2012

  • 2

    Merrill RMCapocaccia RFeuer EJMariotto A: Cancer prevalence estimates based on tumour registry data in the Surveillance, Epidemiology, and End Results (SEER) Program. Int J Epidemiol 29:1972072000

    • Search Google Scholar
    • Export Citation
  • 3

    Rutkowski MJSughrue MEKane AJAranda DMills SABarani IJ: Predictors of mortality following treatment of intracranial hemangiopericytoma. Clinical article. J Neurosurg 113:3333392010

    • Search Google Scholar
    • Export Citation

TrendMD

Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 184 184 15
PDF Downloads 140 140 3
EPUB Downloads 0 0 0

PubMed

Google Scholar