Editorial: The case for a comparative, value-based alternative to the patient-centered outcomes research model for comparative effectiveness research

Full access

Considerable financial and human resources have been directed toward the emerging field of comparative effectiveness research (CER) in the US. Fundamentally, the concept of CER is so logical as to be almost self-evident; namely, that research regarding therapeutic strategies should go beyond efficacy and examine objectively their real-world effects and outcomes. In practice, however, reluctance to consider difficult questions related to the many dimensions of value in health care delivery and corresponding legislative constraints placed on the US CER enterprise risk limiting the ultimate utility of this investigative model. Significant constraints have been codified into the patient-centered outcomes research (PCOR) model of CER, which is emerging as the de facto method for conducting CER in the US. The experience of the authors as clinicians attempting to use CER to improve complex management decisions, for which multidimensional considerations of value represent a critical component of the overall effectiveness of alternate strategies, highlight the inability of PCOR to comprehensively inform this process. This suggests that PCOR may be a suboptimal approach for performing clinically relevant CER. In this editorial, the authors use clinical examples to highlight the limitations of the PCOR approach to CER and to propose an alternate approach, which they term “comparative, value-based effectiveness research” (CVER). The authors believe that the narrow scope and fundamental limitations of PCOR mitigate its overall value to medical decision-makers attempting to optimize overall effectiveness in the real-world setting, while a more comprehensive approach like CVER has greater potential to realize practical benefits for patients, clinicians, and society as a whole.

Abbreviations used in this paper:CER = comparative effectiveness research; CVER = comparative, value-based effectiveness research; PCOR = patient-centered outcomes research; PCORI = Patient-Centered Outcomes Research Institute; QOL = quality of life.

Considerable financial and human resources have been directed toward the emerging field of comparative effectiveness research (CER) in the US. Fundamentally, the concept of CER is so logical as to be almost self-evident; namely, that research regarding therapeutic strategies should go beyond efficacy and examine objectively their real-world effects and outcomes. In practice, however, reluctance to consider difficult questions related to the many dimensions of value in health care delivery and corresponding legislative constraints placed on the US CER enterprise risk limiting the ultimate utility of this investigative model. Significant constraints have been codified into the patient-centered outcomes research (PCOR) model of CER, which is emerging as the de facto method for conducting CER in the US. The experience of the authors as clinicians attempting to use CER to improve complex management decisions, for which multidimensional considerations of value represent a critical component of the overall effectiveness of alternate strategies, highlight the inability of PCOR to comprehensively inform this process. This suggests that PCOR may be a suboptimal approach for performing clinically relevant CER. In this editorial, the authors use clinical examples to highlight the limitations of the PCOR approach to CER and to propose an alternate approach, which they term “comparative, value-based effectiveness research” (CVER). The authors believe that the narrow scope and fundamental limitations of PCOR mitigate its overall value to medical decision-makers attempting to optimize overall effectiveness in the real-world setting, while a more comprehensive approach like CVER has greater potential to realize practical benefits for patients, clinicians, and society as a whole.

Abbreviations used in this paper:CER = comparative effectiveness research; CVER = comparative, value-based effectiveness research; PCOR = patient-centered outcomes research; PCORI = Patient-Centered Outcomes Research Institute; QOL = quality of life.

Comparative effectiveness research has been broadly defined as research to generate and synthesize “evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care.”5 The purpose of CER is “to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care.”5 The initial federal investment of $1.1 billion1 in CER reflects a recognition that effective health care delivery is a societal mandate.

Superficially, the mission of CER appears so logical as to be self-evident; namely, to provide comprehensive resources for patients and clinicians to better inform decisions between alternate management strategies for disease. In practice, however, legislative constraints on the US CER enterprise risk limiting the ultimate utility of this investigative model. In this editorial, we use two clinical examples to highlight the limitations of the current embodiment of CER in the US, and to propose an alternate approach with greater potential benefit for patients, clinicians, and society as a whole.

Alternate Approaches to CER

The addition of erlotinib to gemcitabine chemotherapy for patients with advanced pancreatic cancer has been shown to provide a statistically significant survival benefit over treatment with gemcitabine alone (0.33 months; p = 0.023),9 but the incremental cost of this therapy is calculated at $430,000–$510,000 per quality-adjusted life year.8

Clinicians and patients face a practical question: “What is the balance between the survival benefit and the cost of adding erlotinib to gemcitabine, and how should this affect our medical decision making?” This type of inquiry could logically be expected to be addressed through CER, and much of the initial enthusiasm over this research paradigm derived from its promise to provide more complete answers to such dilemmas. However, careful consideration of this seemingly straightforward question in the context of recent federal legislation highlights the limitations of the current approach to CER and the need for an alternative.

There are two fundamentally different approaches through which CER could be used to address real-world questions, such as the one posed by the erlotinib example. The first approach is a safe form of CER, focused strictly on comparing outcomes both in the setting of clinical trials and in post hoc analyses of real-world therapy. We use the term “safe” because these analyses avoid the technically and ethically difficult issue of cost effectiveness. Safe CER analyses can have 1 of only 2 potential outcomes: identification of either a superior strategy or a condition of equipoise.2 Either is palatable and leaves all involved parties feeling content with their efforts and conclusions.

Recent legislation essentially guarantees that safe CER is the type that will be pursued in the US. Congress enacted legislation to create the Patient-Centered Outcomes Research Institute (PCORI) as the nation's de facto CER entity, and then specifically prohibited PCORI from using “…dollars-per-quality adjusted life year (or similar measure…) as a threshold to establish what type of health care is cost effective or recommended.”10 Any latitude that might have been found in this definition has been eliminated by PCORI Executive Director Joe Selby, who subsequently stated that “You can take it to the bank that PCORI will never do a cost-effectiveness analysis.”11

How much does this safe form of CER, which we believe is synonymous with PCOR, actually help clinicians and patients? The benefit is likely limited. Despite assertions to the contrary, it appears that PCOR is fundamentally a methodology of comparative efficacy. Although consideration is given to results from clinical trials and from real-world settings, the basic issue under study remains the difference in efficacy between alternate strategies. In this scenario, the illustrious efficacy versus effectiveness distinction becomes semantic and the underlying investigative approach is not novel; similar research has been conducted for years as “Phase IV trials,” or “comparator trials.”4 Rather than being the much-anticipated, transformative research paradigm in American health care, PCOR may be arguably little more than a multibillion dollar exercise in rebranding extant models of post–Phase III research.

Many who refute this position highlight two characteristics that they contend to be demonstrative of PCOR's novelty. First, they argue that PCOR places new emphasis on QOL assessment. While QOL considerations should certainly be included in any multidimensional CER analysis, there is little evidence to suggest that the current CER enterprise is achieving this objective. A Medline search on “comparative effectiveness” and “quality of life” from the inception of the US CER enterprise (2009) through February 2012 identifies only 54 nonreview, English-language publications. These publications represent less than one-third of articles on comparative effectiveness, a statistic that makes it difficult to argue that QOL assessment is currently “a major part of CER” in the US. Second, PCOR proponents claim that these investigations focus on real-world settings. This is a challenging position to defend when cost is excluded from the definition of “value,”2 unless this purported real world is one in which both patients and clinicians have unrestricted access to limitless resources.

While PCOR advocates may argue that the quantity of QOL data will increase in the coming years, they cannot refute the fact that PCOR is, by design, incapable of comprehensively informing value-based discussions between clinicians and patients that include the trade-offs between cost and efficacy, or QOL. Such discussions are becoming more common and more important, because in the actual real-world setting of finite assets, both parties are progressively influenced by the cost of therapy.12 Comparative, value-based decision making has become a part of life for the informed American consumer, and it appears imprudent to suggest that patients should be denied the information required to apply similar thought processes to the management of their own health.2,3

This limitation is not inherent to CER but rather to PCOR, its safe incarnation. An alternate approach is one where the “effectiveness” dimension of comparative effectiveness is made capable of critically examining efficacy, QOL cost, and their subvariables in the context of the basic value equation (value = benefits/costs). We term this critical form of CER “comparative, value-based effectiveness research” (CVER) and believe that, because it facilitates semiquantitative comparisons across the domains of cost, benefit, and value, it represents a more robust opportunity for physicians and patients to use objective data to inform the complex process of real-world medical decision making. In contrast, because PCOR excludes cost considerations, it cannot be used in conjunction with this type of mathematical model. The semiquantitative analytical capabilities of PCOR are limited to those performed within a single dimension, while its cross-dimensional comparisons must remain qualitative.

The Role of Semiquantitative Modeling in CVER

A second clinical example demonstrates the unique challenge of conducting CVER. Recently, we employed this CER approach to analyze efficacy, QOL, and cost data for alternate management strategies for acromegaly.7 Our goal was not to identify a single, “best” strategy, but rather to provide comprehensive, multidimensional information for patients and clinicians faced with the challenge of selecting a therapeutic strategy. Our approach included comparisons of all 3 domains for several treatment options, all of which enter into the conversations between patients with acromegaly and their physicians on a daily basis. With various therapies having approximately equivalent control rates but highly variable physical, emotional, and financial costs to the patient, it is essential for physicians to understand the trade-offs that exist with regard to all of these domains, so that they can work with patients to design individualized strategies that are in accordance with the patients' personal values. It borders on negligence to fail to inform a patient that a strategy of daily injections may be associated with a significant out-of-pocket cost, particularly if the patient's financial situation prohibits compliance with this treatment regimen. Yet without cost analysis, the physician risks remaining uninformed regarding these real-world issues that affect management, and lacks adequate data to discuss objectively the relative risks and benefits of other treatment alternatives with which the patient may better comply.

Although this methodology allowed us to review and rank a series of management strategies with regard to each of the 3 domains, developing an overall ranking of the actual comparative effectiveness of these strategies required application of a semiquantitative model of the relative weights of efficacy, QOL, and cost in the medical decision-making process. This type of model creates an objective, logical, and quantifiable framework that allows clinicians and patients to understand the dynamic relationships between these 3 domains so that they can work in concert to make the best possible decisions given a particular set of values and circumstances. However, because no such model has been adequately developed, we could only adopt an approach based on equal weighting in our analysis, and advise clinicians to individualize the model on an ad hoc basis.

This example demonstrates how the ability of CVER-type investigations to comprehensively inform clinical decisions is limited by the absence of objective, semiquantitative models that reflect societal values and preferences, as well as by adequate data to inform their development. More research regarding cost and QOL analysis is needed to facilitate the technical and social aspects of developing such a model, but this is not the major obstacle to the evolution of a clinically relevant CER enterprise. Rather, the primary impediment is the reluctance of the medical community and the public to recognize that the present health care environment, in which a perpetual torrent of novel but costly therapeutic strategies contributes to unsustainable health care spending, mandates that difficult decisions be made regarding the fundamental values and practical aspects of health care delivery. Comparative, value-based effectiveness research provides a framework for confronting these questions in an objective fashion, whereas PCOR does not.

Opponents of this position argue that building such models and using them to inform comparative effectiveness analyses controverts attempts at “personalized medicine.”6 This is only true in the setting of rigid and unyielding adherence to these models, a condition that most reasonable clinicians and patients would be likely to denounce. Instead, semiquantitative models used in conjunction with a CVER approach to comparative effectiveness analysis provide a concrete framework in which the relative risks and benefits of alternate treatment strategies can be compared objectively, and the dynamics and implications of their interactions can be studied on both the individual and population level. Investigations designed to build these models and to gather data that facilitates their application are the necessary first step in developing a strategy for comparative, value-based effectiveness analysis that can stand in contrast to the already-handicapped PCOR model that is currently overtaking the US CER enterprise.

Summary

Health care costs and expenditures in the US continue to rise, and now represent nearly 18% of the US gross domestic product. Most clinicians and patients understand that economic and material resources are finite, and we currently have the opportunity to leverage the federal commitment to CER as a catalyst for positive and meaningful change in our health care system. This means identifying opportunities to improve QOL and to control costs while maintaining efficacy and protecting patient outcomes. It is impossible to realize the full potential of this opportunity without collecting data regarding all dimensions of value in health care and building models that allow for the relationships among these many variables to be analyzed and optimized.

The current implementation of CER in the US (the PCORI) has been crippled from its inception by legislation that restricts the types of value-based data that it can collect and incorporate into its analyses. This, in turn, limits the nature and scope of the conversations that it can inform between clinicians and patients regarding treatment decisions in real-world settings. An alternative to this limited version of CER is the more critical approach that we refer to as CVER. The concept is not novel, and we believe that it is what most clinicians have always envisioned as the way to effect health care reform without compromising patient care. However, if we are unwilling to broach the challenging issues that CVER mandates and instead persist unassumingly in the direction of PCOR, we may soon find ourselves 10 years older, $1 billion poorer, and only marginally wiser than we are today.

Disclosure

Partial support and funding were provided by the AANS William P. VanWagenen Fellowship program (N.F.M.), the Melvin Burkhardt Chair in Neurosurgical Oncology (R.J.W.), and the Karen Colina Wilson Research Endowment within the Burkhardt Brain Tumor and Neuro-Oncology Center at the Cleveland Clinic Foundation (R.J.W).

Author contributions to the study and manuscript preparation include the following. Conception and design: both authors. Analysis and interpretation of data: both authors. Drafting the article: Marko. Critically revising the article: both authors. Reviewed final version of the manuscript and approved it for submission: both authors. Statistical analysis: Marko. Administrative/technical/material support: both authors. Study supervision: Weil.

References

  • 1

    American Recovery and Reinvestment Act of 2009 Pub Law 111–5 (February 17 2009) (http://www.gpo.gov/fdsys/pkg/PLAW-111publ5/pdf/PLAW-111publ5.pdf) [Accessed May 10 2012]

  • 2

    Brook RH: Can the Patient-Centered Outcomes Research Institute become relevant to controlling medical costs and improving value?. JAMA 306:202020212011

    • Search Google Scholar
    • Export Citation
  • 3

    Freemantle NDrummond MF: AHRQ series commentary 3: the United States addresses comparative effectiveness but not cost-effectiveness through the effective health-care program. J Clin Epidemiol 63:4794802010

    • Search Google Scholar
    • Export Citation
  • 4

    Goss CH: Comparative effectiveness research: what happened to incorporating costs of care?. Am J Respir Crit Care Med 183:9739742011

    • Search Google Scholar
    • Export Citation
  • 5

    Institute of Medicine: Initial National Priorities for Comparative Effectiveness Research Washington, DCNational Academies Press2009

    • Search Google Scholar
    • Export Citation
  • 6

    Kamerow D: PCORI: odd name, important job, potential trouble. BMJ 342:d26352011

  • 7

    Marko NFLaSota EHamrahian AHWeil RJ: Comparative effectiveness review of treatment options for pituitary microadenomas in acromegaly. Clinical article. J Neurosurg [epub ahead of print June 22 2012. DOI: 10.3171/2012.4.JNS11739]

    • Search Google Scholar
    • Export Citation
  • 8

    Miksad RASchnipper LGoldstein M: Does a statistically significant survival benefit of erlotinib plus gemcitabine for advanced pancreatic cancer translate into clinical significance and value?. J Clin Oncol 25:450645082007. (Letter)

    • Search Google Scholar
    • Export Citation
  • 9

    Moore MJGoldstein DHamm JFiger AHecht JRGallinger S: Erlotinib plus gemcitabine compared with gemcitabine alone in patients with advanced pancreatic cancer: a phase III trial of the National Cancer Institute of Canada Clinical Trials Group. J Clin Oncol 25:196019662007

    • Search Google Scholar
    • Export Citation
  • 10

    Patient Protection and Affordable Care Act Pub Law 111–148 (March 23 2010) (http://www.gpo.gov/fdsys/pkg/PLAW-111publ148/pdf/PLAW-111publ148.pdf) [Accessed May 10 2012]

  • 11

    Wilkerson J: PCORI Head Vows Not To Do Cost-Effectiveness Studies But Notes Gray Areas Inside Washington Publishers2011. (http://insidehealthpolicy.com/Inside-Health-General/Public-Content/pcori-head-vows-not-to-docost-effectiveness-studies-but-notes-gray-areas/menu-id-869.html) [Accessed May 10 2012]

    • Search Google Scholar
    • Export Citation
  • 12

    Wilson JF: Cancer care: a microcosm of the problems facing all of health care. Ann Intern Med 150:5735762009

If the inline PDF is not rendering correctly, you can download the PDF file here.

Article Information

Contributor Notes

Address correspondence to: Nicholas F. Marko, M.D., Cancer Research UK Cambridge Cancer Institute, Robinson Way, Cambridge CB2 0RE, United Kingdom. email: nicholas.marko@cancer.org.uk.Please include this information when citing this paper: DOI: 10.3171/2012.5.FOCUS12127.
Headings
References
  • 1

    American Recovery and Reinvestment Act of 2009 Pub Law 111–5 (February 17 2009) (http://www.gpo.gov/fdsys/pkg/PLAW-111publ5/pdf/PLAW-111publ5.pdf) [Accessed May 10 2012]

  • 2

    Brook RH: Can the Patient-Centered Outcomes Research Institute become relevant to controlling medical costs and improving value?. JAMA 306:202020212011

    • Search Google Scholar
    • Export Citation
  • 3

    Freemantle NDrummond MF: AHRQ series commentary 3: the United States addresses comparative effectiveness but not cost-effectiveness through the effective health-care program. J Clin Epidemiol 63:4794802010

    • Search Google Scholar
    • Export Citation
  • 4

    Goss CH: Comparative effectiveness research: what happened to incorporating costs of care?. Am J Respir Crit Care Med 183:9739742011

    • Search Google Scholar
    • Export Citation
  • 5

    Institute of Medicine: Initial National Priorities for Comparative Effectiveness Research Washington, DCNational Academies Press2009

    • Search Google Scholar
    • Export Citation
  • 6

    Kamerow D: PCORI: odd name, important job, potential trouble. BMJ 342:d26352011

  • 7

    Marko NFLaSota EHamrahian AHWeil RJ: Comparative effectiveness review of treatment options for pituitary microadenomas in acromegaly. Clinical article. J Neurosurg [epub ahead of print June 22 2012. DOI: 10.3171/2012.4.JNS11739]

    • Search Google Scholar
    • Export Citation
  • 8

    Miksad RASchnipper LGoldstein M: Does a statistically significant survival benefit of erlotinib plus gemcitabine for advanced pancreatic cancer translate into clinical significance and value?. J Clin Oncol 25:450645082007. (Letter)

    • Search Google Scholar
    • Export Citation
  • 9

    Moore MJGoldstein DHamm JFiger AHecht JRGallinger S: Erlotinib plus gemcitabine compared with gemcitabine alone in patients with advanced pancreatic cancer: a phase III trial of the National Cancer Institute of Canada Clinical Trials Group. J Clin Oncol 25:196019662007

    • Search Google Scholar
    • Export Citation
  • 10

    Patient Protection and Affordable Care Act Pub Law 111–148 (March 23 2010) (http://www.gpo.gov/fdsys/pkg/PLAW-111publ148/pdf/PLAW-111publ148.pdf) [Accessed May 10 2012]

  • 11

    Wilkerson J: PCORI Head Vows Not To Do Cost-Effectiveness Studies But Notes Gray Areas Inside Washington Publishers2011. (http://insidehealthpolicy.com/Inside-Health-General/Public-Content/pcori-head-vows-not-to-docost-effectiveness-studies-but-notes-gray-areas/menu-id-869.html) [Accessed May 10 2012]

    • Search Google Scholar
    • Export Citation
  • 12

    Wilson JF: Cancer care: a microcosm of the problems facing all of health care. Ann Intern Med 150:5735762009

TrendMD
Metrics

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 68 68 7
PDF Downloads 44 44 4
EPUB Downloads 0 0 0
PubMed
Google Scholar