In February 2009, President Obama signed into legislation the American Recovery and Reinvestment Act (ARRA) in which $1.1 billion was set aside to support comparative effectiveness research (CER). Funding for CER will be awarded through the Office of the Secretary of Health and Human Services (HHS), the Agency for Healthcare Research and Quality (AHRQ), and the National Institutes of Health. Almost immediately, the Friends of Cancer Research (FCR)—a nonprofit organization promoting research collaboration and public-private partnerships—assembled a committee of senior academic investigators, clinicians, and advocates to provide commentary on CER. The committee published a set of recommendations based on the needs and experiences of the oncology community: Improving Medical Decisions Through Comparative Effectiveness Research: Cancer as a Case Study.
In its statement, the Friends of Cancer Research encourages broad support for the expansion of CER and the early application of research findings to clinical cancer care. These recommendations have now been endorsed by more than 25 medical groups. A few months later, at the behest of the United States Congress, the Institute of Medicine (IOM) convened a committee to help define a research agenda for investigations to be supported through ARRA funds. The recommendations of the IOM committee are contained in its report, Initial National Priorities for Comparative Effectiveness Research, which was released in June 2009 at the same time a second national report was released from the Federal Coordinating Council for Comparative Effectiveness Research.[2,3]
The IOM committee provided a working definition of CER by reconciling the definitions used by the Congressional Budget Office, prior studies by the IOM, the American College of Physicians, and the Medicare Payment Advisory Commission: “…the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health-care at both the individual and population levels.”
In other words, the primary goal of CER is to establish what treatments and healthcare strategies work, for whom, and how best to implement them to improve national health, as well as the health of individual patients.
The fact that cancer is common, results in widespread morbidity and mortality, and that cancer care is very costly and rapidly evolving, assured its prominence in the IOM lists of priorities. Indeed, 7 of the 25 first-tier projects involve cancer research.
The Scope and Objectives of Comparative Effectiveness Research
Controversy abounds over what constitutes CER, but some areas of consensus are starting to emerge. First, both the IOM and the Federal Coordinating Council reports take pains to specify that CER does not require cost data or cost-effectiveness analysis and that CER is not used for access or coverage decisions.[2,3] However, the reports do not exclude pertinent economic considerations; the IOM states that “the overall value of a strategy can be understood best by considering costs and benefits together…When CER examines differences in costs as well as outcomes, its aim is to identify the approach that offers the better value; it does not necessarily promote or favor low-cost care.”
Second, it is widely agreed that studies in CER need to focus on rigorous, head-to-head comparisons to identify best possible health-care outcomes. These comparisons should not be limited to exploring whether one drug or another is more efficacious in a phase III clinical trial setting. For CER to fulfill its promise of improving individual and population health, its research agenda must cast a wider net that includes comparing active interventions and systems of healthcare in addition to assembling healthcare strategies from a synthesis of different data sources.[1-4]
Third, after a treatment or process is shown to be effective, CER should include studies aimed at the more rapid dissemination and implementation of study results, as well as best practices that are already shown to be effective, but are not widely applied.[1-3]
Fourth, CER should emphasize effectiveness rather than efficacy. Research needs to address real-world settings and practical delivery of healthcare among a more representative cross-section of patients than the insured, more educated, younger patients that typify clinical trial enrollees in an academic cancer center.[1-3] Similarly, CER is not intended for drug development trials that are designed to establish the required safety and efficacy metrics for Food and Drug Administration approval. In order to demonstrate efficacy, such trials frequently restrict the study population to a homogenous group without comorbid illness that bears little resemblance to real-world patients, and therefore the results cannot be generalized to community clinics.[1-3]
Fifth, it is freely acknowledged that new research methodologies will be needed to increase the relevance and broaden applicability of CER for the average patient. Some of the new methodologies under discussion include pragmatic clinical trials in community-based practice settings, cluster-randomized interventions, and more nimble Bayesian adaptive designs for clinical trials that allow updating trial design in response to information generated during the investigation.[1-5]
The concept of pragmatic clinical trials with relaxed eligibility criteria, less stringent criteria for adherence, and easily measured outcomes that do not entail central adjudication, is particularly appealing for CER in cancer care at a community level. In these settings, away from tertiary care academic cancer centers, most clinics lack an electronic medical record and the highly developed research infrastructure that helps sustain the expense and administrative burden of fielding clinical trials.[1,4]
Sixth, a federal organizational infrastructure must be developed to coordinate prioritizing research topics across agencies, support the expansion of registries and databases, foster new methodologies, and educate the next generation of investigators in the methods utilized in CER studies.[2,3]
The Need to Measure Quality of Life in Cancer Care
Undertaking any type of CER in oncology, calls for an outcomes measurement that can capture quality-of-life (QOL) measures. Most treatment decisions in cancer care are made with palliative intent or with probabilities of cure that must be weighed against the toxicity side-effects of the treatment. Although great strides have been made in the use of targeted therapies with more favorable side-effect profiles, oncologists remain acutely aware of the clinical tradeoffs between length of life and quality of life. By its nature, comparing the benefits and harms of two or more cancer treatments long ago pushed the field of oncology beyond the metrics of mortality and disease-free survival, and toward considerations of QOL. Measuring QOL and not just survival allows patients, physicians, and policy makers: (1) to choose between treatment options without curative intent by judging whether the tradeoffs in side effects may be worth any gains in life expectancy or mitigation of cancer symptoms in the absence of cure, and (2) to choose between two treatments with similar survival benefits but different constellations of side effects.
Two Types of Quality-of-Life Measures
It is critical at this juncture to distinguish between the two different types of QOL measures encountered in the field of oncology: descriptive health status vs patient preference weights. In the medical literature, both measures are frequently referred to as QOL or health-related QOL, leading to considerable confusion. However, each type of measure has a different theoretical origin in addition to different applications and uses.
Descriptive Health Status
Descriptive health status measures are probably the more accessible and transparent of the two rubrics. Examples of some of the instruments found in cancer care include the various Functional Assessment of Cancer Therapy (FACT) scales, QLQ-C30, and the Expanded Prostate Cancer Index Composite (EPIC).[6-8] These measures look beyond physical function to evaluate quality of life in other areas that contribute to overall health. By considering questions about emotional health, social support, the ability to engage in usual activities, and limitations due to pain, descriptive health status helps to appraise QOL domains besides the physical function captured in performance status or survival assessments.
For instance, EPIC, a prostate cancer-specific health status measure, evaluates physical health but also assesses how patients perform relative to other domains affected by prostate cancer, such as urinary, bowel, and sexual functioning. Such disease-specific domains affected by prostate cancer were found to be important to survivors but had not been included in more generic instruments like the QLQ-C30. Outcome measurements based on health status assessment have their roots in psychometric theory. Questions in each domain are scored as a subscale and then summed to a composite score that can be compared with other studies that have used the same measurement. Implicit in the scaling of health status is that changes of a certain magnitude in one domain are valued equivalently to the same size change in a different domain. In reality, patients may value one domain very differently from another domain. Some prostate cancer patients may regard severe bowel symptoms as worse than severe impotence, but the different values are not reflected in health status scores. A prostate cancer treatment resulting in severe bowel symptoms scores the same as a treatment resulting in severe impotence, when all other domains are held equal.
Other methodological issues develop when comparing health status scores across different diseases or for the same disease assessed by different measurement scales. For example, how does a FACT-B score in a breast cancer trial compare with an EPIC score in a prostate cancer trial, or to a Seattle Angina Questionnaire score from a cardiology trial?[7,8,10] Although generic health status measures can overcome some of these problems, the concern is that they may be insensitive to clinically relevant changes in disease-specific domains that are not surveyed explicitly. It can be difficult to inform either patients or policy makers when studies report QOL outcomes over different time frames using different health status scales containing different domains.[11,12]
Patient Preference Weights
Because the primary goal of CER is to compare results across a broad array of healthcare strategies and medical interventions to determine which offer the greatest benefits, the outcomes measure utilized in CER requires a common denominator that incorporates QOL over a standard time interval. The assessment of quality of life using patient preference weights allows both QOL (morbidity) and length of life (mortality) to be captured in a single unit of measure, known as a quality-adjusted life-year (QALY). QALYs can be used as the common denominator to compare results from healthcare interventions regardless of disease type, time frame, or medical discipline. Unlike descriptive health status measures, outcomes measured in QALYs in a 1-year breast cancer program can be compared with outcomes measured in QALYs for a 3-year prostate cancer trial, which in turn can be compared to QALY results in a cardiology intervention.[12-14]
Perhaps the greatest advantage to employing QALYs in CER, however, is that descriptive health status measures cannot be used for QOLadjustments in analyses of resource utilization. Only preference-weighted measures can be used to reflect quality of life in the economic analysis of resource use to determine effectiveness or cost-effectiveness. The theoretical origins of QALY measurement can be found in game theory and the axioms of expected utility theory in the field of economics.[15,16] The application of QALYs to medicine and healthcare was popularized in the 1970s and has grown steadily to the point that QALYs are now a well-accepted measure of health outcome.[17-19] Although it would be appealing to substitute descriptive health status measures for preference weights in an economic analysis, the two QOL metrics are not equivalent,[20,21] and doing so would violate the theoretical underpinnings of medical resource comparisons.[18,22-24]
1. Improving Medical Decisions Through Comparative Effectiveness Research: Cancer as a Case Study. Arlington, Virginia, Friends of Cancer Research, 2009.
2. Institute of Medicine: Initial National Priorities for Comparative Effectiveness Research. Washington, DC, The National Academies Press, 2009.
3. Federal Coordinating Council for Comparative Effectiveness Research Report to the President and the Congress. Washington, DC, Department of Health and Human Services, 2009.
4. Horn SD, Gassaway J: Practice-based evidence study design for comparative effectiveness research. Med Care 45(suppl 2):S50-S57, 2007.
5. Luce BR, Kramer JM, Goodman SN, et al: Rethinking randomized clinical trials for comparative effectiveness research: The need for transformational change. Ann Intern Med 151:206-209, 2009.
6. Aaronson NK, Ahmedzai S, Bergman B, et al: The European Organization for Research and Treatment of Cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 85:365-376, 1993.
7. Fallowfield LJ, Leaity SK, Howell A, et al: Assessment of quality of life in women undergoing hormonal therapy for breast cancer: Validation of an endocrine symptom subscale for the FACT-B. Breast Ca Res Treat 55:189-199,1999.
8. Wei JT, Dunn RL, Litwin MS, et al: Development and validation of the expanded prostate cancer index composite (EPIC) for comprehensive assessment of health-related quality of life in men with prostate cancer. Urology 56:899-905, 2000.
9. Nunnally JC, Bernstein IH: Psychometric Theory, 3rd ed. New York, McGraw-Hill, Inc., 1994.
10. Spertus JA, Winder JA, Dewhurst TA, et al: Development and evaluation of the Seattle Angina Questionnaire: A new functional status measure for coronary artery disease. J Am Coll Cardiol 25:333-341, 1995.
11. Kattan MW: Comparing treatment outcomes using utility assessment for health-related quality of life. Oncology (Williston Park) 17:1687-1693, 2003.
12. Stiggelbout AM, de Haes JC: Patient preference for cancer therapy: An overview of measurement approaches. J Clin Oncol 19:220-230, 2001.
13. Earle CC, Chapman RH, Baker CS, et al: Systematic overview of cost-utility assessments in oncology. J Clin Oncol 18:3302-3317, 2000.
14. Tengs TO, Wallace A: One thousand health-related quality-of-life estimates. Med Care 38:583-637.
15. Hirshleifer J, Riley JG: The analytics of uncertainty and information. Cambridge,
England, Cambridge University Press, 1992.
16. Von Neumann J, Morgenstern O: Theory of Games and Economic Behavior. Princeton, New Jersey, Princeton University Press, 1947.
17. Weinstein MC, Stason WB: Foundations of cost-effectiveness analysis for health and medical practices. N Engl J Med 296:716-721, 1997.
18. Gold MR, Siegel JE, Russell I, et al (eds): Cost-effectiveness in Health and Medicine. New York, Oxford University Press, 1996.
19. Mark DB, Pan W, Clapp-Channing NE, et al: Occluded Artery Trial I: Quality of life after late invasive therapy for occluded arteries. N Engl J Med 360:774-783, 2009.
20. Bosch JL, Hunink MG: The relationship between descriptive and valuational quality-of-life measures in patients with intermittent claudication. Med Decis Making 16:217-225, 1996.
21. Bryce CL, Angus DC, Switala J, et al: Health status versus utilities of patients with end-stage liver disease Qual Life Res 13:773-782, 2004.
22. Drummond M, Sculpher MJ, Torrance GW, et al: Methods for the Economic Evaluation of Health Care Programmes, 1st ed. New York, Oxford University Press, 2005.
23. Hunink MG, Glasziou P, Siegel JE, et al: Decision Making in Health and Medicine: Integrating Evidence and Value, 1st ed. New York, Cambridge University Press, 2001.
24. Sox HC, Blatt MA, Higgins MC, et al: Medical Decision Making, 1st ed. Boston,
25. Bleichrodt H, Johannesson M: An experimental test of a theoretical foundation for rating-scale valuations [see comment]. Med Decis Making 17:208-216, 1997.
26. Froberg DG, Kane RL: Methodology for measuring health-state preferences--II: Scaling methods. J Clin Epidemiol 42:459-471, 1989.
27. Torrance GW: Utility approach to measuring health-related quality of life. J Chronic Dis 40:593-603.
28. Froberg DG, Kane RL: Methodology for measuring health-state preferences--III: Population and context effects. J Clin Epidemiol 42:585-592, 1989.
29. Brooks R: EuroQol: The current state of play. Health Policy 37:53-72, 1996.
30. Torrance GW, Feeny DH, Furlong WJ, et al: Multiattribute utility function for a comprehensive health status classification system. Health Utilities Index Mark 2. Med Care 4:702-722, 1996.
31. Krahn M, Ritvo P, Irvine J, et al: Construction of the Patient-Oriented Prostate Utility Scale (PORPUS): A multiattribute health state classification system for prostate cancer. J Clin Epidemiol 53:920-930, 2000.
32. Nease RF, Jr., Tsai R, Hynes LM, et al: Automated utility assessment of global health. Qual Life Res 5:175-182, 1996.
33. Littenberg B, Partilo S, Licata A, et al: Paper Standard Gamble: The reliability of a paper questionnaire to assess utility. Med Decis Making 223:480-488, 2003.
34. Lenert LA, Hornberger JC: Computer-assisted quality of life assessment for clinical trials. Proc AMIA Annu Fall Symp 1996: 992-996. Available at http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2233008/pdf/procamiaafs00002.... Accessed April 28, 2010.
35. Ross PL, Littenberg B, Fearn P, et al: Paper standard gamble: A paper-based measure of standard gamble utility for current health. Int J Technol Assess Health Care 19:135-147, 2003.
36. Kilbridge KL, Fraser G, Krahn M, et al: Lack of comprehension of common prostate cancer terms in an underserved population [see comment]. J Clin Oncol 27:2015-2021, 2009.
37. Jemal A, Thomas A, Murray T, et al: Cancer statistics, 2002 [see comment][erratum appears in CA Cancer J Clin 52(2):119, 2002]. CA Cancer J Clin 52:23-47, 2002.
38. Esper P, Mo F, Chodak G, et al: Measuring quality of life in men with prostate cancer using the functional assessment of cancer therapy-prostate instrument. Urology 50:920-928, 1997.
39. Litwin MS, Hays RD, Fink A, et al: The UCLA Prostate Cancer Index: Development, reliability, and validity of a health-related quality of life measure. Medical Care 36:1002-1012, 1998.
40. Potosky AL, Harlan LC, Stanford JL, et al: Prostate cancer practice patterns and quality of life: The Prostate Cancer Outcomes Study.
J Natl Cancer Inst 91:1719-1724, 1999.