A variety of activities are included under the general heading of
outcomes assessment. The terms outcomes assessment and
outcomes studies are sometimes used to describe classic
clinical trials that include unconventional end points, such as
quality of life and cost. However, the same terms may also describe
cohort or observational studies, in which the treatment patients
receive is not dictated by the study, but rather, patients are
monitored in the course of routine clinical care to determine what
outcomes they experience as a result of the treatment selected.
Finally, outcomes assessment is increasingly used to refer to the
process of managing patterns of care in routine clinical practice.
Although these are clearly three distinct activities, they share
several important common features. In all cases, there is an emphasis
on quality of life and economic outcomes, in addition to the more
familiar measures of efficacy employed in traditional oncology
studies. There is also an explicit consideration of the importance of
patient characteristics in determining outcomes. For example, all of
these approaches ask such questions as what is the right drug
for this patient? or what particular types of patients
are most likely to benefit from one treatment strategy versus
another? Finally, all share an unusually broad definition of
what constitutes cancer care, including not just pharmaceutical
agents but also practice setting, type of provider, and follow-up
strategy, for example.
Underlying all of these approaches is a basic model of the
relationship between patient and provider characteristics, treatment
strategies, and outcomes. As delineated in this model, the choice of
interventions is influenced by both patient and provider
characteristics. A providers knowledge, preferences, and
biases, as well as his/her sociodemographic characteristics may
influence therapeutic choices. For example, a number of studies have
demonstrated marked geographic variability in patterns of care for
common cancers, presumably related, at least in part, to geographic
variability in physician beliefs and biases.[1-3] Similarly, patient
characteristics, including clinical factors such as age, stage, and
comorbidity, as well as patient preferences, are important
determinants of treatment choice.
Treatment choice, in turn, influences outcomes. But so do patient
characteristics, and it is critical to control for this in assessing
the relationship between treatment and outcomes. For example, if one
looks only at the relationship between treatment choice and outcome,
one could mistakenly conclude that drug A is less
effective than drug B, when, in fact, patients receiving
drug A were older with more comorbid disease and thus
likely to have poorer outcomes with any therapeutic approach. The
need to control for this confounding by patient characteristics is
the biggest challenge of outcomes research using observational study designs.
Outcomes data can also be used to evaluate providers. Assessments of
the patterns of care and, in particular, providers compliance
with guidelines, are increasingly being used to assess the quality of
care. It is also critical to control for patient characteristics in
this type of analysis.
Several different study designs may be employed to examine the
effectiveness of procedures or interventions. The major advantage of
the randomized, controlled trial is that it eliminates the need to
control for confounding patient characteristics, which are assumed to
be allocated equally to both arms by chance alone. If the array of
outcomes in randomized clinical trials is broadened to include not
only response and survival but also quality of life, patient
preferences, and costs, the trial can provide invaluable information
on the relationship between treatment choice and all of the outcomes
of interest, unconfounded by other factors. It is well known,
however, that patients who agree to participate in randomized cancer
trials are highly selected, and tend to be younger, healthier, and
better educated than the average cancer patient. Furthermore, care
delivered in the course of a clinical trial may be more intensive
than in the usual care setting. As a result, it may be difficult to
generalize the results from such trials to all patients seen in the
clinical practice setting. In contrast, observational studies examine
the results of treatments in a wider array of patients within the
usual care setting, but nearly always involve some degree of
confounding by patient characteristics. These two study designs are
best seen as complementary; each addressing the limitations of the other.
Cohort studies are observational studies in which the treatment is
not specified by the study design. Cohort studies may rely on
preexisting data bases, such as administrative or claims data sets
generated by government or private health insurers. These data bases
have been mined successfully to assess patterns of care and outcomes.
The amount of clinical detail, including tumor stage and specific
treatment types or doses, is often fairly limited in these data
bases, however, so that carefully constructed, prospective, cohort
studies may be required to answer many questions of interest.
Finally, decision analysis is a frequently used study design in
outcomes assessments. These computerized models combine data from a
variety of data sources, including randomized trials and cohort
studies, to generate estimates of the likely impact of alternative
strategies on an array of outcomes, including length of life, quality
of life, and costs.
Physicians and the general public are increasingly seeing the results
of outcomes studies used to measure the quality of care provided by
physicians, hospitals and insurance plans. Physician and hospital
report cards, listing severity-adjusted mortality or
other outcomes are a good example. Both New York and Pennsylvania
have conducted state-sponsored analyses of coronary artery bypass
surgery mortality and the results, listed by provider, have been
published in the lay press.[4,5] It is likely that such measures will
be used with greater frequency in the coming years in other areas of
medicine, including oncology, once the appropriate procedures for
severity-adjustment are developed. The use of patterns of care and
outcomes data for internal fiscal and administrative management by
health care institutions and insurance plans is less visible to the
public but much more widespread. Physician profiling, in which
individual physicians practice patterns are assessed for
appropriateness and cost, is increasingly common.
Outcomes management is a process that integrates outcomes research or
assessment with monitoring of providers in an attempt to produce high
quality, cost-effective medical care. As described by Ellwood in an
influential paper published in The New England Journal of Medicine
in 1988, outcomes management involves several components. The
cornerstone of this approach is the implementation of guidelines.
Ellwood stressed the importance of considering the impact of medical
care on functioning, as well as disease-specific clinical outcomes,
in generating these guidelines, and argued that comprehensive data on
the full array of outcomes resulting from medical care should be
collected in the course of routine clinical care. He also urged that
findings from analyses of these data be disseminated and, in
particular, used in an interactive fashion to improve the quality of
the guidelines. Current outcomes management programs follow this
general paradigm; however, with the recent dramatic changes in the
financing of health care, the concept of outcomes management has
evolved to include a heavier emphasis on considerations of cost.
The two goals of outcomes management are to maximize patient outcomes
and to minimize cost. Inherent in this charge is the need to assess
the value for money produced by any specific medical intervention.
The standard mechanisms of outcomes management are guidelines for the
management of common conditions, and the measurement of provider
performance against a set of quality indicators derived from those guidelines.
This tight link between guidelines and outcomes data is increasingly
being recognized as critical to the success of outcomes management
programs. Achievement of institutional or organizational goals
requires that outcomes data are used to monitor compliance with
guidelines, and that feedback is given to providers about their level
of adherence to those guidelines. Furthermore, it is essential that
information on the impact of interventions on outcomes be fed back to
guideline writers who must continually update and improve the guidelines.
Many physicians would argue that, in their role as the individual
patients advocate, they should support any intervention that
they believe will result in a net benefit for the patient. However,
for most oncologists today, patient advocacy is just one of several
roles in their professional lives. Any physician who is involved
in setting policy through membership in a Pharmacy and Therapeutics
Committee or participation in the formulation of guidelines, for
example, faces the necessity of making decisions about the allocation
of resources that are not unlimited. Similarly, physicians who
provide care to capitated patients or who have assumed some financial
risk for patient care must consider both the outcomes and the costs
of an array of therapeutic strategies. Also, any physician practicing
in a setting where job security depends on attracting patients in a
competitive market must also consider costs. In the late 1990s it is
a rare oncologist, indeed, who can ignore considerations of whether
the outcomes of therapy justify the costs.
The dilemma for the clinician is how to reconcile the need to do what
is best for patients with the need to practice cost-conscious
medicine. High-quality outcomes data are at the heart of the
solution. If we know how alternative choices influence all of the
relevant patient outcomes, we have a better chance of successfully
identifying when we can choose a lower cost option without
meaningfully compromising the quality of care. Even more importantly,
we can justify the decision to pursue high cost options based on
compelling evidence of the value added for patients.
1. Osteen RT, Steele GD, Menck HR, et al: Regional differences in
surgical management of breast cancer. CA Cancer J Clin 42:39-43, 1992.
2. Nattinger AB, Gottlieb MS, Veum J, et al: Geographic variation in
the use of breast-conserving treatment for breast cancer. N Engl J
Med 326:1102-1107, 1992.
3. Lu-Yao GL, McLerran D, Wasson J, et al: An assessment of radical
prostatectomy time trends, geographic variation and outcomes. JAMA
4. Chassin MR, Hannan EL, DeBuono BA: Benefits and hazards of
reporting medical outcomes publicly. N Engl J Med 334: 394-398, 1996.
5. Coronary Artery Bypass Graft Surgery. Technical Report, vol 2.
Harrisburg, Pennsylvania Health Care Cost Containment Council, 1994.
6. Ellwood PM: Shattuck lectureoutcomes management: A
technology of patient experience. N Engl J Med 318:1549-1556, 1988.
7. Smith TJ: A piece of my mind: Which hat do I wear? JAMA