In 1999, the Institute of Medicine released an important
report entitled "Ensuring Quality Cancer Care" that emphasized the
importance of providing patients with quality cancer care and establishing
frameworks with which to measure the care that patients receive. The issues
raised by this report are critical for determining the future of the health
Katherine Kahn, MD, of the RAND Corporation, is a national
leader in the area of quality of medical care, with recent efforts focusing on
breast cancer care. Dr. Kahn shared her findings at the "Ensuring
Quality Cancer Care" symposium of the Robert H. Lurie Comprehensive Cancer
Center of Northwestern University and the VA Chicago Health Care System.
Defining Quality of Care
To measure quality of cancer care, it is essential to consider
the definition of quality of health care. The quality of health care can be
loosely defined according to the framework of the structures providing health
care, the processes involved in caring for patients, and the outcomes of the
care that patients receive.
In evaluating structure, researchers often assess the
organization of the provider, the patient’s access to the provider, and the
comprehensiveness of the care provided by clinicians.
Interactions between patients and different health care
providers within the system are the source of both objective and subjective
data that can be used to evaluate health care processes.
One example of a process measure is the time from an abnormal
Pap smear to a diagnostic resolution of the results from the smear. Subjective
data could include the patient’s impressions of the clinician and the tone of
their discussion before and after the Pap smear. Objective data might include
the length of time between the laboratory’s interpretation of the test
results and the communication of these results to the health care provider.
However, while process measures are generally useful for
measuring quality of care, Dr. Kahn suggests that evaluating the
appropriateness of the care along with process measures can lead to a much
richer picture of the patient’s experience within the health care system.
Measuring outcomes of health care, the third component of
evaluating quality of health care, involves measuring the effects of the health
care process on health. Mortality is one fairly simple outcome to measure. For
example, 5-year survival rates and the length of remission are common health
outcomes used in evaluating cancer care.
However, measurement of outcomes may also encompass a wide
variety of patient characteristics, including the clinical, financial,
psychological, and social impact of the health care process on the patient. A
quality-of-life (QOL) assessment of patients can include all of these different
factors and gives patients an opportunity to provide qualitative feedback to
their health care provider and institution.
Data from QOL assessments may affect the care that future
patients receive. Results from these types of measurements can also be combined
with other measures, such as the cost of care for the patient, provider, and
institution, to analyze the cost-effectiveness of certain treatments for these
different entities in the health care system.
Challenges in Designing Studies
In her presentation, Dr. Kahn emphasized that it is important
to consider how to evaluate the complex interactions among the various
components of the health care system.
She stated that researchers encounter many challenges in
designing a study to measure the quality of cancer care.
First, the study must determine if a problem in care exists. A
baseline measurement of factors and continued surveillance of these factors
will allow researchers to detect a deviation from the standard of care.
Second, once a problem is detected, the researchers are faced
with the delicate task of attributing the problem to a combination of patient,
provider, and hospital characteristics.
A misattribution of the problem to a health care provider could
result in many patients needlessly switching their provider, creating added
hassles for the patient and removing patients from a provider who provides
Finally, researchers must carefully consider the initiatives
they suggest for quality improvement. The data from the study and resulting
intervention must ultimately produce consistent long-term
changes, rather than effects that are short lived.
Although quality of health care studies have tremendous
potential for improving the ways in which health care is delivered, Dr. Kahn
demonstrates that designing an effective quality of health care study demands
Hospital Death Rates Study
In the mid-1980s, the Health Care Financing Administration
(HCFA) released annual reports about hospital death rates for Medicare patients
in an effort to identify hospitals that provided low quality of care. In
response to this effort, Dr. Kahn collaborated with several other researchers
and clinicians in a study that examined the variation in hospital death rates
at a variety of acute-care hospitals in the United States.
Previously, other groups had found that while death rates
varied by hospital, these differences were often attributable to differences in
the severity of illness or level of comorbidities in the patient
Dr. Kahn and her collaborators evaluated whether hospitals with
high age-, sex-, race-, and disease-specific death rates provided lower quality
of care to patients than hospitals with lower death rates.
They also evaluated the relative contributions of severity of
illness and quality of care in statistical models that addressed an individual’s
in-hospital probability of death.
Retrospective medical record review was used to find
information on congestive heart failure and acute myocardial infarction, which
comprised 17.5% of Medicare hospital deaths.
The results indicated that hospitals with high death rates do
not provide a lower quality of care than hospitals with lower death rates: 25%
of the difference in death rates among hospitals was attributed to the higher
average severity of illness of patients with acute myocardial infarctions in
higher-death-rate hospitals. However, differences in severity of illness were
not correlated with the higher death rates of patients with congestive heart
Higher death rates at certain hospitals may be related to other
factors besides quality of care such as longer average inpatient stay.
From this study, the authors concluded that it is difficult to
accurately assess differences in hospital care, even when using carefully
constructed measures of quality of care.
Dr. Kahn reviewed some key issues that may influence the rates
of certain quality measures. For example, in the case of screening rates for a
type of cancer, including women with symptoms (diagnostic evaluations) could
increase the rate of screening. Many outcome measures may also vary with
patient and population characteristics such as age, co-morbidity, access to
screening, treatment choices, eligibility for treatment, and family history of
A researcher would expect survival to be better among women who
have had continuous screenings for breast cancer throughout their lives,
compared with women who rarely had mammograms.
Patient surveys can be misleading. For example, if only the
most functional breast cancer patients respond to a survey about QOL measures,
then the population will appear to be healthier.
Outcome measures and process measures may need to be adjusted
for many of these factors after data collection. These factors should also be
taken into account when selecting the study population and determining the
methods to be used for data collection.
Some settings may not have sufficient numbers of cancer cases
to yield an accurate estimate of a certain measure. For instance, it would be
difficult to assess accurately the quality of cancer care offered by a
physician who sees only one or two cancer cases per year.
One solution to this problem could be to include data from a
number of years or to include data from similar sized practices and similar
Case identification through medical record review at a
particular hospital may result in a very different yield than cancer patient
identification using a national tumor registry.
It is also important to evaluate measures chosen for a study
for reliability and validity. If a QOL measure does not increase as a patient’s
clinical condition improves, the QOL assessment may not accurately reflect the
patient’s mental and physical health.
It is preferable to use measurements that have predictive
validity, internal consistency, and good face validity. However, the only way
to determine if certain measures are reliable and valid is to evaluate the
measures themselves through repeated studies.
A High Priority
Ensuring quality cancer care for patients has become a high
priority for health care providers and policy makers. To determine how to
improve cancer care, clinicians and researchers must first select measures with
which to evaluate the quality of care that is currently provided to
More initiatives are needed to address the numerous challenges
inherent in developing these measures and ultimately evaluating the many
different aspects of care. By devoting additional resources and consideration
to this area, it is likely that both the quantity and quality of life of cancer
patients will be improved.
1. Mandelblatt JS, Ganz PA, Kahn KL: Proposed agenda for the
measurement of quality-of-care outcomes in oncology practice. J Clin Oncol
2. Park RE, Brook RH, Kosecoff JK, et al: Explaining variations
in hospital death rates: Randomness, severity of illness, quality of care. JAMA