In 1997, the Agency for Health Care Policy and Research, now known as the Agency for Healthcare Research and Quality (AHRQ), launched an initiative to promote evidence-based practice in everyday care by establishing 12 Evidence-based Practice Centers. AHRQ establishes contracts with the Evidence-based Practice Centers to develop evidence reports and technology assessments on clinical topics that are common, expensive, and/or significant for the Medicare and Medicaid populations. Through this program, AHRQ partners with private and public organizations to improve the quality, effectiveness, and appropriateness of clinical care by facilitating the translation of research evidence into clinical practice.
Approach to Systematic Reviews
Program Overview
In 1998, the American Society of Hematology (ASH) and the American Society of Clinical Oncology (ASCO) jointly nominated the topic "Uses of Erythropoietin(Drug information on erythropoietin) in Oncology" to AHRQ for an Evidence-based Practice Center systematic review and evidence report. A panel appointed jointly by ASH and ASCO intended to use the resulting evidence report to support development of a clinical guideline for dissemination to their members. AHRQ contracted with the Blue Cross and Blue Shield Association (BCBSA) Technology Evaluation Center Evidence-based Practice Center to conduct the systematic review and evidence report on erythropoietin.
Evidence-based Practice Centers develop evidence reports and technology assessments based on rigorous, comprehensive syntheses and analyses of relevant scientific literature, emphasizing explicit and detailed documentation of methods, rationale, and assumptions. These scientific syntheses may include meta-analyses and cost analyses. Each Evidence-based Practice Center collaborates with other medical and research organizations so that a broad range of experts is included in the development process. More detailed information on the Evidence-based Practice Centers program, the topic nomination process, and the list of centers is available at http://www.ahrq.gov/clinic/epcix/htm.[1] Executive summaries and full copies of completed reports (with bibliographies and evidence tables) are available at the same URL for viewing or complimentary downloading. Complimentary single, printed copies also may be obtained from the AHRQ Publications Clearinghouse (1-800-358-9295).
Systematic Review Methods
Protocols for systematic review are prospectively designed to define study objectives and key questions; search strategy; patient populations of interest; study selection criteria and methods to determine study eligibility; outcomes of interest; data elements to be abstracted and abstraction methods; and methods to assess study quality. Usually, two independent reviewers complete each step of the protocol. Reviewers individually evaluate studies against selection criteria, abstract data separately, and compare their results after each step. Disagreements are generally resolved by consensus but may require resolution by a third reviewer.
A technical advisory group provides ongoing guidance on all phases of each Evidence-based Practice Centers’ review. Six technical advisors participated in the evidence report on use of erythropoietin in oncology patients. ASCO and ASH each appointed two of the six advisors (including the guideline panel co-chairs) and the Technology Evaluation Center Evidence-based Practice Center staff recruited the remaining two.
Evidence-based Practice Centers reviews begin with a comprehensive literature search that attempts to identify all publications of relevant controlled trials. The search strategy for the review on erythropoietin is described briefly in the Executive Summary posted on the AHRQ web site[1] and more completely in the full evidence report.[2] The Medline, Cancerlit, and Embase databases, last searched in December 1998, yielded 2,915 references. We identified 28 additional reports by supplementary searches (eg, Current Contents, bibliographies from manufacturers) through October 30, 1999, for a total yield of 2,943 references.
Next, studies are selected for data abstraction using criteria specified in the protocol. The primary study selection criteria for the erythropoietin review required that studies be designed as controlled trials comparing the outcomes of managing anemia with and without erythropoietin in a patient population relevant to one of four clinical settings. These were (1) anemia due primarily to cancer therapy; (2) anemia due primarily to a malignancy; (3) high-dose myeloablative therapy followed by an allogeneic transplant of hematopoietic stem cells from peripheral blood or bone marrow; and (4) high-dose myeloablative therapy followed by an autologous transplant of hematopoietic stem cells. We defined the setting as anemia primarily due to cancer therapy if trials limited enrollment to patients undergoing concurrent chemotherapy or radiation therapy with conventional nonmyeloablative doses. We defined the setting as anemia primarily due to malignancy if some enrolled patients did not receive concurrent chemotherapy or radiation therapy while on study.[2] Trials were excluded if there were < 10 similarly treated evaluable patients in each arm.
In the available trials, erythropoietin treatment (with transfusion used as necessary) was always compared with red blood cell (RBC) transfusion alone; no trials compared erythropoietin to any other alternative. All randomized controlled trials relevant to any of the four clinical settings were included. Studies that used nonrandomized concurrent or historical controls were included if the reviewers could determine that similar patients were included in the treatment and control groups. Nonrandomized trials were considered to be of lesser quality than randomized controlled trials. Outcomes of interest included
- Magnitude of change in hemoglobin (Hgb) levels.
- Percentage of patients who met criteria for a hematologic response as defined in each study’s protocol.
- Percentage of patients who were transfused.
- Number of RBC units transfused per patient normalized to a 4-week period.
- Quality of life.
- Symptoms of anemia (besides fatigue or other components measured by quality-of-life questionnaires).
- Adverse effects of treatment.
The systematic review addressed the following key questions separately for each clinical setting:
1. Effects of Erythropoietin Treatment—What were the relative effects on outcomes of managing anemia with erythropoietin compared with transfusion alone? In settings other than stem-cell transplants, what were the relative effects of erythropoietin treatment when different Hgb thresholds were used to initiate erythropoietin treatment?
2. Variations in Erythropoietin Regimens—In the included studies, did variations in the erythropoietin treatment regimen (such as dose, frequency, duration, route) affect the outcomes of treatment? Were these variations likely to confound interpretation of the evidence on the relative effects of erythropoietin treatment according to the alternative Hgb thresholds for initiating treatment?
3. Identification of Patient Response—Were there populations or subgroups of patients more or less likely to benefit from erythropoietin treatment? Were there laboratory measurements that either predicted or permitted early identification of patients whose anemia responded to erythropoietin?
4. Data on Adverse Effects—What were the incidence and severity of adverse effects associated with the use of erythropoietin and how did these compare with the adverse effects of transfusion?
Data abstraction for adverse events also was limited to controlled trials so that effects of erythropoietin could be distinguished from effects of disease progression or concurrent therapies for the underlying malignancies.
To supplement the systematic review, we conducted a literature-based meta-analysis of the effect of erythropoietin on the odds of transfusion for patients with anemia or at risk of anemia due primarily to cancer therapy. A random effects model was used to calculate the combined odds ratio of transfusion for the 12 randomized controlled trials that reported numbers or percentages of patients transfused, with or without erythropoietin administered subcutaneously, for treatment-related anemia. The odds ratio expresses the relative likelihood that erythropoietin-treated patients will be transfused compared with the likelihood for controls. Published data were insufficient for literature-based meta-analysis of other outcomes, or of odds of transfusion in other clinical settings.
Sensitivity analysis compared results of higher-quality trials to those of lesser-quality trials. A trial was classified as higher quality when it was randomized and double-blinded and met our criteria concerning limits on the number of subjects excluded from the analysis of results. We required that < 10% of subjects within each study arm were excluded from the analysis and that the ratio of exclusions from each arm was less than 2:1; or, alternatively, that results were reported as an intention-to-treat analysis.
External Review
AHRQ requires that Evidence-based Practice Center reports undergo extensive review by external experts and representatives of stakeholder organizations. Early in each project, these individuals review and provide input to modify the study protocol. Later, they review and comment on the report’s initial draft. However, each Evidence-based Practice Center has ultimate responsibility for the final draft of its reports, subject to AHRQ review.
For the erythropoietin report,[2] the BCBSA Technology Evaluation Center Medical Advisory Panel, which includes nationally recognized experts in technology assessment and hematology/oncology, reviewed a preliminary analysis of the evidence base. Additionally, 20 external reviewers critiqued the study protocol and draft report, and revisions were made based on their comments. Eight reviewers were invited by Technology Evaluation Center based on their expertise in medical oncology, hematology, transfusion medicine, quality-of-life and systematic review methodology. One reviewer directed another AHRQ Evidence-based Practice Center and is a medical oncologist. Ten reviewers were appointed by professional organizations other than ASCO or ASH and by patient advocacy groups. These reviewers included clinical and research specialists involved in the treatment of cancer and/or management of cancer-related anemia and patient advocacy representatives. One external reviewer was from the technical staff of Ortho Biotech, Inc. Lists of the Technology Evaluation Center Medical Advisory Panel members, external reviewers, and technical advisors are included in the evidence report’s appendices.[2]
A 52-year-old man presented with an erythematous lesion in the axilla of unknown duration. Surgical excision was performed. What is your diagnosis?
