Erythropoietin, uncertainty principle and cancer related anaemia.

BACKGROUND: This study was designed to evaluate if erythropoietin (EPO) is effective in the treatment of cancer related anemia, and if its effect remains unchanged when data are analyzed according to various clinical and methodological characteristics of the studies. We also wanted to demonstrate that cumulative meta-analysis (CMA) can be used to resolve uncertainty regarding clinical questions.
METHODS: Systematic Review (SR) of the published literature on the role of EPO in cancer-related anemia. A cumulative meta-analysis (CMA) using a conservative approach was performed to determine the point in time when uncertainty about the effect of EPO on transfusion-related outcomes could be considered resolved. PARTICIPANTS: Patients included in randomized studies that compared EPO versus no therapy or placebo. MAIN OUTCOME MEASURES: Number of patients requiring transfusions.
RESULTS: Nineteen trials were included. The pooled results indicated a significant effect of EPO in reducing the number of patients requiring transfusions [odds ratio (OR) = 0.41; 95%CI: 0.33 to 0.5; p < 0.00001;relative risk (RR) = 0.61; 95% CI: 0.54 to 0.68]. The results remain unchanged after the sensitivity analyses were performed according to the various clinical and methodological characteristics of the studies. The heterogeneity was less pronounced when OR was used instead of RR as the measure of the summary point estimate. Analysis according to OR was not heterogeneous, but the pooled RR was highly heterogeneous. A stepwise metaregression analysis did point to the possibility that treatment effect could have been exaggerated by inadequacy in allocation concealment and that larger treatment effects are seen at hb level > 11.5 g/dl. We identified 1995 as the point in time when a statistically significant effect of EPO was demonstrated and after which we considered that uncertainty about EPO efficacy was resolved.
CONCLUSION: EPO is effective in the treatment of anemia in cancer patients. This could have already been known in 1995 if a CMA had been performed at that time.

Background

A synthetic form of EPO, human recombinant EPO, has been successfully used to treat anemia in patients with chronic renal failure and HIV [1]. Some randomized trials (RCT) assessed the role of EPO in anemia related the cancer [2,3]. However, many of these trials were underpowered and failed to identify clinically meaningful benefits of EPO treatment. As a result, there has been persistent uncertainty about the efficacy of EPO as a treatment for cancer-related anemia, despite the introduction of EPO in a clinical practice almost a decade ago. Systematic reviews (SR) are the best way to offer synthesis of evidence and are of special utility in settings where many small trials fail to achieve a significant result [4].

A recent comprehensive SR [5,6] addressed the role of EPO in treatment-related anemia in cancer patients. This SR included non-randomized studies and performed a meta-analysis (MA) of 12 RCTs showing a significant effect of EPO in reducing the need to transfuse patients who are receiving chemotherapy. Although comprehensive, some issues were not addressed in this previous SR/MA: a formal quantitative synthesis (meta-analysis) was not performed according to different clinical aspects and to main methodological quality dimensions empirically linked to bias [7].

In contrast to this previous report [5,6], in this SR we evaluated only data from randomized studies. We also investigated a broad range of clinical issues, including EPO use according to the level of hemoglobin (hb), platinum-based chemotherapy and tumor type. Finally, to investigate the stability of our conclusions, a broad methodological appraisal of the quality of the trials was performed, specifically examining those dimensions that have been empirically linked to bias [7].

We also performed a conservative cumulative meta-analysis (CMA) [8] to determine the earliest point in time when the use of EPO versus placebo reached a such statistical significance after which the uncertainty [9] about the effect of EPO in cancer-related anemia could have been considered resolved.

Methods

A previous SR served as a basis for location of the articles of interest [5]. We also performed a search of MEDLINE, LILACS and CANCERLIT databases, last update in July of 2001, using the optimal search strategy for RCT for use in MEDLINE [10] and LILACS [11] with the additional terms related to this review – (epoetin OR Erythropoietin) and (cancer OR neoplasm), in all fields.

We included only RCTs that compared EPO versus no therapy or placebo in cancer related anemia. We excluded studies related to transplant setting and myelodysplastic syndrome, since the physiopathology of these diseases is different, more linked to pancytopenia than to anemia [1]. We also excluded trials that used EPO with the primary objective to magnify hb to improve the efficacy of radiotherapy.

Four trials described more than one comparison of different doses of EPO with only one control group [12-15]. We approached these multi-arm studies in two different ways: in the first, each EPO arm was compared to the control arm. In the second we combined all active arms in one, by adding all EPO treated patients, and analyzed it against the control. Both combinations achieved virtually the same result and data are presented using the second method.

Data were abstracted on patient characteristics, type of tumor, treatment details and the major methodological quality dimensions [4,7] (see Additional file 1). The end point studied was the number of patients requiring transfusions. Results in a conventional meta-analysis were pooled using the Peto's Odds Ratio (OR) and relative risk (RR), both in a fixed effect model [16]. We also extracted data on adverse effects of the use of EPO.

Heterogeneity among trials was assessed using chi-square test (chi). The possibility of publication bias was assessed using the funnel plot method [17]. To assess the reasons for any heterogeneity found, we performed a meta-regression [4]. Sensitivity and subgroup analyses according to the main methodological quality dimensions [7] and a number of clinical criteria were also performed (see fig 2 and 3). The results were calculated with corresponding 95% confidence interval (95%CI). When the results from pooled data were significant, we calculated the number of patients need to treat (NNT) [18] in order to prevent one transfusion. However, the readers are advised to exercise due caution because NNT and NNH (number of patients to harm) are more influenced by the baseline risk in a control group than the odds ratio.

Figure 2

Sensitivity analysis using OR. Jadad – value according to Jadad's scale [39]; Adequate random – adequate method of randomization described; Ad all conc – adequate method of allocation concealment used; Double blind trials – double blind trials; Ad desc blind – adequate description of methods of blindness; Plan sample size – planning of the sample size; ITT – intention to treat; Malig – malignancies; CDDP – cisplatin; Disease related anaemia – anaemia due to the effects of the cancer; Trea related anaemia – anaemia due to effects of the treatment (chemotherapy and/or radiotherapy) Hb – hemoglobin level; G/dl – grams/ deciliter. N – number, * Carboplatin used

Figure 3

Analysis and sensitivity analysis using RR. (For legends, see fig 2)

The qualitative conclusion of the authors was assessed using the method described by Gilberts and Colditz [19,20] and adapted to a six point scale, where 1 = control group highly preferred, 2 = control preferred to EPO treatment, 3 = about equal, EPO a disappointment, 4 = about equal, EPO a success, 5 = EPO preferred to control and 6 = EPO highly preferred.

Finally, we performed a cumulative meta-analysis (CMA) [8] using a conservative approach, setting the α error at 1%, with a two tailored "p" in a random effects model [21]. The CMA allowed us to determine the point when the results achieved a level of statistical significance after which we should expect no changes in the effect by performing new trials, and after which placebo controlled trials should not have been performed. In other words: CMA allowed us to determine the threshold point after which uncertainty about this question should have been considered resolved [9].

Results

Thirty-seven randomized articles addressing the use of EPO in cancer related anemia were located and retrieved for full text appraisal. Twenty one papers fit our inclusion criteria. Two articles describing two small trials [22,23] with a total of 79 patients were excluded from our analysis because they did not provide information regarding our end point of interest.

Nineteen articles [2,3,12-15,24-36] describing 21 trials (see Additional file 1) fit our inclusion criteria. One paper [3] described three different trials, two of which [24,26] were reported later, in separate papers. We included data from these latest papers only. Therefore, we included 19 trials from 19 articles with 1896 patients in cancer related anemia. Fourteen trials with a total of 1511 patients described the use of EPO in patients with hb level < 11.5 g/dl and five trials with 369 patients related to hb level >= 11.5 g/dl. Of the latter ones, one trial [14] included patients with hb ranging from 10.6 to 12.0 g/dl. The analysis did not change when we excluded this trial in the sensitivity analysis.

The possibility of publication bias was considered unlikely according to the visual inspection of the funnel plot [17]. The results of the critical appraisal according to the most important methodological quality dimensions of each article are shown in (see Additional file 1). No single trial met all methodological quality criteria. All trials scored five or six on the six point Gilbert [19] and Colditz [20] scales, denoting strong qualitative support in favor of EPO ((see Additional file 1)).

The meta-analysis of the 19 trials (Fig 1 and 3) showed a significant beneficial effect of EPO in diminishing the number of patients requiring transfusion [OR = 0.41; 95%CI: 0.33 to 0.5 ; p < 0.00001], without statistical heterogeneity [chi = 23.46; df = 18; p > 0.1]. When we used relative risk (fig 3) as a summary point estimate in the meta-analysis, the results remained significant [RR = 0.61; 95% CI: 0.54 to 0.68; p < 0.00001], but now with significant statistical heterogeneity: [chi = 37.56; df = 18; p = 0.0044]. However, when the study by Porter et al [29] was removed from the analysis, heterogeneity disappeared from the analysis, and the analysis according to OR or RR did not differ. This is likely because in this small study including 10 patients in each group, event rate was so high with all but one patient receiving transfusion.

Figure 1

Meta-analysis of EPO use in cancer patients n – number of events; N – number of patients; OR – odds ratio; CI – confidence interval.

To further explain heterogeneity, we also performed a metaregression [4]. The variables related to the design and clinical features which possible could affect the treatment effect of EPO were entered in the equation (see Additional file 1) We found no statistically significant association between any of the variables with the treatment effect of EPO when all data were simultaneously entered in a multiregression equation(data not shown). However, when a stepwise regression model was used, two variables emerged of the potential significance which could explain most the heterogeneity of our results: (in)adequacy of allocation concealment (p = 0.026) and Hb cut-off value of 11.5 g/dl (p = 0.046). The between study variation (tau2 estimate) was 0.0372 for the model containing no variables (i.e. the simple meta-analysis); this was reduced to almost zero in the final model. In addition, most likely explanation of heterogeneity seen in our study is related to a small sample size of the individual trials resulting in random fluctuation of the effect between the studies. When we divided the trials according to the hb level, using 11.5 g/dl as a cut-off point, the results remained essentially unchanged (i.e. highly significant in favor of EPO) (fig 2 and 3).

Thus, it appears that the effect of EPO is a rather robust despite heterogeneity noted in the meta-analysis

Sensitivity and subgroup analyses according to clinical and methodological quality dimensions (Fig 2 and 3) showed consistency in the results, favoring the use of EPO. Statistical heterogeneity was noted for some of these comparisons (see fig 2 and 3). It was related to the magnitude of the treatment effect and not to its significance, since in all trials the point estimate of the effect favored EPO. A stepwise metaregression analysis did point to the possibility that treatment effect could have been exaggerated by inadequacy in allocation concealment and that larger treatment effects are seen at hb level < 11.5 g/dl. The heterogeneity was more common when we used RR than when we used OR.

The overall result by pooling all trials indicates that for each 5 cancer patients (95% CI: 4 to 7) using EPO, one patient will avoid transfusion. This result is maintained when we separately analyzed the trials that included patients with hb level < 11.5 g/dl (NNT = 5; 95% CI: 4 to 8) and Hb > 11.5 g/dl (NNT = 5; 95% CI: 3 to 9). These NNT should be analyzed carefully, since they were calculated from a pool of data that in some analysis were not homogeneous.

We also performed a CMA, (fig 4) using a conservative approach (i.e. setting α error at 1%). Our CMA demonstrated that by 1995 a high level of significance [OR = 0.52; 99%CI: 0.28 to 0.97; p = 0.0068]; [RR = 0.69; 99%CI: 0.53 to 0.90; p = 0.0003] about the benefit of EPO on reducing the use of blood transfusions had been achieved. Beyond this point, all new trials only improved the precision, without adding any additional useful information. We estimated that 1240 patients, representing 65% of the total number of cancer patients who participated in EPO studies, were enrolled in 13 placebo-controlled randomized studies that were reported after 1995.

Figure 4

Cumulative meta-analysis of EPO trials. N – number

The previous SR[5]concluded that it was impossible to reasonably distinguish between adverse events related to EPO and concurrent treatments. We also tried to extract data on adverse events, but were unable to make a reasonable approach due to the poor reporting. Therefore, we decided not to tabulate data on adverse events and perform quantitative pooling.

Discussion

According to our overall analysis, administration of EPO is related to an average 59% reduction in odds of requiring a transfusion. A previous SR [5,6] pooled 12 trials that used subcutaneous EPO (all also included in our analysis) in a Bayesian MA reached results similar [OR = 0.38; 95%CI: 0.28 to 0.51] to ours. We could also demonstrate consistency of the results when analyzed according to OR, RR, various quality dimensions and many different clinical settings (fig 2 and 3).

Due to these EPO effects, patients with cancer are expected to have direct benefits through fewer transfusions and lower exposure to anemia effects. Furthermore, all patients will benefit indirectly through conservation of the blood supply.

The use of RR as summary statistics is less robust than OR[4] and in this case resulted in greater heterogeneity than the analysis based on OR (see fig 2 and 3). The estimate the effects of EPO using RR indicated a smaller effect than OR but both results were significant (p < 0.0001 in both analysis). However, when events are common, the OR may overestimate the effect of treatment[4]. Due to these characteristics of the analysis, we choose to show both (OR and RR) as a summary statistics. Nevertheless, we should mention (see the RESULTS) that the most of heterogeneity could be explained by high transfusion rates in the trial by Porter et al. and inadequacy of allocation concealment in EPO trials. Similarly, it appears that the larger treatment effects are seen when EPO is given in the patients with hb > 11.5 g/dl.

Because it was not possible to extract data on quality of life (QOL), this issue was not addressed here. However, a recent systematic review by Bottomley et al [37] indicated that solid evidence about the effect of EPO on this end point is lacking. We should also mention that the high cost of EPO has deterred its widespread acceptance [5] and more studies using economic analysis should be done to address the issue if the use of EPO is cost-effective.

A previous SR[5,6] found no increased incidence of adverse events of EPO when compared with controls. We also did not find any side effect of EPO consistently reported among trials.

Our main goal here is to stress that the basic ethical requirement to randomize patients in clinical trials is that physicians and patients must not have preferences for any of the therapies considered i.e. they should be in the state of uncertainty or equipoise [9]. If uncertainty does not exist, it is felt to be unethical to ask patients to participate in RCTs [9].

To assess if uncertainty regarding a clinical question exists, researchers should evaluate the totality of pre-existing knowledge: if we already knew that one treatment is superior to another, nothing new could be learned and we would unnecessarily expose our patients to already known inferior treatment by performing a new trial. We showed here that if a SR/CMA had been used to formally address the uncertainty regarding the effect of EPO in cancer patients, this uncertainty could have been conservatively considered resolved (Fig 4) since 1995. Our findings suggest that 65% of the cancer patients included in this SR participated in placebo controlled clinical trials that were reported after EPO was shown to be superior to placebo. Had a CMA been performed prospectively, it is possible that clinical trialists may have decided against initiating 13 clinical trials. We should note that we analyzed trials according to date of their report. It is conceivable that some trials were initiated before 1995, but were reported after 1995. However, this only means that the uncertainty about the role of EPO could have been solved even earlier (if these patients were available for CMA or a large RCT). For example, if all 656 patients included in randomized studies up to 1995 had participated in only one, large, randomized study, the uncertainty about EPO could have been solved earlier. An alternative approach to large studies in solving uncertainty is to use CMA to monitor accumulation of treatment effects in clinical trials [38]. The technique of CMA is a powerful scientific instrument to assess the significance of accumulated knowledge [8,38]. In more than 100 CMA performed, there has never been a single case where the point estimate switched to indicate the opposite effect, once the difference between treatments reached statistical significance [8].

Conclusion

EPO is highly effective for cancer-related anemia. This effect could have been detected in 1995 if CMA had been performed at that time. The techniques of CMA should be used more frequently in the future to preserve "the uncertainty principle"[9] and protect patients who consent to be randomized in clinical trials.

Competing interests

No funding or honoraria were provided for this work to any of the authors. CB and BD served as consultants for the US manufacturers of erythropoietin (Amgen, Ortho) and also received their unrestricted support for organization of educational seminars unrelated to the use of erythropoietin.

Authors' contributions

OC conceptualized the study, defined the protocol, extracted and analyzed data, wrote the draft and approved the final version of the paper. JRD extracted data, contributed to the protocol writing, extracted and analyzed data, contributed to the drafts and approved the final version of the paper. CLB analyzed data, took an active participation on the writing of all versions of the draft and approved the final version of the paper. BD conceptualized the study, defined the protocol, extracted and analyzed data, wrote the draft and approved the final version of the paper.

Pre-publication history

The pre-publication history for this paper can be accessed here:

Additional file

Table 1: Characteristics of included studies.

Click here for file