Assessing the risk-benefit profile of ramucirumab in patients with advanced solid tumors: A meta-analysis of randomized controlled trials.

BACKGROUND: Ramucirumab is a widely used cancer drug having gained six regulatory approvals in various advanced solid tumors. Thus, assessing the risk-benefit profile of such a commonly used drug across multiple tumor types is necessary to inform clinical and reimbursement decisions. To objectively assess the risks and benefits of ramucirumab in patients with advanced solid tumors, we performed a systematic review and meta-analysis of published randomized controlled trials (RCTs).
METHODS: A systematic search of PubMed, the Cochrane Library, Google Scholar and conference abstracts for all RCTs of ramucirumab in patients with advanced solid cancer was conducted according to the PRISMA guidelines. Data on treatment-related serious adverse events (SAEs), fatal adverse events (FAEs), primary endpoint, gains in median overall survival (OS), progression-free survival (PFS) with their hazard ratios (HR) and 95% confidence intervals (CIs) and quality of life (QOL) were extracted from each RCT. Summary relative risks (RR) with 95% CI for SAEs and FAEs were calculated by pooling data across the RCTs using random-effects model. Treatment benefit was evaluated descriptively in terms of median and HR with 95% CI of OS and PFS gains as well as improvements in QOL. ESMO-Magnitude of Clinical Benefit Scale (MCBS), a validated tool, was used to objectively quantify clinical benefit of ramucirumab in approved settings.
FINDINGS: Ten RCTs met our inclusion criteria. Compared with the control arm, the use of ramucirumab was associated with a significantly increased risk of developing SAE (RR 1.13, 95% CI 1.05-1.21, incidence 37.5% v 33.5%). The increase in risk of FAE (RR 1.41, 95% CI 0.96-2.07, incidence 1.8% v 1.3%) was not statistically significant. Using the ESMO MCBS tool, clinical benefit with ramucirumab was not substantial in any indication, five of six approvals scoring a "negligible benefit" score of 1 or 2. QOL was either not reported (30%) or not improved (70%) in these RCTs.
INTERPRETATION: In this meta-analysis of RCTs, use of ramucirumab in patients with advanced cancer was associated with increased risk of treatment related serious and possibly fatal adverse events but the magnitude of clinical benefit from ramucirumab was mostly negligible with no trial reporting an improvement in QOL. These relative risks and benefits should be considered in clinical and regulatory decision making. FUNDING: None.

Evidence before this study

Ramucirumab is a commonly used cancer drug with six regulatory approvals for different indications. The clinical benefits and toxicities of the drug are reported for each trial individually before. However, formal risk-benefit analysis for this drug has not been conducted across its portfolio of trials. Toxicities of cancer drugs must be assessed by pooling data from trials across different tumor types because the harms may not be apparent in a single trial.

Added value of this study

In this meta-analysis, we find that the use of ramucirumab was associated with a significantly increased risk of serious adverse events while the clinical benefit from ramucirumab was negligible to moderate based on ESMO-Magnitude of Clinical Benefit Scale. The risk of fatal adverse events was not significantly higher with ramucirumab. No trial reported an improvement in quality of life with ramucirumab.

Implications of all the available evidence

This risk-benefit profile of ramucirumab suggests that it is a low-value cancer drug. This information must be carefully considered during clinical, regulatory and reimbursement decisions.

Alt-text: Unlabelled box

Introduction

Ramucirumab (Cyramza) is a monoclonal antibody against the VEGF2 receptor that exerts antitumor effect as an angiogenesis inhibitor [1]. It was first approved by the US FDA in 2014 for the treatment of metastatic gastric cancer as a monotherapy [2]. It has been subsequently approved by the FDA for the treatment of multiple solid tumors in the advanced setting, including gastric cancer in combination with paclitaxel, non-small cell lung cancer (NSCLC) in combination with docetaxel, colorectal cancer in combination with 5-fluorouracil plus leucovorin plus irinotecan (FOLFIRI), hepatocellular cancer as monotherapy and most recently in epidermal growth factor receptor (EGFR) positive NSCLC in combination with erlotinib [3].

Risk-benefit assessments are important for shared decision making regarding any cancer drug treatment. Since the goal of treatment is not cure but palliation, proper understanding of the treatment benefits and risks are especially important in the metastatic setting [4] where ramucirumab has received all the approvals. Although the common side effects of an anticancer drug are frequently explained to patients in clinical practice, the incidence and risks of serious and fatal adverse events, that have important consequences to the patients and family, are frequently overlooked during these discussions. Physicians are also known to overestimate benefits and underestimate harms of treatment [5]. Proper assessment of risks and benefits are important for informed decision making for all stakeholders-clinicians and patients to make treatment decisions, as well as regulators and policymakers for making approval and reimbursement decisions. This is all the more important for a drug with substantial financial toxicity – the cost of ramucirumab exceeds US $14,000 a month [6].

However, no study to our knowledge has systematically studied and analyzed all the phase 3 RCTs of ramucirumab to objectively assess the risk-benefit profile. For cancer drugs, the clinical benefits can be assessed using gains in survival times or quality of life or using a validated tool such as the Magnitude of Clinical Benefit Scale and the risks can be measured by an assessment of the incidence and relative risks of serious and fatal adverse events in randomized controlled trials. Accordingly, we conducted a systematic review and meta-analysis of all phase 3 RCTs of ramucirumab to objectively quantify the risks (relative risk of serious and fatal adverse events) and benefits (magnitude of clinical benefit) of ramucirumab in patients with solid tumors.

Methods

This systematic review and meta-analysis was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) reporting guideline [7].

Study selection

We conducted a systematic search of PubMed, the Cochrane Library, and Google Scholar for all RCTs of ramucirumab with no date restrictions. The search strategy is provided in the Supplementary appendix. We excluded studies that were done in animals only, non-randomized studies, and non-trial studies such as case reports, editorials, viewpoints and commentaries. The search was first conducted on September, 2019 and updated for new studies on January 2020. Relevant conference abstracts were also searched for updated data. After title and abstract screening by the two authors, the full texts of potentially relevant studies were downloaded and reviewed for the following additional exclusion criteria: (1) not an RCT such as phase 2 single arm trial, (2) RCTs with ramucirumab in both the arms, (3) not reporting data on efficacy or safety, (4) not reporting original data, (5) not efficacy outcome studies such as prognostic or predictive biomarker studies and (6) subgroup analyses. Trials testing ramucirumab both as monotherapy or in combination with other agents were included.

Data extraction

This study was not submitted for institutional review board approval because it did not involve individual patient information and all data extractions were made from publicly available published articles. Data were independently extracted from published reports by the two authors and any discrepancy was resolved by mutual consensus. We collected key trial characteristics: study name, year of publication, tumor type, treatment setting, primary end point, sample size, and details of the treatment and control regimens including the dose of ramucirumab and performance status of included patients. We also searched the FDA database to segregate trials by approval status (yes or no) based on whether the trial led to FDA approval for ramucirumab. For safety outcomes, we extracted the safety population and the number of serious adverse events (SAEs) and fatal adverse events (FAEs). Whenever any of these information were unavailable in the primary publication, we extracted data from clinicaltrials.gov for the respective trials. For efficacy outcomes, we extracted the median progression-free survival (PFS) and median overall survival (OS) in both the arms, as well as hazard ratios (HRs) with confidence intervals (CI) for PFS and OS. We also extracted quality of life (QOL) outcomes wherever available.

Endpoints and definitions

The primary endpoint of the study was to evaluate the risks and benefits of treatment with ramucirumab for patients with solid tumors. Risks would be assessed in terms of pooled incidence and relative risk (RR) of treatment-related SAEs and FAEs for patients receiving ramucirumab compared to the patients in the control arm. FAEs are defined as death caused in all likelihood by the drug and SAEs are defined as adverse events leading to death, life-threatening condition, hospitalization or prolongation of hospitalization, disability or permanent damage, congenital anomaly or birth defect, requiring intervention to prevent permanent impairment or damage, or any other adverse events that may jeopardize the patient and may require medical or surgical intervention (treatment) to prevent one of the other outcomes [8]. The attribution of SAEs or FAEs as treatment-related or disease-related depended on the decision of the primary investigators in the original trials. The data on ‘treatment-related SAEs’ and ‘treatment-related FAEs’ were extracted from the publications (or clinicaltrials.gov when data not available in publications) for the purpose of this meta-analysis.

Treatment benefits would be assessed in terms of gains in QOL as well as medians and hazard ratios for OS and PFS with 95% confidence intervals (CIs). We also used the ESMO-Magnitude of Clinical Benefit Scale (MCBS) to objectively quantify the clinical benefit from ramucirumab. The ESMO-MCBS is a standardized, generic, validated tool to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies at the time of approval [9]. The MCBS grades range from 5 to 1 in the non-curative setting where all the approvals of ramucirumab have been. An MCBS score of 5 and 4 indicate substantial benefit, 3 indicates moderate benefit, and 2 and 1 indicate negligible benefit. These scores were calculated only for RCTs that led to drug approval because calculating “clinical benefit” score would be meaningless for negative RCTs that did not provide evidence of clinical benefit. More information on ESMO-MCBS scores and the associated methodologies is available at https://www.esmo.org/guidelines/esmo-mcbs.

Statistical analysis

For the risk assessment, data on the number of patients with SAEs, the number of patients with FAEs and the number of patients receiving ramucirumab were extracted from the publications of the selected clinical trials and the proportion of patients with SAEs and FAEs was calculated. For the calculation of RR, patients assigned to ramucirumab (+ chemotherapy) cohort were compared with those assigned to placebo/best supportive care (+ chemotherapy) cohort in the same trial. When studies reported zero events in a treatment or control arm, we used a classic half-integer continuity correction for the calculation of relative risk [10].

The pooled estimate for RRs of SAEs and FAEs were calculated using random-effects model to account for the obvious clinical heterogeneity (different tumor types, different lines and different types of treatment) of included studies. Statistically, assumption of homogeneity was considered to be invalid for values of P<0.10 for the Cochrane Q statistic and the inconsistency was quantified with the I2 statistic. Publication bias was assessed visually using funnel plots.

The risks of adverse events from a cancer drug can vary in different tumor types and can increase with increasing dose. Furthermore, the risks can also differ when the drug is used in combination with other agents versus when used alone. Finally, the safety of a drug maybe better in approved indications compared to the unsuccessful trials that did not lead to drug approvals. Hence, we decided to run subgroup analyses to study any differential effect on risk of adverse events based on any of these parameters. These subgroup analyses were pre-specified. Specifically, we conducted subgroup analyses by tumor type, ramucirumab used as monotherapy versus in combination with other chemotherapy, dose of ramucirumab and trials that led to approval of ramucirumab for that indication versus those that did not. However, in absence of statistical evidence to assume heterogeneity, the subgroup analyses would be considered only hypotheses generating.

We decided not to pool the hazard ratio to derive a pooled estimate of hazard ratio for OS or PFS for assessing treatment benefit because such analysis while technically possible would not be scientifically meaningful. A drug may have different efficacies in various tumor types, so it would be inappropriate to derive a single hazard ratio applicable for all tumor types. However, for safety analysis, pooling data across multiple tumor types remains meaningful as has been done previously for other drugs such as bevacizumab [11] or sorafenib [12]. The reasons for why pooling across trials for different settings is appropriate for safety analysis but not appropriate for efficacy analysis have been discussed elsewhere [13]. So, in this study, benefit assessment would be done only descriptively using the medians, HRs and 95% CIs and gains in QOL. These data were also used to calculate the ESMO-MCBS scores. Scales Evaluation forms v 1.1 were used with form 2a used for calculating the scores where OS was the primary endpoint and form 2b used when PFS was the primary endpoint. The method for calculation of these scores are explained elsewhere [14].

All statistical analyses were performed using Stata version 15 (StataCorp).

Role of funding

No funding was obtained for this study.

Results

Study characteristics

Our search revealed a total of 203 potential studies of which 10 studies were finally included in the meta-analysis based on inclusion and exclusion criteria (Fig. 1) [15-24]. All these studies were published between 2014 and 2019 (Table 1). Six trials (60%) had led to ramucirumab's approval [16,17,19,21-23] and four trials did not [15,18,20,24] All the included trials were funded by the industry.

Fig. 1

PRISMA diagram.

Table 1

Characteristics of RCTs included in the systematic review and meta-analysis.

Study name	Authors	Year of publication	Type of cancer	Experimental arm	control arm	Dose of ramucirumab
REGARD	Fuchs et al.	2014	Gastric or junctional 2nd line	Ramucirumab	Placebo	8 mg/kg
RAINFALL	Fuchs et al.	2019	Gastric or junctional adenocarcinoma 1st line	Ramucirumab plus fluoropyrimidine and cisplatin	Placebo plus fluoropyrimidine and cisplatin	8 mg/kg
REVEL	Garon et al.	2014	Non-small-cell lung cancer	Ramucirumab plus docetaxel	Placebo plus docetaxel	10 mg/kg
ROSE/TRIO-012	Mackey et al.	2015	Breast	Ramucirumab plus docetaxel	Placebo plus docetaxel	10 mg/kg
RANGE	Petrylak et al.	2017	Urothelial carcinoma	Ramucirumab plus docetaxel	Placebo plus docetaxel	10 mg/kg
RAISE	Tabernero et al.	2015	Colorectal carcinoma	Ramucirumab plus FOLFIRI	Placebo plus FOLFIRI	8 mg/kg
RAINBOW	Wilke et al.	2014	Gastric or junctional 2nd line	Ramucirumab plus paclitaxel	Placebo plus paclitaxel	8 mg/kg
REACH	Zhu et al.	2015	Hepatocellular carcinoma	Ramucirumab	Placebo	8 mg/kg
REACH-2	Zhu et al.	2019	Hepatocellular carcinoma	Ramucirumab	Placebo	8 mg/kg
RELAY	Nakawaga et al.	2019	Non-small-cell lung cancer	Ramucirumab plus erlotinib	Placebo plus erlotinib	10 mg/kg

FOLFIRI: 5-fluorouracil plus leucovorin plus irinotecan.

Three trials were conducted in gastric or junctional adenocarcinoma, two in the second line and one in the first line setting, two trials in hepatocellular carcinoma (HCC), two in NSCLC, one each in colorectal carcinoma (CRC), breast and urothelial carcinoma (Table 1). Three trials tested ramucirumab as monotherapy against placebo, the rest tested ramucirumab in combination with chemotherapy or erlotinib, with an active control arm of the same regimen minus ramucirumab. All the RCTs were double blind, placebo-controlled. Ramucirumab was used at a dose of 8 mg/kg in 6 RCTs and 10 mg/kg in 4 RCTs.

Assessment of harms

The safety population constituted of 6905 patients (3754 randomized to the ramucirumab arm, 3151 randomized to the control arm). All RCTs included patients with a performance status of 0 or 1 only. Safety data are summarized in Table 2.

Table 2

Safety data.

Study name	Safety N ramucirumab	Safety N control	FAEs ramucirumab	FAEs control	SAEs ramucirumab	SAEs control
REGARD	236	115	5	2	112	51
RAINFALL	323	315	7	7	160	149
REVEL	627	618	15	9	269	262
ROSE/TRIO-012	759	385	2	0	282	114
RANGE	258	265	8	5	66	57
RAISE	529	528	13	10	189	164
RAINBOW	327	329	6	5	153	139
REACH	277	276	7	4	122	89
REACH-2	197	95	3	0	21	5
RELAY	221	225	1	0	34	26

N= number of patients, FAEs= Fatal Adverse Events, SAEs= Serious Adverse Events.

Serious adverse events

SAEs were greater in the ramucirumab arm compared with the control arm in all 10 RCTs. The cumulative incidence of SAEs in patients who received ramucirumab was 37.5% (1408/3754) versus 33.5% (1056/3151) in patients who were in the control arm.

The pooled relative risk for developing SAE with ramucirumab was 1.13 (95% CI1.05–1.21) (Fig. 2). Although subgroup analyses showed some differences based on histology, dose and whether the drug was used in combination with chemo or as monotherapy (Supplementary Table), overall there was no evidence of heterogeneity across the studies (I2 = 11.9%, p = 0.33) and hence the subgroup analyses should only be considered as hypothesis generating observations.

Fig. 2

Forest plot of the relative risk (RR) of serious adverse events (SAEs) associated with ramucirumab versus control.

Fatal adverse events

FAEs were greater in the ramucirumab arm compared with the control arm in nine RCTs and equal to the control arm in one RCT. The cumulative incidence of FAEs in patients who received ramucirumab was 1.8% (67/3754) versus 1.3% (42/3151) in patients who were in the control arm. The pooled relative risk for developing FAE with ramucirumab across the RCTs was 1.41 (95% CI 0.96–2.07) (Fig. 3). There was no evidence to assume heterogeneity across the studies (I2 = 0.0%, p = 0.99). Subgroup analyses based on approved use versus non-approved use, tumor histology and dose and combination versus monotherapy did not reveal any meaningful differences (Supplementary Table).

Fig. 3

Forest plot of the relative risk (RR) of fatal adverse events (FAEs) associated with ramucirumab versus control.

The possibility of publication bias in the meta-analysis of SAE and FAE cannot be ruled out as depicted in the funnel plots (Supplementary Figure).

Assessment of treatment benefit

OS was the primary endpoint in six RCTs and investigator-assessed PFS was the primary endpoint in four (Table 3). Of all the 10 RCTs, only one failed to show a statistically significant improvement in PFS [18]. The gains in median PFS ranged from 0.3 months to 1.5 months, with the exception of RELAY trial which showed a 7 month improvement in PFS.

Table 3

Efficacy data.

Study name	Approval trial?	Primary endpoint	Median OS gains in months	OS HR (95% CI)	Median PFS gains in months	PFS HR (95% CI)	QOL outcomes reported?	QOL outcomes improved?	ESMO-MCBS scorea
REGARD	Yes	OS	1.4 m (5.2 v 3.8)	0.78(0.60–0.99)	0.8 m (2.1 v 1.3)	0.48(0.38–0.62)	Yes	No	1
RAINFALL	No	PFS	0.5 m (11.2 v 10.7)	0.96(0.80–1.16)	0.3 m (5.7 v 5.4)	0.75 (0.61–0.94)	Yes	No	NA
REVEL	Yes	OS	1.4 m (10.5 v 9.1)	0.86(0.75–0.98)	1.5 m (4.5 v 3.0)	0.76(0.68–0.86)	Yes	No	1
ROSE/TRIO-012	No	PFS	0.1 m (27.3 v 27.2)	1.01(0.83–1.23)	1.3 m (9.5 v 8.2)	0.88(0.75–1.01)	Yes	No	NA
RANGE	No	PFS	1.5 m (9.4 v 7.9)	0.89(0.72–1.09)	1.3 m (4.1 v 2.8)	0.76(0.61–0.94)	Yes	No	NA
RAISE	Yes	OS	1.6 m (13.3 v 11.7)	0.84(0.73–0.97)	1.2 m (5.7 v 4.5)	0.79(0.69–0.90)	Yes	No	1
RAINBOW	Yes	OS	2.2 m (9.6 v 7.4)	0.81(0.68–0.96)	1.5 m (4.4 v 2.9)	0.64(0.54–0.75)	Yes	No	2
REACH	No	OS	1.6 m (9.2 v 7.6)	0.87(0.72–1.05)	0.7 m (2.8 v 2.1)	0.63(0.52–0.75)	No	NA	NA
REACH-2	Yes	OS	1.2 m (8.5 v 7.3)	0.71(0.53–0.95)	1.2 m (2.8 v 1.6)	0.45(0.34–0.60)	No	NA	1
RELAY	Yes	PFS	NA (Not reached in both arms)	0.83(0.53–1.30)	7 m (19.4 v 12.4)	0.59(0.46–0.76)	No	NA	3

OS: Overall survival, PFS: Progression-free survival, HR: Hazard Ratio, CI: confidence interval, ESMO-MCBS: European Society for Medical Oncology-Magnitude of Clinical Benefit Scale, NA: Not applicable; QOL: Quality of Life. Median OS gains = median OS in ramucirumab arm minus median OS in the control arm; median PFS gains = median PFS in ramucirumab arm minus median PFS in the control arm. m = months.

ESMO MCBS Scores are calculated only for approved indications to facilitate evaluation of magnitude of clinical benefit for the given drug for the given indication.

OS was significantly improved only in 5 RCTs (50%). All four RCTs with PFS as the primary endpoint did not show significant benefit in OS. The gains in median OS ranged from 1.2 to 2.2 months (Table 3).

Approval trials only

When the analysis was limited to the trials that led to FDA approvals, the gains in median OS for these trials ranged from 1.2 to 2.2 months and gains in median PFS ranged from 0.8 to 7 months. The HR for OS ranged from 0.71 to 0.86 and that for PFS ranged from 0.45 to 0.79.

The RELAY trial was the only RCT with PFS as the primary endpoint that led to regulatory approval and had an ESMO MCBS score of 3 signifying moderate benefit [19]. For the RAINBOW trial with OS as the primary endpoint, the ESMO MCBS score was calculated to be 2 signifying negligible clinical benefit [22]. For the other 4 RCTs with OS as the primary endpoint and leading to regulatory approval (REGARD [16], REVEL [17], RAISE [21] and REACH-223), the ESMO MCBS scores were 1 signifying the most negligible clinical benefit (1 being the least score possible) (Table 3).

Gains in quality of life

QOL outcomes were reported for seven RCTs (70%) and unavailable for three. Of those with reported QOL outcomes, none of the trials showed an improvement in QOL with ramucirumab.

Discussion

In this systematic review and meta-analysis of treatment harms and benefits with ramucirumab in patients with advanced solid tumors, we found that ramucirumab significantly increased the risks of serious adverse events while the treatment benefits were marginal to moderate at best. The incidence of treatment related mortality with ramucirumab was at 1.8%, this risk was not significantly higher than that with control. Overall, this suggests the risk and benefit profile of ramucirumab may be tilting more towards risks than benefit which raises important clinical and policy questions as to the clinical utility of such an expensive drug.

With anticancer drugs in the metastatic setting, the risk benefit judgment is the most crucial aspect of shared treatment decision making. While drug approval is the first line of defense to ensure that the therapy that reach patients have more benefits than harms, it behooves the physicians to engage with patients in a shared decision making to decide whether the treatment benefits from a particular therapy are worth the adverse events. In that context, objective assessment of benefits and harms are important. Our study helps to fill that need for an important cancer drug that has received six regulatory approvals and has substantial costs.

Our study shows that treatment with ramucirumab was associated with a statistically significant 13% increase in the risk of serious adverse events. There was also a 41% increase in the risk of fatal adverse events with ramucirumab, the lack of statistical significance in this risk may not be reassuring as the wide confidence interval is probably reflective of the small number of events. For example, while the incidence of SAEs was 37.5%, that for FAEs was only at 1.8%, although whether these incidence rates are acceptable would depend on individual patient's threshold of risk acceptance.

On the efficacy side, when considering all the approved uses of ramucirumab, the gain in median overall survival ranged from only 1.2 to 2.2 months with an HR ranging from 0.71 to 0.86. These are marginal gains in survival, that need to be judged against the increased risk of toxicities. Using the ESMO-MCBS score, none of the approved uses had a score of 4 or 5 signifying substantial benefit and five of six approved uses had a score of only 1 or 2 signifying negligible benefit. Furthermore, ramucirumab failed to demonstrated any evidence of improving patients’ quality of life in any setting.

Another variable in judging the value of a cancer drug, besides toxicities and clinical benefit, is the cost. In the US, ramucirumab costs upwards of $14,000 a month [6]. For 2018, the worldwide revenue from ramucirumab was $821.4 million and for 2019, it was $925.1million, an increase of 13% [25]. The U.S revenue was $335.3 million for 2019 [25]. Whether such investment is warranted for the given risk-benefit profile is up to the individual regulatory and reimbursement agencies to decide, but analyses such as ours will be helpful for countries in priority setting in making such decisions. Cost-effectiveness analyses of ramucirumab also consistently reveal that ramucirumab is not cost-effective in various tumor types [26,27], including in the only biomarker-based approval of ramucirumab in hepatocellular cancer [28]. Indeed, ramucirumab has also been previously proposed as one of the low-value therapeutics in oncology with regards to its indication for gastric or gastroesophageal junction tumors [29]. Objective assessment of risks and benefits of expensive cancer drugs will help to identify low-value drugs and make priority decisions for the health care systems.

Although drug approvals based on surrogate endpoints are criticized for lacking evidence in overall survival [30], our study shows that drugs that are approved on the basis of statistical benefit in overall survival may also not necessarily have a better risk-benefit profile. Indeed, five of the six approvals for ramucirumab are based on overall survival benefit, but they are all marginal benefits with MCBS scores of 1 or 2.

There are several limitations to our analysis. First, our meta-analysis of harms provides trial level estimates which may not be generalizable to an individual patient in clinic. Further, our risk estimates are summary estimates from good quality well-conducted phase 3 RCTs that included patients with performance status of only 0 or 1. Thus, the real risk in a real-life patient with comorbidities and poor performance status could be higher. Accordingly, the clinical benefit seen in RCTs may not translate to patients in the real-world. In addition, the attribution of SAEs and FAEs as treatment-related or disease-related by the study investigator could bias the results [31]. Furthermore, although all the studies were double-blind, experienced physicians might easily recognize the adverse effects of ramucirumab versus control leading to bias. The lack of transparency of harms information in the publication is another issue; we had to extract this information from clinicaltrials.gov website for three RCTs in our sample. Such selective reporting and publication bias are inherent limitations of such meta-analysis.

In conclusion, our study shows that the risk-benefit balance of treatment with ramucirumab may be tilted more towards risks than benefits due to minimal clinical benefit but substantially increased risk of serious adverse events. These information will be helpful in clinical and regulatory decision making.