Availability of evidence of benefits on overall survival and quality of life of cancer drugs approved by European Medicines Agency: retrospective cohort study of drug approvals 2009-13.

Objective To determine the availability of data on overall survival and quality of life benefits of cancer drugs approved in Europe.Design Retrospective cohort study.Setting Publicly accessible regulatory and scientific reports on cancer approvals by the European Medicines Agency (EMA) from 2009 to 2013.Main outcome measures Pivotal and postmarketing trials of cancer drugs according to their design features (randomisation, crossover, blinding), comparators, and endpoints. Availability and magnitude of benefit on overall survival or quality of life determined at time of approval and after market entry. Validated European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS) used to assess the clinical value of the reported gains in published studies of cancer drugs.Results From 2009 to 2013, the EMA approved the use of 48 cancer drugs for 68 indications. Of these, eight indications (12%) were approved on the basis of a single arm study. At the time of market approval, there was significant prolongation of survival in 24 of the 68 (35%). The magnitude of the benefit on overall survival ranged from 1.0 to 5.8 months (median 2.7 months). At the time of market approval, there was an improvement in quality of life in seven of 68 indications (10%). Out of 44 indications for which there was no evidence of a survival gain at the time of market authorisation, in the subsequent postmarketing period there was evidence for extension of life in three (7%) and reported benefit on quality of life in five (11%). Of the 68 cancer indications with EMA approval, and with a median of 5.4 years' follow-up (minimum 3.3 years, maximum 8.1 years), only 35 (51%) had shown a significant improvement in survival or quality of life, while 33 (49%) remained uncertain. Of 23 indications associated with a survival benefit that could be scored with the ESMO-MCBS tool, the benefit was judged to be clinically meaningful in less than half (11/23, 48%).Conclusions This systematic evaluation of oncology approvals by the EMA in 2009-13 shows that most drugs entered the market without evidence of benefit on survival or quality of life. At a minimum of 3.3 years after market entry, there was still no conclusive evidence that these drugs either extended or improved life for most cancer indications. When there were survival gains over existing treatment options or placebo, they were often marginal. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

Before new prescription medicines are allowed onto the market, they must be tested in studies and show, to the satisfaction of drug regulatory agencies, that their benefits outweigh the harms of drug toxicity. The most informative and valuable studies are those providing robust evidence from well designed randomised controlled trials that a new drug has a significant effect on outcomes that are important to patients and that the magnitude of those effects, compared with other treatment options, are clinically meaningful. Although the goal of cancer treatment is to improve the quantity and quality of life,1 2 3 clinical trials designed to gain regulatory approval for new drugs often evaluate indirect or “surrogate” measures of drug efficacy. These endpoints show that an agent has biological activity, but they are not reliable surrogates for improved survival4 5 6 7 8 9 10 11 or quality of life4 6 11 12 13 in all settings, and two recent systematic reviews suggest that the strength of association between surrogates in cancer clinical trials and life extension is generally low.8 14 Moreover, there is growing concern that the benefits offered by many new treatments for cancer—often discussed and promoted as “breakthroughs”15 16 17 18—are marginal and might not be clinically meaningful to patients, despite rapidly escalating costs.19 20 21 22 23 Consequently, there have been calls to raise the evidence bar for market authorisation of new cancer drugs.15 21 23 24 25 26

No recent studies have systematically examined the evidence base and magnitude of benefit for cancer drugs approved by the European Medicines Agency (EMA).27 28 29 Available data from the US show that only a small proportion of cancer treatments approved by the US Food and Drug Administration (FDA) unequivocally show benefits on survival or quality of life.30 The applicability of this evidence to the European context, however, is not clear. Recent FDA-EMA comparisons show differences in regulatory decisions, route of approval, and availability of cancer drugs that could have important implications for clinical practice and patient safety.31 32 33 34 35 In particular, regulatory provisions for expediting drug development and approval differ between the US and the EU,32 with EU regulation being more restrictive in scope.36 This could lead to divergent outcomes between the two regions.

The proportion of cancer drugs approved by the EMA without any demonstrable benefits on survival or quality of life is not known. In addition, whether treatments approved without evidence of benefit on survival are subsequently shown to improve survival or quality of life in postmarketing studies has not been characterised.

We sought to systematically evaluate the evidence base for all new drugs and new indications for the treatment of solid tumours and haematological malignancies approved by the EMA in the five year period 2009-13. We determined the proportion of drugs with demonstrable benefit on survival or quality of life over available treatment options or placebo, both at time of approval and in the postmarketing period. In addition, we used a validated scale to evaluate the magnitude of benefit of drugs in our sample showing a significant improvement on survival in the treatment of solid cancers.

Methods

Cohort of cancer drugs approved by EMA 2009-13

We searched the publicly available EMA database of European Public Assessment Reports using L01-04 ATC codes to identify “antineoplastic and immunomodulating” agents for solid tumours and haematological malignancies authorised from 1 January 2009 to 31 December 2013. Consistent with previous research,30 our study period ended in 2013, which allowed adequate time for the completion of randomised trials in the postmarketing period. We excluded paediatric indications, indications for the treatment of benign tumours, supportive treatments, and generic products (fig A in appendix). Authorised indications for each drug were identified by searching the EPAR document “Procedural steps taken and scientific information after authorisation.”

We distinguished between different types of regulatory approval. A “first marketing authorisation” indicates that the drug is a new active substance, approved onto the EU market for the first time. An “extension” to a marketing authorisation is when a marketed drug is approved for use in a new patient population, new combination, new line of treatment, or new type of cancer. By law, a “regular marketing authorisation” should be based on comprehensive evidence of quality, safety, and efficacy. “Conditional marketing authorisations” can be granted for drugs intended to treat patients with seriously debilitating or life threatening diseases and are expected to fulfil an unmet medical need. Conditional approvals are granted on the basis of less comprehensive data than required for a regular marketing authorisation, and companies are required to generate additional evidence in the postmarketing period.36 “Orphan drug” designation is granted for the treatment of rare cancers. We also categorised indications according to cancer site and stage of disease (specifically whether treatments were to be used in a curative or non-curative setting).

Data sources and search strategies

To determine the availability of evidence on overall survival and quality of life gains at time of market approval for the drugs in our cohort, we manually searched the EMA website to identify the relevant European Public Assessment Reports (EPARs). EPARs are official regulatory documents in the public domain that summarise the evidence base for a new drug or new indication and provide a justification for the EMA’s decision to grant a marketing authorisation or extension to the marketing authorisation. Clinical trial data supporting approval of each of the unique indications were retrieved from the EPARs. We included only data from those studies identified by the EMA as the main, or “pivotal,” studies supporting approval.

To determine whether drugs that did not show any survival gain at the time of approval were subsequently confirmed to improve survival or quality of life in the postmarketing period, we searched PubMed for randomised controlled trials reporting these outcomes for the relevant indications. Our search strategy included the drug name, approved indication, and search terms for the endpoints of interest (overall survival or quality of life or QoL or HRQoL) (fig B in appendix). We identified randomised controlled trials using the Cochrane Collaboration sensitivity and precision maximising search strategy for identifying randomised trials.37 When possible, we cross checked search results against the European Clinical Trials Database (EudraCT) and the US National Library of Medicine database of clinical trials (ClinicalTrials.gov). Our latest search was on 31 March 2017, which allowed for at least 3.3 years for the completion and publication of clinical studies in the postmarketing period (median 5.4 years, maximum 8.1 years).

For those drugs associated with an overall survival gain at the time of approval or in the postmarketing period, we used the European Society for Medical Oncology’s (ESMO) Magnitude of Clinical Benefit Scale (ESMO-MCBS) to assess the clinical value of these gains as reported in published studies.38 The ESMO-MCBS scale is a tool for evaluating the clinical benefit of new treatments for solid tumours to facilitate the presentation of clear and unbiased statements regarding the relative clinical benefit of new treatments.

Data extraction, outcome variables, and data analysis

Three investigators (AP, EP, and EG) independently extracted data on and descriptively analysed the following trial features: characteristics of the participant population, study design (randomisation, crossover from experimental to control group, and blinding of investigators and participants), experimental and control groups, enrolment, primary and secondary endpoints, magnitude of benefit on survival and quality of life, and narrative interpretation of the findings. We classified comparators as active (in trials comparing drugs A versus B), add on (in trials comparing drugs A+B versus B alone), placebo, or none. Investigators cross checked each other’s data extraction. Difficulties or disagreements in interpretation were resolved by discussion and consensus among the three lead investigators (CD, HN, and AA). All data were also checked by the principal investigator (CD).

We judged that a drug had shown a survival benefit in its approved indication if, as reported in the EPAR or study publication, overall survival was the primary or secondary endpoint in a randomised controlled trial, and the difference in survival between the experimental and control arms was significant according to a prespecified statistical analysis. If an analysis was described as “exploratory” we considered it to be a post hoc analysis. Post hoc analyses (in which the hypotheses being tested were not specified before examination of the data) were accepted if the EMA specifically determined that these analyses provided evidence of a survival benefit. If a published study reporting results of a pivotal trial conflicted with the EMA’s assessment or other information contained in an EPAR, we made a determination consistent with the EPAR, unless the published study reported an updated analysis.

We considered that a drug had shown a benefit on quality of life in its approved indication when a significant difference was reported for an item or subscale derived from a validated quality of life instrument, even in the absence of a significant benefit with respect to the global score.

Determination of ESMO-MCBS score

Our scoring was limited to drugs for solid tumours as the ESMO scales are not intended for evaluation of drugs to treat haematological malignancies.38 Two investigators (AA and CD) independently graded individual trials based on published papers that reported a significant difference in overall survival (as a prespecified primary or secondary study endpoint) using the most recent version of the ESMO-MCBS evaluation forms.39 There was complete agreement between the two investigators in the scoring. As per the established ESMO methods, each score was assigned based on the magnitude of the absolute survival gain and the lower limit of the 95% confidence interval of the hazard ratio, and only significant changes in toxicity or quality of life parameters were used to modify the grades. Only scores of A or B (for treatments of curative intent), or 5 or 4 (for treatments used in the non-curative/palliative setting) are defined as clinically meaningful according to the ESMO framework.

Patient involvement

No patients were involved in setting the research question or the outcome measures, nor were they involved in developing plans for design or implementation of the study. No patients were asked to advise on interpretation or writing up of results. There are no plans to disseminate the results of the research to study participants or the relevant patient community.

Results

From 2009 to 2013, the EMA approved use of 48 oncology drugs in 68 indications. Of the authorised indications for use, 33 were first marketing authorisations and 35 were extensions (see methods for definitions) Table 1 and table A in the appendix show the approved indication and the date and type of market authorisation for each agent. Some 58 uses were granted a regular marketing authorisation, of which seven were designated as orphan drugs. Ten uses were granted conditional marketing authorisation (including four designated as orphan drugs). In total, 11/68 (16%) uses received orphan designation. Approval for one (bevacizumab in combination with docetaxel for the treatment of metastatic breast cancer) was subsequently withdrawn.40

Table 1

Characteristics of approvals of cancer drug, 2009-13. Figures are numbers (percentage) of indications

Characteristics	Solid tumours (n=51)	Haematological tumours (n=17)
Type of marketing authorisation:
First marketing authorisation	24 (47)	9 (53)
Extension	27 (53)	8 (47)
Pathway of first marketing authorisation:
Regular approval	19 (79)	4 (44)
Conditional approval	5 (21)	5 (56)
Orphan designation	3 (6)	8 (47)
Intent of treatment:
Curative	6 (12)	1 (6)
Non-curative	45 (88)	16 (94)

Seventeen drugs were approved for treatment of haematological malignancies and 51 for treatment of solid tumours. Of these 51, 12 (24%) were for breast cancer, seven (14%) for lung cancer, five for bowel cancer, and five (10%) for prostate cancer. Of the 17 drugs approved for treatment of haematological malignancies, four (24%) were for chronic myeloid leukaemia, three (18%) for multiple myeloma, and three (18%) for chronic lymphocytic leukaemia. Most drugs (61/68, 90%) were approved for use in a non-curative setting (table 1).

Characteristics of pivotal studies

Overall, 72 clinical trials supported the approval of 68 novel drug uses. Only 18 of the 68 (26%) were supported by a pivotal study powered to evaluate overall survival as the primary outcome (table 2 and table B in appendix). Survival was evaluated as the primary study endpoint in none of the 10 conditionally authorised uses and in only one of the 11 (9%) with an orphan designation. Overall survival was evaluated as a primary study endpoint in less than a third (18/58, 31%) of all drug indications granted regular marketing authorisation.

Table 2

Characteristics of pivotal trials of cancer drugs. Figures are numbers (percentage) unless stated otherwise

Characteristics	Solid tumours (n=54)	Haematological tumours (n=18)
Pivotal trial design:
Randomised trial	52 (96)	13 (72)
Single arm trial	2 (4)	5 (28)
Comparator:
Active	15 (28)	9 (50)
Placebo	25 (46)	0 (0)
Add on	12 (22)	4 (22)
None	2 (4)	5 (28)
OS as primary endpoint	18 (33)	1 (6)
OS or QoL as secondary endpoint	47 (87)	17 (94)
Median (range) sample size	696 (96-3222)	484 (58-1018)

OS=overall survival; QoL=quality of life.

The remaining drug indications were supported by trials that evaluated a surrogate measure as the primary study outcome (table 2 and table B in appendix). This included progression-free survival for 31 (46%); response rate (including cytogenetic, haematological, and molecular response rates for haematological indications) for 11 (16%); disease-free survival for four (6%); with time to progression, event-free survival, relapse-free survival, and testosterone concentrations accounting for the remainder (4/68, 6%).

Just over half (37/68, 54%) of all drug indications had a supporting pivotal trial evaluating quality of life, but results were reported for only 35. None of the pivotal trials assessed quality of life as a primary endpoint (table 2).

Of 68 drug indications, 60 (88%) were supported by at least one randomised controlled trial, while eight of 68 (12%) were approved solely on the basis of a single arm study (6/10 indications granted a conditional marketing authorisation and 2/58 granted a regular marketing authorisation) (table 2 and table B in appendix). Most (6/8, 75%) authorised uses supported by uncontrolled studies were also orphan designations. On the other hand, more than half of the orphan drugs in our cohort (7/11, 64%) were approved on the basis of a randomised trial.

Availability of evidence on overall survival and quality of life at time of market authorisation

At time of market approval, according to the EPARs, there was significant prolongation of survival in just over a third (24/68, 35%) of all drug indications (including 3/17 drugs to treat haematological malignancies and 21/51 drugs to treat solid tumours). There were gains in survival for seven compared with an active comparator, eight compared with placebo, one compared with best supportive care, and eight as add on treatment. The magnitude of the overall survival benefit ranged from 1.0 to 5.8 months (median 2.7 months) (fig 1). For the 44 (65%) remaining drug indications, there was no conclusive evidence at time of market authorisation that the drugs offered survival benefits, either as add on treatment or compared with placebo or existing treatment options in their authorised use.

Fig 1 Magnitude of overall survival benefit at the time of EMA approval. Figure excludes seven indications for which median overall survival could not be estimated at time of marketing authorisation: mifamurtide for resectable non-metastatic osteosarcoma after complete resection (+chemo); pertuzumab for 1st line HER2+mBC; pomalidomide for 3rd line (+dexamethasone) relapsed and refractory multiple myeloma; rituximab for 1st line CLL (+chemo); trastuzumab for HER2+ BC (+taxane) after adjuvant chemo; trastuzumab for HER2+BC (+adjuvant chemo); and trastuzumab for HER2+locally advanced BC (+neoadjuvant chemo and as monotherapy adjuvantly). Box 1 shows abbreviations used

Box 1: Abbreviations used in tables and figures

5FU=5 fluorouracil; adenoca=adenocarcinoma; AI=aromatase inhibitor; ALCL=anaplastic large cell lymphoma; ALK=anaplastic lymphoma kinase; ALL=acute lymphoblastic leukaemia; AML=acute myeloid leukaemia; AP=accelerated phase; ASCT=autologous stem cell transplant; BC=breast cancer; BCC=basal cell carcinoma; BP=blast phase; chemo: BCS=best supportive care; chemo=chemotherapy; CLL=chronic lymphocytic leukaemia; CML=chronic myelogenous leukaemia; CP=chronic phase; EFGR=epidermal growth factor receptor; EGFRi=epidermal growth factor receptor inhibitor; FOLFIRI: irinotecan/5 fluorouracil/folinic acid; FOLFOX: oxaliplatin/5 fluorouracil/folinic acid; GC=gastric cancer; GIST=gastrointestinal stromal tumours; HL=Hodgkin lymphoma; HR=hormone receptor; mBC=metastatic breast cancer; MCL=mantle cell lymphoma; mCRC=metastatic colorectal cancer; mCRPC=metastatic castration resistant prostate cancer; mGC=metastatic gastric cancer; mGOJ=metastatic gastric or esophageal junction; mNSCLC=metastatic non-small cell lung cancer; mPC=metastatic prostate cancer; mut=mutation; non-squam=non-squamous; PBO=placebo; NHL=non-Hodgkin lymphoma; NR=not reported; PC=prostate cancer; Ph+=Philadelphia chromosome positive; PLD=pegylated liposomal doxorubicin; PNET=pancreatic neuroendocrine tumours; pred=prednisone or prednisolone; RCC=renal cell carcinoma; SCT=stem cell transplantation; squam=squamous; STS=soft tissue sarcoma; TC=thyroid cancer; TCC=transitional cell carcinoma; TKI=tyrosine kinase inhibitor; VEGFi=vascular endothelial growth factor inhibitor; WT=wild-type

There was a significant improvement in quality of life over the control arm in only seven of the 68 (10%) authorised drug indications. Five of these were drug indications for which there was no survival gain at the time of market authorisation, giving a total of 29 of 68 (43%) that did show improvements in either quantity or quality of life at time of market entry.

The remaining 39 (57%) drugs had shown no improvement in survival or quality of life over active treatment, placebo, or as add on treatment. Eleven were evaluated against an active comparator, and 10 of these had comparable efficacy with respect to survival, although in most cases survival data were immature. Twenty were evaluated against placebo or as add on treatment and showed no gains in survival or quality of life. Eight were approved on the basis of a single arm trial.

Availability of evidence on overall survival and quality of life in postmarketing period

Of the 44 drug indications that did not show a survival benefit at time of approval, and with a median of 5.4 years’ follow-up (minimum of 3.3 years and maximum of 8.1 years), three (7%) were subsequently shown to extend life after market entry and five (11%) were associated with some improvements in quality of life (fig 2).

Fig 2 Availability of benefits on overall survival and quality of life of cancer drugs approved 2009-13. Box 1 shows abbreviations used

For all three drug indications with a reported survival gain in the postmarketing period, evidence came from updated analyses of the pivotal studies supporting initial marketing authorisation, and all had been granted regular marketing approval. None of the 10 drugs granted conditional marketing authorisation were subsequently shown to improve quantity or quality of life. One of 68 authorised uses judged by the EMA to extend life at time of market authorisation failed to show such a benefit on longer follow-up according to the published report.41 Thus, with a minimum 3.3 years’ follow-up, in 26 of the 68 (38%) authorised drug indications there was a survival gain (including three of the seven drugs approved for use in a curative setting).

For the five drug indications with a reported improvement in quality of life, evidence came from new studies in the same indicated population with the exception of nilotinib, for which a published paper42 reporting the pivotal study described quality of life gains not reported in the EPAR. Box 2 lists those drugs not shown to extend life, but which were associated with a quality of life benefit with a median of 5.4 years’ follow-up.

Afatinib v pemetrexed/cisplatin (for the treatment of TKI-naive, EGFR mutation +ve, non-small cell lung cancer) significantly delayed the time to deterioration for cough and dyspnoea but not time to deterioration of pain. HRQoL was evaluated against prespecified components of the European Organisation for Research and Treatment of Cancer quality of life questionnaire C30 (EORTC QLQ-C30) questionnaire and lung cancer module QLQ-LC13. The study was open label43

Gefitnib (for the treatment of metastatic non-small cell lung cancer) showed significant benefits on quality of life compared with carboplatin plus paclitaxel, and with docetaxel on some measures, but not compared with placebo. HRQoL was assessed with the functional assessment of cancer therapy-lung (FACT-L), and trial outcome index (TOI), and symptom improvement by the lung cancer subscale (LCS). The active comparator trials were open label. The trial v placebo was double blind44

Tegafur, gimeracil, oteracil (S-1) (in combination with cisplatin for the treatment of advanced gastric cancer) showed a significant improvement over cisplatin/5-fluorouracil for only one of the subscales (physical wellbeing) of the functional assessment of cancer therapy-gastric (FACT-Ga) HRQOL instrument. The study was open label45

Vandetanib v placebo (for the treatment of metastatic medullary thyroid cancer) significantly improved “time to worsening of pain” (a composite endpoint based on patient reported analgesic use and responses to the brief pain inventory) but did not show a significant improvement for the functional assessment of cancer therapy-general (FACT-G) score. The study was double blind46

Crizotinib (for second line treatment of ALK mutation +ve advanced non-small cell lung cancer) showed significant improvements compared with pemetrexed across a range of HRQoL measures, including a significantly greater delay in time to worsening of symptoms. HRQoL was assessed with the European Organisation for Research and Treatment of Cancer quality of life questionnaire C30 (EORTC QLQ-C30) and the lung cancer module QLQ-LC13. The study was open label47

Erlotinib (for first line treatment of EGFR mutation +ve metastatic non-small cell lung cancer) showed clinically relevant and significant improvements compared with gemcitabine/carboplatin in total functional assessment of cancer therapy-lung (FACT-L), trial outcome index (TOI), and lung cancer subscale (LCS) scores. The study was open label48

Nilotinib v imatinib (for patients with newly diagnosed Ph+ chronic myeloid leukaemia) reported a significant quality of life improvement with respect to some categories of low grade adverse events. HRQoL was assessed with generic SF-36 and leukaemia specific functional assessment of cancer therapy-leukaemia (FACT-Leu) surveys. The study was open label42

Ofatumumab (in patients with chronic lymphocytic leukaemia refractory to fludarabine and alemtuzumab) showed significant improvements v “physician’s choice” with respect to fatigue but not side effects of treatment or effects of disease. HRQoL was evaluated according to three prespecified domains of the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire C30 (EORTC-C30) and the chronic lymphocytic leukaemia module QLQ-CLL16 (fatigue, side effects of treatment, and effects of disease). The study was open label49

Pazopanib (for first line treatment of advanced renal cell carcinoma) significantly improved HRQoL compared with sunitinib with respect to 11 of 14 HRQoL domains. HRQoL was assessed with the functional assessment of chronic illness therapy-fatigue (FACIT-F) questionnaire, the National Comprehensive Cancer Network/functional assessment of cancer therapy-kidney symptom index 19 (NCCN-FACT FKSI-19), and the cancer therapy satisfaction questionnaire. The study was open label50

Proportion of EMA oncology approvals meeting ESMO threshold for clinically meaningful survival benefit

Table 3 shows the ESMO-MCBS scores for all the drugs to treat solid tumours that showed benefit on overall survival by the time of our study cut off. After a median of 5.4 years’ follow-up 26 drug indications were associated with a survival gain, but three of these were for haematological cancers, leaving a total of 23 to which the ESMO scale could be applied.

Table 3

ESMO grades for 23 drugs used to treat solid tumours that showed benefit on overall survival after follow-up

Study	Drug	Site	Indication	Trial	Overall survival gain*	HR (CI)	QoL/toxicity†	ESMO score‡
Fizazi (2012)51	Abiraterone	Prostate	mCRPC (+pred) after chemo	COU-AA-301 (prednisone±abiraterone)	4.6	0.74 (0.64 to 0.86)	Yes	4
Ryan (2015)52	Abiraterone	Prostate	mCRPC (+pred) before chemo	COU-AA-302 (abiraterone+prednisone or prednisolone v placebo+prednisone or prednisolone)	4.4	0.81 (0.70 to 0.93)	Yes	4
Van Cutsem (2012)53	Aflibercept	Bowel	2nd line mCRC (+FOLFIRI)	VELOUR (aflibercept, then FOFIRI v placebo, then FOLFIRI)	1.44	0.817 (0.713 to 0.937)	No	1
De Bono (2010)54	Cabazitaxel	Prostate	Hormone refractory mPC (+pred) previously treated with docetaxel	TROPIC (cabazitaxel+prednisone or prednisolone v mitoxantrone+prednisone or prednisolone)	2.4	0.70 (0.59 to 0.83)	No	2
Cortes (2011)55	Eribulin	Breast	3rd line mBC	EMBRACE (eribulin v physician’s choice)	2.5	0.81 (0.66 to 0.99)	No	2
Cappuzzo (2010)56	Erlotinib	Lung	Maintenance therapy in mNSCLC (previous platinum based chemo)	SATURN (erlotinib v placebo)	1	0.81 (0.70 to 0.95)	No	1
Scher (2012)57	Enzalutamide	Prostate	mCRPC after docetaxel	AFFIRM (enzalutamide v placebo)	4.8	0.63 (0.53 to 0.75)	Yes	4
Hodi (2010)58	Ipilimumab	Skin	2nd line unresectable or metastatic melanoma	MDX010-20 (ipilimumab v vaccine alone)	3.7	0.66 (0.51 to 0.87)	No	4
Robert (2011)59	Ipilimumab	Skin	1st line unresectable or metastatic melanoma	CA184-024 (dacarbazine±ipilimumab)	2.1	0.72 (0.59 to 0.87)	No	2
Blackwell (2012)60	Lapatinib	Breast	HER2+ HR- mBC (+ trastuzumab). Previous trastuzumab + chemo	EGF104900 (lapatinib±trastuzumab)	4.5	0.74 (0.57 to 0.97)	No	4
Meyers (2008)61	Mifamurtide	Osteosarcoma	Resectable, non-metastatic osteosarcoma after complete resection (+ chemo)	INT-0133	6 years; 8%	0.71 (0.52 to 0.96)	NR	A
Goldstein (2015)62	Nab-paclitaxel	Pancreatic	1st line (+ gemcitabine) metastatic pancreatic adenoca	CA046 (gemcitabine±nab-paclitaxel)	2.1	0.72 (0.62 to 0.83)	No	2
Douillard (2014)63	Panitumumab	Bowel	1st line KRAS WT mCRC (+ FOLFOX)	PRIME (FOLFOX4±panitumumab)	4.4	0.83 (0.70 to 0.98)	No	3
Ciuleanu (2009)64	Pemetrexed	Lung	Maintenance for mNSCLC (non-squam) after platinum based doublet chemo (w/gemcitabine or taxane)	JMEN (pemetrexed+BSC v placebo+BSC)	2.8	0.79 0.65 to 0.95)	No	3
Paz-Ares (2013)65	Pemetrexed	Lung	Maintenance for mNSCLC (non-squam) after platinum based chemo	PARAMOUNT (pemetrexed+BSC v placebo+BSC)	2.9	0.78 (0.64 to 0.96)	No	3
Swain (2015)66	Pertuzumab	Breast	1st line HER2+ mBC	CLEOPATRA (trastuzumab+chemo±pertuzumab)	15.7	0.68 (0.56 to 0.84)	No	4
Grothey (2013)67	Regorafenib	Bowel	mCRC either after previous treatment with/or ineligible for 5FU-based chemo or VEGFi or EGFRi therapy	CORRECT (regorafenib v placebo)	1.4	0.77 (0.64 to 0.94)	No	1
Bang (2010)68	Trastuzumab	Stomach	1st line HER2+ mGC or mGOJ adenoca	ToGA (5-FU or capecitabine+cisplatin±trastuzumab)	2.7	0.74 (0.60 to 0.91)	No	3
Perez (2014)69	Trastuzumab	Breast	HER2+ BC (+taxane) after adjuvant chemo	NASBP-31 and NCCTG-N9831 (doxorubicin+cyclophosphamide, followed by paclitaxel±trastuzumab)	10 years; 8.8%	0.63 (0.54 to 0.73)	NR	A
Slamon (2011)70	Trastuzumab	Breast	HER2+ BC (+adjuvant chemo)	BCIRG-006 (doxorubicin+cyclophosphamide, followed by trastuzumab v doxorubicin+cyclophosphamide, followed by docetaxel)	5.4 years; 5%	0.63 (NR)	No	B
Verma (2012)71	Trastuzumab emtansine	Breast	HER2+ unresectable or mBC after trastuzumab and/or taxane	EMILIA (trastuzumab emanstine v lapatinib+capecitabine)	5.8	0.68 (0.55 to 0.85)	Yes	5
McArthur (2014)72	Vemurafenib	Skin	Unresectable or metastatic melanoma (BRAF V600 mut)	BRIM-3 (vemurafenib v dacarbazine)	3.9	0.70 (0.57 to 0.87)	No	4
Bellmunt (2009)73	Vinflunine	Urinary	Advanced or metastatic TCC of urothelial tract. Previous platinum regimen	VFL-302 (vinflunine+BSC v BSC)	2.6	0.77 (0.61 to 0.99)	No	3

See box 1 for abbreviations.

*Survival gain in months, or percent of patients surviving at x years.

†Yes=quality of life or improved toxicity reported; no=no quality of life gain or improvement in toxicity reported, or toxicity might be worse.

‡For drug indications with curative intent, grade of A or B indicates trial data show drug offers clinically meaningful benefit. For drug indications with non-curative intent, grade of 4 or 5 indicates trial data show drug offers clinically meaningful benefit. For drugs with non-curative intent, grade of 1, 2, or 3 indicates drug does not reach ESMO threshold for clinically meaningful benefit.

Of these 23 drugs, 11 (48%) reached the ESMO threshold for a meaningful survival benefit; eight out of 20 (40%) for use in the non-curative setting and all three treatments with curative intent.

Results summary

Among 68 cancer drug indications approved by the EMA in the period 2009-13, and with a median of 5.4 years’ follow-up, only 35 (51%) were associated with a significant improvement in survival (26/35) or quality of life (9/35) over existing treatment options, placebo, or as add on treatment. Only two of the 26 drugs shown to extend life also showed benefits on quality of life, and 33 (49%) had not shown any improvement on survival or quality of life (fig 2). Of the 23 drugs with a survival benefit that could be scored with the validated ESMO-MCBS tool, only 11 (48%) were judged to offer a clinically meaningful benefit.

Discussion

Summary of findings

This systematic evaluation of oncology drug approvals by the EMA in 2009-13 shows that most of the drugs (39/68, 57%) entered the market without evidence of improved survival or quality of life. At a minimum 3.3 years after market entry, there was still no conclusive evidence that 33 of these 39 cancer drugs either extended or improved life. Our findings suggest it is extremely rare for new studies or follow-up analyses of pivotal trials in the postmarketing period to report results confirming that new cancer drugs have a positive impact on the two most important outcomes for patients—survival and quality of life. When survival gains over existing treatment options or placebo were shown, they were often marginal and judged to be clinically meaningful in less than half (11/23, 48%) of all cases.

What are potential reasons for the paucity of drug approvals with demonstrable survival advantages over existing treatments? Firstly, only 18 (26%) indications for use in our cohort were supported by trials in which extension of life was the primary outcome, and trials that evaluated survival as a secondary endpoint might not have been powered to detect differences between groups with and without the experimental treatment. Secondly, crossover from control to experimental arm after disease progression (or to similar drugs after the study) is a commonly cited reason for the lack of survival benefits in clinical trials of oncology products.38 It should be noted, however, that for some indications for use in our cohort there were no survival gains even in the absence of crossover.7 50 74 75 76

None of the pivotal studies supporting oncology drug approvals from 2009 to 2013 included quality of life as a primary outcome measure. This is perhaps unsurprising as the EMA does not require marketing authorisation holders to evaluate quality of life, even when drugs are intended for late line palliative care.77 Of the 68 (54%) study indications, 37 were supported by a pivotal trial in which quality of life was evaluated as a secondary endpoint, and for only seven (10%) was there a significant improvement in quality of life at time of market entry. Neither the EPARs nor the published studies were consistent in reporting whether significant improvements for specific quality of life outcomes were also clinically relevant. These are particularly troubling findings as many of the drugs in our sample were approved to treat advanced metastatic disease, when the purpose of treatment is palliative—that is, to improve quality of life—or to extend life while ensuring that any survival gains are not outweighed by a deterioration in quality of life from adverse events or other negative factors related to treatment.78 79

A significant minority of drugs in our sample (8/68, 12%) were approved solely on the basis of a single arm study. Most of the indications supported by single arm studies (6/8) were granted orphan designation, and it is recognised that the conduct of randomised trials to investigate treatments for rare cancers can be challenging. Nevertheless, a review of FDA approvals indicates that randomised controlled trials are feasible even in rare cancers with an incidence of less than 1/100 000/year.80

Comparison with other studies

Apolone and colleagues27 evaluated the evidence base for new cancer drugs approved by the EMA in the period 1995-2004. In that 10 year period the EMA authorised 14 anticancer drugs for 27 indications. An important difference between their earlier study and the present one is that the earlier cohort includes only drugs to treat solid tumours. Nevertheless, our findings indicate some improvements in the quality of the evidence base supporting new oncology drugs. For example, drug uses authorised in the study period 2009-13 were more likely to be supported by trials that included overall survival as a primary endpoint (26% v 8%), and by randomised trials (88% v 66%). There has been a substantial increase in the number of approved treatments for solid tumours compared with the earlier period—an average of 12 a year for 2009-20 compared with three a year for 1995-2004.

In contrast with recent studies focusing on a subset of oncology drug approvals in Europe,32 33 we included all authorised oncology indications regardless of route of approval and systematically analysed survival and quality of life benefits. We also relied exclusively on evidence generated from randomised controlled trials. We did not therefore accept mathematical models considered by health technology assessment bodies such as the National Institute for Health and Care Excellence.81 There is continuing debate around the validity of simulation models that extrapolate findings from short term trials.82 83 Our analysis is the first to investigate whether drugs that did not show a survival benefit at the time of approval in Europe are subsequently shown to improve survival or quality of life in follow-up randomised controlled trials once drugs are on the market.

Study limitations

There are several limitations to this study. Firstly, there were several cases in which results reported in the EPARs were ambiguous or incomplete. When difficulties in interpretation occurred, we ensured our determinations were consistent with the EMA’s conclusions. Following this approach, we determined that significant survival benefits had been shown at time of market authorisation for vinflunine (in transitional cell carcinoma) and trastuzumab (in neoadjuvant breast cancer), despite uncertainty regarding the reported statistical analyses. When the EMA’s opinions on specific questions were not clear or not available, we adopted a generous approach that favoured the drug. For example, we credited panitumumab with showing a survival benefit in the postmarketing period on the basis of an “exploratory” analysis.84

There was also considerable variation in the reporting of quality of life endpoints in both the EPARs and the medical literature. Significant results might have been shown for isolated parameters or single subscales only, and it was not clear whether a net quality of life benefit had been shown. Nor was it consistently reported whether or not the magnitude of improvement was such that it would be considered clinically meaningful for patients. Consequently, we might have overestimated the proportion of drugs that offer benefits on survival or quality of life and the clinical meaningfulness of these. Our findings, however, are consistent with recent findings from the US.30 85

Secondly, we did not consider the appropriateness of clinical trial design, including whether a treatment administered in the control arm of a study was optimal or even informative. This is an important study limitation, which again means we might have overestimated the benefits offered by the drugs in our sample. For example, the control arm of the pivotal trial to support approval of ipilimumab for metastatic melanoma was an experimental treatment. Similarly, the pivotal trial to support lapatinib in combination with trastuzumab for patients with metastatic breast cancer used lapatinib monotherapy as the comparison—a treatment acknowledged in the EPAR to be “without established efficacy.”86 Although this trial design failed to isolate the effects of the test drug, the EMA nevertheless concluded that the reported benefit in survival was clinically relevant and could be attributed to the combination.

Thirdly, our ESMO ratings share the limitations of the published studies on which they are based. For example, the magnitude of clinical benefit shown in a particular trial could be inflated by the choice of an inappropriate comparator or treatment in the control arm that does not represent best standard of care.38 Several maintenance therapy trials for metastatic non-small cell lung cancer, for example, have been criticised for flawed study designs that systematically result in substandard care in the control arms.87 In the case of lapatinib combined with tratuzumab, the pivotal study was scored 4 on the basis of a 4.5 month survival gain over lapatinib monotherapy (with no established efficacy).

Fourthly, our findings do not reflect the totality of the evidence base for a specific authorised indication as we did not take account of studies with negative results for the same drug indication if they were not reported as pivotal studies in the EPAR, nor did we take account of the fact that studies with positive results for a particular indication could conflict with negative findings from other studies in the same indicated patient population.

Implications for clinical practice and regulatory policy

Despite the EMA’s statement that overall survival is the “most persuasive outcome” for studies investigating the clinical safety and efficacy of new oncology drugs and new uses of such drugs, European regulators commonly accept the use of surrogate measures of drug benefit as primary endpoints in pivotal trials for both conditional and regular pathways to market authorisation.11 88 In addition, any demonstration of a statistically significant difference in survival between treatments, however small and regardless of whether such a difference is clinically meaningful to patients, has recently been accepted as the basis for approving new cancer drugs.89 To a large extent, regulatory evidence standards determine the clinical value of, and the quality of the evidence base for, new oncology drugs. Our study suggests these standards are failing to incentivise drug development that best meets the needs of patients, clinicians, and healthcare systems.15 17 19 21

Though surrogate endpoints are argued to have the advantage of allowing quicker drug development and patient access, it is questionable whether studies based on surrogate measures of efficacy provide optimal, or even meaningful, information for patients and clinicians.24 Moreover, our analysis raises the possibility that regulatory and current research practices have created a situation in which critical information about the outcomes that matter most to patients might never be generated once oncology drugs are approved for widespread use. The limited availability of studies showing either benefits to overall survival or quality of life in the postmarketing period underscores the importance of requiring robust evidence of clinical benefit at the time of marketing authorisation.90 91 92

The EMA and other drug regulatory agencies should reconsider when, and to what extent, it is appropriate to approve new cancer drugs on the basis of surrogate endpoints. Furthermore, when gains in survival and quality of life are shown, these gains should be meaningful to patients and clinicians.

Conclusion

Among 68 cancer drug indications approved by the EMA in the period 2009-13, and with a median of 5.4 years’ follow-up, only 35 (51%) were associated with significant improvement in survival or quality of life over alternative treatment options, placebo, or as add on treatment. For 33 (49%), uncertainty remains over whether the drugs extend survival or improve quality of life. Of the 23 drugs with a survival benefit that could be scored with the validated ESMO-MCBS tool, only 11 (48%) were judged to offer a clinically meaningful benefit.

Most new oncology drugs authorised by the EMA in 2009-13 came onto the market without clear evidence that they improved the quality or quantity of patients’ lives, and, when survival gains over available treatment alternatives were shown, they were not always clinically meaningful. Little new information to guide patients, their treating clinicians, or decisions about whether or not to pay for treatments was generated in the postmarketing period. This situation has negative implications for patients and public health.19 93 When expensive drugs that lack clinically meaningful benefits are approved and paid for within publicly funded healthcare systems, individual patients can be harmed, important societal resources wasted, and the delivery of equitable and affordable care undermined.

Clinical trials designed to gain regulatory approval for new drugs often evaluate surrogate measures that do not always reliably predict clinically meaningful outcomes like survival or quality of life

Recent reviews from the US show that only a small proportion of cancer drugs approved by the US Food and Drug Administration improve survival or quality of life

No recent studies have systematically examined the evidence base for and magnitude of benefit for cancer drugs approved by the EMA

Most new oncology drugs authorised by the EMA in 2009-13 came onto the market without clear evidence that they improved the quality or quantity of patients’ lives

After market entry, cancer drugs rarely show benefits on overall survival or quality of life in randomised trials

When survival gains over available treatment alternatives are shown, they are not always clinically meaningful