Trends in endpoint selection and result interpretation in advanced non-small cell lung cancer clinical trials published between 2000 and 2012: A retrospective cohort study.

BACKGROUND: The objective of this review was to investigate trends in clinical trial design, specifically, the primary outcomes used, interpretation of results, and the magnitude of the benefits described in phase III controlled clinical trials in the first-line treatment of patients with advanced non-small cell lung cancer (NSCLC).
METHODS: Seventy-six trials published between 2000 and 2012 were selected from a total of 122 identified in a structured search.
RESULTS: Overall survival (OS) was evaluated as the primary study endpoint in 50 (65.8%) trials, followed by progression-free survival (PFS) in 15 (19.7%), and other variables, such as toxicity, quality of life (QoL), and response rate in 11 (14.5%). Ten (66.7%) out of 15 clinical trials using PFS as the primary endpoint were published between 2010 and 2012. Median overall survival (mOS) was 9.90 months (interquartile range: 3.5) with an increase of 0.384 months per year of publication (P < 0.001). A statistically significant improvement in mOS was obtained in only 13 (18.8%) trials. A total of 41 (53.9%) studies concluded that the result was positive. Of these, only 16 (39.1%) showed a statistically significant benefit in OS. QoL was assessed in 46 trials (60.5%) and of these, 10 (21.7%) reported significant improvements.
CONCLUSIONS: These findings raise important questions about how clinical benefits are measured in clinical trials in advanced NSCLC. Appropriate clinically relevant outcome variables should be established and validated, and post-marketing studies should be requested by regulatory authorities to ensure meaningful clinical benefits in OS and QoL.

Introduction

Advanced non‐small cell lung cancer (NSCLC) is the leading cause of cancer‐related death worldwide.1 Consequently, it has been the subject of extensive investigation over several decades, showing the growing interest of the oncology community and the pharmaceutical industry in this clinical entity.2 However, despite great efforts in research, the clinical benefit of new therapies has been modest,3 and treatment of lung cancer patients remains one of the greatest challenges in oncology.

Although the goal of research in cancer therapy is to improve the quantity and quality of life,4, 5 a growing number of questions have been raised with respect to the magnitude of the benefits shown in clinical trials in oncology. One of the difficulties in evaluating the clinical benefit of new cancer drugs is the numerous outcome measures used in clinical trials, and there is emerging concern regarding the design of such trials in oncology resulting from the increasing trend in measuring effectiveness through surrogate variables (i.e. progression‐free survival [PFS], time to progression [TTP], etc.).6, 7, 8, 9 The evidence supporting the use of surrogate endpoints in oncology is inconsistent6, 8, 10, 11 and, moreover, the reliability of the correlation between surrogate measures and meaningful outcomes like overall survival (OS) or quality of life (QoL) has not always been validated or is uncertain.12, 13 Although OS is still accepted as the gold standard for efficacy endpoints in lung cancer studies,14 its use as the primary endpoint appears to be declining in clinical trials investigating advanced NSCLC. Furthermore, an increasing trend toward positive interpretations of the results of clinical trials in oncology despite modest or questionable clinical benefits has also been reported.15

The aim of the study was to investigate trends in clinical trial design with respect to the primary outcomes used, the interpretation of the results, and the magnitude of the benefits described from first‐line treatment of patients with advanced NSCLC in studies published between 2000 and 2012.

Methods

Search strategy and study selection criteria

A structured search was conducted for randomized phase III clinical trials (RCTs) in the first‐line treatment of patients with advanced NSCLC published between 2000 and 2012. MEDLINE (accessed via PubMed and Ovid SP); Embase (Ovid SP); the Cochrane Central Register of Controlled Trials, Cochrane Library (CENTRAL); Evidence‐Cancer (via NHS); and TRIP (via BV‐SSPA) were searched using the following terms: lung cancer, non‐small, advanced, non‐surgical, treatment, controlled clinical trial, and phase III. To be included in the study, trials had to have at least two comparative arms of systemic chemotherapy or molecular‐targeted agents in patients with advanced NSCLC. In trials with two or more experimental arms, the arm selected for evaluation in this review was the one that obtained the highest OS or PFS, depending on the primary outcome endpoint of the trial. Only RCTs published in journals were selected. Exclusion criteria were: RCTs studying only non‐advanced disease or any stage of small‐cell lung cancer other than advanced stage; surgical or radiotherapy intervention studies or cancer screening and prevention studies; meta‐analyses or reviews reporting data from multiple RCTs; preliminary studies for which subsequent phase III studies were available; retrospective series; congress abstracts; and studies in a language other than English.

Data extraction, analysis, and management

Two investigators independently extracted data from each selected publication using a specifically designed form to record the following information: publication year, primary and secondary endpoints, assessment of QoL, median overall survival (mOS, months), median progression‐free survival (mPFS, months) per arm, and any statistically significant differences observed between treatment arms. We also recorded the authors’ conclusions on the experimental arm and summarized them as a positive result, negative result, or similar result for experimental versus control arms.

Results

Characteristics of selected trials

As shown in Figure 1, of the 122 trials retrieved for further evaluation, 46 were excluded after applying the study eligibility criteria. The 76 trials finally selected for this review (a list of the 76 clinical trials is given as online supporting information) involved a total of 40 765 patients with advanced NSCLC randomly assigned to 152 treatment arms. The characteristics of these 76 selected trials are summarized in Table 1.

Figure 1

Flowchart depicting the trial selection process for the review.

Table 1

Characteristics of the 76 selected clinical trials

Variable	N (%)
Publication year
2000	3 (3.9)
2001	5 (6.6)
2002	7 (9.2)
2003	7 (9.2)
2004	7 (9.2)
2005	7 (9.2)
2006	7 (9.2)
2007	6 (7.9)
2008	5 (6.6)
2009	4 (5.3)
2010	7 (9.2)
2011	5 (5.3)
2012	7 (9.2)
No. of randomized patients
< 500	46 (60.5)
501–1000	17 (22.4)
> 1000	13 (17.1)
Median (range)	432 (126–1725)
Primary endpoint
OS	50 (65.79)
PFS	15 (19.74)
Other (RR, QoL, toxicity)	11 (14.47)
Follow‐up median (range), months	17.6 (2.7–60)
Not recorded	29
Authors’ conclusion
Positive result	41 (53.9)
Negative result	25 (32.9)
Similar result	10 (13.2)

OS, overall survival; PFS, progression‐free survival; RR, response rate; QoL, quality of life.

First and secondary endpoints in randomized controlled trials of non‐small cell lung cancer

OS was used as a primary study endpoint in 50 (65.8%) of the 76 selected clinical trials for the first‐line treatment of advanced NSCLC published between 2000 and 2012, followed by PFS in 15 trials (19.7%), and other variables, such as toxicity, QoL, and response rate, in 11 (14.5%). It should be noted that 10 of the 15 clinical trials using PFS as the primary endpoint were published between 2010 and 2012.

OS data were available in 69 (90.8%) trials, and of these, only 13 (18.8%) reported a statistically significant OS benefit in the experimental arm. The remaining 56 (81.2%) trials did not demonstrate a statistically significant increase in OS compared to alternative treatments. We also examined total gains in OS obtained over the study period: mOS for the 76 trials was 9.90 months (interquartile range [IQR] 3.5), with a total mean improvement in OS of 0.384 months per year (P < 0.001).

PFS data were available in 67 (88.2%) of the 76 trials, and a significant difference between arms was reported in 26 (38.8%). The mPFS reported in trials during the study period was 4.90 months (IQR 1.9).

A total of 41 (53.9%) studies were considered to have concluded that the results were positive, based on the authors’ recommendations for the treatment to be adopted in clinical practice or in further studies. These positive conclusions were supported by a statistically significant result in the primary study endpoint in only 25 (61%) clinical trials. Of these, OS was the primary outcome in 11; surrogate measures, such as PFS, were the primary endpoint in a further 11 clinical trials; and finally, toxicity, QoL, and response rate were the primary endpoints in the three remaining clinical trials that were able to show statistically significant differences in the primary study endpoint.

In 16 (39%) of the 41 clinical trials that concluded that a positive result had been obtained in the experimental arm, no statistically significant differences in the primary study endpoint were shown, and the results were supported by secondary measures, such as lower toxicity (8/16), improvement in response rate (1/16), significant improvement in QoL (2/16), statistically significant improvement in PFS (3/16), and statistically significant improvement in OS (2/16).

QoL was evaluated in 46 (60.5%) trials and a statistically significant improvement in this endpoint was found in 10 (21.7%). A total of 10 different measurement instruments were identified. The specific Functional Assessment of Cancer Therapy Lung Cancer questionnaire (FACT‐LCS5) was used in 23 (50%) trials, the European Organization for Research and Treatment of Cancer Quality of Life questionnaire C30 (EORTC QLQ C‐30) in 20 (43.5%), and the specific EORTC lung cancer questionnaire (EORTC QLQ LC‐13) in 17 (37.0%).

Discussion

In recent years, there has been debate over the increasing use of surrogate variables, such as PFS, instead of the OS primary endpoint in the design of clinical trials in oncology.6, 8 In our study, 65.8% (50/76) of the trials published between 2000 and 2012 used OS as the main variable, followed by PFS in 19.7% (15/76), and other variables in 14.5% (11/76). It should be highlighted that 66% of the RCTs that included PFS as the primary endpoint were published from 2010 to 2012, the last two years of our 12‐year study period. Our work therefore confirms the increasing trend toward the use of PFS as the primary endpoint in RCTs in advanced NSCLC and the declining use of OS or QoL.

Some authors believe this trend to be the result of an increasing willingness of regulatory authorities to accept the use of surrogate variables as primary study endpoints,15 the common use of crossover trial designs and subsequent lines of treatment that may dilute the overall survival benefit, or longer follow‐up times in trials using OS as the primary endpoint.11 The Food and Drug Administration recently accepted the use of PFS as a primary endpoint in trials studying the treatment of advanced NSCLC but with reservations regarding the accuracy, reproducibility, and clinical relevance of this surrogate variable,14 and concludes that OS is still the standard clinical benefit outcome variable that should be used to establish the efficacy of a therapy in patients with advanced NSCLC. The fact is that the advantages gained from faster drug approval and patient access to new agents are questionable if they rely on results of studies based on surrogate variables that do not translate into benefits in terms of OS or QoL.

Many of the surrogate variables used as primary outcome measures in clinical trials are poorly correlated with survival or QoL, or the correlation has not been tested.12, 16, 17 Moreover, most regulatory approvals based on these surrogate variables do not usually require post‐marketing studies to be conducted to confirm or validate effectiveness and safety results that allow translating PFS benefit into meaningful clinical benefit for patients, that is, OS or QoL.

There is currently an interesting and necessary debate regarding when it should be demonstrated that a cancer drug leads to an improvement in OS or QoL. Some argue that this should occur before market authorization while others believe that in some indications, such as advanced NSCLC, a drug could receive provisional approval based on surrogate endpoints, and its benefit on OS or QoL should be demonstrated in post‐marketing trials. However, according to two recent studies, this rarely happens.18, 19 One of these studies that evaluated evidence of the OS or QoL benefits of cancer drugs approved by the European Medicines Agency stated that results reported in the post‐marketing period confirming a positive impact on survival and QoL are very rare for new cancer drugs that have been approved based on a benefit in surrogate variables.19

Our findings suggest that positive conclusions describing a meaningful clinical benefit in patient outcome are extremely common, even when no statistically significant differences in the primary study endpoint are obtained. Similarly, studies frequently reach a positive conclusion without having demonstrated a positive impact on the two most important outcomes for patients, OS and QoL. In our study, of the 41 clinical trials with positive conclusions, only 16 were based on a statistically significant OS gain.

A total of 41 (53.9%) clinical trials concluded their results in the experimental arm were positive, a figure similar to that reported by Sacher et al. in their review of phase III clinical trials in advanced NSCLC.15 They found positive conclusions in only 31% of clinical trials conducted between 1980 and 1990, compared to 53% in clinical trials carried out between 2001 and 2010. These results confirm the increasing trend toward positive interpretations of the results of clinical trials in NSCLC, even when no statistically significant differences in the main variable were shown. In our study, the percentage of trials showing a statistically significant improvement in the primary endpoint was 32.8%, a result very similar to the 31% described by Sacher et al. for the 2001–2010 study period. In this regard, Sacher et al. pointed out that the percentage of trials reporting a statistically significant benefit in the primary endpoint had not changed over time (29% in 1980–1990 and 31% in 2001–2010). However, the percentage of trials reporting a positive result despite not showing a statistically significant benefit in their primary outcome measure had clearly increased over time (30% of positive trials in 1981–1990 vs. 24% in 1991–2000 vs. 53% in 2001–2010; P = 0.001).15

QoL in cancer patients has been found to be a strong predictor of survival and toxicity. However, this outcome variable was evaluated in only 60.5% of clinical trials included in the present study. Moreover, only 21.7% of these trials reported a significant improvement in QoL. This result is better than that reported by Tanvetyanon et al. in their systematic review of QoL associated with chemotherapy for NSCLC, where only 7.1% of trials analyzed demonstrated a significant difference in QoL.20 Overall, QoL results have shown modest improvement and further research on this outcome in NSCLC patients, using more uniform methodologies, is necessary.

There are some limitations to this study. Firstly, publication bias is one of the main limitations of review studies and along with this, selection bias is one of the main criticisms of reviews and meta‐analyses. However, we minimized this bias by designing and applying a structured and well‐defined search strategy. In addition, five different databases were searched and two researchers independently selected the studies. Secondly, it was not always possible to obtain all of the data required to evaluate the methodology and outcomes of the studies. In addition, PFS and TTP were considered as surrogate endpoints of survival and as interchangeable variables, in common with other authors.10, 13, 21 Thirdly, the time period studied should be extended in order to include clinical trials investigating immunotherapy regimens. This will be taken into account in future research.

The present review shows that most of the clinical trials in advanced NSCLC published between 2000 and 2012 did not demonstrate a statistically significant improvement in OS or QoL over alternative treatments.

There is a lack of studies examining the design of clinical trials in advanced NSCLC treatment. Our findings raise important questions about how clinical benefits are measured in clinical trials in advanced NSCLC. Appropriate clinically relevant outcome variables should be established and validated, and post‐marketing studies in OS and QoL should be requested by regulatory authorities to ensure meaningful clinical benefits.

Disclosure

No authors report any conflict of interest.

Appendix S1. A list of the 76 phase III clinical trials included in this review is available as online supplementary material.

Click here for additional data file.