Can contemporary trials of chemotherapy for HER2-negative metastatic breast cancer detect overall survival benefit?

PURPOSE: Although several trials have demonstrated improved progression-free survival (PFS) with first-line regimens for HER2-negative metastatic breast cancer (mBC), overall survival (OS) benefit is elusive. We calculated required sample sizes to power for OS using published data from recent mBC trials. PATIENTS AND METHODS: Randomized superiority trials of first-line chemotherapy/targeted therapy for HER2-negative mBC including >150 patients, meeting the primary efficacy objective, and published in 2000-2018 were identified. The sample sizes required to power for PFS and OS were calculated retrospectively for each trial using observed results and study/recruitment follow-up durations (α=0.05, two-sided log-rank test, 80% power), and summarized as a factor (x) relative to actual sample size.
RESULTS: Nine of 13 identified trials reported all information required for retrospective sample size calculation. Six had sample sizes larger than required to demonstrate a significant PFS benefit but all would have required larger sample sizes to demonstrate significant OS benefit with the observed results. In ten trials, the required sample size was ≥5-fold larger to power for OS than PFS.
CONCLUSION: Designing trials to test potential new treatments for HER2-negative mBC is challenging, requiring a balance of regulatory acceptability, feasibility, and realistic medical assumptions to calculate sample sizes. Powering for OS is particularly difficult in heterogeneous populations with long postprogression survival, potential crossover, heterogeneous poststudy therapy, and evolving treatment standards. Validated surrogate endpoints are critical. Ongoing trials of cancer immunotherapy (new mode of action) in triple-negative mBC (more homogeneous, shorter OS and postprogression survival, fewer treatment options) may show a new pattern.

Introduction

In metastatic breast cancer (mBC), selection of the most appropriate endpoint for clinical trials is becoming increasingly important when evaluating new first-line therapies. In HER2-positive mBC, for which a number of targeted agents exist, several trials across treatment settings have demonstrated overall survival (OS) benefits from HER2-directed therapies.1 In HER2-negative mBC, however, where the target is less clear and patient selection is more challenging, progression-free survival (PFS) benefits have rarely translated into statistically significant OS benefits. To date, no Phase III trial evaluating antiangiogenic agents, cyclin-dependent kinase 4/6 inhibitors, mTOR inhibitors, or poly(adenosine diphosphate–ribose) polymerase inhibitors has shown a statistically significant OS improvement. OS is considered an unambiguous endpoint and is the global gold standard for demonstrating clinical benefit. However, extending life is not necessarily valuable if accompanied by significant quality of life (QoL) deterioration. Other disadvantages of OS as a primary endpoint are bias caused by treatment evolution during long studies, the diluting effect of crossover, numerous heterogeneous subsequent treatment lines, and the need for large patient numbers and/or long follow-up before obtaining results. This is particularly problematic in first-line trials, in which patients typically receive multiple treatment lines after progression.2,3 Consequently, authorities including the European Medicines Agency accept PFS as a relevant endpoint and approve drugs based on PFS benefit.

The correlation between PFS and OS appears to be less robust in settings with longer postprogression survival and/or effective subsequent therapies,4–6 whereas in later treatment lines, the likelihood of showing an OS benefit increases.7,8 However, a recent analysis of 40 randomized controlled trials in HER2-negative hormone receptor-positive mBC indicated a significant association between PFS/time to progression (TTP) and OS, irrespective of treatment line.9 To explore this topic further, we used published data from contemporary HER2-negative mBC trials to calculate the sample sizes required to power for OS compared with sample sizes actually used. Based on our findings, we discuss the challenges of designing trials in HER2-negative mBC, where powering for OS is sometimes unrealistic, unfeasible, or unfundable, with the aim of improving future trial planning and design.

Design

Clinical trials were identified from a systematic search of MEDLINE (details in Table S1) using the following criteria: randomized superiority trials; first-line chemotherapy or targeted therapy for HER2-negative mBC; >150 patients; meeting the primary efficacy objective (“positive” trials); and published in English in a peer-reviewed journal between January 1, 2000 and February 15, 2018.

The sample sizes required to power for PFS/TTP and OS were calculated retrospectively for each trial using the observed median PFS/TTP and median OS in the treatment groups for treatment effect, the actual recruitment period, and the actual total study duration (α=0.05, two-sided log-rank test, 80% power). Dropout rates were not considered for sample size calculation. nQuery Advisor (version 7.0; Statistical Solutions Ltd, Cork, Ireland) was used for sample size calculations. If information on the total study duration was missing, we chose a simple pragmatic assumption that the study period was one-third longer than the recruitment duration.

The retrospectively calculated sample sizes were summarized as a factor (x) relative to the actual sample size. x <1 would require x-fold fewer cases to show a significant benefit, whereas x >1 required x-fold more cases.

Results

Analysis data set

Thirteen trials met the selection criteria (Table 1). Of these, nine reported all information required for retrospective sample size calculation. In four reports (all published before 2006), insufficiently described study duration made it difficult or impossible to understand fully the statistical assumptions for sample size calculation. Only one trial had OS as the primary endpoint.

Table 1

Overview of trials included in the analysis. Trials are ordered according to date enrollment began (earliest first)

Trial	Recruitment period	Control arm (A)	Experimental arm (B)	Missing information for retrospective sample size calculation	Reported sample size calculation			Primary endpoint	Data cutoff for primary analysis	HR for PFS/TTP (95% CI)	HR for OS (95% CI)	Median duration of follow-up, months
					Assumed HR	Power, %	Events/patients
HEPI 01310	Sep 1990–Nov 1992	Cyclophosphamide, methotrexate, fluorouracil	Cyclophosphamide, epirubicin, fluorouracil	Yesa	0.73	80	155/420	TTP	NR	0.73 (0.59–0.92)	0.87 (0.70–1.10)	>20
SBG 940311	Feb 1995–Jan 1999	Epirubicin	Vinorelbine + epirubicin	Yes	NR	NR	NR	PFS	NR	0.75 (0.61–0.92)	NR	42 (A)/43 (B)
Jassem et al 200112	Nov 1996–Apr 1998	Fluorouracil, doxorubicin, cyclophosphamide	Doxorubicin + paclitaxel	Yesa	0.67	80	192/260	TTP	NR	0.74 (0.56–0.98)	0.68 (0.51–0.93)	29
von Minckwitz et al 200513	Nov 1996–Sep 2001	Cyclophosphamide, methotrexate, fluorouracil	Bendamustine, methotrexate, fluorouracil	Yes	NR	NR	NR/296	TTP	NR	NR	NR	NR
Bontenbal et al 200514	Mar 1997–Apr 2002	Fluorouracil, doxorubicin, cyclophosphamide	Doxorubicin + docetaxel	No	0.67	80	201/260	TTP	NR	0.67 (0.51–0.88)	0.70 (0.52–0.94)	27 (OS)/14 (TTP)
Albain et al 200815	Aug 1999–Apr 2002	Paclitaxel	Gemcitabine + paclitaxel	No	0.75	80	377/526	OS	NR	0.70 (0.59–0.85)	0.82 (0.67–1.00)	NR
E210016–18	Dec 2001–May 2004	Paclitaxel	Bevacizumab + paclitaxel	No	0.75	85	546/685	PFS	Feb 9, 2005 (PFS)/Oct 21, 2006 (OS)	0.48 (0.40–0.61)	0.87 (NR)	NR
Sparano et al 200919	Sep 2004–Nov 2006	Docetaxel	Pegylated liposomal doxorubicin + docetaxel	No	0.77	>80	485/720	TTP	NR	0.65 (0.55–0.77)	1.02 (0.86–1.22)	NR
RIBBON-1b,20,21	Dec 2005–Aug 2007	Capecitabine	Bevacizumab + capecitabine	No	0.75	80	405/600	PFS	Jul 31, 2008	0.69 (0.56–0.84)	0.88 (0.69–1.13)	16 (PFS)/23 (OS)
AVADOc,22	Mar 2006–Apr 2007	Docetaxel	Bevacizumab + docetaxel	No	0.70	80	430/669	PFS	NR	0.77 (0.64–0.93)	1.03 (0.70–1.33)	25
ATX (BOOG 2006-06)23	Jun 2007–Dec 2010	Paclitaxel + bevacizumab induction, then bevacizumab maintenance	Paclitaxel + bevacizumab + capecitabine induction, then bevacizumab and capecitabine maintenance	No	0.81	80	NR/303	PFS	Apr 26, 2013	0.52 (0.41–0.67)	0.92 (0.72–1.19)	41
IMELDA24	Jul 2009–Mar 2011	Docetaxel + bevacizumab induction, then bevacizumab maintenance	Docetaxel + bevacizumab induction, then capecitabine + bevacizumab maintenance	No	0.70	80	244/290	PFS	Oct 4, 2013	0.38 (0.27–0.55)	0.43 (0.26–0.69)	30 (A)/32 (B)
MERiDiAN25,26	Aug 2012–Dec 2013	Paclitaxel	Paclitaxel + bevacizumab	No	0.67	85	326/480	PFS	Nov 30, 2014 (PFS)/Apr 28, 2017 (OS)	0.68 (99% CI 0.51–0.91)	0.94 (0.75–1.18)	15 (PFS)/24 (A) and 23 (B) (OS)

Notes:

Total study duration missing.

Data reported for capecitabine cohort (anthracycline/taxane cohort not included because of heterogeneity of chemotherapy backbone).

Data reported for comparison of bevacizumab 15 mg/kg vs placebo (bevacizumab 7.5 mg/kg not included in this analysis as there was no significant improvement in the primary endpoint) but events/patients for reported sample size calculations include all three treatment arms as reported in the statistical design section of the publication.

Abbreviations: NR, not reported; OS, overall survival; PFS, progression-free survival; TTP, time to disease progression.

In most trials, the HRs showed a stronger treatment effect on PFS/TTP than OS (Figure 1). Four trials showed statistically significantly improved OS.12,14,15,24

Figure 1

Summary of PFS/TTP and OS HRs across trials. Vertical bars represent 95% CIs, except for MERiDiAN, which shows the 99% CI reported for this coprimary endpoint.

Abbreviations: OS, overall survival; PFS, progression-free survival; TTP, time to progression.

Retrospective sample size calculation

Table 2 shows the retrospectively calculated sample sizes required to show PFS/TTP and OS benefit with the observed data compared with the actual sample sizes. According to these calculations, six of 13 trials had sample sizes larger than required to demonstrate a significant PFS benefit. However, all would have required a larger sample size to demonstrate a significant OS benefit with the observed results. The increase in sample size ranged from 1.2-fold to 2,460-fold. In nine of the 12 trials with OS information, the calculated required sample size to demonstrate a significant OS benefit with the observed OS results was at least fivefold greater than the actual sample size. Figure 2 summarizes the sample size increase required to show a significant OS benefit with the reported data.

Table 2

Summary of trial outcomes

Trial	Total no of patients in trial	Observed median, months (arm A vs arm B)			Retrospectively calculated sample size		Factor (x)
		PFS	TTP	OS	PFS/TTP	OS	OS sample size/N	OS/(PFS/TTP) sample size
HEPI 01310	460	–	6.3 vs 8.7	18.2 vs 20.1	360	5,906	12.8	16.4
SBG 940311	387	8.2 vs 10.1	–	18.0 vs 19.1	788	11,988	31.0	15.2
Jassem et al 200112	267	–	6.2 vs 8.3	18.3 vs 23.3	506	1,402	5.3	2.8
von Minckwitz et al 200513	345	–	6.7 vs 8.2	–	792	–	–	–
Bontenbal et al 200514	216	–	6.6 vs 8.0	16.2 vs 22.6	1,022	476	2.2	0.5
Albain et al 200815	529	–	4.0 vs 6.1	15.8 vs 18.6	168	1,404	2.7	8.4
E210016,17	722	5.8 vs 11.3	–	24.8 vs 26.5	74	10,396	14.4	140.5
Sparano et al 200919	751	–	7.0 vs 9.8	20.6 vs 20.5	294	1,847,626	2460.2	6284.4
RIBBON-120,21	615	5.7 vs 8.6	–	22.8 vs 25.7	264	6,000	9.8	22.7
AVADOa,22	488	8.2 vs 10.1	–	31.9 vs 30.2	806	21,136	43.3	26.2
ATX (BOOG 2006-06)23	312	8.4 vs 11.2	–	23.1 vs 24.2	440	24,178	77.5	55.0
IMELDA24	185	4.3a vs 11.9a	–	23.7a vs 39.0a	32	214	1.2	6.7
MERiDiAN25,26	481	8.8 vs 11.0	–	25.8 vs 28.8	866	6,728	14.0	7.8

Notes:

Median values are not comparable with the other trials, first because PFS and OS were calculated from the time of randomization to maintenance therapy rather than the start of first-line therapy, and second because only patients with response or stable disease after induction therapy were included in the randomized population.

Abbreviations: OS, overall survival; PFS, progression-free survival; TTP, time to disease progression.

Figure 2

Additional patients required to show an OS benefita.

Note: aOne study19 is not shown on the figure as the numbers are so large (x=2,460.2, retrospectively calculated increase in sample size =1,846,875).

Abbreviation: OS, overall survival.

In all but one trial, a larger sample size would be required to show OS than PFS benefit. In 10 of the 12 trials with available OS results, the sample size required to power for OS was at least fivefold larger than that needed to power for PFS.

Discussion

Our analyses suggest that in the first-line HER2-negative mBC setting, it is a high hurdle to conduct a trial with adequate power to detect an OS improvement. Sample sizes to power for OS are usually extremely large and substantially larger than required to power for PFS.

The generally larger PFS than OS treatment effect in HER2-negative mBC is consistent with a recently reported study across various tumor types.27 Our findings are also consistent with reports in the literature suggesting that demonstrating an OS benefit is becoming increasingly unrealistic in contemporary clinical trials.2 A trial without crossover may answer the question of OS most cleanly. However, if the investigational agent has shown clear activity, the possibility of crossover has to be discussed. An Independent Data Monitoring Committee may feel obliged to stop a trial because of a clear signal, but it will then be impossible to conclude on the secondary endpoint of OS. Furthermore, a second trial of the same agent cannot be conducted after proven benefit because it is difficult to consent patients to be randomized between an experimental agent and a control arm known to be inferior. At times of rapid innovation, endpoints allowing prompt application of therapy optimization to standard clinical care are required. Therefore, it is important to determine whether progression-based endpoints are suitable for demonstrating utility. Available endpoints include PFS, TTP, and time to treatment failure. These allow earlier provision of study results and can be more sensitive indicators of treatment benefit because they are not affected by further treatment lines or crossover.28,29 Another benefit is comparability, as PFS is currently the most commonly used primary endpoint in Phase III trials. However, there is no clear evidence that PFS is a surrogate for OS.

In a recent analysis of PFS and OS in 58 randomized Phase II/III trials evaluating first-line systemic therapy for HER2-negative hormone receptor-positive mBC, several factors besides first-line therapy were reported to influence OS.30 These included prior endocrine therapy, prior (neo)adjuvant chemotherapy, types and lines of postprogression therapy, as well as disease characteristics associated with prognosis. Geographic region also influenced OS, presumably because of differences in healthcare patterns, management, and access in different countries.

In our analysis, the trial in which the actual sample size and the retrospectively calculated sample size for OS were most similar was IMELDA, a maintenance trial evaluating the addition of capecitabine to maintenance bevacizumab after bevacizumab/taxane induction therapy as a new treatment approach.24 In these patients already demonstrating chemosensitivity to induction therapy, switching to capecitabine before progression potentially anticipates development of resistance. In IMELDA, a significant OS benefit was demonstrated with a relatively small sample size but the retrospectively calculated sample size suggested that a larger sample size was needed. This is explained by differences in the methodology used for sample size calculation compared with the trial analysis method. Importantly, sample size calculation is only an estimation. Conversely, trial outcome is not proof and there is 5% error for demonstrating a significant benefit.

There appeared to be a gradual increase in median OS in the investigational arm over time (Table 2). Such cross-trial comparisons have obvious limitations, particularly when including maintenance vs treatment strategies. Nevertheless, median OS with experimental therapy remained <2 years in all trials evaluating chemotherapy alone, crossing the 2-year threshold only with the introduction of targeted therapy (bevacizumab). This presumably reflects not only treatment effect but also earlier diagnosis, better disease management, and an increase in the number of subsequent therapy options available. Indeed, similar increases in median OS can be seen in the control arm.

Given these challenges, how should we test effectiveness most appropriately in the first-line HER2-negative mBC setting? While Health Technology Assessment bodies worldwide accept PFS as a meaningful endpoint for clinical trials, progression-based outcomes are not recognized in Germany by the Institute for Quality and Efficiency in Health Services and the Joint Federal Committee (G-BA). These organizations focus on QoL, safety, OS, and morbidity, whereas PFS alone is not considered a meaningful endpoint, nor (in contrast with the clinical view) as an aspect of morbidity. The rationale for the G-BA’s stance is that superior progression-based outcomes evaluated by imaging are not considered to represent relevant benefits for patients. Patient relevance is accepted only if progression is recorded, for example, through symptoms perceptible to the patient. However, guidelines recommend assessing tumor burden every 8 weeks to allow prompt detection of metastatic progression, discontinuation of ineffective treatment with associated side effects, and prevention of tumor-associated symptoms that could be avoided by a change of treatment or strategy.31

Irrespective of surrogacy for OS, many believe that PFS is an important and relevant outcome for patients, associated with improved overall QoL, physical functioning, and emotional well-being.32 Extending PFS was ranked as more important than tumor shrinkage, limiting side effects, or treatment frequency in a questionnaire-based survey. Self-rated QoL was the highest after respondents had been told that their disease was responding to treatment. Therefore, progression-based parameters should generally be accepted as patient-relevant endpoints. Furthermore, changing therapy at progression affects patients’ lives. A new therapy may be associated with new side effects and/or a new treatment schedule and mode of administration. The consequences of disease progression are depressive reactions, grief, and despair. The possibility of tumor control is the most important reason for patients agreeing to systemic therapy.33 Fear of disease progression is the most commonly reported psychological burden in patients.34

We acknowledge that PFS is not a perfect endpoint, potentially being influenced by assessment intervals, choice of target lesions, and measurement technology. Some of these challenges are overcome by Independent Central Review, which is important for accepting PFS as an endpoint. Regarding the limitations of OS, several elegant biostatistical methods have been developed to account for crossover, such as inverse probability of censoring weighting (IPCW) and the rank-preserving structural failure time (RPSFT) model.35,36 However, these approaches are not flawless: IPCW assumes that there are no unknown or unmeasured confounding factors that could influence crossover and OS, whereas RPSFT assumes that the effect of treatment is constant across time and/or treatment lines. No single validated standard for statistical correction of crossover has been established in settings with long postprogression survival.

With the increasing use of maintenance therapies, eg, in ovarian cancer, alternatives to PFS and OS have emerged, including intermediate endpoints such as time to second progression or time to first or second subsequent therapy.37 These endpoints merit consideration in future trial designs in HER2-negative mBC. For trials evaluating endocrine therapies, time to first chemotherapy can also be a valuable endpoint, with clear patient relevance. Alternative endpoints used in other tumor types include quality-adjusted time without symptoms or toxicity and quality-adjusted PFS. However, it is essential that any endpoint is clearly defined and that the precise definition is used consistently across trials measuring the effect of treatment.38 Changes in molecular markers may also be of interest as surrogate endpoints.

In an attempt to quantify the medical benefits of new drugs, composite scales including pharmacoeconomic parameters have been introduced, such as the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS)39 and the American Society of Clinical Oncology Value Framework.40 A recent survey indicated that many trials demonstrating statistically significant improvements in efficacy did not meet the ESMO-MCBS clinical benefit threshold,41 particularly trials in the palliative setting.

The challenge of large sample sizes required to show OS improvement in clinical trials has been accompanied by increased interest in real-world data (RWD). In some cases, RWD evaluation has suggested improved OS from a treatment despite the lack of OS benefit in prospective randomized clinical trials.42 The main advantages of RWD are the very large sample sizes available for analysis and inclusion of broader, more heterogeneous patient populations with common comorbidities than is possible in a clinical trial, reflecting populations presenting in routine oncology practice. However, there are many limitations and even with sophisticated statistical methodology, RWD are exposed to important potential biases.43 Therefore, RWD can be viewed only as complementary to randomized clinical trials, not as an alternative.

A limitation of our analysis is the focus on chemotherapy and antiangiogenic agents. Numerous ongoing trials in the first-line HER2-negative mBC setting are evaluating cancer immunotherapy agents, which have a different mode of action and thus may exhibit different effects on PFS and OS. Furthermore, many of these trials focus on triple-negative mBC, a slightly more homogeneous population with shorter OS expectancy, shorter postprogression survival, and fewer treatment options after progression. All of these factors may affect the ability to demonstrate a significant OS effect, and, therefore, the patterns observed in our analysis may not predict future trials of cancer immunotherapy. Interestingly, several ongoing Phase III trials of immunotherapy in triple-negative mBC evaluate OS as the (co)primary endpoint. Another potential criticism is that the proportion of patients completing treatment is not taken into account. This information is missing in some of the publications, particularly in the older trials, but may have an impact on outcomes.

Conclusion

Although there are many reasons why OS is an attractive endpoint in trials of first-line therapy for HER2-negative mBC, it has limitations. Designing trials to test potential new treatments for HER2-negative mBC is challenging and requires a balance of regulatory acceptability, feasibility, and realistic medical assumptions to calculate sample sizes, which can be particularly difficult in heterogeneous study populations with long postprogression survival and heterogeneous subsequent therapies. The magnitude of OS benefit likely to be considered as clinically (as well as statistically) significant depends on disease biology and risk. For example, in patients with triple-negative mBC, a 3-month improvement in median OS is undoubtedly meaningful, whereas in hormone receptor-positive mBC, a larger (6-month) improvement may be required to provide convincing meaningful benefit. In the current environment amid soaring costs and fierce competition,44 it is probably unrealistic to aim for trials demonstrating statistically significant OS improvement in this setting, except for trials in very specific poor prognosis populations. Ultimately, identification of robust alternative endpoints reflecting relevant patient benefits remains critical.

Data availability

All data used for the analyses reported in this paper are taken from the cited publications.

Table S1

Search strategy for identification of eligible trials

Set#	Searched for	Results
S1	MESH.EXACT.EXPLODE(“Breast Neoplasms”)	260,430
S2	ti,ab((breast OR mamma*) NEAR/2 (cancer* OR carcinoma* OR tumo* OR neoplasm* or neoplasm* or malignanc*)) AND dstat.exact(“Publisher” OR “In Process” OR “PubMed not MEDLINE” OR “In Data Review”)	32,411
S3	s1 or s2	292,808
S4	ti,ab(metasta* or mBC or dissemin* or spread or advanced) AND s3	66,764
S5	MESH.EXACT.EXPLODE(“Neoplasm Metastasis”) AND s3	31,899
S6	s4 or s5	78,519
S7	s6 AND rtype.exact(“Randomized Controlled Trial”)	2,832
S8	all(randomized) AND s6	5,803
S9	all(placebo) AND s6	475
S10	s7 or s8 or s9	5,920
S11	(s7 or s8 or s9) AND rtype.exact(“Clinical Trial, Phase III” OR “Clinical Trial, Phase II”)	969
S12	(ti,ab(phase p/2 III[*1] or phase p/2 3[*1] or phase p/2 II[*4] or phase p/2 2[*4]) AND s10)	1,468
S13	(s12) and (dstat.exact(“Publisher” OR “In Process” OR “PubMed not MEDLINE” OR “In Data Review”))	126
S14	(s11 or s13)	1,095
S15	((s11 or s13)) and (pd(2014-2018))	348