Factors associated with successful phase III trials for solid tumors: A systematic review.

BACKGROUND: It is known that the success rates of phase III trials for solid cancers are low. The aim of this study was to investigate factors related to trial design and operation that were associated with the probability of the success of phase III trials for solid cancers based on the latest comprehensive data.
METHODS: Relevant clinical trials, started between September 2007 and December 2017, were retrieved from ClinicalTrials.gov. Then, variables related to the selected trials such as types of primary endpoint and duration of trial enrollment were collected from the literature and ClinicalTrials.gov. Based on the collected data, a multivariate logistic regression analysis was conducted to find factors associated with the successful results.
RESULTS: Four hundred phase III trials were found eligible for the study. Unsuccessful trials were 207 and successful trials were 193. As a result of multivariate logistic regression analysis, factors that presented a statistically significant relationship were primary endpoint (Odds ratio [OR]: 2.79 [95% CI: 1.59-4.89]), control arm (OR: 3.06 [95% CI: 1.39-6.73]), start year of trial (OR: 3.28 [95% CI: 1.87-5.77]), and duration of trial enrollment (OR: 0.77 [95% CI: 0.60-0.99]).
CONCLUSION: Type of primary endpoints (time-to-event endpoints other than overall survival), control arm (treatments with lower evidence level, placebo or best supportive care), and duration of trial enrollment (faster enrollment speed) were associated with phase III trial success.

Confidence interval

Disease free survival

Gastrointestinal

Immune checkpoint inhibitor

Non–small cell lung cancer

Overall survival

Odds ratio

Progression free survival

Standard of care

Introduction

Many medical needs remain unmet in the field of oncology and competition in the development of new anticancer agents is increasing. For drugs in the field of oncology, long periods of clinical development are required compared with other diseases due to difficulties in recruiting patients, longer time needed to establish efficacy and low success rates for phase III trials and regulatory approval [[1], [2], [3]]. In particular, the success rate of drug approval for solid cancers is known to be lower than that for hematological malignancies in the field of oncology [4]. Thus, there could be apprehensions about the increasing development cost and poor development efficiency for solid cancers, which have substantially impacted companies’ management.

Previous studies have reported on the phase III trial design factors (biomarker strategy, types of primary endpoint) that are associated with trial result and drug approval [5,6]. Because the trial design is an important factor that affects the success of each trial, these preliminary studies are considerable. However, these reports have limitations such as being descriptive without appropriate statistical methods, targets being mostly molecular-targeted drugs, and results being based on univariate analysis. In addition, the trial operation in phase III, other than the main trial design, might be another potential factor affecting the result. There was a review report of phase III trials that targeted first-line non–small-cell lung cancer (NSCLC) with a similar trial design (first-line, monotherapy, control arm of platinum-based chemotherapy, etc.) for the same drug class of anti-PD-1 antibodies, nivolumab, and pembrolizumab, where pembrolizumab was successful and nivolumab was not [7]. This may suggest factors other than drugs and the main trial design, such as operation methods, that potentially affects the trial result. In fact, to the best of our knowledge, there has been no study that focused on both trial design and operation method for phase III trials and examined their effects on the trial result. Furthermore, recently, rapid changes have been seen in the available anticancer agents, with the approval of several immune checkpoint inhibitors, and the environment surrounding the development of anticancer agents has been changing compared to a time when the previous studies were conducted.

This study aimed to investigate factors related to phase III trial design and operation that were closely associated with the probability of success of phase III trials for solid cancers based on the latest comprehensive data to provide new knowledge toward the improved plan and conduct of phase III trials.

Materials and methods

Trial selection

We used ClinicalTrials.gov as the search engine to extract clinical trials with a start date between September 27, 2007 (registration on ClinicalTrials.gov has been required for studies that were initiated after September 27, 2007) and December 31, 2017 (publication of the most recent phase III trial results could be expected) while considering the following conditions—condition or disease: oncology NOT leukemia NOT multiple myeloma NOT lymphoma; study type: interventional studies; study results: all studies; recruitment status: active, not recruiting/completed/terminated; study phase: phase 3; and study start: from September 27, 2007 to December 31, 2017. In addition, we used the inclusion criteria shown in Table 1 to select the target trials (randomized phase III trials for patients with solid cancer).

Table 1

Inclusion and exclusion criteria for trial selection.

Inclusion criteria

□ Trials that targeted patients with solid cancer; □ Randomized trials with at least 150 patients; □ Phase III trials (phase II/III trials are considered phase III trials).

Exclusion criteria

□ Trials for which the result cannot be obtained; □ Trials that do not involve drug intervention (e.g., surgery, radiation therapy, etc.); □ Trials where the primary endpoint does not include overall survival (OS) or other time-to-event endpoints such as progression free survival (PFS) or disease free survival (DFS); □ Trials on biosimilar or generic drugs; □ Trials of target patients who do not have solid cancer (precursor diseases and pathologies that lead to cancer).

Then, we first identified the availability of the trial outcome. When phase III trial papers were available through “Publications automatically indexed to this study by ClinicalTrials.gov Identifier” of ClinicalTrials.gov, we obtained them from relevant journals. If the papers were not available through the site, we used the ClinicalTrials.gov Identifier to identify relevant papers and abstracts from PubMed and Google Scholar. If there was information that could not be confirmed in the identified papers (including supplemental information) or if papers were not published but trial results and related information could be collected on ClinicalTrials.gov, we collected necessary data from ClinicalTrials.gov (including protocols and SAP referenced on Clinicaltrials.gov). We further selected target trials based on the exclusion criteria (Table 1).

Definition of variables

Definition of objective variables. When the primary endpoint of a phase III trial was statistically significant, we considered the trial to be “successful” and when it was not, we considered it to be “unsuccessful.” Statistical significance here meant the p value being below the predetermined significance level. If the predetermined significance level was unknown, we used significance level of 5% for classification. If the p value obtained in the trial was unknown, as long as the result was clear in the published information such as “did not meet primary endpoint,” “no significant difference,” etc., we referred to such information as well.

The standard of success was as follows: if there was a single primary endpoint of either overall survival (OS) or time-to-event endpoint like progression free survival (PFS) or disease free survival (DFS) other than OS, a statistical significance in the primary endpoint was considered successful, and if there were two or more primary endpoints including OS and other time-to-event endpoints, as long as there was a statistically significant difference in OS, it was considered successful.

The standard for unsuccessful was as follows: if there was a single primary endpoint of either OS or time-to-event endpoint other than OS, lack of a statistical significance in the primary endpoint was considered unsuccessful, and if there were two or more primary endpoints including OS and other time-to-event endpoints, lack of a significant difference in OS was considered to indicate unsuccessful.

Definition of explanatory variables. Factors related to phase III trial design were presence/absence of biomarker strategy identifying subjects with biomarkers, cancer type (gastrointestinal (GI) cancer, NSCLC, breast cancer, other), control arm (strong standard of care [SOC] [Category 1 in the latest national comprehensive cancer network {NCCN} guidelines], SOC [2A in the latest NCCN guidelines], other [best supportive care {BSC}, placebo, or other]), drug class (immune checkpoint inhibitor [ICI], targeted therapy, or other), regimen (monotherapy, combination), and primary endpoint (OS [including OS as a co-primary endpoint], Non-OS [time-to-event endpoint like PFS or DFS other than OS]). Factors related to phase III trial operation were sponsor (other, industry), start year of trial (2007–2011, 2012–2017), and duration of trial enrollment (if the available enrollment date was only the month, for convenience, we entered the first day of the month). Biomarker strategy, cancer type, drug class, regimen, primary endpoint, and sponsor were selected because they were also used in relevant previous studies [5,6,12]. Control arm, start year of trial, and duration of trial enrollment were selected for their possible effect on the phase III outcome and the data availability.

A biomarker strategy meant here that a biomarker was used for selecting targeted populations in the eligibility criteria or in the analysis for the primary endpoint [8,9]. Targeted therapy in the drug class was defined as drugs that block the growth and spread of cancer by interfering with specific molecules (molecular targets) that are involved in the growth, progression, and spread of cancer such as signal transduction inhibitors, apoptosis inducers, and angiogenesis inhibitors [10]. ICI in the drug class was defined as those blocking immune checkpoint proteins like CTLA4, PD-1, and PD-L1. The lists of biomarker strategy, drug class including targeted therapy and ICI, and control arm can be found in the Data Supplement.

Statistical methods

First, the number of successful trials and unsuccessful trials were tallied for each category of the explanatory variables, and success rates were calculated. Difference in the success rate among categories was examined using chi-squared test or Mann–Whitney U test depending on the types of data. In addition, we descriptively calculated the overall success rate and the success rate per start year of the trial.

To evaluate the relationship of the phase III trial design and operation with the success of phase III trials, we conducted a multivariate logistic regression analysis using the binary outcome (successful or unsuccessful) of the phase III trials as an objective variable and all the phase III trial design and operation factors as explanatory variables. We calculated the adjusted odds ratio for each explanatory variable and examined factors associated with the success of phase III trials. A p value of less than 0.05 was considered statistically significant. Considering the impact of missing values on the result, we also performed a sensitivity analysis excluding factors with many missing values. Chi-squared test, Mann–Whitney U test, and logistic regression analysis were performed using EZR on R commander version 1.41, October 1, 2019 [11].

Results

Trial selection and characteristics

The number of trials identified through the data extraction on October 10, 2019 via ClinicalTrials.gov was 2085. Among these trials, 902 trials were chosen through the inclusion criteria. After considering the exclusion criteria, 400 trials were selected for the analysis—207 unsuccessful trials and 193 successful trials (Fig. 1). The selected trials are considered as completed phase III trials for solid cancers with fairly good trial design and reliable trial outcome. The overall success rate was 48.3%. When divided by the start year of trial, the success rate was approximately 30%–40% between 2007 and 2011, whereas it was approximately 50%–70% between 2012 and 2017 (Fig. 2).

Fig. 1

Flow diagram of the trial selection.

Fig. 2

Success rate by the start year of the trial.

Imbalance was observed for characteristics excluding the drug class between the groups of successful and unsuccessful phase III trials. For the duration of trial enrollment, the median for unsuccessful trials was 2.47 years, which was about 0.74 years longer than that for successful trials (Table 2). Missing values were only confirmed in the duration of trial enrollment, and the rate of missing values was 14.8% (59/400).

Table 2

Characteristics of the trials.

Categorical variable	Number of unsuccessful trials (%) N = 207	Number of successful trials (%) N = 193	Success rate	P value
Biomarker strategy (%)
NO	167 (80.7)	129 (66.8)	43.6%	0.002
YES	40 (19.3)	64 (33.2)	61.5%
Cancer type (%)
GI cancer	67 (32.4)	39 (20.2)	36.8%	0.043
NSCLC	37 (17.9)	39 (20.2)	51.3%
Breast cancer	26 (12.6)	34 (17.6)	56.7%
Othera	77 (37.2)	81 (42.0)	51.3%
Control arm (%)
Strong SOC	66 (31.9)	33 (17.1)	33.3%	<0.001
SOC	91 (44.0)	85 (44.0)	48.3%
Otherb	50 (24.2)	75 (38.9)	60.0%
Drug class (%)
ICI	31 (15.0)	34 (17.6)	52.3%	0.184
Targeted drug	98 (47.3)	103 (53.4)	51.2%
Other	78 (37.7)	56 (29.0)	41.8%
Regimen (%)
Mono	66 (31.9)	94 (48.7)	58.8%	0.001
Combo	141 (68.1)	99 (51.3)	41.3%
Primary endpoint (%)
OSc	131 (63.3)	74 (38.3)	36.1%	<0.001
Non-OSd	76 (36.7)	119 (61.7)	61.0%
Sponsor (%)
Industry	152 (73.4)	165 (85.5)	52.0%	0.004
Other	55 (26.6)	28 (14.5)	33.7%
Start year of trial (%)
2007–2011	137 (66.2)	78 (40.4)	36.3%	<0.001
2012–2017	70 (33.8)	115 (59.6)	62.2%
Numerical variable	Median duration of enrollment for unsuccessful trials (Min, Max)N = 162	Median duration of enrollment for successful trials (Min, Max)N = 179
Duration of trial enrollment (year)	2.47 (0.48, 7.67)	1.73 (0.36, 9.25)	–	<0.001

GI: Gastrointestinal, NSCLC: Non–small cell lung cancer, SOC: Standard of care, ICI: Immune checkpoint inhibitor, OS: Overall survival.

Approximately 20 cancer types were included in the category of “other”. The main cancer types categorized as “other” were prostate cancer (33 trials), melanoma (25 trials), ovarian cancer (20 trials), renal cell carcinoma (14 trials), and head and neck cancer (13 trials).

Best supportive care, Placebo, less than category 2A in NCCN guidelines.

Including OS as the co-primary endpoint.

Time-to-event endpoint other than OS (e.g., PFS etc).

Results of logistic regression analysis

We calculated the adjusted odds ratio for each factor through multivariate logistic regression analysis. Factors that presented statistical significance were control arm (strong SOC vs. other) (odds ratio [OR]: 3.06 [1.39–6.73], p = 0.0053), primary endpoint (OS vs Non-OS) (OR: 2.79 [1.59–4.89], p < 0.001), start year of the trial (2007–2011 vs 2012–2017) (OR: 3.28 [1.87–5.77], p < 0.001), and duration of trial enrollment (OR: 0.77 [0.60–0.99], p = 0.040) (Table 3).

Table 3

Result of multivariate logistic regression analysis.

Factor	Reference		Odds ratio (95% CI)	P value
Biomarker strategy	NO	YES	1.19 (0.62–2.26)	0.60
Cancer Type	GI	NSCLC	1.58 (0.73–3.46)	0.25
		Breast Ca	1.43 (0.58–3.54)	0.44
		Other	1.34 (0.69–2.61)	0.38
Control arm	Strong SOC	SOC	1.63 (0.85–3.15)	0.14
		Other	3.06 (1.39–6.73)	0.0053*
Drug class	ICI	Targeted Drug	0.67 (0.28–1.63)	0.38
		Other	0.90 (0.36–2.28)	0.83
Regimen	Mono	Combo	0.96 (0.52–1.79)	0.91
Primary endpoint	OSa	Non-OSb	2.79 (1.59–4.89)	<0.001*
Sponsor	Other	Industry	1.28 (0.63–2.62)	0.50
Start year of trial	2007–2011	2012–2017	3.28 (1.87–5.77)	<0.001*
Duration of trial enrollment	–	–	0.77 (0.60–0.99)	0.040*

GI: Gastrointestinal, NSCLC: Non–Small-Cell Lung Cancer, SOC: Standard of care, ICI: Immune checkpoint inhibitor, OS: Overall survival, CI: Confidence interval.

*Statistically significant (P < 0.05).

Including OS as the co-primary endpoint.

Time-to-event endpoint other than OS.

Results of sensitivity analysis

We conducted a multivariate logistic regression analysis by excluding the factor of duration of trial enrollment, in which missing values were confirmed in 14.8% of data, as a sensitivity analysis. Primary endpoint, control arm, and start year of the trial, which presented a significant relationship with the probability of success in the main analysis, were also significant in the sensitivity analysis, showing consistency (Table 4).

Table 4

Result of multivariate logistic regression analysis (excluding factors of “duration of trial enrollment”).

Factor	Reference		Odds ratio (95% CI)	P value
Biomarker strategy	NO	YES	1.26 (0.71–2.22)	0.43
Cancer Type	GI	NSCLC	1.44 (0.73–2.87)	0.30
		Breast Ca	1.26 (0.57–2.81)	0.57
		Other	1.19 (0.66–2.17)	0.56
Control arm	Strong SOC	SOC	1.62 (0.89–2.94)	0.11
		Other	2.50 (1.24–5.05)	0.010*
Drug class	ICI	Targeted drug	0.96 (0.46–2.01)	0.92
		Other	1.01 (0.47–2.17)	0.97
Regimen	Mono	Combo	0.76 (0.44–1.31)	0.32
Primary endpoint	OSa	Non-OSb	2.78 (1.67–4.62)	<0.001*
Sponsor	Other	Industry	1.39 (0.76–2.54)	0.29
Start year of trial	2007–2011	2012–2017	3.05 (1.84–5.05)	<0.001*

NSCLC: Non–Small-Cell Lung Cancer, GI: Gastrointestinal, SOC: Standard of care, ICI: Immune checkpoint inhibitor, OS: Overall survival, CI: Confidence interval.

*Statistically significant (P < 0.05).

Including OS as the co-primary endpoint.

Time-to-event endpoint other than OS.

Discussion

In this study, the overall success rate of phase III trials of anticancer agents was 48.3%. In previous studies, it was reported to be 40%–50% [1,3,12], and our finding is consistent with these results. However, in this study, the success rate increased since 2012. The results of unsuccessful trials are not often published compared with those of successful trials; it was reported that in a previous study, 60% of unsuccessful trials were not published [13]. Alternatively, even when the trial results are published, there might be a delay until publication, resulting in a period of 2–3 years between the completion of a trial and publication. In particular, the time from trial completion to publication of unsuccessful trials was 1.3 years later than the time of successful trials [14,15]. Therefore, such publication bias may be a reason for the apparently different success rates between the years 2007–2011 (30%–40%) and 2012–2017 (50%–70%) for the analytical target trials. Even if this bias exists, we included “start year of the trial” as an explanatory variable in the multivariate logistic regression analysis to adjust the OR.

Shorter duration of trial enrollment was associated with successful phase III in this study. It is known that many factors play a role in low rates of trial participation such as financial barriers, lack of resources, uncertainty of risk-benefit ratio, and types of control arm [[16], [17], [18]]. Operational factors such as the number of trial sites, which is a controllable factor, might also affect the duration of trial enrollment. In addition, the result of trials in the previous phase might affect it. For example, higher response rate or other attractive efficacy data in the previous phase might lead to investigators’ higher motivation for patient enrollment. These might be confounding factors to this outcome. Although there could be many confounding factors, it is of great significance to actively consider and accelerate the enrollment so that the external medical environment such as approval of new subsequent therapy would not affect the original trial hypothesis. Accelerating enrollment is also useful in terms of getting innovative drugs and new indications to patients faster.

The type of primary endpoint was reported to be associated with the results of phase III trials in previous studies [3,6,14]. And in the present study, it was also a significant factor, both in the multivariate logistic regression analysis and the exploratory sensitivity analysis. This result would be generalized and show that trials tend to be unsuccessful when OS is set as a primary endpoint. OS is known to be easily influenced by subsequent treatment due to treatment switching in patients that were originally in the control arm [19]. This may be one of the reasons that OS as a primary endpoint was associated with unsuccessful phase III results. A trial design that has a time-to-event endpoint other than OS, such as PFS, would be preferable based on this study's result. However, when using an endpoint such as PFS, particularly in an open-label trial, an evaluation bias could exist. Moreover, it may not correlate to OS, the most reliable endpoint for evaluating anticancer agents [16]. When using only a time-to-event endpoint other than OS as the primary endpoint, a strategy to resolve or mitigate such problems and disadvantages is necessary. Evaluating OS as a key secondary endpoint might be an option. At the same time, we need to deepen our understanding as to the correlation between surrogate endpoints and OS.

In this study, we confirmed that trials using a control arm of placebo, BSC, or drug with lower evidence level were associated with a high probability of success compared with trials using a control arm of strong SOC. Unmet medical needs are expected to be higher in disease areas where there is no established standard therapy or where existing medications have a low evidence level. The result of the present study would encourage development of new medications in those areas. There are a limited number of treatments in disease areas with small number of patients such as rare cancers [20]. Also, available treatments are limited in highly heterogeneous types of cancer including those being resistant to conventional chemotherapy. It might be meaningful to explore the possibility of using real world data as historical control data in the development of new therapies for rare cancers, and to consider a trial design using biomarkers to select patient populations in order to easily show a clinically significant difference and to reduce the sample size.

Although the type of primary endpoint and control arm were identified as factors associated with the probability of success in phase III trials, they should be selected based on individual drug's characteristics and expected clinical positioning. The results of the present study should be referred as exploratory data for considering phase III trial design for solid tumors.

Previous studies have identified the biomarker strategy as a factor that is related to the result of phase III trials. However, it was not significant in this study. In previous studies, molecular-targeted drugs were the main target, and conclusions were made based on a univariate analysis without adjusting for relevant confounding factors [5,6]. This study comprehensively considered a variety of drug classes and utilized a multivariate logistic regression analysis. These may be the reasons for differences in the results. A biomarker strategy is an important approach to understand drug characteristics and improve its efficacy. In fact, in a meta-analysis of registration trials of anticancer agents approved by Food and Drug Administration and that of oncology phase I and phase II trials, it was reported that higher overall response rate and longer PFS was expected with a biomarker strategy and many drugs have been approved for patients in whom effects are anticipated based on biomarkers [8,[21], [22], [23]]. Thus, the said strategy was not denied. We need to further investigate the outcome of biomarker strategy in the future.

Limitations of this study were as follows. First, not all trial results during the target period were published and our result could likely be affected by publication bias. Second, evaluation of the evidence level of the control arm was based on the most recent NCCN guidelines rather than those available at the start of each trial. In addition, it is highly likely that there could be unknown confounding factors (such as the result of clinical trials in the previous phase) affecting the phase III outcome and consequently the results of the present study. Further studies are needed to clarify these issues.

Conclusion

We comprehensively collected the latest available data of phase III trials for solid cancers and conducted a multivariate logistic regression analysis to investigate factors associated with successful phase III trials from the viewpoints of trial design and operation. Factors that were identified to be related with the probability of success were the type of primary endpoint (time-to-event endpoint other than OS), type of control arm (placebo/BSC/control drug with low evidence level), and duration of trial enrollment (shorter duration). The results of the present study will serve as a useful reference in considering trial design and enrollment plan in future clinical trials.

Availability of data and material

All data generated or analyzed during this study are included in this published article [and its supplementary information files].

Declaration of interests

All authors have no conflicts of interest that are directly relevant to this research. Yasushi Otsuka is an employee of Alexion Pharma GK, Tokyo, Japan.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author contributions

Yasushi Otsuka: Conceptualization, Methodology, Data curation, Formal analysis, Investigation, Project administration, Visualization, Writing – original draft, Writing – review & editing, Masayuki Kaneko: Methodology, Validation, Writing – review & editing, Mamoru Narukawa: Methodology, Resources, Supervision, Writing – review & editing