Model-Based Meta-Analysis for Multiple Myeloma: A Quantitative Drug-Independent Framework for Efficient Decisions in Oncology Drug Development.

The failure rate for phase III trials in oncology is high; quantitative predictive approaches are needed. We developed a model-based meta-analysis (MBMA) framework to predict progression-free survival (PFS) from overall response rates (ORR) in relapsed/refractory multiple myeloma (RRMM), using data from seven phase III trials. A Bayesian analysis was used to predict the probability of technical success (PTS) for achieving desired phase III PFS targets based on phase II ORR data. The model demonstrated a strongly correlated (R2 = 0.84) linear relationship between ORR and median PFS. As a representative application of the framework, MBMA predicted that an ORR of ∼66% would be needed in a phase II study of 50 patients to achieve a target median PFS of 13.5 months in a phase III study. This model can be used to help estimate PTS to achieve gold-standard targets in a target product profile, thereby enabling objectively informed decision-making.

Study Highlights

WHAT IS THE CURRENT KNOWLEDGE ON THE TOPIC?

✓ Depth of response is known to be associated with prolonged outcomes in multiple myeloma; however, the specific association between response rate and median progression‐free survival has not been investigated.

WHAT QUESTION DID THIS STUDY ADDRESS?

✓ This analysis evaluated whether a model‐based meta‐analysis framework could be used to predict progression‐free survival with reasonable confidence to estimate the probability of technical success in a phase III clinical trial in relapsed/refractory multiple myeloma, based on response rate data from phase II studies.

WHAT THIS STUDY ADDS TO OUR KNOWLEDGE

✓ This analysis provides the first demonstration of the feasibility of such an approach in RRMM, and highlights a strong correlation between overall response rate and median progression‐free survival based upon data from seven phase III studies on therapies with diverse mechanisms of action in this setting.

HOW THIS MIGHT CHANGE CLINICAL PHARMACOLOGY OR TRANSLATIONAL SCIENCE

✓ Utilization of this approach may allow early planning and initiation of phase III trials based on early phase II data, ultimately potentially facilitating more efficient drug development for multiple myeloma with increased probability of success.

Multiple myeloma (MM) is the second most common hematologic malignancy, accounting for an estimated 1.8% of all cancer diagnoses and 2.1% of deaths from cancer in the United States in 2017.1 Despite recent advances in patient outcomes, including significant improvements in overall survival (OS) following the introduction and increasing use of novel therapies,2, 3 MM remains a generally incurable disease,4 characterized by multiple relapses and substantial burdens on patients and caregivers.5, 6, 7, 8 Thus, there is an ongoing unmet need for additional novel treatment options that will extend patients’ progression‐free survival (PFS) within a specific line of therapy, as well as their OS.

To demonstrate that novel treatment regimens offer improved outcomes vs. currently available options, they must ultimately be investigated in randomized phase III clinical trials. However, conducting a phase III trial is a costly and lengthy process, and the failure rate of phase III trials in oncology is high.9 Furthermore, with improving outcomes in MM, the length of time required to obtain mature PFS and OS data from clinical trials in both the newly diagnosed and relapsed/refractory settings is increasing. In this context, ways of both improving the success rate of clinical trials and making faster‐but‐smarter decisions on whether to proceed to phase III investigation would be valuable, and quantitative predictive approaches could fulfil this unmet need.

It is well established in patients with MM that depth of response correlates with duration of response, which in turn is associated with prolonged disease control (PFS) and OS.10, 11 Additionally, quantitative relationships have been described between early measures of change in M‐protein (the circulating measure of tumor burden in MM) and OS in patients administered the proteasome inhibitor carfilzomib.12 However, the specific association between overall response rate (ORR) and median PFS based on data from antimyeloma agents across mechanisms of action has not been investigated. Determining such an association could potentially enable an early insight into anticipated outcomes based upon response rates, despite PFS data not yet being mature, which could in turn enable earlier decisions to be made with regard to the phase III clinical investigation of a regimen.

We considered a hypothetical scenario in which a phase III registration clinical trial is being planned, comparing a novel agent vs. a recognized standard of care in adult patients with relapsed/refractory MM (RRMM), with the planned primary end point of PFS. In this virtual scenario, there is currently an ongoing nonrandomized phase II study of the novel agent for which PFS is an end point; however, due to the long duration of the study (and perhaps the durable responses being achieved), data for the PFS end point will not be available for over 1 year. We investigated whether PFS could be predicted with reasonable confidence based upon response rates from the ongoing phase II study so that the probability of technical success (PTS) of the proposed phase III study could be estimated, in order to inform development decision‐making.

To do this, we developed a model‐based meta‐analysis (MBMA) framework to predict PFS from ORRs observed in clinical trials of patients with RRMM. Several phase III clinical trials of a range of different combination regimens for patients with RRMM have recently been reported, with available data including ORR and PFS. Using these data, we hypothesized that if we could demonstrate a correlation and characterize a relationship between response rates and PFS across these trials evaluating antimyeloma drugs with diverse mechanisms of action, there would be a reasonable probability of the relationship between response rates and PFS being disease‐specific and not treatment‐specific. Given such a finding, estimates of the probability for achieving a specified clinically relevant PFS benefit in a phase III study could then be made by predicting the phase III PFS results using response rates determined from a phase II study using a Bayesian framework.

Thus, using our hypothetical scenario, we illustrate herein the application of such an MBMA framework for estimating the PTS for achieving desired PFS targets in phase III trials from ORR observed in phase II studies in RRMM.

METHODS

Data used in the MBMA framework

A systematic literature review in RRMM was conducted by van Beurden‐Tan et al.13 We identified all phase III clinical trials in adult MM patients who had received at least one prior therapy that reported PFS data between 2014 and 2017.

MBMA

MBMA was used to predict the PTS for achieving a clinically meaningful phase III PFS benefit from phase II ORR data. Linear regression in R was used to determine the relationship between PFS and ORR (complete response (CR) + very good partial response (VGPR) + partial response (PR)) or VGPR or better (CR+VGPR), using available data from phase III trials in RRMM. Both ORR and PFS were treated as continuous variables in the MBMA. ORR was considered as a surrogate end point that could be used to predict long‐term PFS benefit. The relationship between PFS and ORR was built through the following model: where β0 and β1 represent intercept and slope estimated from linear regression, respectively. The following three steps were then used to apply the MBMA framework to predict median PFS from ORR and calculate the PTS of achieving a desired target median PFS. In step 1, it was assumed that the observed ORR is the truth, and then the distribution of ORR was determined using normal approximation correspondingly; for example, denoting the observed ORR from phase II as , then , where n 2 is the sample size of the phase II study. In step 2, the distribution of median PFS was derived based on ORR results using the linear relationship between PFS and ORR. Finally, in step 3 the probability of median PFS being greater than that required for the target product profile (TPP) and the minimal detectable value (MDV) was calculated based on the distribution of median PFS. This could therefore be interpreted as the PTS of the phase III study, assuming an adequately powered confirmatory trial.

Bayesian estimation of PFS and phase III PTS from phase II ORR

Per the MBMA, the relationship between PFS and ORR was established through linear regression. In order to account for the fact that the parameters are estimated based on small samples, this method utilized a prior distribution of parameters to account for the uncertainty of the parameter estimate from linear regression. The relationship between PFS and centralized ORR was built through the following model: where , is the average ORR, and and β1 represent intercept and slope estimated from this linear regression. Assuming prior distribution of as follows: the parameters estimated from linear regression can be used as a prior mean for , which is recognized as an empirical Bayesian approach. We assume σ2 follows an inverse Gamma distribution with parameters , i.e.,Different combinations of can be picked to reflect the confidence of a prior parameters assumption. It can be considered a noninformative prior if variance is large and an informative prior if variance is small.

Following the subsequent three steps per those in the MBMA method, this Bayesian analysis was used to predict the PTS for achieving a clinically meaningful phase III PFS benefit based on an early (phase II) benefit observed with ORR by accounting for the uncertainty of the parameter estimate.

In the context of predicting the PTS for achieving a clinically meaningful phase III PFS benefit based on phase II ORR data, there are two parameters of specific interest: one is and the other is . Assuming that the parameter vector has the following prior distribution: then is the estimated logit(ORR) from phase II and is asymptotically normally distributed as follows: where N is the sample size from phase II. is the estimated log(hazard rate) in the phase III study and is asymptotically normally distributed as follows: where E is the number of PFS events observed in phase III. Usually, the control arm treatment effect assumption in a randomized study, and the treatment effect assumption for the null hypothesis in a single‐arm study, with a small number of patients (N; phase II study) and a small number of events (E; phase III study), can be used to define treatment effect and variance in prior distribution. In this method, the predictive power was calculated according to equation (8) in Hong and Shi (2012).14

RESULTS

Clinical trials

A total of seven randomized, controlled, phase III trials in RRMM, covering data from 4,924 patients, were identified for inclusion in the MBMA (Table 1). These included the double‐blind, placebo‐controlled TOURMALINE‐MM1 study (NCT01564537) of ixazomib‐lenalidomide‐dexamethasone (ixazomib‐Rd) vs. placebo‐Rd,15 the open‐label ASPIRE study (NCT01080391) of carfilzomib‐Rd vs. Rd,16 the open‐label ENDEAVOR study (NCT01568866) of carfilzomib‐dexamethasone vs. bortezomib‐dexamethasone,17 the double‐blind, placebo‐controlled PANORAMA‐1 study (NCT01023308) of panobinostat‐bortezomib‐dexamethasone vs. placebo‐bortezomib‐dexamethasone,18 the open‐label ELOQUENT‐2 study (NCT01239797) of elotuzumab‐Rd vs. Rd,19 the open‐label POLLUX study (NCT02076009) of daratumumab‐Rd vs. Rd,20 and the open‐label CASTOR study (NCT02136134) of daratumumab‐bortezomib‐dexamethasone vs. bortezomib‐dexamethasone.21 These studies encompassed regimens incorporating a broad representation of the diverse mechanisms of actions of current MM therapies, including proteasome inhibitors, immunomodulatory drugs, histone deacetylase inhibitors, and monoclonal antibodies. Data from 12 treatment arms were used to evaluate the relationship between response rates and PFS; the two investigational treatment arms in the POLLUX and CASTOR studies were omitted from the MBMA, as the median PFS had not been reached at the time of data reporting when this analysis was conducted.

Table 1

Phase III studies in patients with RRMM included in the MBMA

Drug class of investigational agent	Study	Study arm	N	ORR (%)	≥VGPR (%)	Median PFS (months)
Proteasome inhibitor	TOURMALINE‐MM115	Ixazomib‐Rd	360	78	48	20.6
		Placebo‐Rd	362	72	39	14.7
	ASPIRE16	Carfilzomib‐Rd	396	87.1	69.9	26.3
		Rd	396	66.7	40.4	17.6
	ENDEAVOR17	Carfilzomib‐dexamethasone	464	77	54	18.7
		Vd	465	63	29	9.4
HDAC inhibitor	PANORAMA‐118, 38	Panobinostat‐Vd	387	60.7	27.6	11.99
		Placebo‐Vd	381	54.6	15.7	8.08
Monoclonal antibody targeted against SLAMF7	ELOQUENT‐219	Elotuzumab‐Rd	321	79	33	19.4
		Rd	325	66	28	14.9
Monoclonal antibody targeted against CD38	POLLUX20	Daratumumab‐Rd	286	93	74	N/a
		Rd	283	76	44	18.4
	CASTOR21	Daratumumab‐Vd	251	82.9	59.2	N/a
		Vd	247	63.2	29.1	7.2

HDAC, histone deacetylase; N/a, not available; Rd, lenalidomide‐dexamethasone; Vd, bortezomib‐dexamethasone.

Relationship between response rates and PFS

Linear regression analysis of the relationship between ORR and median PFS was conducted using the data from the seven phase III clinical trials in RRMM. The relationship is shown in Figure 1 a. The model demonstrated a strongly correlated (R2 = 0.84) linear relationship between ORR and median PFS, with the relationship characterized as PFS (months) = –23.46 + 0.56*ORR. A second model was constructed for the correlation between achieving at least a very good partial response (≥VGPR) and median PFS (Figure 1 b). Again, a linear relationship was determined, characterized as PFS (months) = 2.62 + 0.34*≥VGPR, but the correlation was not as strong (R2 = 0.75) as for the relationship between ORR and median PFS. In addition to ORR and ≥VGPR rates, median time to response was reported in six of the seven phase III clinical trials. However, it was not feasible to construct a model for this parameter, as median time to response was rapid in both arms of all six studies, at ≤4 months. Therefore, no conclusions could be drawn with regard to how time to response correlates with PFS. Further, median OS had not been reached at the time of this MBMA in a majority of the trials; thus, it was not feasible to construct a model to assess the relationship between response rates and median OS.

Figure 1

Relationships between (a) ORR and (b) ≥VGPR rate and median PFS, using data from seven phase III studies in patients with RRMM. The blue lines show the linear regression and the gray bands represent the 95% confidence intervals.

To illustrate the application of the MBMA framework for estimating the PTS for achieving desired PFS targets in phase III trials from ORR observed in phase II trials, the planned phase III assumptions were as follows: 300 PFS events, TPP PFS hazard ratio of 0.67 (median PFS improvement of 15 vs. 10 months) with 90% power and type I error of 5%, and an MDV TTP hazard ratio of 0.794 (median PFS improvement of 12.6 vs. 10 months). It was also assumed that phase II ORR data were obtained from 50 patients. As demonstrated in Figure 2, MBMA predicted that an ORR of ∼66% would be needed in a phase II study of 50 patients to achieve a median PFS of 13.5 months in a phase III study. Based on this example, the probability of achieving the TPP median PFS target of 15 months would be 34% and the probability of achieving the MDV median PFS of 12.6 months would be 60%.

Figure 2

Illustrative example of predicting PFS using ORR in the MBMA model. Probability of achieving target median PFS of 15 months is 34% (purple area) and probability of achieving minimum detectable PFS is 60% (blue area) for an ORR of 60% estimated in a study of 50 RRMM patients.

Demonstration of the predictive ability of this MBMA framework was performed by comparing predicted to observed PFS for ixazomib‐Rd in patients with RRMM from the phase III TOURMALINE‐MM1 study (NCT01564537) based on a model developed without the results of this study included. After removing data from both arms of TOURMALINE‐MM1 from model fitting, the new model resulted in a linear relationship between ORR and PFS characterized as: PFS (months) = –23.2609 + 0.5547*ORR, which is very similar to the final model reported based on all studies. Based on the model fitted without TOURMALINE‐MM1 data, if observed ORR in the treatment arm is 78%, predicted PFS is 20 months, which is very close to the final reported PFS of 20.6 months.15

Bayesian analysis for predicting PTS

Using the assumptions for the phase II and phase III studies described in the previous section, a Bayesian analysis was also done to predict the probability of achieving the TPP and MDV median PFS in a phase III study for various observed ORRs from a phase II study.

Based on the relationship established between PFS and centralized ORR (), the uncertainty of the parameter estimate was further accounted for in the PTS calculation as follows: which is a noninformative prior since a relatively large variance is assumed. If an informative prior is desired, can be considered. The noninformation prior was used.

The predictive probability of achieving the TTP and MDV median PFS in a phase III study is summarized in Table 2 and demonstrated in Figure 3 for various ORRs observed in a phase II study of 50 patients. The probability of achieving the TPP median PFS of 15 months, as well as the probability of achieving the MDV median PFS of 12.6 months, increased as the observed ORR from the phase II study increased from 64% to 74% ( Figure 3).

Table 2

Probability of achieving the target median PFS of 15 months and minimal detectable median PFS of 12.6 months in a phase III study for various observed ORRs (64–74%) in a phase II study

Observed ORR	Probability of achieving target PFS (median 15 months)	Probability of achieving minimal detectable PFS (median 12.6 months)
64%	17%	44%
66%	28%	58%
68%	42%	72%
70%	56%	83%
72%	70%	91%
74%	82%	95%

Figure 3

Predicted probability of achieving the target median PFS and minimal detectable median PFS in a phase III study for various observed ORRs in a phase II study.

An alternative Bayesian prediction approach was also used for predicting the phase III PTS based on an observed phase II ORR (Figure 4). It was assumed that the two parameters of interest, and , had the following prior distribution, which is a noninformative prior with eight patients from phase II, with ORR of 60%, and two patients from phase III with median PFS of 10 months. Usually, the control arm treatment effect assumption in a randomized study, and the treatment effect assumption for the null hypothesis in a single‐arm study, can be considered as treatment effect assumptions in prior distribution. In this example, the null hypothesis of an ORR of 60% in the phase II single‐arm trial was used as the prior assumption for ORR, and a median PFS assumption of 10 months in the control arm of the phase III trial was used as the prior assumption for PFS.

Figure 4

Bayesian predictive probability of achieving (a) minimal detectable median PFS and (b) target median PFS, based on different correlation strengths between ORR and median PFS.

Using the assumptions for the phase II and phase III studies described in the previous section, is asymptotically normally distributed as follows: and is asymptotically normally distributed as follows:

As demonstrated in Figure 4 a, this approach predicted that the probability of achieving the MDV median PFS is 38%, 47%, and 63% based on observation of an ORR of 70% in the phase II study by assuming a weak ), moderate ), and very strong ) correlation between ORR and PFS, respectively. If we assume there is a strong correlation ) between ORR and PFS, which is also supported by the MBMA, the predicted probability of achieving the MDV median PFS is 57% based on the observation of an ORR of 70% in the phase II study. Of note, as the strong correlation scenario is aligned with the observed strength of correlation from the MBMA (Figure 1 a), the alternative scenarios of weak to very strong correlations may be interpreted as sensitivity analyses. The predicted probability of achieving the TPP median PFS of 15 months is illustrated in Figure 4 b.

The normal approximation to binomial distribution was used in the PTS calculation. In general, normal approximation is adequate if sample size is large enough (n > 20) and probability P (e.g., ORR) is not near 0 or 1. One commonly used rule is that both n*p (i.e., number of responders) and n*(1‐p) (i.e., number of nonresponders) must be ≥5. Thus, the model utility for predicting PFS and the PTS might be extended to ORR data from smaller cohorts, such as expansion‐cohort results from a phase I study with a sample size of ∼30, as long as the ORR is between 15% and 85%. However, it may not be appropriate to extrapolate from the reported range since no data support the association between ORR and PFS outside this range. Thus, we recommend predicting PFS and the PTS using our models only when the observed ORR is 50–85%.

DISCUSSION

Model‐informed approaches are being increasingly applied in oncology drug development to enhance benefit vs. risk through optimal dosing, inform objective decision‐making in early clinical development, and enable winning designs of confirmatory trials, with the ultimate objective of increasing the PTS. To this end, the value of drug‐independent disease models linking short‐term measures of reduction in tumor burden to survival outcomes has been reviewed previously, with the vast majority of examples being in solid tumor malignancies.22, 23, 24 In RRMM, although a quantitative framework linking reduction in M‐protein to survival has been developed based on longitudinal patient‐level data on the proteasome inhibitor carfilzomib,12 this framework awaits qualification of drug/mechanism‐independence of the estimated relationship. The recent rapid evolution of the therapeutic armamentarium in this disease has resulted in the availability of antimyeloma drugs that exert their effects via a variety of mechanisms of action, ranging from targeted cytotoxic agents (e.g., proteasome inhibitors, histone deacetylase inhibitors) to immunomodulatory drugs and biotherapeutics (e.g., elotuzumab, daratumumab), with approved clinical regimens representing permutations of their combinations. Accordingly, a generalizable framework linking early measures of disease response to long‐term primary end points in registration‐enabling trials will be invaluable to guide the design and interpretation of early clinical data with a line of sight to proof‐of‐concept ahead of phase III initiation.25 To this end, we herein describe development of the relationship between ORR and PFS in RRMM using an MBMA framework constructed from publicly available data on contemporary phase III trials in this disease and illustrate application in a typical drug development setting.

The results of these analyses demonstrate the feasibility of developing a quantitative drug‐independent framework for RRMM to predict PFS based upon an end point (ORR) for which data are available earlier. Of note, two of the seven phase III trials (PANORAMA‐118 and ELOQUENT‐219) used European Group for Blood and Marrow Transplantation (EBMT) criteria26 to evaluate response, whereas the other five used International Myeloma Working Group (IMWG) uniform response criteria.27 The EBMT criteria are the older of the two sets of criteria, the IMWG criteria having evolved from them; both sets of criteria are internationally accepted, and previous studies have shown similar response rates using both.28 Of note, both arms of the two trials that used EBMT criteria provided data that were consistent with the overall findings. The findings show that MBMA approaches can provide an enhanced quantitative knowledge management framework that can be used to predict long‐term survival end points (such as PFS and OS) based on the response rates seen in early‐phase studies. As the analyses are based on results from large numbers of patients, the power to detect small but clinically significant effects is increased.29 Such approaches could allow early planning and potentially early initiation of phase III trials before all phase II end points have reached maturity, by estimating the PTS of achieving gold‐standard efficacy targets in the target product profile. This information would thereby enable objectively informed “go/no‐go” decisions at the molecule level as well as the cross‐molecule/portfolio level when comparing assets being developed for a common indication, ultimately offering the potential to speed up the development of novel therapies with improved benefits for patients.

MBMA is acknowledged as a valuable tool in the drug development process that is widely used for integrating data to enable informed and efficient development decisions,29, 30, 31 for example in dose–response analyses.32 The approach has been successfully used in other cancer types and in other therapeutic areas to assist in the development of new agents and, in combination with network meta‐analysis, to model the findings of head‐to‐head trials.33, 34, 35 For example, an MBMA was recently reported on median OS among patients with advanced hepatocellular carcinoma receiving antiangiogenic therapy that identified predictors of median OS and enabled clinical trial simulations of phase II comparative studies.35 Similarly, in rheumatoid arthritis, this approach was used to assess the relationship between short‐term and long‐term treatment effects measured by the American College of Rheumatology (ACR) 50 responses and to assess the feasibility of predicting 6‐month efficacy from short‐term data.34 In this study, the findings quantitatively supported the empirical clinical development paradigm of using 3‐month efficacy data to predict long‐term efficacy and to inform the probability of clinical success based on an early efficacy readout.

To our knowledge, this is the first application of an MBMA model in RRMM and for the prediction of long‐term outcomes in this setting based on early‐phase response rates. The MBMA model developed in these analyses is robust, being based on data from nearly 5,000 patients with similar disease characteristics enrolled in seven phase III clinical trials in RRMM.15, 16, 17, 18, 19, 36, 37, 38 The regimens used in these studies incorporate agents from all the main novel therapeutic classes used in the treatment of MM, including proteasome inhibitors, immunomodulatory drugs, histone deacetylase inhibitors, and monoclonal antibodies, and so the strong correlation identified between ORR and median PFS (R2 = 0.84) is supportive of a disease‐specific relationship for RRMM that is unlikely to be sensitive to the type of treatment. It was not feasible to extend our MBMA approach to examine the relationship between ORR and median OS due to the latter not being reached at the time of analysis in the majority of the phase III trials. OS is not an end point that is readily available in clinical trials in RRMM due to improving long‐term outcomes in this setting. Consequently PFS is the preferred primary end point in phase III trials and was thus chosen as the parameter of greatest relevance for this MBMA model.

It should be noted that the relationship between ORR and PFS estimated in this analysis for RRMM, using data derived from patients who had primarily received 1–3 prior lines of therapy, may not necessarily be applicable in different MM treatment settings, such as later in the disease course or in the newly diagnosed setting. A separate MBMA may be required if a different relationship exists between ORR and median PFS in a different patient population. For example, the relationship between percent reduction in tumor size at 8 weeks and survival is estimated to be steeper in first‐line vs. second‐line treatment of nonsmall‐cell lung cancer.39 A related challenge is that in order to determine a robust relationship across the disease continuum, the MBMA approach requires the availability of data from multiple studies in the same setting, in similar patients, and using a similar treatment paradigm, e.g., long‐term treatment or the treat‐to‐progression approach. In the absence of a substantial amount of data, the predictive power of an MBMA model may be limited, particularly given that MM is a highly heterogeneous disease with various different patient subgroups with differing prognoses and sensitivities to treatment.40 Thus, had patient and disease characteristics not been broadly similar across the seven phase III trials included in our MBMA, the predictive power of our MBMA may have been diluted due to greater patient and disease heterogeneity in the analysis population.

Nevertheless, the findings reported here highlight the applicability of this approach in the RRMM setting for estimating the phase III PTS based on early‐phase response data. While further validation may be required in order to continuously refine and increase confidence in this MBMA framework for RRMM, it is hoped that, ultimately, this and other complementary model‐informed drug development approaches will facilitate more efficient phase III drug development in terms of reducing time and costs, and increasing the likelihood of success.

Conflict of Interest

All authors are employees of Millennium Pharmaceuticals, Inc., a wholly owned subsidiary of Takeda Pharmaceutical Company Limited.