Models of excellence: improving oncology drug development.

Simulations based on disease-progression models and phase II trial results can predict phase III results and have the potential to improve oncology drug development by informing end-of-phase II decisions (EOP2Ds). Many barriers impede effective use of modeling and simulation (M&S) for EOP2Ds in oncology: concerns about model validity, lack of access to M&S results and patient-level data, limited awareness of M&S among academic oncologists, and inexperience fitting M&S into the drug development timeline.

Introduction

Simulations based on disease progression models and phase II trial results can predict phase III results and have the potential to improve oncology drug development by informing end-of-phase II decisions (EOP2D). Many barriers impede effective use of modeling and simulation (M&S) for EOP2D in oncology: concerns about model validity, lack of access to M&S results and patient-level data, limited awareness of M&S among academic oncologists, and inexperience fitting M&S into the drug development timeline.

The statistician George E.P. Box famously wrote that “essentially, all models are wrong, but some are useful”.[1] He was making the point that, while models make predictions that never perfectly reflect reality, they can still be powerful tools for guiding decisions. Modeling and simulation (M&S) have been used to guide decision-making for non-oncology drugs in development, and have the potential to do the same for the largest active area of development, oncology drugs.

Disease progression models are mixed effects mathematical models that describe the relationship between a quantitative measure of disease status and time. These models support use of M&S to guide decision-making in drug development. The potential of this approach was recently demonstrated in studies of Alzheimer’s disease and rheumatoid arthritis, in which models based on validated scales of symptom severity enabled efficacy comparisons to existing drugs.[2,3] In both examples, M&S was used not only to support a go/no-go decision regarding further development, but also to guide the optimal design of the subsequent trial, saving resources and enhancing the probability of success.

In 2009, two landmark studies introduced disease progression models in oncology. In solid tumor oncology, the usual measure of disease status is tumor size, conventionally defined as the sum of longest diameters of target lesions measured on routinely performed cross-sectional imaging studies. Wang (and his FDA colleagues) used data from four registration trials in advanced non-small cell lung cancer (NSCLC) to develop drug-specific models of tumor size, as well as a drug-independent model linking overall survival (OS) to baseline prognostic factors and early change in tumor size.[4] Similarly, Claret et al used data from a phase II trial of capecitabine and a phase III trial of 5-fluorouracil/leucovorin in metastatic colorectal cancer to develop drug-specific models of tumor size, as well as a drug-independent model linking OS to baseline prognostic factors and early change in tumor size.[5] These studies implied that data from phase II trials could be used to predict accurately a range of outcomes for hypothetical phase III trials, and inform the design of potential phase III trials, providing valuable information for end-of-phase II decisions (EOP2D).

In the article by Claret et al[6] published in this issue of Clinical Pharmacology & Therapeutics, the authors present results of phase III trial simulations based on the aforementioned models of tumor size and overall survival in advanced NSCLC. They utilized data from a previously completed, 3-arm, randomized, open-label, phase II trial of carboplatin/paclitaxel (C/P) plus motesanib (an oral anti-angiogenic drug, administered on either a continuous or intermittent schedule) or bevacizumab in patients with advanced NSCLC. They simulated OS data for 700 patients receiving each of the following: C/P plus continuous motesanib (median OS 11.0 months); C/P plus intermittent motesanib (median OS 11.0 months); C/P plus bevacizumab (median OS 10.8 months); and C/P alone (median OS 9.3 months). The predicted hazard ratio (HR) for OS in hypothetical trials of C/P plus continuous motesanib versus C/P alone was 0.87 (95% CI, 0.71–1.1), with 60% of hypothetical trials having a statistically significant survival advantage for the motesanib arm (P<0.05). The actual phase III trial of C/P plus continuous motesanib versus C/P plus placebo (MONET1) demonstrated median OS of 13.0 vs. 11.0 months with an HR of 0.90 (95% CI, 0.78–1.04; P=0.14).

The authors conclude that “the results of our simulations…are consistent with the MONET1 results”. While this is certainly true if one uses the HR as the criteria for consistency, it is not necessarily true if one examines the OS data. The median OS for C/P plus motesanib in MONET1 (13.0 months) is greater than the upper bound of the 95% prediction interval (12.3 months) for the simulated results. Similarly, the median OS for C/P plus bevacizumab in the registration trial for bevacizumab (12.3 months) is greater than the upper bound of the 95% prediction interval (12.1 months) for the simulated results.[7] Thus, the model underestimates survival in patients treated with motesanib or bevacizumab in combination with chemotherapy. Since these anti-angiogenic drugs inhibit tumor growth by different mechanisms than cytotoxic chemotherapy, it is perhaps not surprising that the model may need to be refined to reflect more accurately their different treatment effects. As the authors point out, the disparity between observed and predicted survival might also result from known prognostic factors that are absent from the original model (e.g., histologic subtype, presence of brain metastases, age) and/or differences in covariate distributions between the phase II and phase III trials.

The authors should be commended for demonstrating the potential value of M&S to support EOP2D in oncology. Nonetheless, there is an important detail in this example that prevented M&S from being used to its maximum potential. The authors acknowledge that the simulations were performed “while MONET1 was ongoing”, which means that the decision to move forward with phase III development was made without these results available. MONET1 had a target accrual of 1,060 patients (530 per arm) based on having 80% power to detect an HR of 0.80 with a Type I error rate of 0.03.[8] However, the simulation results demonstrate a power of only 60% with 1,400 patients (700 per arm) and a Type I error rate of 0.05. In MONET1, the motesanib arm had a median OS that was two months longer than the placebo arm (13.0 vs. 11.0 months) with an HR of 0.90 but the results did not reach statistical significance, suggesting that the study was underpowered. A simulation-based power calculation prior to launching the phase III trial could have been used to predict the number of patients necessary to achieve 80% power, which would have substantially exceeded 1,400 patients. It is unclear whether the sponsor would have spent the resources necessary to complete the trial in these circumstances, especially since bevacizumab was already commercially available for use in combination with C/P.

What are the barriers to more successful applications of M&S to improve oncology drug development? One barrier is the accuracy and precision of the models, although the published models are an excellent starting point for future investigation. As models are applied to novel settings and performance is continually re-evaluated, opportunities to incorporate previously unrecognized covariates arise and the models improve. A second barrier is the paucity of publicly available information about M&S done in the private sector. The study by Claret et al was the result of collaboration between quantitative pharmacologists at a large pharmaceutical company and their external consultants. In the current drug development climate, divisions dedicated to “quantitative pharmacology” or “modeling and simulation” exist at most large pharmaceutical companies, and a multitude of consulting companies have sprung up to support their efforts as well as those of their smaller counterparts. Given the large number of individuals employed to conduct these sorts of analyses, the absence of more examples in the literature is striking. It is understandable that sponsors have little incentive to publish results from M&S prior to a potential registration application, but results should be published eventually.

A third barrier is the lack of public access to patient-level data from completed trials. To partially solve this problem, we propose the creation of a new public database for federally funded clinical trials data with submission required by NIH policy. The goal of this database would be to create a resource for M&S, as well as for re-analysis of completed trials. A model for how to do this successfully could be the dbGAP database of genotypes and phenotypes, which was established in 2007 by the National Center for Biotechnology Information to facilitate the progress of clinical applications of genetics research.[9]

A fourth barrier is the limited awareness of M&S in the academic oncology community. Non-industry sponsored oncology clinical trials around the world are typically government-funded and conducted through cooperative groups, with lead investigators at academic institutions. M&S could be used to prioritize proposed studies, as well as to improve the efficiency of such studies. A professional campaign to increase awareness of M&S in the academic oncology community and foster collaboration between oncologists and pharmacometricians would be a good first step, but resources to support the collection and verification of quantitative data will also be required.

A final barrier is inexperience with fitting M&S into the conventional drug development timeline. EOP2D are typically made very quickly after phase II results become available, while results of M&S based on these data might take several months to become available. One potential solution to this dilemma is the increased use of combined Phase II/III trials, in which an interim analysis is planned and conducted at the end of the phase II portion while accrual continues on the phase III portion. In this setting, M&S could be undertaken side-by-side with conventional statistical analyses and used to guide the decision about whether to continue forward with the phase III trial. These analyses could even be repeated at intervals with preliminary phase III data in order to inform changes to target accrual and possible decisions about early termination, similar to adaptive designs that are increasingly used in oncology trials.[10] Figure 1 schematically illustrates this proposed paradigm.

Figure 1

Phase II/III trials with an adaptive component during phase III. M&S: modeling and simulation.

Given the high failure rate of phase III trials in oncology and the current economic climate for funding new trials, it is more important than ever to use all tools available to optimize the efficiency and success rate of drug development. As Claret et al shows, M&S has the potential to enhance oncology drug development by informing EOP2D. However, M&S in oncology drug development will not fulfill its potential if we we do not recognize and overcome the significant but remediable barriers to success.