Estimation of renal cell carcinoma treatment effects from disease progression modeling.

To improve future drug development efficiency in renal cell carcinoma (RCC), a disease-progression model was developed with longitudinal tumor size data from a phase III trial of sorafenib in RCC. The best-fit model was externally evaluated on 145 placebo-treated patients in a phase III trial of pazopanib; the model incorporated baseline tumor size, a linear disease-progression component, and an exponential drug effect (DE) parameter. With the model-estimated effect of sorafenib on RCC growth, we calculated the power of randomized phase II trials between sorafenib and hypothetical comparators over a range of effects. A hypothetical comparator with 80% greater DE than sorafenib would have 82% power (one-sided α = 0.1) with 50 patients per arm. Model-based quantitation of treatment effect with computed tomography (CT) imaging offers a scaffold on which to develop new, more efficient, phase II trial end points and analytic strategies for RCC.

RCT Entities:   Population Interventions Outcomes

INTRODUCTION

Novel anticancer therapeutics fail in phase III trials with higher frequency than drugs developed in other fields of medicine (1-2). Because phase III trials are the most expensive to conduct, this high failure rate disproportionately increases the total costs for development of anticancer therapy. The need to modernize decision-making in the phase II to phase III transition has been recognized for nearly a decade(3), and the issue of improving phase II clinical trial design has garnered increasing attention(4-12).

Measuring the effects of treatment for solid tumors has been difficult to standardize(13-15). The most widely used method, the Response Evaluation Criteria in Solid Tumors (RECIST)(16), achieves reproducible results by estimating the quantitative change in tumor burden and then categorizing disease response as: complete response (CR), partial response (PR), stable disease (SD), or progressive disease (PD). The cut-points to separate disease response into the different categories were intended to minimize miscategorization and were first established when digital imaging was a new technology. At that time there were few available standard treatments for advanced solid tumors. A categorical system was efficient for screening new agents, especially drugs that were expected to shrink tumors within short timeframes. Now that solid tumors have become more treatable, and testing novel agents in combination programs or in subsets of patients has become more common, the inefficiencies of categorical tumor assessment are becoming less sustainable(11, 17).

Efforts to improve phase II trials are especially timely for RCC where 7 new agents in 6 years have been FDA-approved for metastatic disease(18-19). A promising approach to evaluate new therapies, recently demonstrated in lung and colorectal cancer(20-22), is to model the course of disease progression from tumor measurements collected in previous clinical trials. Relying upon the sum of the longest dimensions of the measured target lesions rather than RECIST categorization, this approach offers an efficient, adaptable method for evaluating treatments with the current commonly used, primarily CT-imaging data.

As each disease has different, characteristic, patterns of growth, we have developed a longitudinal tumor growth model for RCC based on measurements collected in the phase III trial that supported regulatory agency approval for sorafenib (23). We then validated this model with measurements collected in the phase III trial that led to approval of pazopanib(24). To determine the potential for evaluating therapeutics based on a quantitative assessment of treatment effect we performed simulations to assess the power of randomized phase II trials to detect different magnitudes of treatment effect above the estimated effect of sorafenib.

RESULTS

Longitudinal growth model of RCC

In the TARGET study sample used to generate the initial model, 36% of patients had one tumor lesion, 20% had two, and 44% had the sum of three lesions available for serial measurement. Initially, linear and exponential models were used to fit the placebo data for estimating tumor growth rate. The linear model described the data better than the two exponential models (Table 1), with lower value of Akaike information criterion (AIC) and lower objective function value (OFV). Models with a drug effect (DE) parameter were explored with the placebo data but no such parameter could be estimated (lack of convergence), suggesting that there was no average reduction in tumor size in the placebo arm.

Model	AIC	OFV
Placebo model
TS(t) = BASE + PR × t	7872.4	7868.4
TS(t) = BASE × e(PR × t)	7996.8	7992.8
TS(t) = BASE × tPR	8256.2	8252.2
Combined placebo and treatment model
TS(t) = BASE − DE × t + PR × t	18459.4	18453.4
TS(t) = BASE × e−DE × t + PR × t	18209.5	18203.5
TS(t) = BASE × t−DE + PR × t	18534.5	18528.5

The final model simultaneously estimated mean baseline tumor size (BASE), the tumor growth rate parameter (PR), drug effect (DE) in the sorafenib arm and residual variance using all available data (both placebo and treatment arms) of the TARGET study. To estimate sorafenib’s effect on renal cell carcinoma growth entailed an exponential treatment effect parameter. Estimated parameters and their relative standard errors (%SEs) are presented in Table 2. Note that NONMEM fits a non-linear, mixed effects model incorporating the fixed effects (BASE, PR, DE) as well as random, inter-individual variability in these parameters. The %SEs reported from NONMEM are all less than 25%.

Parameter	NONMEM		Bootstrap
	Estimate	%SE	Estimate	%SE
Tumor progress model
PR (mm/day)	0.158	11.6	0.158	15.2
BASE (mm)	62.7	2.81	62.7	2.73
DE (1/day)	0.00443	11.1	0.00443	13.9
Inter-individual variability	[shrink %]
PR (additive)	0.22 [17.9]	21.4	0.23	27.5
BASE (%)	71.5 [3.25]	5.0	71.4	4.80
DE (additive)	0.005 [46.1]	23.2	0.005	27.5
Residual error	[shrink %]
Proportional (%)	8.9 [30.8]	11.7	8.9	20.23
Dropout model
Baseline hazard	0.00806	14.6	-	-
Dropout hazard	0.00635	21.6	-	-

One artifact of hierarchical modeling approaches such as this one is known as Bayesian “shrinkage.”(25) Excessive shrinkage could lead to spurious identification of patient-specific factors associated with tumor progression. Although we did not pursue detailed evaluation of patient-specific factors, we calculated shrinkage in interindividual variability (η), which is reported in Table 2. As clinical and molecular differences among patients and their tumors respectively might lead to distinct observable differences in baseline tumor size and progression rate, mixture distributions were explored for BASE and PR parameters in the placebo dataset. However, these models were not stable and model diagnostics reflected significant mis-specification with the mixture distribution models. Thus our model makes the standard assumption that the random effects follow a normal distribution. Finally, bootstrapping was also used to evaluate the uncertainty in parameter estimates, with median values from 1,000 replicates shown (Table 2). The median parameter values resulting from the bootstrap procedure agreed with the estimates from our final model. This suggests that the parameters in the final model were reasonably well determined and the model was stable. From 2000 bootstrap runs, 1000 (placebo model) and 947 (combined placebo and sorafenib model) minimized successfully and were included in the bootstrap analysis.

Consideration of informative dropout

In these studies, investigators used their clinical assessments and RECIST criteria to determine when patients had disease progression. This means that the patients’ tumor measurement data could be censored without additional CT measurements (clinically determined progression), or with solely new, small, but difficult-to-measure lesions that indicate RECIST progression, but not contribute significantly to the sum of the longest dimensions of tumor measurements. The probability of dropout (termination of the longitudinal data due to disease progression or censoring) is therefore dependent on this unobserved tumor growth since the most recent RECIST measurements. To determine whether this non-random dropout affected the model we estimated the dropout hazard (β2), which indicated an increased hazard of dropping out with larger tumor size. The last observation was also considered to be related to withdrawal, but the dropout model did not support inclusion of both the last observation and the unobserved tumor size due to their high correlation. With lower objective function value and shrinkage, the unobserved predicted tumor size was included in the dropout model. However, this dropout model had minimal effect on the renal cancer growth model parameter estimates (Table 2). This suggests that the development of new metastatic lesions does not contribute to significant deviation from predictions of this linear growth model. The likelihood of disease progression is captured by the increasing sum of longest dimensions of the original target lesions.

Model performance and external validation

Selected cases of placebo-treated and sorafenib-treated patients demonstrate agreement between predicted and observed data (Figure 1). We performed an external validation of the model with data collected from 145 patients assigned to placebo in the randomized phase III trial of pazopanib. The visual predictive check (VPC) (Figure 2) depicts the distribution of the sum of the largest dimensions of all target lesions for each patient in the placebo arm over time. The median, 5th, and 95th percentiles of the tumor size distributions are captured by the model-based predictions through approximately 168 days (or 4 CT imaging evaluations). Importantly, the model captures the population “dip” in tumor size at the 95th percentile over the first, second, and third CT imaging evaluation intervals as the subjects with the largest tumor burdens drop-out.

Fig 1

Plots of representative individual patients. Placebo-treated patients are in panel A and sorafenib-treated patients in panel B. Circles are observed tumor sizes for the individual patient, solid lines are individual model-predicted tumor sizes, and dashed lines are model-predicted tumor sizes for the entire cohort of placebo- (A) or sorafenib-treated (B) patients.”

Fig 2

Visual predictive check of the joint tumor size/informative dropout model over 180 days. The circles represent the observed sum of largest dimensions of target lesions for each placebo-treated patient from the pazopanib trial at each timepoint. The solid line is the median of the simulated data, and the 90 percent prediction intervals are encompassed by the dashed lines.

Power calculations for drug effect (DE) parameter as an endpoint in future trials

The DE parameter in the tumor growth model provides a quantitative representation of the effect of drug on tumor growth on a continuous scale. By definition, the placebo takes a value of DE = 0. In a placebo-controlled trial the difference in drug effect = DEtreatment – 0. For sorafenib in the TARGET trial, DE = 0.00443/day. One way to interpret the model is that for a patient with a baseline tumor size of 62.7 mm in the placebo arm, on average the tumor will increase to 62.7+0.158*30 = 67.4 mm after one month, a 7.5% increase. In the sorafenib arm, after one month the expected size is 62.7e-0.00443*30+0.158*30 = 59.6 mm, a 5% decrease. This may not be as intuitive to the trialist as progression-free survival, but is a novel quantitative term to reflect the impact of drug in terms of % reduction in tumor size. In the TARGET trial, HR for disease progression was 0.44 (5.5 months sorafenib arm, vs. 2.8 months in the placebo arm). A larger DE will result in a lower HR.

As standard-of-care treatments become available and serve as comparators, clinical trials can be designed to detect evidence of improvement in DE over that achieved with sorafenib. Given the variance on the DE estimate for sorafenib from the 749 subjects for whom data were available, we assessed power for relative improvements in the drug effect over sorafenib in simulated, randomized, 2-arm, phase II trials having 50 patients per arm with a one-sided Type I error rate (alpha) = 0.1 (Figure 3). The ratio of DE for the comparator compared to sorafenib ranged from 1.2 to 2.0 for which power ranged from 30-91%. Randomized studies (50 patients/arm) had 82% power to detect a significant difference in drug effect when the comparator had a true DE 1.8 times (or 80% greater than) that of sorafenib, and 91% power to detect a significant difference in DE when the comparator had a true DE 2.0 times (or 100% greater than) that of sorafenib.

Fig. 3

Simulation-based power calculation for a randomized phase II trial of a hypothetical new comparator (could be new agent or new agent added to sorafenib) versus sorafenib, using model-estimated drug effect (based on CT scans conducted every 6 weeks up to 24 weeks) as the primary endpoint. We considered a hypothetical new drug with drug effect 1.2 to 2.0 times greater (i.e.., 20-100% greater) than that of sorafenib.

DISCUSSION

We have presented a modeling framework to quantify the magnitude of DE for therapeutics tested in advanced RCC, based on CT imaging measurements collected in phase III trials. The model was first developed with imaging data from the multicenter trial of sorafenib in RCC, and then validated with imaging data from the multicenter trial of pazopanib in RCC. Simulations using the DE model parameter suggest that a randomized 2-arm trial (with 50 patients/arm) would have sufficient power to detect new treatments with at least 80% greater DE than sorafenib. As with any endpoint, larger clinical trials would be required to detect smaller improvements in DE over sorafenib.

This initial modeling effort to approach RCC progression evaluation on a continuous rather than categorical scale takes into account the full set of longitudinal measurements. This model serves as a modifiable scaffold upon which new advances in predictive and prognostic factors could be incorporated to make clinical trials in advanced RCC more efficient. The model that best fit this large dataset relies on three fundamental parameters: the baseline burden of disease represented by the sum of the longest dimensions of the measurable lesions on CT imaging (BASE), and the combination of DE and PR, reflecting the growth rate determined by the changes in total tumor burden assessed over time. It is intuitively obvious that differences in “how much tumor you have” and “how fast it is growing” are the primary variables for assessing treatments in advanced disease. Indeed these are the same parameters used in disease progression models for colorectal cancer and non-small cell lung cancer (20-21). Based on readily collected empirical imaging data, this model doesn’t address the more theoretical, mechanistic considerations of tumor fractions growing and regressing during the course of treatment(26). However, models of this structure have been robust for application to both first-line and second-line treatment settings in other tumor types (27). The BASE parameter is typically larger in second-line than first-line treatment, but the linear progression rate is within the same range. As these empirical models have recently been developed, potentially important predictors such as prior therapy response, molecular markers, and co-morbid conditions have not yet been incorporated.

For the model in this paper, we referred to the exponential decay parameter as drug effect (DE). In the non-small cell lung cancer model, developed primarily with cytotoxic therapeutics, this term is called the shrinkage rate (SR)(20), and in the colorectal cancer model, developed solely with cytotoxic therapeutics, the term is drug constant cell kill rate (K)(21). This RCC model has been developed and validated solely with inhibitors of VEGFR2. These agents may decrease the measured size of tumors in part through killing tumor cells, but slowing the pace of further tumor growth, and diminishing intratumoral hydrostatic pressure are other means by which these drugs may decrease size and rate of progression of tumor masses (which can reduce associated symptoms). This therefore justifies the more general term, DE.

As for many different solid tumor types, the advancement of therapy for advanced RCC has entered a new era. Several distinct chemical entities have clear evidence of benefit compared with placebo(28), and more recently these have begun to be compared directly against established standard therapies (19). However, the need to detect evidence of clinically significant benefit for new drugs in disease subsets, and to detect benefit of combination treatments with factorially increasing combination possibilities requires new approaches to the conduct of phase II clinical trials(7, 17, 29). For RCC the disease progression model in this study may improve clinical research efficiency in a number of ways. First, as in the conduct of trials using RECIST, the solely required technology is readily available CT imaging. Second, studies to detect meaningful differences in treatment effects among randomly assigned groups are likely to require the fewest patients with a quantitative parameter endpoint for which inter-subject variance can be estimated and incorporated. Third, this and subsequent models can be shared and optimized publicly. For example, if a tumor tissue marker (such as VHL genotype) or novel imaging modality (such as PET or CT volume measurements) significantly improves the capacity to assess tumor growth response to treatment, that parameter could be readily incorporated into the model and used by investigators in subsequent trials.

The findings from this modeling study are consistent with the results from other efforts and support innovative approaches to new endpoints in RCC clinical trials. Simulations of data from completed phase III clinical trials of pazopanib(30) and sunitinib(31) have suggested improved power in phase II trials of RCC for alternative endpoints to standard response rate or PFS based on RECIST. These studies and our investigation, all conducted on VEGF-signaling pathway inhibitors, support the use of a change in tumor size measurement at a fixed early timepoint, such as 8 weeks as a more sensitive endpoint for testing new RCC therapeutics than the conventional RECIST response rate. By obviating the need for the longer term follow up of PFS, studies with this endpoint might also be completed more rapidly and inexpensively. Although a model-based endpoint of DE promises additional efficiency it would be simplistic and premature to compare its performance characteristics with PFS. As recently published in this journal(32), a complete modeling and simulation study, incorporating data from multiple drugs tested in a randomized trial would be needed to assess the relationship between the model parameter, DE, and PFS.

There is room for improvement upon this initial model-building effort. Due to the limitations of the studies in which imaging data were collected, the model describes tumor growth and not patient survival patterns. However, Heng, et al.(33) have suggested that progression-free survival and overall survival are well correlated. The variances not accounted for by the model include not only disease heterogeneity, but also the flaws in the data collection and transmission process. Although the model structure would be unlikely to change significantly, the parameter estimates would likely be more precise if measurements collected directly from images by a central reviewer were used rather than data extracted from radiology reports and transcribed by research associates. Finally, a recurrent criticism of models derived from sum of longest dimensions measures is that disease progression defined by new lesions is inadequately captured. Our assessment of an informative drop-out effect suggests this might be relevant and worth further development but does not impact the power of this model to detect meaningful treatment effects in a modest-sized cohort of patients randomly assigned to different treatment arms.

Although solid tumor progression models may share the same general structure across disease types, the estimated parameters of these equations will be disease, and disease sub-type specific. Analyzing data in two of the largest advanced RCC trials ever conducted, this model implies that further advances in development of RCC therapeutics may be made through serial studies that collect CT imaging measurement data to examine the quantitative estimates of drug effect. This model enables further refinements and improvements through testing of novel variables and contributions of novel technology. By relying upon routinely collected imaging data (CT scans), this model offers a novel opportunity to advance clinical research in RCC.

METHODS

Trials and Data

Model development was performed with tumor measurement data, as collected and reported by investigators from 749 patients (375 placebo arm; 374 sorafenib arm) enrolled in the phase III Treatment Approaches in Renal cancer Global Evaluation Trial (TARGET)(23). According to local standards for adherence to RECIST(15), target lesions were identified and measured by investigators. To represent the typical quality of smaller scale, locally performed, phase II trials, the complete, independently reviewed imaging data were not used. Model validation was performed with centrally reviewed data from the 145 patients assigned to placebo treatment in VEG105192, the multicenter phase III study of pazopanib(24). In both studies, imaging assessments were conducted every 6 weeks for the first 24 weeks and every 8 weeks thereafter. Conduct of these studies was approved by the University of Chicago Institutional Review Board before the project commenced.

Data extraction

To maximize informativeness of investigator-level data, we generated a program script in the R software environment (Supplemental Information) to extract tumor measurements in a format most consistent with RECIST 1.1(16). The script produced a dataset for import into the nonlinear mixed effects modeling software NONMEM (version VII, level 1, ICON, Ellicott City, MD, USA)(34). This dataset headings included: patient ID, visit, lesion #, lesion size, sum of lesion sizes by RECIST 1.1, sum of lesions, a flag for whether or not a lymph node measurement was incorporated, and the number of the visit. New lesions were identified, but not incorporated into the initial model building. Patients were also flagged for clinical progression without a confirmatory CT scan. If a lesion decreased to 0 and was not tracked consistently, that lesion was not included in the patient’s sum of longest diameters, unless it was the only baseline lesion.

Tumor growth and sorafenib effect model development

The longitudinal tumor size data (sum of the longest diameters of the serially tracked target lesions) were fitted with nonlinear mixed effects modeling. The model was built in three stages. Initially, both linear and exponential models were used to fit the placebo dataset for estimating tumor growth rate. The second step was to add the drug effect data and estimate the drug effect parameters with the placebo effects fixed. The final model simultaneously estimated baseline tumor size (BASE), the tumor growth rate parameter (PR), drug effect (DE) and residual variance on all available data (both placebo and treatment arms) of the TARGET study.

The first-order conditional estimation (FOCE) algorithm for calculating the likelihood in NONMEM was used for parameter estimation. The tumor progression model describes tumor size as a function of time and accounts for the natural growth of the tumor and drug effect (DE). Linear, exponential and power functions were used to fit tumor size data as candidate models describing both tumor shrinkage and tumor growth. Mixture distributions were explored to model potential subgroups with different tumor growth and/or responses to treatment. A model using a combination of an exponential-decay (shrinkage) and linear-growth (progression) was developed, as it was the best for describing the tumor growth and drug effect. Model structural selection was guided using the objective function value difference as well as the AIC. An alpha value for significance was set a-priori to alpha<0.01 (objective function value difference of 6.63 for one degree of freedom). Akaike Information Criterion was also used to adjust for degrees of freedom but the absolute difference was assessed and not statistical significance.

The final model fitted is TSi(t) is the tumor size at time t for the ith individual, BASE is the baseline tumor size, DEis the exponential tumor shrinkage rate parameter due to the drug effect and PR is the tumor growth rate constant, all for the ith individual. BASE,PR, and DEincorporate random deviations for the ith individual about the respective population mean parameter. For additional details of model development see Supplemental Methods.

The final tumor growth model parameter estimates were further evaluated internally using a nonparametric bootstrap. The resampling was performed 1000 times. The median values and the 2.5th and 97.5th percentiles of the parameter estimates obtained by this analysis were compared with those of the final model.

Model Validation and Power Assessment

External model validation

The model was evaluated with visual predictive check (VPC)(35-37) of the 145 placebo-treated subjects from the VEG105192 multicenter phase III study of pazopanib as described above(24). These placebo data were not included in the initial model development. The VPC was generated using 1000 simulations from the joint tumor growth and dropout model to assess the predictive performance. A graphical comparison was made between observed data and the model predicted median and 90% prediction interval (90% PI).

Power calculations with drug effect as the endpoint

Randomized, two-arm (50 patients per arm) phase II trials comparing sorafenib and a hypothetical comparator (with drug effect as the primary endpoint) were simulated (with 1,000 replicates) to estimate the power to detect a significant difference between arms (α= 0.10). Specifically, simulated data for tumor size at 6 weeks, 12 weeks, 18 weeks and 24 weeks were generated using the baseline tumor size and progression rate from the validated placebo model, with drug effects ranging from 0% to 100% greater than that for sorafenib [0.00443 (sorafenib effect), 0.005316, 0.006202, 0.007088, 0.007974, 0.00886 (twice the sorafenib effect)]. Simulated data used the same estimates of interindividual variability and residual error as fitted for sorafenib. Population estimates of drug effect for the two 50 patient arms in each simulated trial (hypothetical comparator vs. sorafenib) were compared using a z-test, and estimated power was the percentage of trials with a statistically significant difference between the two arms.