Validation of Progression-Free Survival as a Surrogate Endpoint for Overall Survival in Malignant Mesothelioma: Analysis of Cancer and Leukemia Group B and North Central Cancer Treatment Group (Alliance) Trials.

PURPOSE: The aim of this study was to investigate whether progression-free survival (PFS) can be considered a surrogate endpoint for overall survival (OS) in malignant mesothelioma.
MATERIALS AND METHODS: Individual data were collected from 15 Cancer and Leukemia Group B (615 patients) and 2 North Central Cancer Treatment Group (101 patients) phase II trials. The effects of 5 risk factors for OS and PFS, including age, histology, performance status (PS), white blood cell count, and European Organisation for Research and Treatment of Cancer (EORTC) risk score, were used in the analysis. Individual-level surrogacy was assessed by Kendall's tau through a Clayton bivariate Copula survival (CBCS) model. Summary-level surrogacy was evaluated via the association between logarithms of the hazard ratio (log HR)-log HROS and log HRPFS-measured in R2 from a weighted least-square (WLS) regression model and the CBCS model.
RESULTS: The median PFS for all patients was 3.0 months (95% confidence interval [CI], 2.8-3.5 months) and the median OS was 7.2 months (95% CI, 6.5-8.0 months). Moderate correlations between PFS and OS were observed across all risk factors at the individual level, with Kendall's tau ranging from 0.46 to 0.47. The summary-level surrogacy varied among risk factors. The Copula R2 ranged from 0.51 for PS to 0.78 for histology. The WLS R2 ranged from 0.26 for EORTC and PS to 0.67 for age.
CONCLUSIONS: The analyses demonstrated low to moderate individual-level surrogacy between PFS and OS. At the summary level, the surrogacy between PFS and OS varied significantly across different risk factors. With a short postprogression survival and a moderate correlation between PFS and OS, there is no evidence that PFS is a valid surrogate endpoint for OS in malignant mesothelioma. The Oncologist 2017;22:189-198Implications for Practice: For better disease management and for more efficient clinical trial designs, it is important to know if progression-free survival (PFS) is a good surrogate endpoint for overall survival in malignant mesothelioma. With a relatively large database of 17 phase II trials and 716 patients from Cancer and Leukemia Group B and North Central Cancer Treatment Group, we conducted statistical analyses and found that there is no evidence to suggest that PFS is a valid surrogate endpoint for OS for malignant mesothelioma. Future research work is needed to find alternative surrogate endpoints for OS. © AlphaMed Press 2017.

Introduction

Mesothelioma is a rare cancer that originates in the mesothelium and is often caused by exposure to asbestos [1]. The annual number of new cases in the United States is 2,000–3,000 [2]. Mesothelioma is frequently diagnosed at an advanced stage; therefore, patients have a poor prognosis. The median survival after surgery is approximately 1 year [1]. Most patients die in a short period after disease progression. The combination of a platinum agent (carboplatin or cisplatin) and an antifolate agent (pemetrexed or raltitrexed) is standard therapy for patients with advanced disease [3], [4], [5].

Overall survival (OS), the time from study registration or randomization to death, is the most important clinically meaningful endpoint for evaluating drug efficacy in oncology clinical trials. However, OS may not always be the optimal primary endpoint for efficacy assessment. It often requires a longer follow‐up period, and any potential effect of first‐line therapy may be attenuated by effective sequential therapies, making it difficult to evaluate the true treatment effect of the first‐line therapy. When newer agents display promising early efficacy, many trials allow the patients on standard therapy crossover to the investigational agent after progression. In such cases, the use of a surrogate endpoint, such as progression‐free survival (PFS), in place of OS is an appealing option. In this study, we assessed the capability of PFS as a surrogate endpoint for OS for malignant mesothelioma based on data from the clinical trials conducted by Cancer and Leukemia Group B (CALGB) and North Central Cancer Treatment Group (NCCTG) trials. CALGB and NCCTG are now part of the Alliance for Clinical Trials in Oncology.

A surrogate endpoint, such as PFS, can be used to evaluate treatment effect much like an established endpoint, such as OS, in evaluating an experimental treatment. In clinical cancer research, there is a general interest in validating or invalidating PFS as a surrogate endpoint for OS [6], [7]. Rather than pursuing a strict definition of surrogacy, such as the Prentice criteria [8], we adopted a more practical definition on surrogacy [9] and the corresponding statistical methods to validate the PFS surrogacy [10]. Specifically, surrogacy evaluation can take place at three different levels: individual level (correlation between PFS and OS within a patient), trial level (correlation between the treatment effect on PFS and that on OS), and summary level (correlation between the effect of risk factor on PFS and that on OS). When the data are from randomized clinical trials, the surrogacy of PFS as a potential endpoint to replace OS in evaluating new treatment is often based on the first two measures: individual‐level and trial‐level surrogacy. Indeed, such approach has been successfully used to validate PFS as a surrogate endpoint for OS in colorectal cancer [6], in which treatment effects based on PFS and OS were used for evaluation.

The primary goal of the current study was to evaluate the utility of PFS as a surrogate endpoint for OS in mesothelioma by using clinical trials data from CALGB and NCCTG. For rare disease, such as malignant mesothelioma, clinical trials data are mostly from small single‐arm trials; we are not able to estimate the trial‐level surrogacy, which directly answers whether a treatment‐induced change in PFS predicts a treatment‐induced change in OS. In such situations, we have to rely on the individual‐level correlation, including the summary‐level correlation based on risk groups, for surrogacy assessment. We feel that different approaches or measures for individual‐level correlation are warranted in surrogacy evaluation, although our approach is not for the purpose of estimating the trial‐level correlation.

Materials and Methods

Trials and Data

CALGB and NCCTG conducted 17 trials (15 CALGB and 2 NCCTG) on malignant mesothelioma between 1984 and 2009 (Table 1) [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27]. Follow‐up data reported before February 2014 were included. Of the 17 trials, 15 trials were single‐arm phase II trials and 2 were two‐arm phase II randomized trials. Each arm of the two‐arm trials was considered as an independent trial in our analysis. Because mesothelioma is a relatively rare disease, the number of patients enrolled in each trial was small, ranging from 10 to 67. Each participant signed an institutional review board‐approved, protocol‐specific informed consent in accordance with federal and institutional guidelines. Data were collected by CALGB and NCCTG statistics and data centers.

Table 1.

Summary of 17 mesothelioma trials from Cancer and Leukemia Group B and North Central Cancer Treatment Group

Abbreviations: CI, confidence interval; G‐CSF, granulocyte colony‐stimulating factor; GM‐CSF, granulocyte‐macrophage colony‐stimulating factor; OS, overall survival; PFS, progression‐free survival.

The following data were extracted for each trial: gender, age at registration, Eastern Cooperative Oncology Group (ECOG) performance status (PS), previous radiation status, previous chemotherapy status, previous surgery status, weight loss, mesothelioma histology, white blood cell count (WBC), date of last observation (or date of death, if patient died), survival status, date of progression, progression status, baseline primary tumor, baseline nodal involvement, and baseline distant metastasis.

PFS Surrogacy Evaluation with Single‐Arm Trial Data

In this analysis, PFS is defined as the time from study registration or randomization to disease progression or death from any cause, whichever comes first. The common approach of validating PFS surrogacy is to use data from large randomized clinical trials in which patients were randomly assigned to the same (or similar) two treatment arms. Most CALGB and NCCTG mesothelioma trials are single‐arm trials, and the treatments varied from trial to trial. Hence, the common approach of evaluating PFS surrogacy on treatment effect is no longer applicable. Risk factors, such as ECOG PS and histology, are known prognostic factors for PFS and OS in mesothelioma [28], [29]. Like treatment effect, these risk factors affect both PFS and OS. We postulated that these risk factors affect PFS and OS in a manner similar to that of treatment effect (e.g., pemetrexed plus cisplatin vs. cisplatin alone). Therefore, for the data of mostly single‐arm trials, it is reasonable to use the risk factors, such as PS and histology, to substitute treatment effect for PFS surrogacy evaluation. To verify this idea, we examined the effect of treatments (experimental vs. control) on PFS and OS and the effect of PS (1/2 vs. 0) by using the extensive‐stage small cell lung cancer (ESCLC) data assembled for another study [30], in which both treatment effect and risk factors were available. As seen in the supplemental online data (Appendix A), PFS surrogacy at the individual level and the summary level is very similar for the PS effect and the treatment effect in the ESCLC dataset. Although the summary‐level analysis based on the risk factor effect is consistent with the trial‐level analysis based on the treatment effect in the example of small cell lung cancer, there is a fundamental difference between the surrogacy valuation based on true treatment effect and that based on risk factor effect.

To investigate PFS surrogacy induced by risk factors on PFS and OS, we focused on the following risk factors identified in two well‐known prognostic models for mesothelioma [28], [29]: ECOG PS [31], histology, age, and WBC. All patients were classified into binary risk groups: PS (≥1 or 0), age (≥70 or <70 years), histology (nonepithelial or epithelial), and WBC (>0.83 or ≤0.83 × 109/L). For the surrogacy analysis, we also considered European Organisation for Research and Treatment of Cancer (EORTC) risk score [28] as a risk factor; this is a composite score of several risk factors, including WBC, PS, and histology. On the basis of EORTC risk score, patients were classified into a good prognosis group (≤1.27) and a poor prognosis group (>1.27).

Statistical Analysis

Survival Analysis.

The distributions of PFS and OS were estimated by using the Kaplan‐Meier method. Estimation procedures and hypothesis tests were stratified by the classification criteria for trials (Fig. 1). The effects of risk factors on PFS and OS were estimated as logarithms of the hazard ratio (log HR), respectively: log HROS and log HRPFS log, through Cox proportional hazards model with the risk factor of interest as a single predictor. The log HR is approximately equal to the risk reduction for small effects. The HR estimates were used for assessing the agreement between PFS effect and OS effect at the summary level. There was no intention to conclude whether any specific risk factor is statistically significant at either the summary level or overall. For this reason, the issue of multiplicity was not addressed.

Figure 1.

Kaplan‐Meier curves of OS and PFS.

Abbreviations: EORTC, European Organisation for Research and Treatment of Cancer; OS, overall survival; PFS, progression‐free survival; WBC, white blood cell count.

Surrogacy Criteria Analysis.

To evaluate whether PFS is a valid surrogate endpoint for OS, we aimed to demonstrate that (a) PFS was strongly associated with OS (individual‐level surrogacy) and (b) the treatment effect (in this case the risk factor effect) for PFS and for OS was strongly associated (summary‐level surrogacy). The association of log HROS and log HRPFS log was assessed to evaluate the summary‐level surrogacy. A weighted least‐square regression for log HROS and log HRPFS log was performed to evaluate the relationship of OS and PFS, with weights equal to the sample size of the trial [6], [7]. Another approach used a bivariate survival model derived by Buyse et al. [10], in which both summary‐level and individual‐level surrogacies between PFS and OS were estimated. The bivariate survival model was constructed from a Clayton Copula with Weibull marginal distribution. For individual‐level surrogacy, the association parameter δ was transformed into a scale parameter, Kendall's tau, which belongs to the interval [−1, 1]. A value of tau near 1 suggests a strong positive association between PFS and OS, whereas a value near −1 indicates a negative association. Finally, a leave‐one‐out cross‐validation was performed to assess the surrogacy between PFS and OS [32]. Specifically, we calculated the mean squared prediction error (MSPE) by using the leave‐one‐out cross‐validation procedure by averaging the square of the difference between observed HROS and predicted HROS over all trials. The percentage of patients with one risk factor missing ranged from 0% for age to 14% for EORTC risk score; the patients with missing risk factors were excluded from the analysis of that particular risk factor. Approximately 15% patients had received prior chemotherapy at the enrollment of these trials. The primary analysis included all patients, but a sensitivity analysis including only patients who received no prior chemotherapy was also conducted. All statistical analyses were done in SAS 9.4 (SAS, Cary, NC, USA, http://www.sas.com) and R 3.3.2 (R core team, Vienna, Austria, http://www.R-project.org).

Results

Patient Characteristics and Outcomes

The data were from the 716 patients enrolled in the 17 mesothelioma clinical trials. As shown in Table 2, the median age was 65 years (range, 23–92 years); 81% of patients were male. For the entire cohort, the median follow‐up time was 42 months. The median PFS and OS were 3.0 months (95% confidence interval [CI], 2.8–3.5 months) and 7.2 months (95% CI, 6.5–8.0 months), respectively. Two thirds (67.2%) of the patients had epithelioid histology and 25.8% of the patients had a PS score of 0. EORTC risk score was calculated as follows: risk score = 0.55 (if WBC >8.3 × 109/L), 0.60 (if performance status ≥1), 0.52 (if histologic diagnosis is probable or possible), 0.67 (if histologic subtype is sarcomatous), and 0.60 (if gender is male). A total of 616 (86%) patients had all risk factors observed, and we were able to compute the EORTC risk score; 39.3% of these patients were classified into the poor prognosis group (>1.27) and 60.7% were in the good prognosis group (≤1.27).

Table 2.

Summary of baseline patient characteristics

aGood prognosis if EORTC ≤1.27 and poor prognosis if EORTC >1.27.

Abbreviations: CI, confidence interval; EORTC, European Organisation for Research and Treatment of Cancer; OS, overall survival; PFS, progression‐free survival; PS, performance status.

Effects of Risk Factors on PFS and OS

Figure 1 shows the PFS and OS curves of all patients for five risk factors, including EORTC risk group (poor or good prognosis), PS (≥1 or 0), age group (≥70 or <70 years), histology (nonepithelial or epithelial) and WBC (≥8.3 or <8.3 × 109/L). Each plot represents Kaplan‐Meier curves of PFS and OS by different levels of a risk factor. For all patients, the HRs for EORTC risk group (poor and good prognosis) were 1.36 for PFS (95% CI, 1.15–1.6) and 1.67 for OS (95% CI, 1.41–1.97). The median PFS and median OS were 2.6 months and 5.3 months for the poor prognosis group, respectively, and 3.5 months and 8.6 months for the good prognosis group. A clear separation of curves is evident in each plot for both endpoints, and the benefit ordering is consistent between OS and PFS, which suggests an association between OS and PFS in malignant mesothelioma.

We also calculated the summary‐level PFS and OS HRs for each risk group. Supplemental online Figure 1 displays the estimated HRs and their 95% CIs by each trial for EORTC risk group. Similar forest plots (supplemental online Fig. B2–B5) for other risk groups can also be found in supplemental online Appendix B. C9631 was excluded from these plots because with only 10 patients its HR for risk factors was inestimable or unreasonably big. These plots indicate moderate agreement between PFS HRs and OS HRs in terms of direction and effect size across risk factors. A separate set of analyses was conducted to include only the 607 patients who did not receive prior chemotherapy at enrollment. Supplemental online Figure 1 shows similar Kaplan‐Meier plots for PFS and OS for patients receiving no prior chemotherapy.

Figure 2.

WLS regression of log HROS on log HRPFS.

Abbreviations: log HROS, logarithm of the hazard ratio for overall survival; log HRPFS, logarithm of the hazard ratio for progression‐free survival; WBC, white blood cell count; WLS, weighted least‐square.

Individual‐Level Surrogacy

The Clayton bivariate Copula survival (CBCS) model was used to evaluate individual‐level surrogacy. Across all trials, Kendall's tau ranged from 0.45 to 0.47 for all risk factors, indicating that PFS and OS were only moderately correlated at the individual‐level (Table 3). Supplemental online Table C1 shows similar results for the patients receiving no prior chemotherapy.

Table 3.

Results from Clayton bivariate Copula survival model

Copula R2 is a measure for summary‐level surrogacy based on Clayton bivariate Copula survival model.

Kendall's tau is a measure for individual‐level surrogacy.

Good prognosis if EORTC ≤1.27 and poor prognosis if EORTC >1.27.

Abbreviations: CI, confidence interval; EORTC, European Organisation for Research and Treatment of Cancer; PS, performance status; WBC, white blood cell count.

Summary‐Level Surrogacy

Two measures were used to assess summary‐level surrogacy. According to the modeling of the CBCS model, the summary‐level surrogacy was moderate across all the risk groups, with Copula R2 values ranging from 0.51 for age to 0.78 for histology (Table 3).

For each risk group, a weighted least‐squares (WLS) regression model can be fitted with summary‐level HRs and weighted with the size of the trial (Fig. 2). For example, the linear regression equation for EORTC risk group was as follows (Fig. 2, Table 4):

Table 4.

Results from weighted least‐square model

C9631 was removed from WLS analysis because with only 10 patients it made the hazard ratios for several risk factors inestimable or unreasonably big.

Intercept estimated from WLS regression model.

Slope estimated form WLS regression model.

WLS R2 is a measure for summary‐level surrogacy based on WLS model.

MSE of β1 based on WLS model.

MSPE of β1 based on leave‐one‐out validation.

Abbreviations: CI, confidence interval; EORTC, European Organisation for Research and Treatment of Cancer; MSE, mean square error; MSPE, mean square prediction error; PS, performance status; WBC, white blood cell count. WLS, weighted least‐square.

This model indicates that the risk increase associated with poor prognosis group was approximately 49% lower on OS than on PFS. The closer the slope of the regression line is to 1, the better the agreement between the effect of that risk group on PFS and OS.

The WLS R2 ranged from 0.26 to 0.67; the lowest was for EORTC and PS and the highest was for age (Table 4). In the WLS analysis, we excluded one obvious outlier that significantly affected the goodness of fit if the data point were included. C9631 was removed from WLS analysis because with only 10 patients it made the HRs for several risk factors inestimable or unreasonably big. The shrinkage of PFS surrogacy for OS was further evaluated by using leave‐one‐out cross‐validation. The values of MSPE for all risk factors are generally bigger than the corresponding mean square error values. Such shrinkage of prediction accuracy on new data is expected [32]. The lowest MSPE was for age (0.07) and highest was for WBC (0.15) (Table 4). Similar results were observed for summary‐level surrogacy for the patients receiving no prior chemotherapy, as seen in supplemental online Figure C2, Table C1, and Table C2.

Summary‐Level Surrogacy Based on Artificially Paired Treatment Arms

As seen in Table 1, two studies (C8435 and C9131) are randomized phase II trials and the cohort consists of 138 patients. The small sample size prevented us from carrying out the analysis on summary‐level surrogacy. For a purely exploratory exercise, we computed summary‐level correlation by creating artificial pairs of treatment arms. Specifically, the two arms of the randomized trial each formed a pair. C9631 was excluded because it had only 10 patients. The rest of 14 single‐arm trials formed 7 pairs based on the closeness between the means of the EORTC risk scores. For each pair, the trial/arm with better PFS was coded as the experimental arm, and the trial/arm with worse PFS was coded as the control arm. On the basis of on the bivariate Copula model, the individual‐level tau is 0.46 and the trial‐level correlation Copula R2 is 0.77. On the basis of the weighted least square model, the summary‐level correlation WLS R2 is 0.52. These estimates again suggest a moderate PFS surrogacy for OS when data from single arm trials were paired into treatment groups by the proximity of their EROTC scores.

Discussion

The goal of this project was to evaluate whether PFS can be used as a surrogate endpoint for OS in malignant mesothelioma clinical trials. Individual patient data from a total of 716 patients enrolled in 17 CALGB and NCCTG phase II trials were used to evaluate the surrogacy of PFS for OS at both the patient and summary levels. Our analysis suggests that PFS is associated with OS, but the measures on both individual‐level and summary‐level surrogacy are not strong enough to allow the use of PFS as a surrogate endpoint for OS, regardless of which risk factor was considered.

In the individual‐level analysis, the concordance between PFS and OS, as measured by Kendall's tau values, ranged from 0.45 to 0.47, suggesting a low to moderate correlation. In the summary‐level analysis, the proportion of OS effect explainable by the corresponding PFS effect was measured by Copula R2 and WLS R2. Although in the bivariate Copula model, the Copula R2 was moderately high for some risk factors, such as histology (0.78) and WBC group (0.76), it was low for the remainder, ranging from 0.51 to 0.69. In the WLS analysis, there was also a weak correlation between PFS and OS across all the trials; R2 values ranged from very low (0.26) for EORTC risk group and PS to moderate (0.67) for age. As an exploratory exercise, the trial‐level surrogacy was estimated by the artificial randomized trials data created by pairing single‐arm trials on the means of the EORTC risk scores; again, the PFS surrogacy was moderate and similar to those estimated from the risk factor effect approach.

Our analyses have several limitations. Most of the trials we analyzed contained only one treatment arm, and the treatment varied from trial to trial. The PFS surrogacy for OS has usually been evaluated with multiple trials, with the same two treatment regimens across these trials [6], [7]. One can argue that the effect of baseline prognostic factors, such as PS, on survival endpoints is different from the treatment effect on the same endpoints and that these baseline factors are not the causes of survival difference. Although the use of the effect of risk factors as a substitute for treatment effect in evaluating PFS surrogacy is supported by the analysis of small cell lung cancer data reported in supplemental online data (Appendix A) and this approach has been used by a different team of investigators for similar analysis [33], [34], the validity of this approach needs further confirmation from both methodological and applied research.

In addition, most of these trials are small, with sample sizes ranging from 10 to 67, causing bigger errors associated with summary‐level estimates. Moreover, none of the 17 trials contained pemetrexed. Active agents, such as pemetrexed, may have an effect on the observed correlation between PFS HRs and OS HRs, and the direction and the magnitude of the impact are not assessable with the existing data. These limitations may have mischaracterized the real surrogacy relation between PFS and OS. However, because mesothelioma is a rare disease, it is unlikely that there will be many large clinical trials for this population. This analysis includes a large number of patients enrolled on multiple cooperative group trials, and hence the results likely outweigh the limitations.

The trial‐level surrogacy cannot be estimated from single‐arm trial data because no patients of the same trial were assigned to different treatment arms. To evaluate PFS surrogacy with single‐arm trial data, we had to resort to the summary‐level surrogacy, which is obtained by replacing treatment arms with levels of risk factors and then applying the standard surrogacy analysis. The concordance of PFS surrogacy evaluated with the risk factor effect and the treatment effect in the small cell lung cancer analysis suggests that the surrogacy tends to take place at multiple levels. However, we need to emphasize that the summary‐level correlation is fundamentally different from the trial‐level correlation. This statement holds for the exploratory exercise in which artificial single‐arm treatment trial was paired into pseudo‐randomized treatment groups. In other words, the creation of artificial “treatment groups” is another way of estimating the summary‐level correlation, and it does not inform the true trial‐level correlation. The summary‐level correlation carries no information on trial‐level surrogacy in the conventional sense.

Meanwhile, when the risk factor effect is used for surrogacy analysis on single‐arm trials data, the individual correlation tau estimated from the bivariate Copula model is related but, in general, distinct from the individual‐level surrogacy obtained from randomized trials data. For similar reasons, the square error and the MSPE in Table 4 may have a different interpretation when these statistics are estimated by using randomized trials data. The analysis includes all 17 mesothelioma trials conducted in CALGB and NCCTG, and most the investigative therapies were negative. This may dampen the generalizability of our findings.

Additional issues for validating PFS as a surrogate endpoint in clinical trials have been discussed by other authors [35], [36], [37], [38], such as PFS definition varying among studies and assessment bias in open‐label studies. Many of the trials performed in mesothelioma have been small phase II trials with investigator review of response and PFS. These issues are not unique to our analysis but make it more difficult to validate PFS as a surrogate endpoint for OS in malignant mesothelioma.

Conclusion

More mesothelioma patients are expected to receive maintenance or second‐line treatments, and there is a need to demonstrate that PFS or other short‐term endpoints can be used as a valid surrogate endpoint for OS. Because of these limitations of our database and the analysis approach, we shall refrain from a definitive conclusion that PFS is not a valid surrogate endpoint for OS in future mesothelioma trials. A definitive answer to the surrogacy question in this disease requires further investigation on the data from multiple large randomized trials, with disease progression confirmed by an independent central review [39], [40]. Hence, pending more definitive studies, we cannot recommend using PFS as a surrogate endpoint for OS in malignant mesothelioma trials.

See http://www.TheOncologist.com for supplemental material available online.