Relative value assessment: characterizing the benefit of oncology therapies through diverse survival metrics from a US perspective.

OBJECTIVES: The introduction of innovative, high-cost oncology treatments, coupled with mounting budgetary pressures, necessitates value trade-offs across cancer types. Defining value is critical to informing decision-making. A cost-value analysis tool was used to assess relative clinical value from a US perspective using multiple outcome metrics for a variety of metastatic cancers.
METHODS: Literature published (January 1, 2000-August 31, 2016) was reviewed to identify outcome metrics for approved treatments for metastatic cancers. Data were extracted or derived for median and mean overall survival (OS), landmark survival rates, and other survival metrics, and compared across treatments vs their respective trial comparators, with and without considering costs.
RESULTS: Reported survival metrics varied by agent within cancer type. For treatment of prostate cancer, abiraterone yielded the highest improvement in 1-year survival rate (13.7%, previously treated), whereas enzalutamide yielded the highest median OS improvement (4.8 months, previously treated) and sipuleucel-T, the highest mean OS improvement (3.6 months, previously untreated) vs their respective trial comparators. For treatment of non-small cell lung cancer vs their respective trial comparators, nivolumab yielded the highest improvement in mean OS (11.9 months) and 3-year survival rate (12.6%), each in previously treated squamous disease, whereas afatinib yielded the highest median OS improvement (4.1 months, previously untreated EGFR del19 and L858R mutants). Cost-value analysis results varied with the applied survival metric.
CONCLUSIONS: Although median OS is the traditional gold standard oncology efficacy metric, it fails to capture long-term survival benefits-the ultimate goal of cancer treatment-offered by new treatment modalities. Diverse metrics are needed for comprehensive value assessments of cancer therapies.

Plain language summary

The efficacy of a cancer therapy is traditionally measured by median overall survival (OS; the time point at which half of the patients receiving therapy have died). However, newer therapies such as immuno-oncology agents have shown long-term survival benefits in some patients that are not adequately captured by median OS. We reviewed survival outcomes in pivotal clinical trials for therapies approved for metastatic cancers in the US between January 1, 2000 and August 31, 2016.

Improvements in median OS, average OS and landmark survival (the proportion of patients alive after receiving therapy for a specific time) with approved therapies were assessed relative to trial comparator arms. Relative clinical value was considered by applying the total cost of therapy per patient to the relative survival improvements.

Survival advantage and relative value of approved agents differed depending on the survival metric assessed. Median OS may undervalue treatments that provide long-term survival benefit, and average OS and landmark survival may provide more accurate measures of such long-term survival. When assessing the value of oncology therapies, decision-makers should consider a diverse range of oncology survival metrics.

Introduction

Substantial unmet needs remain in cancer, and long-term survival continues to be an elusive goal for most patients with advanced cancer.1 Further complicating treatment goals are the continually rising costs associated with cancer care, which are outpacing other types of healthcare costs.2,3 A variety of factors are driving the budgetary impact of cancer care, including increasing incidence with a growing and aging population,4,5 a shift toward more costly personalized care,4 increased cancer survivorship,5–7 and increasing availability and adoption of new branded therapies.8,9

Against this backdrop of increasing cancer incidence and healthcare costs, many healthcare budget holders are facing mounting pressure to control costs while funding optimal care for a growing population.10 Value trade-offs across cancer types are, thus, becoming increasingly important in resource allocation decision-making. Defining value is critical in determining what treatments should be reimbursed. Each stakeholder group (patients, caregivers, physicians, and payers) defines the value of cancer treatment differently, resulting in an ongoing debate and a lack of agreement regarding the meaning of value.3,11 Oncology treatment guidelines generally do not account for treatment costs. Nevertheless, multiple traditionally cost-agnostic provider organizations have created “scorecard”-based frameworks to define the value of oncology treatments, including the American Society of Clinical Oncology (ASCO),12,13 the European Society for Medical Oncology (ESMO),14 and the Memorial Sloan Kettering Cancer Center (MSKCC).15 As of 2015, National Comprehensive Cancer Network (NCCN) guidelines have included Evidence Blocks, a visual representation for each agent or regimen of five key value measures, one of which is affordability. The Institute for Clinical and Economic Review has also incorporated economic analyses, but, like the NCCN Evidence Blocks, these use an expert panel-driven rather than a scorecard approach. These initiatives demonstrate increasing concern over escalating healthcare costs.

One challenge in assessing value is identifying the appropriate efficacy metrics. Traditional “gold-standard” efficacy metrics such as median overall survival (OS) or progression-free survival (PFS) focus on a single point, where 50% of the population has had an event. Thus, they are not well suited to reflect the long-term survival benefits of new treatment modalities, such as immuno-oncology agents, which demonstrate prolonged survival in a proportion of patients.16 Despite this, several common value frameworks utilize median OS as a key driver of value scoring. To capture long-term survival benefits, and thus assess the full value of newer treatment modalities, there is a need to report and evaluate nontraditional efficacy endpoints such as landmark and mean OS.17,18

The objectives of this study were to: 1) compare the value of approved oncology agents, in particular new immuno-oncology agents, as assessed by a variety of efficacy metrics vs median OS, within and between tumor types and 2) apply a recently developed relative value assessment (RVA) tool for conducting both outcomes assessments and cost-value analyses.19 These analyses were conducted from a US perspective. Application of the RVA tool to clinical trial data may be useful in assessing managed care decision-making.

Methods

This analysis included any therapy approved by the US Food and Drug Administration (FDA) between January 1, 2000 and August 31, 2016, inclusive, for the treatment of stage III/IV of the following cancers: breast cancer, bladder cancer, colorectal cancer (CRC), lymphomas, gastric cancer, glioblastoma multiforme, hepatocellular carcinoma, myelodysplastic syndromes, melanoma, multiple myeloma, non-small cell lung cancer (NSCLC), prostate cancer, renal cell carcinoma (RCC), small cell lung cancer, and squamous cell carcinoma of the head and neck. Single-arm trials were excluded, as survival metrics were evaluated for each drug vs its direct trial comparator. Two kinds of subanalyses were performed: within-tumor analyses and pan-tumor analyses. The within-tumor analyses focused on NSCLC and prostate cancer, as these reflect common cancers, for which baseline survival rates are among the lowest and highest, respectively, across all cancers.20 The pan-tumor analyses included a selection of tumor types identified as the most common causes of cancer-related deaths in the US (breast cancer, CRC, melanoma, NSCLC, and RCC).20

Survival metrics

For each agent and its indicated tumor type (prostate, NSCLC, breast cancer, melanoma, RCC, and CRC), survival metrics were obtained from FDA product labels and European Medicines Agency (EMA) Public Assessment Reports, or relevant clinical trial publications in cases where these were not in the FDA label or EMA Public Assessment Report (see Table S1). Kaplan–Meier (KM) OS curves were extracted and digitized using the Engauge software package (http://markummitchell.github.io/engauge-digitizer/), allowing for the calculation of mean OS (calculated as the area under the KM curve) and landmark survival at 1, 2, and 3 years for each agent and tumor type. After extracting survival metrics, the difference between an intervention and its trial comparator was calculated.

A confounding factor in the calculation of mean OS is the maturity of data cutoff, which may particularly disadvantage newer agents that have less mature OS data available, particularly those that provide long-term OS benefits. To address this, KM OS curves were digitized, and reconstructed individual patient-level data were generated based on the code from Guyot et al.21 Parametric distributions, including spline models, were fitted to the reconstructed individual patient-level data to provide extrapolation beyond reported cutoffs at 15 years. The parametric curve with the best fit was determined by goodness-of-fit statistics; specifically, those with the lowest Akaike information criterion were chosen and then the visual best fit was used to validate this selection. A 15-year cutoff was used for all extrapolated curves; an assumption was that at the end of year 15 survival would be 0%. Mean OS and landmark survival were recalculated using this extrapolated curve set for therapies in melanoma, RCC, and NSCLC (see Table S2), which were the first three approved indications of immuno-oncology agents.

Cost-value analyses

In addition to calculating survival metrics, relative clinical value across tumor types (breast cancer, melanoma, NSCLC, RCC, and CRC) was assessed in cost-value analyses using the RVA tool by plotting total drug cost vs a given survival outcome. Calculation of total drug costs was based on monthly costs multiplied by therapy duration, as noted by data in the product label, determined based on median duration of administration, median PFS, or median time to progression, ranked in descending order of preference, as available. Loading dose for applicable agents, as indicated by the product label, was also incorporated into total cost. Total drug costs were obtained from the IHS Markit Life Sciences PharmOnline International database.22 All costs are expressed in US dollars.

For all therapies with available data, the relative value of each product for a given clinical metric was assessed against average value as determined by a regression line calculated from the distribution of the cost-to-outcome ratio as represented by total drug cost (x-axis) vs one of the clinical metrics (y-axis). A value of zero was assigned to the regression line intercept to standardize disease progression across the different evaluated metrics, which then could be used to identify products that could be considered “above average value” (falling above the line) or “below average value” (falling below the line) using those metrics.

Comparisons of cost-value within a single metric were undertaken using a “relative additional patients alive at 1 year per dollar spent” metric, calculated based on reported cutoffs from each drug’s pivotal trial for NSCLC. The cost of avoiding death at 1 year was calculated by multiplying the number needed to treat to avoid a death at 1 year by the per-patient treatment cost. This number was inverted to calculate the number of additional patients alive at 1 year per dollar spent. The agent with the lowest value was set equal to one and used to calculate the relative additional persons alive at 1 year for all other agents.

Results

In the individual tumor analyses, efficacy results for individual agents varied by survival metric. For treatment of prostate cancer, for which all metrics were based on reported KM OS curve data, abiraterone yielded the highest improvement in 1-year survival rate (13.7%, previously treated, Figure 1A), whereas sipuleucel-T yielded the highest mean OS improvement (3.6 months, previously untreated, Figure 1B) and enzalutamide yielded the highest median OS improvement (4.8 months, previously treated, Figure 1C) among all agents vs their respective trial comparators.

Figure 1

Prostate cancer survival improvement. (A) Improvement in 1-year survival, (B) improvement in mean OS, and (C) improvement in median OS for therapies in prostate cancer for each agent vs its respective trial comparator, based on reported Kaplan–Meier OS curves.

Abbreviations: 1L, first line; 2L, second line; 223RaCl2, radium 223 dichloride; 3L, third line; Abira, abiraterone; Cabazi, cabazitaxel; Enza, enzalutamide; OS, overall survival; Sip-T, sipuleucel-T.

For treatment of NSCLC, nivolumab yielded the highest improvement in 1-year survival rate (18%, previously treated squamous disease) and mean OS (5.0 months, previously treated squamous disease) among all agents vs their respective trial comparators (figures not presented), whereas afatinib yielded the highest improvement in median OS (4.1 months, previously untreated EGFR del19 and L858R mutants, Figure 2A) based on reported KM curves. In the extrapolated analysis, which helps to account for differences in data maturity, nivolumab again yielded the highest improvement in 3-year survival rate (12.6%, previously treated squamous disease, Figure 2B) and in mean OS (11.8 months, previously treated squamous disease, Figure 2C).

Figure 2

Non-small cell lung cancer survival improvement. (A) Improvement in median OS based on reported Kaplan–Meier OS curves, (B) improvement in 3-year OS, and (C) improvement in mean OS for each agent vs its respective trial comparator, based on fitted Kaplan–Meier OS curves that extrapolate survival beyond the reported cutoffs; excludes interventions where relevant Kaplan–Meier OS curves were not identified (ie, afatinib, nintedanib, and pemetrexed [2L]). Any drug compared with placebo or best supportive care (offers a lower clinical benchmark against which it is easier to demonstrate relative value) was excluded (ie, pemetrexed [maintenance], docetaxel, and erlotinib [2/3L]).

Abbreviations: 1L, first line; 2L, second line; 3L, third line; Afa, afatinib; Bev, bevacizumab; Criz, crizotinib; Erlot, erlotinib; Gefit, gefitinib; Nab-pac, nab-paclitaxel; Neci, necitumumab; Nivo, nivolumab; NSQ, nonsquamous; OS, overall survival; PD-L1, programmed death ligand 1; Pemet, pemetrexed; Pembro, pembrolizumab 2 mg/kg; Ramu, ramucirumab; SQ, squamous.

In the case of immuno-oncology agents used to treat NSCLC (nivolumab in previously treated disease, irrespective of programmed death ligand 1 [PD-L1] expression and pembrolizumab in previously treated ≥ 1% PD-L1-positive disease; see Table S1), the greatest survival benefits vs their respective trial comparators were apparent when mean OS and 3-year survival rate improvements (based on extrapolated curves) were used as the comparative metrics (Figures 2B and C). By comparison, when median OS improvement based on reported curves was used to compare agents (Figure 2A), the benefits of immuno-oncology drugs vs their respective trial comparators were comparable with those of many targeted alternatives in NSCLC. Furthermore, the magnitude of variation among NSCLC agents across the different survival metrics was greater than that observed in prostate cancer, where immuno-oncology agents were not used.

Results of the pan-tumor cost-value analyses are shown in Figures 3–6. Presentation of these data as a single-variable plot, in terms of the relative number of additional patients alive at 1 year per US dollar spent on a range of treatments for NSCLC, is provided in Figure S1.

Figure 3

Improvement in 1-year survival rate over respective trial comparators vs total treatment cost for branded monotherapies for breast cancer, colorectal cancer, melanoma, non-small cell lung cancer, and renal cell carcinoma based on reported Kaplan–Meier overall survival curves.

Note: Regression line represents average value given cost. Gray shaded area below line represents below average value given cost.

Abbreviations: 1L, first line; 2L, second line; 3L, third line; 5-FU, 5-fluorouracil; Aflib, ziv-aflibercept; Axit, axitinib; Bev, bevacizumab; BSC, best supportive care; Cabo, cabozantinib; Cape, capecitabine; Cetux, cetuximab; Criz, crizotinib; Dabraf, dabrafenib; Doce, docetaxel; EGFR, epidermal growth factor receptor; Erib, eribulin; Erlot, erlotinib; Evero, everolimus; FOLFIRI, folinic acid, fluorouracil, irinotecan; FOLFOX, folinic acid, fluorouracil, oxaliplatin; Gefit, gefitinib; Ifo, ifosfamide; ILF, infusional 5-FU; Ipi, ipilimumab; ITT, intent-to-treat; Lapat, lapatinib; LV, leucovorin; M(c), maintenance (continuous); M(s), maintenance (switch); Nab-p, nab-paclitaxel; Neci, necitumumab; Nivo, nivolumab; NSQ, nonsquamous; Panit, panitumumab; Pazop, pazopanib; Pembro, pembrolizumab; Pemet, pemetrexed; Ramu, ramucirumab; Regor, regorafenib; Soraf, sorafenib; SQ, squamous; Sunit, sunitinib; Tems, temsirolimus; Tipi, tipiracil; Tramet, trametinib; Trastuz, trastuzumab; Triflu, trifluridine; Vem, vemurafenib; Vin, vinorelbine; WT, wild type; XELOX, capecitabine + oxaliplatin.

Figure 4

Improvement in 1-year survival rate over respective trial comparators vs total treatment cost for combination regimens comprising two branded therapies in breast cancer, melanoma, and renal cell carcinoma based on reported Kaplan–Meier overall survival curves.

Note: Regression line represents average value given cost. Gray shaded area below line represents below average value given cost.

Abbreviations: 1L, first line; 2L, second line; Cape, capecitabine; Cobi, cobimetinib; Dabraf, dabrafenib; Evero, everolimus; Ipi, ipilimumab; Lapat, lapatinib; Len, lenvatinib; Nivo, nivolumab; Pertuz, pertuzumab; Ph, phase; Tramet, trametinib; Trastuz, trastuzumab; Vem, vemurafenib.

Figure 5

Improvement in median OS over respective trial comparators vs total treatment cost for breast cancer, colorectal cancer, melanoma, non-small cell lung cancer, and renal cell carcinoma based on reported Kaplan–Meier OS curves.

Note: Regression line represents average value given cost. Gray shaded area below line represents below average value given cost.

Abbreviations: 1L, first line; 2L, second line; 3L, third line; 5-FU, 5-fluorouracil; Afat, afatinib; Aflib, ziv-aflibercept; Axit, axitinib; Bev, bevacizumab; BSC, best supportive care; Cabo, cabozantinib; Cape, capecitabine; Cetux, cetuximab; Cobi, cobimetinib; Criz, crizotinib; Dabraf, dabrafenib; Doce, docetaxel; EGFR, epidermal growth factor receptor; Erib, eribulin; Erlot, erlotinib; Evero, everolimus; FOLFIRI, folinic acid, fluorouracil, irinotecan; FOLFOX, folinic acid, fluorouracil, oxaliplatin; Gefit, gefitinib; Ifo, ifosfamide; ILF, infusional 5-FU; Ipi, ipilimumab; ITT, intent-to-treat; KN, KEYNOTE; Lapat, lapatinib; Lenvat, lenvatinib; LV, leucovorin; M(c), maintenance (continuous); M(s), maintenance (switch); Nab-p, nab-paclitaxel; Neci, necitumumab; Nivo, nivolumab; NSQ, non-squamous; OS, overall survival; OxMdG, oxaliplatin/modified de Gramont chemotherapy; Panit, panitumumab; Pazop, pazopanib; Pembro, pembrolizumab; Pemet, pemetrexed; Pertuz, pertuzumab; Ph, phase; Ramu, ramucirumab; Regor, regorafenib; Soraf, sorafenib; SQ, squamous; Sunit, sunitinib; Tems, temsirolimus; Tipi, tipiracil; Tramet, trametinib; Trastuz, trastuzumab; Triflu, trifluridine; Vem, vemurafenib; Vin, vinorelbine; WT, wild type; XELOX, capecitabine + oxaliplatin.

Figure 6

Improvement in mean OS over respective trial comparators vs total treatment cost for melanoma, non-small cell lung cancer, and renal cell carcinoma based on fitted Kaplan–Meier OS curves that extrapolate survival beyond the reported cutoffs. Regression line represents average value given cost. Gray shaded area below line represents below average value given cost.

Abbreviations: 1L, first line; 2L, second line; 3L, third line; Bev, bevacizumab; Cabo, cabozantinib; Criz, crizotinib; Dabraf, dabrafanib; Doce, docetaxel; EGFR, epidermal growth factor receptor; Erlot, erlotinib; Evero, everolimus; Gefit, gefitinib; Ifo, ifosfamide; Ipi, ipilimumab; ITT, intent-to-treat; Lenvat, lenvatinib; M(c), maintenance (continuous); M(s), maintenance (switch); Nab-p, nab-paclitaxel; Nivo, nivolumab; NSQ, nonsquamous; OS, overall survival; Pazop, pazopanib; Pembro, pembrolizumab; Pemet, pemetrexed; Ph, phase; Ramu, ramucirumab; Soraf, sorafenib; SQ, squamous; Sunit, sunitinib; Tems, temsirolimus; Tramet, trametinib; Vem, vemurafenib; Vin, vinorelbine; WT, wild type.

Therapies used for tumors and lines of therapy with higher baseline median survival were generally associated with the greatest absolute median survival gain compared with their trial comparators; that is, in these settings, the same relative risk reduction would result in a larger absolute gain in median OS (eg, first-line melanoma and breast cancer).23 Using 1-year survival as the metric, some of the agents with greatest benefit vs their trial comparators were for NSCLC, where baseline survival is very low.23 However, two (docetaxel and pemetrexed) of the three NSCLC agents with the greatest survival gain vs their respective comparators (docetaxel, pemetrexed, and nivolumab) had best supportive care as their trial comparator, providing a low clinical benchmark against which greater efficacy gains can be demonstrated. Similarly, based on the cost-value analyses, nivolumab + ipilimumab appeared less favorable than nivolumab monotherapy in first-line melanoma in terms of the mean OS benefit over their respective trial comparators. The comparator arm for nivolumab + ipilimumab in CheckMate 069 was ipilimumab,24 which is a higher clinical benchmark than cytotoxic chemotherapy (dacarbazine, the comparator for nivolumab monotherapy in CheckMate 066),25 making demonstration of relative value more challenging. As with this example, branded combination therapies may be rated as having a lower relative clinical value than their constituent monotherapies. Thus, the relative value of branded combination agents can appear comparable with or even lower than that offered by their constituent monotherapies; therefore, it may be more appropriate to evaluate the value of branded combination regimens separately (Figure 4).

Results varied with the applied survival metric, with some agents achieving a higher cost-value for some metrics, but not others, as can be seen by differences in relative value from 1-year survival rates based on reported KM curves (Figures 3 and 4), median OS based on reported KM curves (Figure 5), and mean OS based on extrapolated curves (Figure 6). For example, the intervention with the greatest increase in median OS was pertuzumab + trastuzumab for breast cancer, at a cost that appeared to provide above-average efficacy benefits compared with other available therapies (as indicated by its relative position over the regression line in Figure 5). In contrast, when looking across agents at increases in 1-year survival (Figure 4), pertuzumab + trastuzumab for breast cancer provided a much less favorable efficacy profile (5.2% increase in 1-year survival), and at a cost (more than $196,000 per patient) far in excess of the average value offered (as indicated by its position relative to the regression line). This discrepancy may illustrate limitations in median OS as an efficacy metric, as it cannot take into account any long-term survival benefits after the median has been reached, unlike mean OS and landmark survival analyses.

Discussion

This analysis demonstrated that therapy benefits differed depending on the survival metric used, with evaluations of oncology therapies based on improvements in median OS yielding some very distinct results vs other survival metrics. Although median OS is among the most commonly used metrics in oncology trials, it fails to take into account long-term survival benefits after the median has been reached and, thus, is not well suited to assess the value of immuno-oncology therapies, for which efficacy is typified by KM survival curves showing delayed but sustained separation.

Alternative survival metrics include mean OS and landmark survival rates. In the pan-tumor analysis, some differences in cost-value patterns were also observed with the 1-year survival metric (Figure 3) compared with the mean OS metric (Figure 6). Although median OS provides an estimate that cannot be biased by a small number of outlying cases, it calculates survival for the whole cohort based on only the first 50% of patients who do not survive.26,27 Mean OS offers a more comprehensive assessment of long-term survival benefits, as it captures the complete survival curve, including the “tail”. In terms of value for money, many healthcare payers and health technology assessment agencies consider mean OS as the standard metric for cancer treatment cost-effectiveness analyses.27 However, a limitation of assessing value using mean OS calculations based on the study duration deemed appropriate by regulatory bodies for each treatment’s Phase III pivotal trial is that duration is not consistent among trials, which may confound comparisons. Although mean OS offers a more comprehensive assessment of long-term survival benefits, using this metric requires extrapolation of KM OS curves beyond reported cutoffs. Extrapolation in this study was undertaken for therapies indicated for NSCLC, RCC, or melanoma (the first three cancers with an approved immuno-oncology agent). Use of digitized KM curves, rather than actual data points, where patient-level data were not available, is a limitation of our analysis. Furthermore, parametric curve selection, particularly with an immature data set, can be subject to interpretation, as distinct curves with similar goodness-of-fit statistics can introduce variability in outcomes.

Landmark survival rates can be highly effective for comparing therapies by enabling survival comparisons after the median has been reached. However, data maturity and cutoffs varied among trials included in this assessment. As selection of a later time point for landmark survival rates could severely constrain the comparator set, 1-year survival rates based on reported KM OS curves (where available) were used. Using such an early time point, however, may not account for long-term survival benefits of selected therapies, and indeed may even precede the median OS cutoff for some tumor types. Utilizing fitted parametric KM curves to provide survival data beyond the reported cutoffs and the selection of landmark survival at a later time point can provide a consistent set of more mature landmark survival readouts, but is subject to limitations, as discussed above.

In addition to survival, health-related quality-of-life (HRQoL) is an important metric that provides detailed information from the patient perspective, and is increasingly being reported in clinical studies. However, unlike survival outcomes, HRQoL is not reported consistently across studies. Furthermore, diversity in HRQoL metrics used across trials, variability in timing of PRO data reporting, and the percentage of patients missing PRO data limit comparability within and across tumor types.28 Therefore, HRQoL metrics were not included in this analysis.

Our results showed variation in the relative value of the agents assessed, particularly between median OS and other metrics. Nevertheless, the findings showed no clear ideal metric, with variations observed in the relative value of treatments, even between landmark survival rates and mean OS metrics. Moreover, a higher relative survival outcome on a given metric did not necessarily translate into greater economic value. These discrepancies emphasize the need to incorporate a wide range of metrics into assessments of the efficacy benefits of different agents in a pan-tumor as well as in individual tumor comparison of value, or to be able to appropriately target value metrics to the agents of interest.

Pan-tumor value comparisons should be interpreted with caution owing to heterogeneity between trials (particularly variations in comparators, but also in populations, study design, outcomes, and timing of assessments). Furthermore, baseline survival expectations differ among tumor types and lines of therapy and must be considered when evaluating relative value so as not to discriminate unfairly against agents that treat cancers with the highest unmet need. Although the ASCO Value in Cancer Care Task Force has expressed concern about cross-trial comparisons due to potential biases, the task force has also suggested that the ASCO value framework tool could be used to facilitate individual patient treatment decision-making based on data from multiple trials for a specific indication.13

Another factor that may affect relative value is variation in patient crossover rates from the control to the active agent arm, which can confound OS results. For example, in Figure 4, the two regimens with the greatest survival benefits (lenvatinib + everolimus and cobimetinib + vemurafenib) were from trials where crossover was not allowed, whereas the regimen with the next highest survival benefit (nivolumab + ipilimumab) was from a trial where more than 55% of patients in the ipilimumab arm crossed over to receive nivolumab + ipilimumab. Pembrolizumab, which also had ipilimumab as its trial comparator for first-line and second-line treatment of melanoma,29 appeared to offer a mean survival benefit vs ipilimumab of a similar magnitude to that offered by nivolumab + ipilimumab vs ipilimumab for first-line treatment of melanoma; however, crossover was not permitted in the pembrolizumab trial.

An additional limitation of RVA is that it involves naïve trial comparisons, in distinct cancer types and lines of therapy, and in different patient populations. This includes making comparisons of therapies approved in biomarker-selected vs nonbiomarker selected populations. For example, in Figure 5, the anti-PD-1 agent pembrolizumab appears to produce a survival benefit in previously treated NSCLC that is within the range of the anti-PD-1 agent nivolumab in previously treated squamous and non-squamous NSCLC, but at a lower cost. However, as of the analysis end date, pembrolizumab was only approved in PD-L1+ patients in this line of therapy, whereas nivolumab was approved in an all-comers population. As an alternative, more complex meta-analytic approaches (if feasible given available data resources), instead of this type of pan-tumor value assessment tool, could be used to reduce bias. The results of the individual tumor analyses for NSCLC and prostate cancer can address some of these limitations to an extent, and provide an alternate means for assessing the relative value assessment approach.

Another limitation in this cost-value analysis is that each therapy was assigned a cost based on reported monthly cost, multiplied by therapy duration (median duration of administration, median PFS, or median time to progression, based on available data), which varied between trials. Although costs could be underestimated by using median instead of mean therapy duration, this bias was assumed to be relatively consistent across agents because similar methods were used to estimate costs for all comparators. Furthermore, assigned cost-value only incorporated drug cost as adjusted for duration and loading dose, and did not take into account symptom burden, treatment safety profile, HRQoL, health-care resource utilization, or use of postprogression therapies. These factors are particularly important with newer, frequently less toxic therapies compared with traditional chemotherapy. Additionally, the cost-value analysis did not take into account drug cost discounting, which might be available among certain payers, or compassionate use programs or patient rebates. As with any complex analysis that entails input of a large and diverse body of evidence, only FDA-approved drugs during the study period (January 1, 2000—August 21, 2016) are included; later approvals were not included in the analysis. Finally, the predefined inclusion criteria may have resulted in the evaluated agents not being representative of the entire treatment landscape, thus potentially introducing selection bias.

Despite the limitations noted here, from which no value assessment tool is exempt, the RVA tool utilized in this study provides a framework for comparing oncology therapies both within and across tumor types. As such, it adds to the range of value tools and metrics available that include treatment costs for valuing oncology treatments.12–15,30

US payers often look to NCCN and ASCO clinical guidelines as part of their reimbursement decision-making,31 which either do not currently account for treatment costs (ASCO) or provide a five-level affordability rating (NCCN). Several of the existing value scorecards come from provider bodies (ASCO, NCCN, and MSKCC), suggesting that traditionally cost-agnostic stakeholders are becoming increasingly concerned over the escalating costs of care. As the cost of care continues to rise, payers may be increasingly forced to make value trade-offs, and this tool can be used to support funding prioritization.

While some of these tools base scores heavily on median OS or PFS benefits, the RVA tool incorporates a wide range of efficacy metrics that, unlike median OS, can account for long-term survival benefits. As shown in this study, valuing oncology therapies based on improvements in median OS alone potentially undervalues new treatment modalities associated with long-term survival benefits. Ongoing review and discussion of the range of indices to define and measure cost-effectiveness is needed to reach a consensus, or at least more broadly accepted criteria, for defining value, particularly for novel therapies.

Further research could include a systematic methodological standardization for parametric curve selection for the calculation of mean OS and long-term landmark OS endpoints alongside aligning best value metrics and reconciling when results differ. The RVA approach demonstrated here can compare a wide range of efficacy metrics for treatments across a broad range of cancers that can be utilized to better inform decisions by payers and providers in managed care.

Conclusions

In the era of expanding healthcare costs and budgetary pressure, payers will increasingly focus on areas of high and expanding costs, of which cancer care is prominent. Physicians are also becoming attuned to treatment costs, and it is notable that many of the value frameworks and scorecards are provider-led. Although median OS is the traditional gold standard oncology efficacy metric, it should not be considered in isolation of other metrics, because it does not capture long-term survival benefits, the ultimate goal of cancer treatment. The metric selected to define the value of an oncology therapy must be carefully considered. Value frameworks incorporating diverse metrics, such as the one presented here, are needed for comprehensive value assessments of cancer therapies, particularly for new treatment modalities such as immuno-oncology.

Additional persons alive at 1 year vs respective trial comparator (relative to crizotinib) for NSCLC.

Notes: Based on the number needed to treat for an additional person to be alive at 1 year, based on reported cutoffs from each drug’s pivotal trial. Includes FDA-approved interventions in NSCLC for which the relevant Kaplan–Meier overall survival curves were available; trials with comparators of placebo or best supportive care were excluded.

Abbreviations: 1L, first line; 2L, second line; 3L, third line; Bev, bevacizumab; Carbo, carboplatin; Chemo, chemotherapy; Cis, cisplatin; Criz, crizotinib; Doce, docetaxel; Erlot, erlotinib; FDA, Food and Drug Administration; Gefit, gefitinib; Nab-p, nab-paclitaxel; Neci, necitumumab; Nivo, nivolumab; NSQ, nonsquamous; Pac, paclitaxel; Pembro, pembrolizumab; Pemet, pemetrexed; PT, platinum; Ramu, ramucirumab; SQ, squamous.

Table S1

Data included in the survival and cost analyses

Study treatment	Histology	Line of therapy	Trial	Comparator
Melanoma
Dabrafenib1–3	NA	1L	BREAK-3	Dacarbazine
Dabrafenib + trametinib4	NA	1L	COMBI-V	Vemurafenib
Dabrafenib + trametinib1,5–7	NA	1L	COMBI-D	Dabrafenib
Ipilimumab8–10	NA	2L	MDX010-20	Gp100
Nab-paclitaxel11	NA	1L	CA031	Paclitaxel
Nivolumab12,13	NA	2L+	CheckMate 037	Dacarbazine or carboplatin + paclitaxel
Nivolumab14	NA	1L	CheckMate 066	Dacarbazine
Nivolumab + ipilimumab15,16	NA	1L	CheckMate 069	Ipilimumab
Pembrolizumab17–19	NA	1L/2L	KEYNOTE-006	Ipilimumab
Pembrolizumab20,21	NA	2L+	KEYNOTE-002	Investigator’s choice chemotherapy
Trametinib5,22,23	NA	1L/2L	METRIC	Dacarbazine or paclitaxel
Vemurafenib24–27	NA	1L	BRIM-3	Dacarbazine
NSCLC
Afatinib28–30	NSQ	1L	LUX LUNG 3	Cisplatin-based chemotherapy
Bevacizumab + paclitaxel + carboplatin31–33	NSQ	1L	E4599	Paclitaxel + carboplatin
Crizotinib34	NSQ	2L	PROFILE 1007	Docetaxel or pemetrexed
Docetaxel35	NA	2L+	TAX 320	Ifosfamide or vinorelbine
Erlotinib36–39	NSQ	1L	EURTAC	Standard chemotherapy
Erlotinib37,40,41	NA	2L/3L	BR.21	Placebo
Gefitinib42,43	NA	1L	IPASS	Carboplatin + paclitaxel
Nab-paclitaxel44	NA	1L	CA031	Solvent-based paclitaxel
Necitumumab + gemcitabine + cisplatin45–47	SQ	1L	SQUIRE trial	Gemcitabine + cisplatin
Nivolumab48–50	SQ	2L	CheckMate 017	Docetaxel
Nivolumab50,51	NSQ	2L+	CheckMate 057	Docetaxel
Pembrolizumab (2 mg/kg, ≥1% PD-L1)52–54	NA	2L+	KEYNOTE-010	Docetaxel
Pemetrexed55–58	NSQ	1L	FDA study	Cisplatin + gemcitabine
Pemetrexed55,56,59	NSQ	1L maintenance	PARAMOUNT	Placebo
Pemetrexed60	NSQ	1L maintenance (switch)	JMEN	Placebo
Ramucirumab61	NA	2L	REVEL	Docetaxel
Renal cell carcinoma
Axitinib62–64	NA	2L	AXIS	Sorafenib
Cabozantinib65,66	Clear cell	2L+	METEOR trial	Everolimus
Everolimus67–71	NA	2L	RECORD-1	Placebo
Lenvatinib + everolimus72–74	NA	2L+	Study 2 FDA label	Everolimus
Nivolumab75,76	Clear cell	2/3L	CheckMate 025	Everolimus
Pazopanib77,78	NA	1/2L	VEG105192	Placebo
Sorafenib79–83	NA	2L	TARGET	Placebo
Sunitinib84–88	NA	1L	A6181034	IFN-alpha
Temsirolimus89–91	NA	1L	3066K1-304-WW	IFN-alpha
Breast cancer
Nab-paclitaxel92–94	NA	2L+	CA012-0	Paclitaxel
Everolimus67,69–71	NA	2L+	BOLERO-2	Placebo
Capecitabine + docetaxel95,96	NA	2/3L	Xeloda SO14999	Docetaxel
Eribulin mesylate97–100	NA	3L	EMBRACE	Physician’s treatment of choice
Trastuzumab + docetaxel101–105	NA	1L	Herceptin M77001	Docetaxel
Ado-trastuzumab emtansine106–109	NA	2L+	EMILIA	Lapatinib + capecitabine
Pertuzumab + trastuzumab + docetaxel110–114	NA	1L	CLEOPATRA	Placebo + trastuzumab + docetaxel
Trastuzumab emtansine + placebo115	NA	1L	MARIANNE	Trastuzumab + taxane (docetaxel/paclitaxel)
Lapatinib116,117	NA	1L	EGF 30008	Letrozole + placebo
Lapatinib + capecitabine118–121	NA	2L+	EGF100151	Capecitabine
Colorectal cancer
Bevacizumab122,123	NA	1L	NO16966	XELOX/FOLFOX-4
Bevacizumab122,124,125	NA	1L	AVF2107	ILF + placebo
Capecitabine95,96,126	NA	1L	Xeloda pooled (SO14695, SO14796)	5-Fluorouracil (5-FU)/Leucovorin (Mayo regimen)
Cetuximab (RAS wild-type population)127–129	NA	1L	CRYSTAL	FOLFIRI
Cetuximab (KRAS wild-type population)129–131	NA	1L	CRYSTAL	FOLFIRI
Regorafenib132–134	NA	2L+	CORRECT	Placebo + BSC
Trifluridine + tipiracil + BSC135–137	NA	3L+	RECOURSE trial (TPU-TAS-102-301 trial)	Placebo + BSC
Panitumumab + FOLFOX 4138–141	NA	1L	PRIME (KRAS wild-type)	FOLFOX4
Ziv-aflibercept + FOLFIRI142–144	NA	2L	VELOUR	FOLFIRI + placebo
Prostate cancer
Abiraterone145–149	NA	2L	COU-AA-301	Placebo
Abiraterone145,147,148,150–153	NA	1L	COU-AA-302	Placebo
Cabazitaxel154	NA	2L	TROPIC	Mitoxantrone
Enzalutamide155–158	NA	1L	PREVAIL	Placebo
Enzalutamide155–157,159,160	NA	2L	AFFIRM	Placebo
Radium-223 dichloride161–163	NA	1L	ALSYMPCA	Placebo
Sipuleucel-T164,165	NA	1L	IMPACT	Placebo

Abbreviations: 1L, first line; 2L, second line; BSC, best support care; IFN, interferon; NA, not applicable; NSCLC, non-small cell lung cancer; NSQ, non-squamous; PD-L1, programmed death ligand 1; SQ, squamous.

Table S2

Parametric curves selected for OS extrapolation calculations

Trial	Study treatment	Comparator	Parametric curve selected – treatment	Parametric curve selected – comparator
Melanoma
BREAK-3	Dabrafenib	Dacarbazine	Log-normal	Log-normal
COMBI-V	Dabrafenib + trametinib	Vemurafenib	Generalized gamma	Log-logistic
COMBI-D	Dabrafenib + trametinib	Dabrafenib	Log-logistic	Log-logistic
MDX010-20	Ipilimumab	Gp100	Generalized gamma	Log-logistic
CA031	Nab-paclitaxel	Paclitaxel	Log-logistic	Log-normal
CheckMate 037	Nivolumab	Dacarbazine or carboplatin + Paclitaxel	Log-normal	Gamma
CheckMate 066	Nivolumab	Dacarbazine	Gompertz	Exponential
CheckMate 069	Nivolumab + ipilimumab	Ipilimumab	Log-normal	Log-normal
KEYNOTE-006	Pembrolizumab	Ipilimumab	Log-normal	Gamma
METRIC	Trametinib	Dacarbazine or paclitaxel	Log-logistic	Gompertz
BRIM-3	Vemurafenib	Dacarbazine	Gompertz	Gamma
NSCLC
E4599	Bevacizumab + paclitaxel + carboplatin	Paclitaxel + carboplatin	Gamma	Generalized gamma
PROFILE 1007	Crizotinib	Docetaxel or pemetrexed	Exponential	Log-logistic
TAX 320	Docetaxel	Ifosfamide or vinorelbine	Log-normal	Log-logistic
BR.21	Erlotinib	Placebo	Generalized gamma	Log-normal
IPASS	Gefitinib	Carboplatin + paclitaxel	Weibull	Generalized gamma
CA031	Nab-paclitaxel	Solvent-based paclitaxel	Log-logistic	Log-normal
CheckMate 017	Nivolumab	Docetaxel	Log-logistic	Exponential
CheckMate 057	Nivolumab	Docetaxel	Weibull	Weibull
KEYNOTE-010	Pembrolizumab (2 mg/kg, ≥1% PD-L1)	Docetaxel	Exponential	Generalized gamma
FDA study	Pemetrexed	Cisplatin + gemcitabine	Log-normal	Log-logistic
PARAMOUNT	Pemetrexed	Placebo	Gamma	Log-logistic
JMEN	Pemetrexed	Placebo	Log-logistic	Log-logistic
REVEL	Ramucirumab	Docetaxel	Generalized gamma	Gamma
Renal cell carcinoma
AXIS	Axitinib	Sorafenib	Weibull	Gamma
METEOR trial	Cabozantinib	Everolimus	Log-logistic	Weibull
RECORD-1	Everolimus	Placebo	Log-logistic	Log-normal
Study 2 FDA label	Lenvatinib + everolimus	Everolimus	Gompertz	Log-normal
CheckMate 025	Nivolumab	Everolimus	Exponential	Exponential
VEG105192	Pazopanib	Placebo	Gamma	Weibull
TARGET	Sorafenib	Placebo	Gamma	Gamma
A6181034	Sunitinib	IFN-alpha	Gamma	Weibull
3066K1-304-WW	Temsirolimus	IFN-alpha	Log-normal	Log-normal

Abbreviations: FDA, US Food and Drug Administration; IFN, interferon; NSCLC, non-small cell lung cancer; OS, overall survival; PD-L1, programmed death ligand 1.