Approving cancer treatments based on endpoints other than overall survival: an analysis of historical data using the PACE Continuous Innovation Indicators™ (CII).

BACKGROUND: There is an active debate about the role that endpoints other than overall survival (OS) should play in the drug approval process. Yet the term 'surrogate endpoint' implies that OS is the only critical metric for regulatory approval of cancer treatments. We systematically analyzed the relationship between U.S. Food and Drug Administration (FDA) approval and publication of OS evidence to understand better the risks and benefits of delaying approval until OS evidence is available. SCOPE: Using the PACE Continuous Innovation Indicators (CII) platform, we analyzed the effects of cancer type, treatment goal, and year of approval on the lag time between FDA approval and publication of first significant OS finding for 53 treatments approved between 1952 and 2016 for 10 cancer types (n = 71 approved indications).
FINDINGS: Greater than 59% of treatments were approved before significant OS data for the approved indication were published. Of the drugs in the sample, 31% had lags between approval and first published OS evidence of 4 years or longer. The average number of years between approval and first OS evidence varied by cancer type and did not reliably predict the eventual amount of OS evidence accumulated.
CONCLUSIONS: Striking the right balance between early access and minimizing risk is a central challenge for regulators worldwide. We illustrate that endpoints other than OS have long helped to provide timely access to new medicines, including many current standards of care. We found that many critical drugs are approved many years before OS data are published, and that OS may not be the most appropriate endpoint in some treatment contexts. Our examination of approved treatments without significant OS data suggests contexts where OS may not be the most relevant endpoint and highlights the importance of using a wide variety of fit-for-purpose evidence types in the approval process.

Introduction

All cancer treatments aim to counteract the disease process. Although few are curative, most can extend life or ameliorate suffering. Patients are often eager to obtain access to new treatments that show tangible effects on tumor size and disease progression, even if overall survival (OS) data are not yet available. However, these treatments also come with significant side effects, are expensive, and are potentially harmful. The tension between providing timely access to promising treatments and ensuring treatment efficacy has given rise to a debate about the acceptability of so-called surrogate endpoints for regulatory approval of oncology treatments.

Some policy commentators have advocated for broader use of surrogate endpoints to support regulatory approvals. For example, a 2012 report by the President’s Council of Advisors on Science and Technology recommended that the U.S. Food and Drug Administration (FDA) “should make full use of accelerated approval for all drugs meeting the statutory standard of addressing an unmet need for a serious or life threatening disease, and demonstrating an impact on a clinical endpoint other than survival or irreversible morbidity, or on a surrogate endpoint, likely to predict clinical benefit” [1]. Similarly, the 21st Century Cures Act, signed into law in December 2016, encourages greater reliance on surrogate endpoints in drug development [2].

Other authors argue that surrogate endpoints are often poorly correlated with clinical outcomes and provide insufficient evidence for regulatory approval. One analysis of cancer drug approvals from 2008 to 2012 (n = 54, includes hematologic malignancies and approvals for new indications) found that two-thirds were approved based on surrogate endpoints, and, of these, only 14% were subsequently shown to improve OS at a median follow-up of 4.4 years [3]. Although the authors of that study suggested better enforcement of postmarketing studies as a primary solution, other authors argue that surrogate endpoints should not be used at all, at least until they have been validated as appropriate predictors of clinical outcomes within a given treatment context [4,5].

Many studies supporting or discouraging the use of surrogates employ a case study approach or restrict analyses to relatively narrow timeframes or treatment contexts [6,7]. A broader analytical approach is needed: the advantages and disadvantages of surrogate endpoints must be evaluated systematically. Moreover, although the short-term benefits of using surrogate endpoints are clear—faster, less expensive trials, and earlier access for patients—the long-term costs and benefits are more difficult to assess.

Drawing from a database of OS findings for treatments of 12 solid tumors, the Patient Access to Cancer care Excellence (PACE) Continuous Innovation Indicators™ (CII) provides visualizations and insights into how evidence of increased OS has accumulated in the past and continues to evolve over time [8]. To date, we have reviewed more than 12,000 literature references and curated more than 3000 pieces of evidence. The CII ‘Approval Date Analysis’ tool allows users to calculate the time between FDA approvals and the publication of evidence that a treatment improves OS. This tool allows for analyses of historical data that are not constrained to short time frames or single diseases. In this paper, we use the CII to explore the factors that influence the time in years between the date of U.S. drug approval and the date of first published OS evidence, as well as to gain a better understanding of the consequences—both positive and negative—of approving drugs before OS data are mature.

Methods

The PACE CII provides data and visualizations drawn from published scientific studies for 12 solid cancers from the early 1900s through the present (data current through November 1, 2016, and updated biannually). The full methodology of the CII has been published previously [8]. In brief, individual studies are evaluated to determine whether they provide evidence that one or more treatments significantly improve OS compared to a control. Each significant OS result, or ‘Piece of Evidence’ (PoE), is then assigned a weight depending on the quality of the evidence, the disease state of the treatment population, and the goal of treatment. These weighted values, called ‘evidence scores’ or ‘E-scores,’ can be aggregated and compared across different drugs, cancer types, and other relevant variables. The Approval Date Analysis tool uses information about FDA approvals sourced from FDA.gov and other publicly available sources to present E-score data in the context of the date that a given drug was approved for marketing in the United States. Approval dates are listed for the first time that a drug was approved for a given cancer type. The CII is agnostic to line extensions, new formularies, and new combinations of previously approved agents. The CII interactive tools, as well as downloadable data, can be found online at www.scoringprogress.com. All analyses herein are based on the data from this online tool and were conducted using R version 3.3.2.

We focused our analyses on the 10 most common solid tumors: breast cancer, colorectal cancer, endometrial cancer, gastric cancer, liver cancer, melanoma, non-small-cell lung cancer (NSCLC), pancreatic cancer, prostate cancer, and renal cancer. We used the CII Approval Date Analysis tool, data downloaded from the CII platform, and records of nonsignificant OS findings in the CII database to explore the relationship between FDA approval and publication of OS evidence for single agent treatments for the 10 included cancers. This yielded a dataset comprising 71 approvals for 53 treatments between 1952 and 2016.

The CII calculates ‘approval distance’ for each PoE as the year of publication minus the year the drug was approved for a given indication. We created a dataset that contained approval distances for the first PoE in the CII database for each treatment by cancer type. We used descriptive statistics and data visualizations to explore how these approval distances varied by the cancer type studied, treatment goal, and year of FDA approval. To determine whether the treatments approved alongside the publication of OS data are ultimately supported by more evidence than the treatments with a longer approval distance (i.e. those approved based on surrogate endpoints), we also assessed the relationship between approval distance and further innovation, as indicated by the accumulation of additional PoEs over time. This was performed using a linear regression of the E-score increase per year for a given treatment compared with the approval distance.

To gain deeper insight into the potential benefits and risks of approving drugs when OS data are delayed, we conducted qualitative analyses of approved drugs for which no positive OS evidence yet exists. For these analyses, we utilized PubMed, Drugs@FDA, and other web sources (e.g. National Cancer Institute Physician Data Query, Drug Dictionary, and A to Z List of Cancer Drugs), as well as published data on the treatments.

Results

Our analyses show that most cancer treatments were approved before any data showing improvement on OS were published. Approval distances vary by the cancer type, treatment goal, and approval year. However, the approval distance did not predict the amount of positive evidence that would eventually develop. By comparing the data from the CII to other data sources, we identified 14 treatments that have been approved but have not yet been shown to improve OS. A close examination of these drugs suggested that many have demonstrated clinical significance through noninferiority or other endpoints.

Many cancer treatments have long approval distances

Across the 71 drug/cancer type combinations in the dataset, 11.3% were approved after evidence of survival had been published, resulting in a negative approval distance; 29.6% were approved in the same year the first PoE was published, resulting in an approval distance of zero; and 59.2% were approved before any PoE was published, resulting in approval distances greater than zero. Approval distances ranged from −15 to 46 years (mean = 4.9, median = 1.0).

For drugs with approval distances of 1–3 years, it is possible that OS data were available to the FDA for approval but not published until later, or that OS data matured shortly after FDA approval. However, for 31.0% of drugs in the sample, the approval distances were longer than 3 years (Table 1). Indeed, 12 drugs (16.9%) were initially approved by the FDA more than 10 years before evidence of OS improvements was published. Among those drugs with long approval distances (Table 1) are many staples of cancer treatment, including leucovorin (approval distance: 46 years), cyclophosphamide (45 years), doxorubicin (29 years), fluorouracil (32 years [colorectal], 17 years [pancreatic]), tamoxifen (15 years), fulvestrant (12 years), bicalutamide (11 years), goserelin (8 years), paclitaxel (7 years), capecitabine (4 years), and pemetrexed (4 years).

Table 1

Approval details for drugs with long (>4 years) approval distances.

Treatment	Indication	Year approved	Evidence type supporting approvala	Year of first OS evidence	Approval distance
Leucovorin	Colorectal	1952	bTumor response rate [9]	1998	46
Cyclophosphamide	Breast	1959	bSignificant benefit for more than 1 monthd [10]	2004	45
Diethylstilbestrolc	Breast	1960	bTumor regression; survival difference between responders and nonresponders [11]	1999	39
Fluorouracil	Colorectal	1962	bClinically useful response [12]	1994	32
Doxorubicin	Breast	1974	b50% Regression for at least 1 month [13]	2003	29
Estramustinec	Prostate	1981	bObjective response rate [14]	2004	23
Fluorouracil	Pancreatic	1962	bObjective response rate [15]	1979	17
Tamoxifenc	Breast	1977	bTumor shrinkage or radiological regression of metastases; subjective improvement > 3 months [16]	1992	15
Fulvestrant	Breast	2002	Response rate; time to progression	2014	12
Bicalutamidec	Prostate	1995	bPartial regression; progression; or stable disease [17]	2006	11
Capecitabine	Colorectal	2001	Response rate; equivalent survival	2012	11
Panitumumab	Colorectal	2006	Progression free survival	2014	8
Sorafenib	Renal	2005	Progression free survival	2013	8
Strontium 89	Prostate	1993	bPain; quality of life; toxicity [18]	2001	8
Goserelinc	Prostate	1989	bEquivalence to orchiectomy; objective response [19]	1997	8
Paclitaxel	Breast	1994	Response rate	2001	7
Epirubicin	Breast	1999	Relapse free survival	2005	6
Lapatinib	Breast	2007	Time to progression	2013	6
Capecitabine	Breast	1998	Response rate	2002	4
Gemcitabine	Breast	2004	Time to progression; overall response rate	2008	4
Ixabepilone	Breast	2007	Progression free survival	2011	4
Pemetrexed	NSCLC	2004	Response rate	2008	4

Unless otherwise noted, evidence supporting approval was derived from the Drugs@FDA database (https://www.accessdata.fda.gov); however, approval letters and details are typically unavailable for approvals prior to 1998. In those cases, we provide the references for original papers or historical reviews containing information about the clinical trial evidence most likely available at the time.

Evidence derived from the historical literature instead of FDA approval letter. For a general overview of the decades leading to the prominence of OS as the primary endpoint for most FDA approvals, see Chapter 8 of ‘The Death of Cancer’ [20].

Antihormonal treatments were typically approved based on noninferiority trials compared to surgical removal of the hormone source. Trials seeking an OS benefit were typically conducted later in additional treatment contexts. See Cancer Research UK website for a summary of the introduction of tamoxifen into clinical practice (http://scienceblog.cancerresearchuk.org/2012/10/15/high-impact-science-tamoxifen-the-start-of-something-big/ [Last accessed: 20 September 2017]).

Defined as significant shrinkage of tumor, masses and organomegaly, reduction in fever, weight gain, rise in hemoglobin, and remission of symptoms to the extent the patients would tolerate a nonhospital environment if previously institutionalized.

NSCLC, non-small-cell lung cancer.

Approval distance varies by cancer type and treatment goal

There were large differences in approval distance patterns by cancer type. Figure 1 shows the histograms of approval distances for each cancer that had at least one PoE. (Endometrial cancer was included in the study; however, no approved drugs were present in the database with a PoE for endometrial cancer.) As this figure shows, OS evidence for breast cancer and colorectal cancer has frequently been published years after a drug was initially approved (median approval distances were 4.0 and 2.5 years, respectively). By contrast, for melanoma and liver cancer, approval distances tended to be shorter (median approval distance was zero years for each cancer).

Figure 1

Histograms of approval distances by cancer type. Histograms of approval distances by the type of cancer in which a significant survival benefit was first shown. The red line indicates an approval distance of zero: points to the right of the line were approved before OS data were published. Cancers with the lowest mean approval distances appear at the top of the figure, and those with the highest mean approval distances appear at the bottom.

Differences in approval distances across cancers may be in part due to differences in treatment goals and survival times associated with these goals. Figure 2 shows CII Value Matrices for cancers with relatively short (liver and melanoma) and long (colorectal and breast) approval distances. The right side of each matrix places each piece of evidence in the context of the population being treated. The left side of each matrix categorizes each PoE on the basis of the treatment goals. One striking difference is the much larger prevalence of evidence in cell J (adjuvant and preventative treatments) for colorectal and breast cancer. The patients receiving adjuvant treatment are likely to live for many years. By contrast, the majority of evidence for drug treatments for liver cancer and melanoma is in cell D (treatments designed to stop progression of advanced cancer). Survival for patients receiving these treatments is typically measured in months. Evaluating efficacy using OS can be achieved within a much shorter timeframe for this type of treatment than for adjuvant treatments.

Figure 2

CII Value Matrices for cancers with short (liver and melanoma) and long (colorectal and breast) average approval distances. Value Matrices representing treatments supported by published OS evidence for four cancer types: (a) liver cancer, (b) melanoma, (c) colorectal cancer, and (d) breast cancer. The Value Matrix framework depicts each treatment as a circle classified by the treatment goal (left axis) and disease state (right axis). Circle colors denote treatment modalities for surgery (blue), radiation (green), chemotherapy (brown), class level evidence (purple), and other (pink). Darker shades indicate the treatments supported by greater amounts of evidence, and larger circles indicate the treatments that are more widely used in clinical practice.

Indeed, when we compared the approval distances for single agents used to treat breast cancer that first (or only) showed OS evidence in cell D to those that first (or only) showed OS evidence in cell J (see Figure 3), the three treatments that were used exclusively in the adjuvant setting had approval distances of 6 years or more (median = 29 years). By contrast, the median approval distance for the 16 treatments with first OS evidence to stop progression of advanced disease was 3.5 years.

Figure 3

E-scores for breast cancer treatments for advanced disease (a) and in the adjuvant setting (b). (a) E-scores for the 16 breast cancer treatments in the CII where evidence was first (or only) published in cell D (to stop progression in advanced cancer). (b) E-scores in relation to the year of approval for the three breast cancer treatments where OS evidence was first (or only) published in cell J (adjuvant or preventative treatments). The vertical black line illustrates the point at which each drug was approved, and the black dots indicate each point in time when OS evidence was published for each drug.

Approval distances over time

Figure 4a shows the approval distances of all the drugs with mature, positive OS data in the dataset by year of approval. The black line indicates drugs with an approval distance of zero. Drugs above the line were approved before OS evidence was published. As the figure shows, approvals that co-occur with or post-date published OS evidence appear to have become much more common since the 1990s. The red line on the graph indicates the year 2016: it is not yet possible to observe lag times that might eventually appear above this line (i.e. in the future). This illustrates an important limitation for exploring approval distances: the more recently a drug was approved, the less time evidence has had to accumulate. This figure also demonstrates that recently published studies have provided foundational OS data for treatments approved from the mid-20th century to the present.

Figure 4

(a) Approval distance by year of FDA approval and (b) number of approved drugs with no significant survival data by year. (a) Scatterplot of approval distance by year of FDA approval. Darker points indicate that multiple data points exist in the same location. Horizontal black line indicates an approval distance of zero (i.e. OS evidence published in the same year that the treatment was initially approved). Red line represents the year 2016. Approved drugs that currently lack significant survival data do not appear in this figure, and are instead indicated in Figure 4b. (b) Line graph of the number of approved drugs with no significant survival data by year. Blue bars represent the number of treatments with evidence demonstrating significant noninferiority compared to controls, red bars represent the number of treatments with nonsignificant OS evidence, and green bars represent the number of treatments with no published OS evidence as of the writing of this manuscript.

Approved treatments without significant OS data

By comparing the lists of FDA-approved cancer treatments from 1952 to 2016 to the OS data in the CII, we identified 14 approved drugs that have not yet been supported by the positive OS data. Figure 4b illustrates the number of treatments that fall into each of these categories by the year of approval. Table 2 lists these drugs and shows the evidence available for each one.

Table 2

FDA-approved treatments that lack positive OS data in the CII database.

Generic name	Year first approved	Type of OS evidence available	Approval indication	Other significant evidence
Ceritinib	2014	None	NSCLC	Response rate, duration of response
Alectinib	2015	None	NSCLC	Response rate, duration of response
Osimertinib	2015	None	NSCLC	Response rate, duration of response
Palbociclib	2015	None	Breast	PFS
Gefitinib	2015	Non-inferior	NSCLC	Noninferior OS to docetaxel, response rate, duration of response
Toremifene citrate	1997	Non-inferior	Breast	Noninferior OS to tamoxifen
Pazopanib	2009	Non-inferior	Renal	Noninferior OS to sunitinib, PFS, response rate
Aldesleukin	1992	Nonsignificant	Renal	Response rate, duration of response
Mitoxantrone	1996	Nonsignificant	Prostate	Time to treatment failure, time to progression, quality of life
Peginterferon alpha-2b	2001	Nonsignificant	Melanoma	Relapse free survival
Sunitinib malate	2006	Nonsignificant	Renal	PFS, response rate
Everolimus	2009	Nonsignificant	Renal	PFS
Crizotinib	2011	Nonsignificant	NSCLC	PFS, response rate
Axitinib	2012	Nonsignificant	Renal	PFS

NSCLC, non-small-cell lung cancer; OS, overall survival; PFS, progression-free survival.

Four treatments in Table 2 have no OS results in the CII database. All four were approved through the accelerated approval pathway since 2014 based on response rate or progression-free survival (PFS). Postmarketing studies have yet to determine whether these drugs improve OS. While the field is waiting for the emergence of sufficient OS evidence, patients and physicians must make treatment decisions based on limited information about a drug’s efficacy.

Three treatments in the table have been demonstrated to be statistically noninferior to control treatments and, in some cases, have shown superior performance on safety and quality-of-life measures. Thus, they improve patient quality of life without any negative consequences for survival. The utility of gefitinib for treating a subgroup of NSCLC cancers with EGFR mutations has been recognized by the FDA [21], and pazopanib is currently being studied as part of combination regimens for soft tissue sarcomas.

Data showing nonsignificant differences in OS are available in the literature for seven of the treatments in Table 2, suggesting that these drugs may not significantly improve OS compared to control treatments. However, a closer look at these studies shows that most of these treatments have demonstrated superiority to control treatments on some other clinically relevant measure. For example, there is evidence that aldesleukin may produce durable, complete remissions in a small minority (approximately 5%) of patients, and mitroxantrone is an appealing option for palliative care because it has been shown to improve time-to-treatment failure, time to progression, and quality of life when used in combination with hydrocortisone [22]. Sunitinib, everolimus, and axitinib were all approved for the treatment of renal cancer on the basis of positive PFS findings. Recent work has suggested that long survival times, together with the use of trials that allow patients to switch treatments upon disease progression (also known as ‘crossing over’), can confound OS data from clinical trials in renal cancer, and that differences in PFS are clinically relevant and should be considered a primary endpoint in this treatment context [23].

Crizotinib, while initially showing nonsignificant survival evidence, continues to be studied with the expectation that it may improve OS for some patient subgroups [24]. Furthermore, recent evidence suggests that radiotherapeutic local ablative therapy can be used to ‘weed out’ tumors that develop resistance to crizotinib, potentially allowing patients to receive this treatment for longer periods of time, leading to substantial increases in survival [25]. This type of treatment combination effect is typically not reflected in phase 3 trial results but can be captured with more sophisticated result-tracking tools such as the CII.

Approval distance does not predict accumulation of further evidence

To explore how positive OS evidence has accumulated, we calculated the average E-score increase per year since the first PoE was published for all drugs in the database approved before 2010. More recently approved drugs were excluded from this analysis, because the limited number of years since approval inflated E-score per year estimates for these drugs. There was considerable variability in the rate of evidence accumulation: E-scores per year ranged from 0.05 to 1.45 for each treatment. A linear regression of E-score per year compared with approval distance yielded a marginally significant result (p = 0.051) with an R2 value of 0.091. This suggests that the length of time between approval and first published evidence of OS is not the primary factor that determines the rate at which OS evidence will accumulate in subsequent years.

Discussion

Long lag times between FDA approval and publication of OS data are common – including for many key treatments

Using the PACE CII data and analytical tools, we presented a systematic analysis of the historical impacts of approving cancer treatments based on surrogate endpoints. Taken together, these analyses suggest that the availability of evidence of OS at or close to the time of FDA approval is not a reliable indicator of a drug’s eventual impact. As an example, fluorouracil was first approved for use against breast, colorectal, gastric, and pancreatic cancers by the FDA in 1962. The evidence that it improved OS in any of the 10 solid tumors we examined was not published until 1979, when it was shown to nearly double survival in pancreatic cancer. Fluorouracil has since become a staple of combination chemotherapy treatments, with evidence of OS improvements in 5 of the 10 cancers studied here, and further evidence from studies of at least 28 different combination treatment regimens. Our analyses show that fluorouracil is not atypical among key cancer drugs in having a long approval distance. We showed that although cancer type, treatment intent, and time of approval may all influence approval distances, the time between FDA approval and publication of OS evidence does not predict the accumulation of further evidence.

One measure of the importance of a treatment to patients is inclusion on the World Health Organization’s (WHO) Model List of Essential Medicines [26]. The WHO considers that the drugs on this list are necessary to provide adequate health care and suggests that they should be available and affordable worldwide. Among the 15 WHO-listed treatments approved for use by the FDA against any of the 10 cancers we studied with at least one PoE as a single-agent treatment in the CII database, the median approval distance was 7 years. Eighty percent of the drugs had lag times of 3 years or more. If each drug were not approved until the date when OS evidence was first published, the cumulative number of years without access to these medicines in the United States would have been 182 years.

Our results suggest that some analyses that have been critical of the use of endpoints other than OS in the approvals process may underestimate the time it takes for positive survival data to emerge. For example, as mentioned in the introduction, a recent study used follow-up times between 3 and 7 years to infer that a majority of drugs approved on surrogate endpoints did not yield eventual evidence of survival [27]. However, 21% of the treatments in our analysis had an approval distance of at least 8 years. Our analysis shows that, for some drugs and treatment contexts, waiting for ultimate clinical endpoints (i.e. OS) could potentially restrict access to effective drugs for years. This is especially true for diseases with long survival times even in the comparator arm. In addition to the long follow-up time required, a long post-progression survival period—which is often overlooked when using common surrogate measures such as progression-free survival—can confound the measurement of OS, resulting in a nonsignificant OS finding even when survival is clearly improved [28]. However, although a lack of published OS evidence may not mean that a treatment is not effective, a system in which drugs are available for years without a clear understanding of their benefits is confusing and potentially harmful to patients. Because these treatments are costly, long lags between approval and final efficacy evidence also present a financial risk to patients and health care payers.

Phase 3 trials seeking true differences in OS may not be appropriate in all treatment contexts

The goal of clinical research is to understand which treatments work best in which contexts. Nonetheless, in some contexts, it may not be appropriate for a trial to seek to measure the true differences in OS between two treatments, for example, by prohibiting treatment switching between arms. Although measuring OS is always appropriate, researchers should acknowledge that OS results may not always reflect the true effect of an experimental treatment. This may be the case, for example, when treatment switching (also known as crossover, typically from a control treatment to the experimental treatment after disease progression) is ethically required [29,30]. This leads to a paradox where the most effective treatments (i.e. those that are most likely to have extensive treatment switching from the comparator) can be less likely to generate significant OS effects [23], in some cases being accepted as standard of care even when no significant effect was detected 5 years later [31]. Indeed, there have been attempts to prohibit treatment switching to preserve the fidelity of trial results, such as in the BRIM3 phase 3 trial of vemurafenib for BRAF-positive melanoma [32]. This led to a highly visible debate on the ethics of clinical trial designs [33], and, ultimately, the data safety monitoring board of the trial amended the protocol to allow treatment switching [34]. Similarly, others have acknowledged that it is ethically required to permit treatment switching in studies of targeted agents that cause significant tumor shrinkage along with symptom relief, such as crizotinib, when compared to cytotoxic chemotherapy, despite the confounding effects this has on the evaluation of OS [35]. Another example of the need for caution in interpreting nonsignificant OS findings is a phase 3 trial of gefinitib that permitted treatment switching. Although it did not show significantly improved OS compared to chemotherapy for EGFR-positive NSCLC, it significantly improved other endpoints such as response rate and progression-free survival. Moreover, survival in both arms was substantially higher than what would have been expected from chemotherapy alone, suggesting a significant improvement in OS based on the use of gefitinib in both arms [36].

Personalized treatments that target small patient subgroups can be particularly challenging and slow to test because of the practical and ethical challenges of identifying a sufficient number of patients, including controls. In fact, traditional, large randomized clinical trials with OS endpoints may be misleading or even counterproductive when studying targeted agents in mixed populations [37]. More biomarker-driven trials with appropriate patient selection could help mitigate this problem while speeding the development of and access to new medicines. Clinical trials also exclude the sickest patients from testing, leaving these patients understudied and underserved by the treatments available on the market. Thus, to complement randomized clinical trials, real-world evidence studies are needed during the initial postmarketing phase. These studies can provide key information on the safety and efficacy of drugs for populations typically excluded from registration trials. Such studies can also speed approvals for secondary indications. This approach would be particularly beneficial if early marketing access were granted to treatments on the basis of endpoints that are relatively fast to measure, thereby accelerating the pace of learning from real-world evidence.

Balancing access and risk

How should treatments be assessed to minimize risk without unduly limiting patient access? We believe two major research developments are needed. First, effectiveness data should be collected more quickly using more realistic patient populations by using real-world data and continuous learning systems in drug research. By providing the outcome data more quickly, the use of these data sources could reduce the risk of using surrogate endpoints for approval without limiting the patient access to promising treatments. Availability of real-world evidence and its consideration by regulatory bodies could especially speed the approval of new indications for treatments that already received initial marketing authorization. Learning from real-world evidence can also mitigate the risks to patients of rare or long-term toxicities caused by treatments that may not have been detected during registration trials. For example, the FDA Sentinel Initiative has used postmarketing data for pharmacovigilance surveillance and adverse event reporting since 2008 [38]. Although an examination of long-term toxicities is beyond the scope of this study, examining whether the rates of late emerging toxicities and adverse events are more prevalent for treatments with longer approval distances should be explored. Such a study would further inform the magnitude of the risk of making treatments available based on relatively limited evidence from randomized trials.

Second, as our analysis shows, endpoints other than OS can often indicate clinical value. Care should be given to consider the specific goals of a treatment within the context of a cancer and treatment population, and fit-for-purpose endpoints should be selected that provide the best and most timely measure of the treatment goals. The development of such patient-centric endpoints should be a high priority for the field.

As other authors have noted [6], the term surrogate endpoint is insufficient to describe the complex relationship between ultimate clinically relevant outcomes and proxy measures that can be used to predict such outcomes. The term may overlook the clinical significance of these metrics in their own right, as well as the important roles they can play in contexts where OS data are particularly time consuming or difficult to obtain. All evidence matters: the decision to approve a drug should be based on a careful analysis of the known risks and benefits as well as of the uncertainties, within the context of a particular treatment population, and with attention to effect sizes as well as statistical significance.

Different patient populations (and individual patients) have different tolerances for risk; their diverse needs must be considered early in the regulatory process, as well as in health technology assessments that can determine patient access in many countries. Learning from patients may help both clinical trialists and regulators to determine the most appropriate measures, endpoints, and follow-up metrics for new treatments in different diseases and treatment contexts. A refined framework in which clinical trials routinely use a wide variety of fit-for-purpose measures would be helpful to capture the complexity of relevant outcomes.

Conclusions

The history of cancer treatments provides important context for evaluating current debates within the field. The analyses described herein demonstrate the key role that endpoints other than OS play in the approvals process and the importance of specific clinical context such as disease and treatment goal in determining the most appropriate endpoints to consider. Analyses based on the PACE CII reveal a more nuanced picture than do reviews that focus on a few key milestone treatments or that investigate drugs in a limited treatment context, and they highlight the importance of using real-world data to expedite the availability of survival data, as well as of designing fit-for-purpose endpoints that consider treatment goals and other contextual factors. These actions can improve the regulatory process by improving efficiency and minimizing the risks to patients inherent in withholding promising treatments until long-term outcome data are available.

The CII is publicly available at www.scoringprogress.com. Readers are encouraged to use the tool to conduct tailored analyses with their own assumptions and arrive at evidence-based estimates of risk compared with the benefit of regulatory approval that precedes evidence of OS.