Comparing Findings From a Friends of Cancer Research Exploratory Analysis of Real-World End Points With the Cancer Analysis System in England.

PURPOSE: This study compared real-world end points extracted from the Cancer Analysis System (CAS), a national cancer registry with linkage to national mortality and other health care databases in England, with those from diverse US oncology data sources, including electronic health care records, insurance claims, unstructured medical charts, or a combination, that participated in the Friends of Cancer Research Real-World Evidence Pilot Project 1.0. Consistency between data sets and between real-world overall survival (rwOS) was assessed in patients with immunotherapy-treated advanced non-small-cell lung cancer (aNSCLC). PATIENTS AND METHODS: Patients with aNSCLC, diagnosed between January 2013 and December 2017, who initiated treatment with approved programmed death ligand-1 (PD-[L]1) inhibitors until March 2018 were included. Real-world end points, including rwOS and real-world time to treatment discontinuation (rwTTD), were assessed using Kaplan-Meier analysis. A synthetic data set, Simulacrum, on the basis of conditional random sampling of the CAS data was used to develop and refine analysis scripts while protecting patient privacy.
RESULTS: Characteristics (age, sex, and histology) of the 2,035 patients with immunotherapy-treated aNSCLC included in the CAS study were broadly comparable with US data sets. In CAS, a higher proportion (46.7%) of patients received a PD-(L)1 inhibitor in the first line than in US data sets (18%-30%). Median rwOS (11.4 months; 95% CI, 10.4 to 12.7) and rwTTD (4.9 months; 95% CI, 4.7 to 5.1) were within the range of US-based data sets (rwOS, 8.6-13.5 months; rwTTD, 3.2-7.0 months).
CONCLUSION: The CAS findings were consistent with those from US-based oncology data sets. Such consistency is important for regulatory decision making. Differences observed between data sets may be explained by variation in health care settings, such as the timing of PD-(L)1 approval and reimbursement, and data capture.

INTRODUCTION

Randomized controlled trials (RCTs) are the gold standard for assessing safety and efficacy of drugs; however, RCTs have limitations such as controlled settings, selective patient populations, and short-term study outcomes that might not be generalizable to heterogenous real-world populations and settings.[1-3] There is a need to understand the effectiveness of medical interventions in representative real-world populations and for accelerated clinical evidence, which can be met by improved data analytics and unprecedented availability of real-world data (RWD).[4,5] Regulatory bodies including the US Food and Drug Administration, the European Medicines Agency, and health technology assessment agencies, such as the National Institute for Health and Care Excellence, are increasingly acknowledging the importance of RWD.[6-8] However, for RWD to be used widely to supplement or augment clinical trials, the validity of real-world clinical end points in specific RWD must be established.

CONTEXT

Key Objective

The Cancer Analysis System (CAS) is a national cancer registry with linkage to national mortality and other health care databases in England. Real-world end points, including overall survival and time to treatment discontinuation, were analyzed from the CAS database and compared with diverse US oncology data sources used in the Friends of Cancer Research Real-World Evidence Pilot Project 1.0.

Knowledge Generated

The CAS analysis demonstrates the consistent performance and validity of real-world end points across geographically and structurally diverse settings. It also validates the findings from the CAS database by way of comparison with multiple US data sets.

Relevance

Evaluation and comparison of the strengths, limitations, and validity of specific real-world data for addressing defined clinical questions can facilitate development of guidelines on fit-for-purpose real-world data and defining standards for real-world evidence studies intended to inform regulatory decisions.

A pilot RWD project conducted by Friends of Cancer Research (Friends) evaluated the reliability of real-world end points among programmed death-1 or programmed death ligand-1 (PD-[L]1) inhibitor–treated advanced non–small-cell lung cancer (aNSCLC; patients with advanced-stage IIIB-IV or early-stage recurred or progressed non–small-cell lung cancer [NSCLC]) patients by convening six oncology-focused US health care data sets from participating organizations, including Cancer Research Network, Cota Healthcare, Flatiron Health, IQVIA, OptumLabs Data Warehouse, and PCORnet.[5] The findings of the pilot project demonstrated that worthwhile data can be aggregated from diverse research-ready RWD and that real-world end points measured from these data are consistent with each other and are directionally similar to those observed in RCTs,[9] especially in relation to overall survival. That said, none of the US-based data sets that participated in the pilot captured mortality systematically in all patients and the approaches to measure mortality varied across the US health care data sources.

The present study expands on the work of the pilot study by including the Cancer Analysis System (CAS) in England. CAS is a cancer registry that covers more than 99% of all patients with cancer in England and contains data on patient and tumor characteristics, treatments, hospitalizations, and mortality. It combines data from the Cancer Outcomes and Services Dataset (COSD), collected by Public Health England's National Disease Registration Service, with linkage to other national data sets, including the Systemic Anticancer Therapy (SACT), Hospital Episode Statistics, the National Radiotherapy Dataset, and mortality data from the Office of National Statistics (ONS).[10] The CAS database is a vital resource for understanding patient care and outcomes in England and for supporting drug research and development globally. The present study aims to validate and compare real-world end points extracted from CAS with those from the six US oncology data sets from Friends' original pilot. Considering the population and health care system differences between the US data sets and CAS, along with the complete capture of mortality data in CAS, this study will help in the validation of real-world end points across diverse populations and health care settings.

PATIENTS AND METHODS

Study Design and Objectives

The present study followed the protocol of the US-based pilot project.[5] It used a retrospective observational cohort design in patients with aNSCLC (stage IIIB-IV) leveraging the CAS database. However, because of differences in content and structure of the CAS data and US data sets, some distinctions in the study design should be noted. For example, the CAS study cohort included only incident stage IIIB-IV histologically confirmed NSCLC patients, unlike several of the US-based data sets, which also included early-stage patients with documented evidence of progression. The overarching objective of the present study was to apply the original Friends' pilot study protocol to an ex-US setting and a different type of RWD, namely, a national cancer registry linked to national mortality and other health care databases. The study also had the following secondary objectives:

To describe and compare the demographic and clinical characteristics of patients with aNSCLC treated with PD-(L)1 immune checkpoint inhibitors.

To assess real-world end points (real-world overall survival [rwOS] and real-world time to treatment discontinuation [rwTTD]) in patients with aNSCLC treated with PD-(L)1 immune checkpoint inhibitors, overall and by clinical and demographic characteristics.

To highlight the performance and consistency of real-world end points, particularly rwOS, in CAS (with complete ascertainment of mortality) with the US data sets.

Patient Cohort

The study cohort flow diagram is shown in Figure 1. Histologically confirmed incident stage IIIB-IV NSCLC (referred to as aNSCLC) patients diagnosed between January 1, 2013, and December 31, 2017, who initiated treatment with approved PD-(L)1 immune checkpoint inhibitors (ie, nivolumab, pembrolizumab, or atezolizumab) until March 2018 (most recent available SACT data[11]) were eligible for analysis. Patients in CAS were excluded if they had unconfirmed stage at diagnosis and a concomitant nonmelanoma malignancy or had systemic cancer treatment recorded more than 30 days before the aNSCLC diagnosis. Patients retrieved from CAS had a shorter minimum potential follow-up of 3 months (until March 2018) compared with a minimum potential follow-up of 6 months for US-based data sets. Index date was defined as the initiation of therapy containing any PD-(L)1 inhibitor.

FIG 1.

Study cohort flow diagram. CDA, Cancer Drugs Fund; NSCLC, non–small-cell lung cancer; SACT, Systemic Anticancer Therapy.

Data Sources

Data were collected retrospectively from the CAS database. The CAS database comprises several linked databases. For the present study, COSD, SACT, and ONS were used. COSD contains patient demographics (eg, age, sex, ethnicity, and geographic region) and tumor characteristics (eg, staging, morphology, and performance status). The SACT dataset[11] is the national mandatory collection of systemic anticancer therapy from all National Health Service England chemotherapy providers. The CAS database contains patient-level data, which is subject to strict data protection rules. The process to access the CAS data has historically been long and complex. To simplify access to the CAS database for research, Health Data Insight CIC has developed a programming tool called Simulacrum, which contains artificial patient data and allows analytical programs to be developed before running queries on the CAS database. Simulacrum mimics the structure and types of CAS data, enabling faster analyses by enabling debugging and validation of programming code before running analyses on the CAS database. The details of the data sets or health care data organizations, which participated in the US-based pilot study, are described in the original publication.[5]

End Point Definition

End point definitions used in this analysis were closely aligned with the US pilot study protocol and are given in Appendix Table A1 where end points had to be adapted.[5]

TABLE A1.

Operational Definitions

Statistical Analysis

Descriptive statistics (counts [%] or means [with standard deviations]) were performed for demographic and clinical characteristics. The lines of therapy were derived using an algorithm for NSCLC developed with input from clinicians.[12] Continuous variables were summarized using medians and interquartile ranges, and frequencies were calculated for categorical variables. Kaplan-Meier analysis was performed for time-to-event end points (rwOS and rwTTD); these end points were subsequently summarized using median time (in months) with the associated 95% CIs. Spearman's rank-order correlation was used to estimate the correlations between rwOS and rwTTD.

RESULTS

Patient Identification and Demographic Characteristics

Table 1 shows the demographic and clinical characteristics of patients with aNSCLC treated with PD-(L)1 inhibitors in the CAS and six US oncology data sets.[5] Overall, 2,035 patients with aNSCLC were included in the analysis. The median age at advanced NSCLC diagnosis was 67 (interquartile range [IQR]: 60.0-73.0) years and 68.3 (IQR: 61.6-73.7) years at PD-(L)1 inhibitor initiation (Appendix Table A2). Median age was comparable with the US pilot study; however, the proportion of PD-(L)1–treated patients in the 65-74 years age group was higher in CAS (47.7%), compared with 30%-40% in the US data sets. Gender distribution in CAS (54.3% male) was within the range of the US data sets. In terms of ethnicity, the patient population in CAS was more homogeneous (91.6% White) than in the US data sets (65%-87%). The majority of CAS patients (81.7%) were diagnosed with stage IV disease, and 18.3% with stage IIIB. In the four US data sets with data on disease stage at initial diagnosis, 62%-91% of patients were diagnosed with stage IV disease. Distribution of histology was broadly similar to the US data sets (71.2 v range 66%-74% in US data sets), with a higher proportion of nonsquamous histology at diagnosis (Appendix Table A2). In the CAS, a considerably higher proportion (46.7%) of PD-(L)1 inhibitors were given as first-line treatment than in US data sets (18%-30%). Almost similar proportion (44.1%) of PD-(L)1 inhibitors in CAS were given in second line and relatively few (7.9% of patients) in third line or beyond. The majority (93.4%) of CAS patients treated with a PD-(L)1 inhibitor (in any therapy line) did not receive a subsequent therapy line during the study observation period. A subgroup analysis of patients below 50 years showed that the majority of patients received PD-(L)1 inhibitor as their second-line therapy (51.7%; n = 45) and 91% (n = 79) did not receive a subsequent line after their PD-(L)1 inhibitor treatment (Appendix Table A4). The median time from advanced diagnosis to PD-(L)1 inhibitor initiation of 4.9 (IQR: 1.6-11.7) months in CAS was shorter than in US data sets (ranging from 6 to 8 months), likely because of higher first-line PD-(L)1 usage in CAS. At 10.6 (IQR: 3.9-16.0) months, the median follow-up time from PD-(L)1 inhibitor initiation was slightly longer in CAS than in US data sets (6-9 months).

TABLE 1.

Description of Demographic and Clinical Characteristics of Patients With aNSCLC Treated With PD-(L)1 Checkpoint Inhibitor

TABLE A2.

Description of Demographic and Clinical Characteristics of Patients With aNSCLC Treated With PD-(L)1 Checkpoint Inhibitor

TABLE A4.

LOT for Age Group Below 50 Years

Real-World End Points

Table 2 shows the median time-to-event estimates for real-world end points in CAS. The median rwOS (Fig 2) was 11.4 months (95% CI, 10.4 to 12.7), consistent with the range observed in US data sets (from 8.6 to 13.5 months). The median rwTTD for CAS was 4.9 months (95% CI, 4.7 to 5.1), within the range of US data sets (3.2-7.0 months).

TABLE 2.

Median Time and 95% CI for Real-World Extracted End Points Comparing CAS With US Data Sets

FIG 2.

Comparison of median rwOS (with 95% CI) in CAS with six US data sets in patients with aNSCLC treated with PD-(L)1 inhibitors. aNSCLC, advanced non–small-cell lung cancer; CAS, Cancer Analysis System; OS, overall survival; PD-(L)1, programmed death-1 or programmed death ligand-1; rwOS, real-world overall survival.

Median rwOS and 95% CI segmented by treatment setting and patient demographic characteristics are given in Table 3. Median rwOS in CAS was higher for the age groups 65-74 and ≥ 75 years (both around 12.2 months). Unexpectedly, survival improved with age among patients in CAS and a similar pattern was reported only in US data set E. In CAS, median rwOS at 13.5 months was higher in female than in male patients (9.95 months), consistent with the pattern seen in most of the US data sets. Within the CAS database, stage III patients had higher median rwOS (15.8 months) than many US data sets (B, C, and D). For CAS patients, median rwOS was higher for nonsquamous than squamous aNSCLC (11.8 months) within the range of US data sets (from 8.7 to 14.2 months). Median rwOS decreased almost linearly with increasing therapy line in CAS, whereas in US data sets, there was greater variability in survival across therapy lines with gains in survival also observed in higher lines. In the first-line setting, the median rwOS in CAS was 13.4 months (95% CI, 11.9 to 15.6). In the second-line setting, it was 10.2 months (95% CI, 8.9 to 11.8). In the US-based data sets, there was a wide range in first-line median rwOS from 9.4 to 20.8 months. In the second-line setting, where the US range for median rwOS was narrower 7.9-11.7 (95% CI, 6.5 to 10.5-10.9 to 12.8), the CAS results were also consistent with the observed range (Appendix Table A3). Unlike the US data sets, mortality information is systematically completed for all patients in CAS through linkage with ONS. Correlations between rwOS and rwTTD for CAS and US data sets are shown in Appendix Table A5 (0.7 in CAS compared with the US range of 0.6-0.9).

TABLE 3.

Median rwOS and 95% CI, Segmented by Treatment Setting and Patient Characteristics Comparing CAS With US Data Sets

TABLE A3.

Median rwOS and 95% CI, Segmented by Treatment Setting and Patient Characteristics

TABLE A5.

Correlations Between rwOS and rwTTD in CAS Compared With US Data Sets Using Spearman's Rank Correlation Coefficient

DISCUSSION

This study compared the findings from the US-based Friends' pilot project with a UK Cancer Registry (CAS database) to characterize the validity of real-world end points for addressing clinically relevant questions on treatment effectiveness, determination of unmet need, and use of PD-(L)1 inhibitors across diverse health care settings. This study is based on the UK National Health Service, which operates on a fundamentally different health care model (ie, free for all at the point of use). Furthermore, this study helps to assess the performance of real-world end points because of the completeness of patient and mortality information and validate the CAS database by way of comparison with diverse US data sets.

A comparison of the present study with the US-based pilot study highlights a few key differences between CAS and the US data sets in terms of population and database characteristics. CAS captures clinical characteristics related to TNM staging, whereas this information is generally not available in US-based claims data. Thus, unlike most US-based data sets, which also included patients early-stage progressed NSCLC, the present study included only incident stage IIIB-IV NSCLC, with more than 80% of patients diagnosed in stage IV. In CAS, the median follow-up time from PD-(L)1 inhibitor initiation was longer than that in the US data sets, mainly attributed to the large proportion of PD-(L)1 inhibitors in patients within first-line setting. The estimated overall median rwOS in CAS was within the range of estimates observed in the US data sets although the observed US range was relatively wide. Variations in rwOS in the US data sets are likely due to challenges in accessing mortality data, as deaths are not routinely recorded in most electronic health care records or insurance claims. In contrast to CAS where mortality is ascertained through regularly updated linkage with official national mortality statistics, some US-based RWD rely on published obituary or insurance data to supplement the gaps in mortality information.[13] Combining data from multiple research-ready real-world sources under a common framework can enhance the reliability of real-world end points. Therefore, combining different data types and sources allows timely availability of data, addresses missing data, and improves completeness to create robust data sources.[13]

RCTs demonstrate efficacy of PD-(L)1 inhibitors under optimal settings, whereas real-world evidence (RWE) provides realistic estimates of effectiveness in routine clinical practice. There can be differences in the results of RWD and RCTs; however, considering the complexities in health care systems and high degree of variation in treatment response in the real world, RWD may differ from RCTs but still be valid.[14]

The CAS database has several key strengths including the use of synthetic data to facilitate data access while maintaining patient confidentiality, linkage between a national population–based cancer registry of England with a national systemic therapy database (SACT) and other national databases, and most importantly, the inclusion of mortality data obtained via linkage with ONS. Mortality data are reliably captured through regular updates as structured fields and undergo regular evaluations of validity. This includes a tracing process annually, which ensures the completeness of the annual mortality update. Completeness of survival or mortality data emphasizes and validates the overall findings of the study.

This study also highlights some limitations of the CAS database. First, the CAS database currently does not contain biomarker data although this information is expected in the future. Information on patients' biomarker status would be relevant in interpreting the results of this study, given that response to treatment with PD-(L)1 inhibitors has been shown to correlate with the NSCLC molecular profile. Previous studies have failed to demonstrate unequivocal survival benefits of PD-(L)1 inhibitor monotherapy in patients with epidermal growth factor receptor (EGFR) or anaplastic lymphoma kinase (ALK) genetic alterations compared with standard chemotherapy.[15] Second, disease progression and treatment response are also not directly captured. Treatment-related intermediate end points, such as time to next treatment or death, can be derived from the available data and used to make inferences about possible disease progression. However, such intermediate treatment-related end points are generally limited to patients who have received a subsequent therapy. Compared with most electronic health care records, CAS has a longer lag time as it combines data from national cancer registry and other national databases, which contributed to a shorter observed follow-up time in our study. Because of the high completeness of mortality information in CAS, stronger insights can be gleaned into the value of nontraditional end points, such as time to treatment discontinuation (TTD). The strong correlation between overall survival and TTD in these data further elucidates the role of TTD and its potential use as an earlier indicator of clinical benefit. This could prove to be useful in evaluating the effectiveness of therapies in a broader range of RWD sets, particularly in instances where mortality information may be less accurate or incomplete.

Compared with the US-based study, a higher proportion of patients in CAS received treatment with PD-(L)1 inhibitor in the first-line setting. This was largely due to the time period studied, which reflects the later approval and reimbursement—and in the case of nivolumab also, more restricted use through the Cancer Drugs Fund—of PD-(L)1 inhibitors in England. The earlier approval and reimbursement of pembrolizumab in England, relative to the later approval and conditional reimbursement (through CDF) of nivolumab, has also led to a disproportionate number of patients treated with pembrolizumab in our study. Patients who received other PD-(L)1 inhibitors (ie, nivolumab) through the CDF are under-represented in our analysis because clinical outcomes data were not accessible for these patients.

RCTs are widely used by regulators because of their strong internal validity, despite having limited generalizability to real-world settings.[16] The CAS database represents more than 99% of the population of England and is nationally representative for patients with cancer. Thus, the results of this study are broadly representative of patients with aNSCLC treated with PD-(L)1 inhibitors in England over the observed time period. National coverage makes CAS a valuable resource for assessing real-world end points, and it may also be used for research in rare cancers, where RCTs may be less feasible. Furthermore, studies similar to ours can establish reference ranges for direct comparison of RWD with clinical trial data in oncology paving the way for alternative study designs.[17]

RWD collection practices for treatment and clinical parameters have improved over time in terms of data completeness and quality, and this trend is expected to continue.[18] In 2021, CAS is expected to incorporate biomarker data, which can provide additional insights into clinical characteristics of patients. Extension of initiatives such as Friends to multicountry settings will further support the development of best practices for the generation and evaluation of RWE to supplement RCTs in regulatory decision making and inform the development of future regulatory guidance.[14] RWE can support regulatory decision making about new or expanded medication indications in a number of ways, from the more established use of external controls for single-arm trials, which have also been used as the basis of US Food and Drug Administration–accelerated approvals, to the growing interest in the use of pragmatic trials and nonrandomized RWE from health care databases, particularly in situations where real-world outcomes and clinical practice patterns differ significantly from the tightly controlled RCT settings.[6,19] Studies similar to ours that establish research processes and evaluate the strengths, limitations, and validity of specific RWD for addressing defined clinical questions can support development of guidelines on fit-for-purpose RWD and defining standards for RWE studies intended to inform regulatory decisions.[19]

In conclusion, this study corroborates and extends the conclusions of the original pilot study that RWD can generate clinically meaningful and timely evidence on the efficacy of new cancer treatments used across diverse real-world settings. It further describes the usefulness of readily extractable end points, such as TTD, to assess clinical benefit.

Despite the variation in local clinical practice and data collection, there was considerable consistency in the findings between CAS and the US data sets. The observed differences in results could be largely explained by underlying differences in health care settings, including the timing of and variation in approvals and reimbursement, and data structure. This supports the premise that RWE can be informative for clinical, payer, policy, and regulatory decision making.