Real-world data from expanded access programmes in health technology assessments: a review of NICE technology appraisals.

OBJECTIVES: To quantify and characterise the usage of expanded access (EA) data in National Institute for Health and Care Excellence (NICE) technology appraisals (TAs). EA offers patients who are ineligible for clinical trials or registered treatment options, access to investigational therapies. Although EA programmes are increasingly used to collect real-world data, it is unknown if and how these date are used in NICE health technology assessments.
DESIGN: Cross-sectional study of NICE appraisals (2010-2020). We automatically downloaded and screened all available appraisal documentation on NICE website (over 8500 documents), searching for EA-related terms. Two reviewers independently labelled the EA usage by disease area, and whether it was used to inform safety, efficacy and/or resource use. We qualitatively describe the five appraisals with the most occurrences of EA-related terms. PRIMARY OUTCOME MEASURE: Number of TAs that used EA data to inform safety, efficacy and/or resource use analyses.
RESULTS: In 54.2% (206/380 appraisals), at least one reference to EA was made. 21.1% (80/380) of the TAs used EA data to inform safety (n=43), efficacy (n=47) and/or resource use (n=52). The number of TAs that use EA data remained stable over time, and the extent of EA data utilisation varied by disease area (p=0.001).
CONCLUSION: NICE uses EA data in over one in five appraisals. In synthesis with evidence from well-controlled trials, data collected from EA programmes may meaningfully inform cost-effectiveness modelling. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This study is the first to assess whether health technology assessments rely on data from expanded access programmes.

Our search was limited to health technology appraisals performed by National Institute for Health and Care Excellence between 2010 and 2020.

Combining automated and manual screening can efficiently facilitate health policy analyses.

Introduction

Novel drug therapies are important drivers of increased healthcare spending. In the UK, the National Institute for Health and Care Excellence (NICE) conducts technology appraisals (TAs) to evaluate cost-effectiveness of technologies (eg, drugs, medical devices) and to determine their impact on healthcare budgets.1 These evaluations are conducted using a variety of data sources, such as randomised controlled trials (RCTs) or observational studies.2 3 In this research, we explore the use of data in NICE TAs from another source: expanded access (EA) programmes.

A positive appraisal determination from NICE forms the main pathway for novel pharmaceutical technologies to access the National Health Service (NHS) and become available for patients across the UK. The health technology assessment (HTA) usually starts with the submission of evidence on clinical effectiveness and costs by the pharmaceutical company. The submission is scrutinised by an independent Evidence Review Group (ERG), which critically reviews the manufacturer’s submission and performs additional exploratory analyses of cost-effectiveness; in some cases, the ERG even reanalyses clinical data.1 4–6

Patients, patient advocacy groups and physicians working within the NHS also contribute to NICE’s appraisals. The resulting qualitative input is considered in the formal analyses conducted by the manufacturer and the ERG. The entire evidence is assessed by NICE’s Appraisal Committee and forms the basis of their appraisal determination.6 More detailed information on NICE’s processes can be found on their guidance website (https://www.nice.org.uk/about/what-we-do/our-programmes/nice-guidance).

HTA bodies are particularly keen to know how technologies will use resources, yield benefit and attribute risks in the real-world patient population for which treatment will potentially be reimbursed.7 Real-world data (RWD) are ‘information on healthcare that is derived from multiple sources outside typical clinical research settings’, such as electronic health records, claims and billing databases or patient registries.8 RWD is typically generated after a drug comes to market (postapproval). At the time of the reimbursement decision; however, most of the available data stem from clinical trials (preapproval). Noteworthy, payers may use (real-world) data from patients who have been treated outside of clinical trial settings, but prior to marketing authorisation.1–3 These patients can receive treatment via EA programmes.

EA is a pathway to access investigational medicine for patients who suffer from life-threatening conditions, who cannot enter clinical trials and have exhausted all approved treatment options. It is also known as ‘compassionate use’, ‘early access’ or ‘non-trial preapproval access’.9 The primary intent of EA programmes is to provide patients and physicians in dire need with potential treatment options outside of clinical trials. Secondary, such programmes may potentially collect RWD in a regulatory preapproval setting, but the generation and useability of evidence derived from these programmes remain a topic of debate.10–16

Data from EA programmes may be used for various purposes in the appraisal process, for example, to inform formal safety or efficacy analyses, to inform resource use and associated costs in real-world settings, to estimate the size of the patient population or to gain insights into the treatment experience from patients or physicians who participated in an EA programme. These data are increasingly accepted to support evidence of clinical efficacy by regulators, especially when collecting data in controlled settings is infeasible, such as in (ultra-)rare diseases, or is deemed unethical, in the case of extremely large treatment effects.10 However, the use of EA data by payers or HTA bodies remains unquantified. Understanding the role of EA data in TAs may clarify the value of these data for payers, pharmaceutical industry, physicians and patients and is relevant for cost-effectiveness decision-making and evaluation of HTA policy. Therefore, we here investigate the usage of EA data in NICE decision-making by reviewing all appraisals presented to NICE between 2010 and 2020.

Methods

Documents relating to all TAs conducted are provided on the NICE website. We investigated TAs published between 1 January 2010 and 1 January 2021. Terminated, withdrawn or replaced appraisals were removed as documentation was unavailable. A schematic overview of our workflow is provided in figure 1.

Figure 1

Screening and selection of technology appraisals from NICE. EA, expanded access; HST, highly specialised technology; MTA, multiple technology appraisals; NICE, National Institute for Health and Care Excellence; STA, single technology appraisal.

We wrote a computer script (ie, a web scraper)10 to automatically list and download all documentation (eg, manufacturer submissions, ERG report, final appraisal determination) available through NICE’s website. Subsequently, the script extracted the text from these documents and automatically screened whether the text contained ‘expanded access (EA) terms’, like ‘Compassionate Use’, ‘Expanded Access’ ‘Early Access’, etc, as well as all possible spellings thereof. A detailed protocol, including all search terms, is available in online supplemental file A. The data and code from the paper are available on the GitHub from the first author, https://githubcom/TobiasPolak. When at least one of these ‘EA terms’ were present, two authors (TBP and DGJC) independently and manually, reviewed the context of the term.

We primarily labelled the data usage with one or more of the following categories:

Safety: EA data were used to evaluate the safety profile.

Efficacy: EA data were used to evaluate the efficacy profile.

Resource use: EA data were used to inform cost parameters.

Trivial: EA data were not used or trivially mentioned in the appraisal.

Patient and physicians also share their treatment experience. As the impact of these accounts is harder to quantify, we did not include them in our main analysis but secondarily labelled:

Treatment experience: when patients or physicians cited experience within the EA programme.

Discordance was resolved by discussion between the two reviewers. To give the reader a sense of these different types of usage, examples are provided in the Results section. Additionally, we provide a narrative summary of the five appraisals that contain the most occurrences of the search terms to illustrate the use of EA data qualitatively. Finally, TAs were classified as single technology appraisal, multiple technology appraisal or highly specialised technology (HST). All TAs were categorised according to their area of disease.

Patient and public involvement

No patients were involved during the planning and writing of this work; all data were derived from NICE TAs.

Statistics

The Spearman rank correlation test was used to detect time trends in the yearly number of appraisals using EA data. We performed a Pearson χ2 test to assess whether the proportion of appraisals that included EA data differed by disease area. For all significance testing, we set the two-sided significance level at 0.05.

Results

We screened all 496 TAs conducted between 1 January 2010 and 1 January 2021. This ranged from Technology Appraisal 185 to Technology Appraisal 667 and from HST1 to HST13. n=116 appraisals were excluded (for details, see figure 1). The remaining 380 appraisals had 8925 documents that were downloaded and screened.

In 54.2% (206 of 380 appraisals), at least one reference to EA was made. In total, 80 out of 380 (21.1%) of the TAs used EA data to inform safety (n=43), efficacy (n=47) or resource use (n=52). As a single TA could have multiple labels, there is overlap between safety, efficacy and resource use. This is depicted in figure 2A. Additionally, in 54 appraisals (14.5%), the EA programme was cited by patients or physicians as treatment experience.

Figure 2

Technology appraisals (TAs) using expanded access (EA) data to support safety, efficacy and/or resource use. (A) Venn-diagram displaying the overlap of safety, efficacy, and/or resource use labelling of TAs. (B): Bar chart of TAs published between 1 January 2010 and 1 January 2021 that did (‘yes’) or did not (‘no’) include data EA programmes to support safety, efficacy and/or resource use.

Although there is a significant increase over time in the absolute use of EA data by payers (=0.73 and p=0.011; figure 2B), there is no evidence of a significant increase in use of EA data over time relative to the total number of appraisals conducted (=0.32 and p=0.332).

Significant differences ( = 38.8, p=0.001) exist in the disease areas that did versus those that did not include EA data. Oncology and haematology together account for 66% of the appraisals with EA data, whereas they make up 50% of the entire fraction of appraisals. On the other hand, disease areas such as cardiology, gastroenterology, endocrinology, dermatology, rheumatology and ophthalmology jointly make up 24.5% of all appraisals, whereas they merely account for 2.6% of the appraisals that included EA data. These results are found in table 1.

Table 1

Technology appraisals that did (‘yes’) or did not (‘no’) include expanded access (EA) data to support the profile of safety, efficacy and/or resource use, classified on disease area

	Included EA data		Total†	P value*
	No†	Yes†
Disease area				0.001
Benign haematology	5 (1.7%)	3 (3.8%)	8 (2.1%)
Cardiology	14 (4.7%)	0 (0%)	14 (3.7%)
Dermatology	12 (4.0%)	1 (1.3%)	13 (3.4%)
Endocrinology	12 (4.0%)	0 (0%)	12 (3.2%)
Gastroenterology	13 (4.3%)	0 (0%)	13 (3.4%)
Haematology	34 (11%)	21 (26%)	55 (14%)
Internal medicine	23 (7.6%)	9 (11%)	32 (8.4%)
Neurology	14 (4.7%)	6 (7.6%)	20 (5.3%)
Oncology	106 (35%)	32 (41%)	138 (36%)
Ophthalmology	18 (6.0%)	0 (0%)	18 (4.7%)
Psychiatry	3 (1.0%)	1 (1.3%)	4 (1.1%)
Pulmonology	6 (2.0%)	4 (5.1%)	10 (2.6%)
Rheumatology	22 (7.3%)	1 (1.3%)	23 (6.1%)
Surgery	4 (1.3%)	1 (1.3%)	5 (1.3%)
Urology	1 (0.3%)	1 (1.3%)	2 (0.5%)
Vascular medicine	13 (4.3%)	0 (0%)	13 (3.4%)
Total	300 (79%)	80 (21%)	380 (100%)

*Pearson chi-square test.

†n (%).

Examples

To give the reader a better sense of the main labels ‘safety, efficacy, resource use’ as well as the secondary ‘treatment experience’ label, we here provide illustrative examples from the TAs that were supported by EA data.

Safety

Safety data from EA programmes are often described rather qualitatively, supporting results from clinical trials. For example, in the appraisal of gefitinib for the first-line treatment of locally advanced or metastatic non-small-cell lung cancer, the appraisal committee noted:

The favourable safety profile of gefitinib demonstrated in the phase III studies is consistent with that observed in everyday settings. In addition to the data from clinical trials, the Early Access Program for gefitinib in Caucasian patients indicated that gefitinib is well tolerated by patients with advanced or metastatic NSCLC. The majority of ADRs associated with gefitinib are mild in nature and those most commonly reported are grade 1/2 diarrhoea and skin reactions.

Manufacturer submission, Safety and tolerability, TA192

Alternatively, safety signals from EA programmes can be quantitatively incorporated in cost-effectiveness analyses. When evaluating ocrelizumab for treating relapsing–remitting multiple sclerosis, the committee noted that an important safety signal from the compassionate use programme is lacking from the current analysis:

The committee heard that there has been the 1 case of PML (progressive multifocal leukoencephalopathy, red.) following treatment with ocrelizumab in the compassionate-use programme in Germany, (…). It concluded that the economic model should have included a risk of PML for ocrelizumab.

Appraisal consultation, Adverse events in the economic model, TA533

Efficacy

Efficacy data from EA programmes can also be used, together with data from clinical trials, to estimate overall efficacy of the technology appraised. In the evaluation of lutetium (177Lu) oxodotreotide for treating irresectable or metastatic neuroendocrine tumours, response rates were obtained from the ‘Erasmus study’. The Erasmus study was a compassionate use programme conducted at the Erasmus Medical Centre. The data from this programme are summarised as:

In a single centre non-controlled phase I/II open-label study (The Erasmus study, red.), conducted in 810 Dutch patients with different somatostatin receptor positive tumour types, the objective response rate (ORR) for the full analysis set (FAS) population with GEP-NETs and bronchial NETs (360 patients) was 44% (95% confidence interval [CI] 38%–49%).

Manufacturer submission, Executive summary, TA539

NICE requires that benefits of technologies are evaluated using quality-adjusted life years (QALYs), as NICE’s decision to recommend or not recommend a product for reimbursement depends (among other things) on the willingness-to-pay for an incremental year in perfect health—the so-called cost-per-QALY approach. In the evaluation of cabazitaxel for hormone-relapsed metastatic prostate cancer treated with docetaxel, the EA programme was used to gather quality of life data not collected during the routine clinical development:

The company did not collect data on health-related quality of life in TROPIC (the RCT, red.), so it took utility values from the UK Early Access Programme (EAP) for cabazitaxel. The programme measured the health-related quality of life (using the EQ-5D) of men who had been treated with cabazitaxel after docetaxel.(…)

(…) One hundred and twelve patients participated in the UK EAP at 12 UK Cancer Centres. All had mCRPC with disease progression during or after docetaxel and were similar in baseline patient characteristics to the population in TROPIC. (…) Safety assessments were performed prior to each cycle and HRQL recorded at alternate cycles using the EQ-5D-3L questionnaire and visual analogue scale (VAS).

Committee papers, Health-related quality of life, TA391

Resource use

EA data can also be used to inform other parameters in cost-effectiveness modelling. Such models are often based on Markov chains, which describe the state of the disease that patients are in at a given time point. These models require cost per state and transition probabilities or rates between states. Registries, or other RWD sources, are frequently used to estimate such data. In the appraisal of sofosbuvir–velpatasvir–voxilaprevir for treating chronic hepatitis C, transition probabilities from decompensated liver cirrhosis to death are modelled via a beta-distribution and the input parameters are provided from the EA programme:

Variable: From decompensated cirrhosis to death

Distribution and parameters: Beta; =46.5; =147.2

Source: EAP data (expanded access programme, red.)

Manufacturer submission, Ssensitivity analyses, TA507.

A different, direct resource use example is given in the evaluation of ipilimumab for previously treated irresectable malignant melanoma. The dosing of ipilimumab is weight dependent. Hence, to estimate the number of vials needed for treatment of UK patients, an estimate of the (UK) patient population weight is required. This weight is calculated via:

Patient level analysis of the weight of UK clinical trial patients in MDX010-20 (n=55), and the weight of UK patients in the ipilimumab compassionate use program (n=258), from these weights, the mean number of vials required (assuming no vial sharing) is calculated.

Results from these analyses showed that the dose of ipilimumab given per patient per induction has a large impact on the ICER with the minimum dose given in the trial and compassionate use programme (3×50 mg) resulting in an ICER of £38 387 per QALY gained and the maximum dose (2×200 mg) given resulting in an ICER of £88 788 per QALY gained.

Manufacturer submission, Intervention and comparators costs, TA268

Treatment experience

NHS professionals share their opinions and experience on the technology appraised in expert committee meetings. In the appraisal of patisiran for treating hereditary transthyretin amyloidosis, the Head of the National Amyloidosis Centre is asked ‘how data on real-world experience in this condition compare with clinical trial data?’. His response is:

The experience of my colleagues at the NAC treating patients through compassionate access (over one year) and Early Access to Medicine Schemes has been extremely favourable. Remarkable clinically significant improvements of well-being and function have occurred in a majority of cases, including regaining the ability to walk unaided.

Clinical expert statement, HST10

Patients, caregivers or patient group representatives are also provided the opportunity to share their experience with the appraised treatment. The assessment of nusinersen for treating muscular atrophy sparked comments from parents with children who suffer from this disease:

My son is currently receiving Spinraza at Gosh for type 1c SMA. He was lucky enough to be included into the expanded access program for a select group of children. Since receiving his treatment we have watched the transformation of a seriously weakening child to a thriving boy who has gained significant progress in his motor function and health, we are continually amazed by his progress. He starts preschool in the coming weeks, an achievement we never thought possible. (…)

Patient/caregiver stakeholder comment, TA588

The above provides qualitative examples of EA usage in NICE appraisals. To further illustrate how EA data are appraised by the manufacturer, ERG and NICE committee, and what the advantages and limitations of its use may be, a detailed discussion of the top-5 appraisals in which the search terms most frequently occurred are found in online supplemental file B. This includes representative examples in the areas of haemato-oncology (eg, prostate cancer, follicular lymphoma) and rare diseases (eg, spinal muscular atrophy).

Discussion

In this review, we combined automated documentation searches with double, independent manual review to screen NICE documentation on the usage of EA data for HTA. We have found that data from EA programmes are frequently included: 21.1% of the TAs used EA data to evaluate safety, efficacy/effectiveness or resource use of the appraised technology. The use of data from EA programmes appears to remain stable over the years. Additionally, patients and physicians share their treatment experience from an EA programme in 14.2% of the appraisals.

The disease areas of the appraisals that included EA data differed significantly from the overall distribution of disease areas from all appraisals investigated between 2010 and 2020. Oncology and haematology account for the lion’s share (66%) of EA data usage, yet account for half (50%) of all TAs conducted. Although ‘the life-threatening or seriously debilitating’ prerequisite for EA is often present in haemato-oncologic malignancies, cardiac or ophthalmologic illnesses can also be severely limiting.17 18 Cardiology and ophthalmology account for 8.4% of all TAs, but none (0%) of these programmes used EA data (or even mentioned it). There is a range of possible explanations for this discrepancy. Perhaps, drug developers in these areas may be less familiar with collecting and using EA data, or cardiologists and ophthalmologists may be less acquainted with EA than haemato-oncologists—simply because EA may be less warranted in these disease areas.19 20

Compared with regulatory submissions to the EMA and the FDA, submissions to NICE more frequently include EA data. The EMA and FDA used EA data to support efficacy in 49 regulatory approvals over 25 years (2 annually).10 In this work, we find that NICE used EA to inform cost-effectiveness in 76 over 11 years (7 annually). One reason for this may be that payers have a higher uptake of RWD in their decision-making. Furthermore, they also assess comparative effectiveness rather than efficacy. Modelling cost and comparative effectiveness by definition necessitate a variety of input parameters, every one of them potentially coming from different sources, such as EA.

Whether using EA data (or other non-randomised data) for payer decision-making is wise, depends in part on the robust design and execution of the EA programme, and the relevance to the decision problem.21 The instances in which the FDA and the EMA assessed efficacy mainly based on EA data, are scarce, and characterised by (1) a high unmet medical need (2) a rare disease population and (3) large treatment effects.10 Additionally, we witnessed twice (TA391, TA491) that health-related quality-of-life data were not gathered during the conventional clinical trials but were captured in the EA programme. Although data from EA programme can bridge an evidence gap, HRQoL data should simply have been collected during all stages of clinical development. For safety, the use of registries, postapproval safety studies, or pharmacovigilance during EA, is useful to detect infrequently occurring adverse events. Indeed, we identified such an example in TA533, where the compassionate use programme led to the identification of a rare but serious adverse event. Overall, the evidence for assessing safety and efficacy should primarily come from regulatory studies and can be synthesised with RWD or other non-randomised sources, such as EA programmes.

Including EA can have several advantages, as it can increase sample size, add robustness, inform additional parameters—such as HRQoL—or aid to estimate effects for patients who were excluded from the trial but were included in the EA programme. Such patients are generally older and frailer,7 22 23 and, thus, collecting data in these populations help to extrapolate results on safety and efficacy found in RCTs. Estimates of resource use parameters that are derived from clinical trials, such as adherence, monitoring or the number of hospital visits, can even be more distinct from real-world settings. Therefore, EA data can play a useful role in informing resource use parameters. Furthermore, modelling resource use requires estimates of a large number of input parameters, such as costs, incidence and also transition parameters that determine the amount of time spent in a disease state. Some of these parameters can only be estimated from studies with lengthy follow-up periods, so that patient or population registries or EA programmes would be best suited to inform decision-making on these model inputs. Finally, trial values may not be sufficiently informative, as they are typically multinational and do not contain data relevant to a particular national health system.

The regulatory status of data collection during EA programmes is a matter of debate.10 11 14 15 24 25 In Europe, individual Member States regulate EA programmes.26 Different countries may issue conflicting statements that can be at cross with EMA decision-making.10 This also resonates in appraisals. For example, we read in the appraisal of cemiplimab for treating metastatic or locally advanced cutaneous squamous cell carcinoma:

While formal data collection is not permitted from a regulatory standpoint, the safety of cemiplimab at the flat 350mg dose in a real-world setting will be monitored.

Manufacturer submission, Safety overview, TA592

This begs the questions who decides what formal and informal data collection is and whether all examples put forth in this paper where impermissible for regulators. Regardless of regulatory requirements, it can be a source of frustration when EA data are not available, as one advisory group (AG) noted:

The lack of any efficacy data from the compassionate use program is particularly disappointing,

AG response to company comments, AG conclusions, TA535

Although the primary intent of EA programmes is treatment provision and not to conduct research, it seems awkward to treat patients with investigational medicine and not to collect data to inform safety and efficacy. Furthermore, it is difficult to precisely determine where treatment-intent ends and research-intent starts. The changing nature of EA programmes from sole treatment-intent to treatment-intent with data collection is a current topic of debate among bioethicists.12 14 27 We stress that data collection during EA should be lightweight and must not disproportionally burden patient and physicians—hence, a smart design should facilitate data to be collected.12 If so, EA programmes can be the first source of RWD to inform HTA evaluations gathered in a preapproval setting—this makes EA data different from general RWD sources (eg, electronic health records or claims and billing data), as the latter will typically only start generating evidence once the drug has been approved. Results from EA programmes can be obtained via peer-reviewed publications, if published. Alternatively, data can be requested via the medical company using data sharing platforms, such as Vivli.28 Finally, data may be available through local investigators (see HST7, Supplementary Files B).

Limitations and future research

Our work has several limitations. First, we only reviewed TAs from one HTA body: NICE. Formally, NICE’s decisions are only valid within their UK jurisdiction, but informally they lead the way for other European HTA bodies—either via setting an example or via reference pricing. We have chosen NICE for our review as they have the longest history of HTA assessment and ample documentation publicly available. For other HTA bodies, results may be different. Future research should confirm whether our results uphold for other HTA bodies. Preliminary findings presented at a conference concluded that using EA data gathered within French compassionate use programmes had a positive impact on reimbursement discussions.29 Second, we may have missed use-cases of EA data in payer submission as companies or reviewers may have used other terms to indicate EA programmes (or failed to have done so). Our automated algorithm facilitates high throughput of document screening in health policy analysis, but it may have missed cases that would have been identified in manual evaluation. Therefore, our estimates should be interpreted as a lower bound of EA use in NICE appraisals. Finally, we were unable to exactly quantify the added value of EA data. As we lack a counterfactual, we do not know what would have happened without the inclusion of EA data. Additionally, it is difficult to measure the impact of EA data, as it is not always clear how these data have exactly been used: the use of EA data—and the appraisal thereofin HTA by the manufacturer, ERG or NICE committee are difficult to quantify due the complexity and extent of the discussions described in the documentation. Although we have provided the reader with both high-level quantitative statistics and with illustrative qualitative examples from our data set, future research could attempt to systematically analyse these topics.

Conclusion

EA data are used in over one in five (21.1%) NICE appraisals, and this number appears to remain stable over time. In general, adding data from EA can yield more real-world information. Especially to estimate the resource use, preapproval EA data can play a vital role informing postapproval real-world usage. In synthesis with evidence from well-controlled regulatory studies, data collected from EA programmes may meaningfully inform NICE decision-making. Further research is required to understand when EA data can and should be included in HTA.