Evaluation of Advanced Therapy Medicinal Products by the National Institute for Health and Care Excellence (NICE): An Updated Review.

This review discusses the methodological challenges to the evaluation of advanced therapy medicinal products (ATMPs) by the UK National Institute for Health and Care Excellence (NICE). We analysed the health technology appraisals (both published and in development) of ATMPs conducted by NICE in England until July 2021. A total of 14 health technology appraisals of ATMPs were included, of which two were in development and 12 had been published. There were ten gene therapy products (talimogene laherparepvec [TA410], strimvelis [HST7], tisagenlecleucel [TA554 and TA567], axicabtagene ciloleucel [TA559], voretigene neparvovec [HST11], autologous anti-CD19-transduced CD3+ cells [TA677], betibeglogene autotemcel [ID968], onasemnogene abeparvovec [HST15] and OTL-200 [ID1666]), one tissue engineered product (holoclar [TA467]) and three somatic cell therapy products (darvadstrocel [TA556] and autologous chondrocyte implantation [ACI] [TA477 and TA508]). Only three of these technologies were not recommended by NICE, although four were only recommended within the Cancer Drugs Fund. There was large uncertainty in the assessment of clinical effectiveness, as evidence relied mostly on small, single-arm, open-label studies. There were also several concerns in the cost-effectiveness evaluations, such as limited information on health-related quality of life, immature survival data due to short follow-up, and unclear curative potential. Substantial risk may be incurred with ATMPs, which have high upfront and possibly irrecoverable costs but uncertain long-term benefits. In conclusion, the challenges raised by the economic appraisal of ATMPs, albeit not unique, may be exacerbated by the uncertainty related to the often scant evidence base. Adaptations of the conventional decision-making process rather than completely new methods may improve appraisals of ATMPs.

Key Points for Decision Makers

Introduction

Advanced therapy medicinal products (ATMPs) are medicines that replace or regenerate human cells, tissues or organs to restore or establish normal function. The European Medicines Agency (EMA) identifies three groups of ATMPs: gene therapy products, tissue engineered products and somatic cell therapy products [1]. In addition, some ATMPs may contain one or more medical devices as an integral part of the medicine, which are referred to as combined ATMPs (e.g., cells embedded in a biodegradable matrix or scaffold).

Although potential breakthroughs in this area of clinical research are eagerly anticipated, unregulated use of novel therapies with unproven benefits can be not only ineffective but also potentially harmful [2]. Furthermore, technology evaluations of ATMPs may present additional challenges compared with those of conventional pharmaceutical treatments, and straightforward application of standard appraisal methods recommended by the UK National Institute for Health and Care Excellence (NICE) might be difficult or even inappropriate [3]. Acknowledging the need for flexible and adaptable methods to ensure that innovative technologies, such as ATMPs, are “fairly, efficiently and robustly” evaluated, NICE is currently considering substantial changes to its methods of health technology evaluation [4]. The aims of this study are to review evaluations of ATMPs conducted by NICE and discuss the associated challenges and methodological issues.

Methods

We searched for appraisals of ATMPs, both published and in development, on the NICE database of technology appraisal guidance [5] and highly specialised technologies guidance in July 2021 [5]. We cross referenced this with the list of ATMPs published by the EMA [6]. For each technology, we reviewed the final appraisal or evaluation determination document and associated documents. We extracted data about several domains pertaining to clinical effectiveness and cost effectiveness, broadly following the structure recommended by the guide to the process of technology appraisal published by NICE (clinical effectiveness, cost effectiveness, uncertainty, discounting, subgroups, additional benefits, generalisability to the national health service [NHS], innovation, service reformulation, inequalities, patient access schemes, end of life) [7]. ATMPs for which marketing authorisation was withdrawn were not considered eligible for this study.

Results

To date, NICE has published guidance on 14 ATMPs (Table 1): ten gene therapy products (talimogene laherparepvec [TA410] [8], strimvelis [HST7] [9], tisagenlecleucel [TA554 and TA567] [10, 11], axicabtagene ciloleucel [TA559] [12], voretigene neparvovec [HST11] [13], autologous anti-CD19-transduced CD3+ cells [TA677] [14], betibeglogene autotemcel [ID968] [15], onasemnogene abeparvovec [HST15] [16] and OTL-200 [ID1666] [17]), one tissue engineered product (holoclar [TA467] [18]) and three somatic cell therapy products (darvadstrocel [TA556] [19] and autologous chondrocyte implantation (ACI) [TA477 and TA508] [20, 21]). The appraisal of sipuleucel-T was excluded since the guidance and associated documents were removed when the EMA withdrew marketing authorisation [22].

Table 1

Summary of characteristics of the advanced therapy medicinal products included in this review

Number and ATMP name	Tisagenlecleucel [TA 554]	Axicabtagene ciloleucel [TA 559]	Tisagenlecleucel [TA 567]
Conditions	B-cell acute lymphoblastic leukaemia	Diffuse large B-cell lymphoma and primary mediastinal large B-cell lymphoma	Diffuse large B-cell lymphoma
Recommendation	Tisagenlecleucel therapy is recommended for use within the CDF as an option for treating relapsed or refractory B-cell acute lymphoblastic leukaemia in people aged ≤ 25 years, only if the conditions in the managed access agreement are followed	Axicabtagene ciloleucel therapy is recommended for use within the CDF as an option for treating relapsed or refractory diffuse large B-cell lymphoma or primary mediastinal large B-cell lymphoma in adults after two or more systemic therapies, only if the conditions in the managed access agreement are followed	Tisagenlecleucel therapy is recommended for use within the CDF as an option for treating relapsed or refractory diffuse large B-cell lymphoma in adults after two or more systemic therapies, only if the conditions in the managed access agreement are followed
Type of AMT	CAR T-cell therapy	CAR T-cell therapy	CAR T-cell therapy
Administration and cost	Single dose £282,000	Single dose but price not known	Single dose £282,000
Clinical effectiveness	Tisagenlecleucel is clinically effective, but the benefit vs. blinatumomab or salvage chemotherapy is based on naïve indirect comparisons as direct comparator data are lacking	Axicabtagene ciloleucel is clinically effective, but the lack of comparator data makes assessing comparative effectiveness challenging	Tisagenlecleucel is clinically effective, but immature survival data and the lack of trial data directly comparing tisagenlecleucel with salvage chemotherapy means the size of this benefit is difficult to establish. Tisagenlecleucel is associated with frequent AEs, and the costs associated with managing and treating these should be reflected in the cost-effectiveness modelling
Cost effectiveness (ICER)	Over £30,000 per QALY for blinatumomab Over £45,000 per QALY for salvage chemotherapy	Over £50,000 peer QALY for salvage chemotherapy	£42,991 per QALY gained with a 2-year cure point, £49,963 per QALY gained with a 3-year cure point, £55,403 per QALY gained with a 4-year cure point
Uncertainty	Only three single-arm studies (phase II) with small sample sizes and different populations that are difficult to compare. Uncertainty about whether and how many pts would need an allogeneic stem cell transplant after tisagenlecleucel. Lack of data to determine the costs of treating side effects. Lack of valid and reliable data on survival beyond 30 months and curative effect	Data from a small single-arm study (phase I/II) and data for comparators are of low quality. Limited follow-up and no direct data comparing axicabtagene ciloleucel with salvage chemotherapy. Uncertainty in extrapolation of long-term survival and cure points. Uncertainty about whether and how many pts will need intravenous immunoglobulin	Data from single-arm study with short follow-up and a small observational study. Lack of reliable data on long-term survival and cure points. Lack of data about need for intravenous immunoglobulins for B-cell aplasia and stem cell transplant
Discounting	3.5% for costs and benefits	3.5% for both costs and benefits because evidence is immature, so the duration of health benefits cannot robustly show cure	3.5% for both costs and benefits
Subgroups	Numbers too small to analyse Philadelphia chromosome positive and negative separately	No mention	Considered those not eligible for stem cell transplant, but this population is difficult to define
Additional benefits	No benefits beyond those captured by QALY calculation	No benefits beyond those captured by QALY calculation	No benefits beyond those captured by QALY calculation
Generalisability to NHS	Uncertain generalisability to routine practice in the NHS	Generalisable to patients and clinical practice in the NHS	Findings are generalisable to clinical practice in the NHS
Innovation	Innovative treatment, but reference case still applicable	Innovative treatment, but reference case still applicable	Innovative treatment, but reference case still applicable
Service reformulation/change in practice	Not discussed	Will need a phased implementation in the NHS due to new infrastructure and training required to administer treatment and manage side effects	Not discussed
Inequalities	No issues identified	No issues identified	No issues identified
PAS	Company agreed a PAS with the Department of Health, but the level of the discount is commercial in confidence. The most plausible cost-effectiveness estimates for tisagenlecleucel are above what is considered an acceptable use of NHS resources, so it is recommended for use within the CDF to collect further data on subsequent stem cell transplant rates and immunoglobulin usage	Company agreed a PAS with the Department of Health, but the level of the discount is commercial in confidence. The most plausible cost-effectiveness estimates for axicabtagene ciloleucel vs. salvage chemotherapy are above what is considered an acceptable use of NHS resources, so it is recommended for use within the CDF to collect further data on PFS, OS and immunoglobulin usage	Company agreed a PAS with the Department of Health, but the level of the discount is commercial in confidence. ICER higher than what is considered an acceptable use of NHS resources, so recommended for CDF to collect more data on PFS, OS and immunoglobulin usage
End of life	Tisagenlecleucel extends OS by > 3 months, but it cannot be considered a life-extending treatment at the end of life as the life expectancy of people with relapsed or refractory acute lymphoblastic leukaemia is uncertain	Axicabtagene ciloleucel meets both criteria to be considered a life-extending treatment at the end of life	Tisagenlecleucel meets both criteria to be considered a life-extending treatment at the end of life

ACI autologous chondrocyte implantation, ADA–SCID adenosine deaminase deficiency–severe combined immunodeficiency, AE adverse event, AMT advanced medicinal therapy, ARSA arylsulphatase A, ATMP advanced therapy medicinal product, BSC best supportive care, BTK Bruton’s tyrosine kinase, CAR chimeric antigen receptor, CD Crohn’s disease, CDF Cancer Drugs Fund, EMA European Medicines Agency, HLA human leukocyte antigen, HRQoL health-related quality of life, HSC homologous stem cell, HSCT homologous stem cell transplant, HST highly specialised technologies, ICER incremental cost-effectiveness ratio, LSCD limbal stem cell deficiency, NHS national health service, NICE National Institute for Care and Excellence , OS overall survival, PAS patient access scheme, PFS progression-free survival, PFU plaque-forming units, PROs patient-reported outcomes, pt(s) patient(s), QALY quality-adjusted life-year, RCT randomised controlled trial, SMA spinal muscular atrophy, TA technology appraisal, TDT transfusion-dependent β-thalassaemia, VAT value-added tax

Talimogene laherparepvec is recommended in adults as an option for treating unresectable, regionally or distantly metastatic (stage IIIB, IIIC or IVM1a) melanoma that has not spread to bone, brain, lung or other internal organs, only if:

-Treatment with systemically administered immunotherapies is not suitable and

-The company provides talimogene laherparepvec with the discount agreed in the PAS

Holoclar (ex vivo expanded autologous human corneal epithelial cells containing stem cells) is recommended as an option for pts with moderate to severe LSCD after eye burns, only if:

-It is only used to treat one eye and

-Pts have already had a conjunctival limbal autograft or

-There is not enough tissue for a conjunctival limbal autograft, or it is contraindicated and

-The company provides it with the discount agreed in the PAS.

Moderate to severe LSCD is defined by the presence of superficial corneal neovascularisation in at least two corneal quadrants, with central corneal involvement, and severely impaired visual acuity. Holoclar is recommended in people with moderate to severe LSCD after eye burns for treating both eyes only:

-In the context of research and

-When there is not enough tissue for a conjunctival limbal autograft.

Such research should be designed to generate robust evidence of the clinical and cost effectiveness of Holoclar in treating two eyes in pts without enough tissue for a conjunctival limbal autograft

ACI recommended as an option for treating symptomatic articular cartilage defects of the knee only if:

-Pt has not had previous knee repair surgery

-Osteoarthritic damage to the knee is minimal (as assessed by clinicians experienced in investigating knee cartilage damage using a validated measure for knee osteoarthritis)

-Defect is > 2 cm2

-Procedure is done at a tertiary referral centre

ACI using chondrosphere is recommended as an option for treating symptomatic articular cartilage defects of the knee in adults only if:

-Pt has not had previous surgery to repair articular cartilage defects

osteoarthritic damage to the knee is minimal (as assessed by clinicians experienced in investigating knee cartilage damage using a validated measure for knee osteoarthritis) and

-Defect is > 2 cm2

ACI likely to be cost effective only in subgroups of pts:

-Pts who have not had previous knee repair

-Pts who have minimal osteoarthritic damage to the knee

-Pts with articular cartilage defects of > 2 cm2

ACI likely to be cost effective only in subgroups of pts (as per TA477):

-Pts who have not had previous knee repair

-Pts who have minimal osteoarthritic damage to the knee

-Pts with articular cartilage defects of > 2 cm2

Onasemnogene abeparvovec is recommended as an option for treating 5q SMA with a biallelic mutation in the SMN1 gene and a clinical diagnosis of type 1 SMA in babies, only if:

They are aged ≤6 months, or

They are aged 7–12 months, and their treatment is agreed by the national multidisciplinary team. It is only recommended for these groups if:

Permanent ventilation for > 16 h/day or a tracheostomy is not needed

The company provides it according to the commercial arrangement.

For babies aged 7–12 months, the national multidisciplinary team should develop auditable criteria to enable onasemnogene abeparvovec to be allocated to babies in whom treatment will give them ≥70% chance of being able to sit independently. Onasemnogene abeparvovec is recommended as an option for treating presymptomatic 5q SMA with a biallelic mutation in the SMN1 gene and up to three copies of the SMN2 gene in babies. It is recommended only if the conditions in the managed access agreement are followed

OTL-200 is not recommended, within its marketing authorisation, for treating metachromatic leukodystrophy characterised by biallelic mutations in the ARSA gene that reduce ARSA enzyme activity in children who have:

Late infantile or early juvenile types, with no clinical signs or symptoms

The early juvenile type, with early clinical signs or symptoms, and who can still walk independently and have no cognitive decline.

This recommendation is not intended to affect treatment with OTL-200 that was started in the NHS before this guidance was published. Children having treatment outside this recommendation may continue without change to the funding arrangements in place for them before this guidance was published, until they and their NHS clinician consider it appropriate to stop. This decision should be made jointly by the clinician and the child or young person or their parents or carers

Seven of these contained positive recommendations for use in specific indications (talimogene laherparepvec, strimvelis, voretigene neparvovec, holoclar, onasemnogene abeparvovec and the two ACI), four recommended that the ATMP should be included in the Cancer Drugs Fund (CDF) (tisagenlecleucel, axicabtagene ciloleucel and autologous anti-CD19-transduced CD3+ cells) and three did not recommend the ATMP for use in the NHS (darvadstrocel, betibeglogene autotemcel and OTL-200).

The incremental cost-effectiveness ratios (ICERs) for ATMPs recommended under the technology appraisal guidance [5] were within the £20,000–30,000 per quality-adjusted life-year (QALY) threshold usually considered by NICE, whereas the ATMPs with ICERs of £30,000–50,000 were recommended for the CDF. For the three ATMPs approved under the highly specialised technologies guidance [5], which applies a threshold of £100,000 per QALY, the ICERs for strimvelis were under the threshold irrespective of discount rate, but those for voretigene neparvovec were over the threshold when a 3.5% discount rate was applied. In the case of onasemnogene abeparvovec, using a discount rate of 1.5%, the ICER was under the weighted threshold of £186,000 per QALY.

Technology appraisals also considered whether there were externalities or additional benefits beyond those captured by QALYs. These additional benefits for individuals included avoidance of negative impact of alternative treatments (holoclar) and improved independence, social participation and ability to achieve full potential (strimvelis, voretigene neparvovec and onasemnogene abeparvovec). Positive externalities were taken into account qualitatively in the recommendations made for highly specialised technologies, such as reduced financial impact on families and society overall due to cost savings for the NHS and non-NHS government departments (e.g., social care). Chimeric antigen receptor (CAR) T-cell therapies were deemed to have no additional benefits, whereas the appraisals of darvadstrocel, betibeglogene autotemcel and ACIs did not comment on additional effects and externalities.

Discussion

This review suggests that appraisals of ATMPs present specific, and arguably greater, challenges to NICE committees than do other technologies. Uncertainty pervades clinical and cost effectiveness in all appraisals because of the sparse or even absent evidence on utility values, long-term effects and costs of treatment versus comparators. These issues are partly due to the nature of the conditions, which are typically rare and severe, and partly due to the technologies, which are novel and yet to be fully understood. We explore the key issues related to the evaluation of clinical and cost effectiveness of ATMPs, and we discuss the potential implications of adoption of ATMPs for the NHS and society overall.

Clinical effectiveness

Overall, evidence on the clinical effectiveness of ATMPs was limited and weak, but this varied substantially across technologies. The main difficulty in determining the clinical effectiveness of ATMPs was related to the limitations of the studies available, which were mostly single-armed, non-blinded studies with small sample sizes and short follow-up. The poor quality of the studies available may be because ATMPs are typically prescribed as end-of-line treatment to people with rare diseases, which limits sample size. However, this does not necessarily explain why studies were non-blinded and single-armed, which raises concerns about the pharmaceutical industry sponsoring those studies. The limited evidence available raised important issues for assessing clinical effectiveness. First, short follow-up meant that survival data were immature, so the survival estimates and overall duration of effects were highly uncertain. In this context, scenario analyses exploring the effects of different assumptions about long-term benefits might be advantageous, as shown by their use for the evaluation of onasemnogene abeparvovec [4]. Second, single-arm studies did not provide direct evidence of efficacy against comparators, and finding suitable comparator(s) with valid and reliable data available was difficult. For instance, data from previous studies often did not reflect contemporary practice because of the fast evolution of therapeutics, or they included populations with different disease stages and prognoses. Comparison with historical cohorts that studied the natural progression of disease was also naïve (e.g., OTL-200). This rendered quantification of benefits and potential harms associated with different treatment modalities challenging, which translated into uncertainty in the evaluation of clinical effectiveness.

The criteria for life-extending end-of-life treatments, which include (1) life-extending treatments for people with a short life expectancy (normally < 24 months) and (2) providing a gain in overall survival of > 3 months [5], were introduced to allow the QALYs gained by treatments that improve patient survival in terminal stages of disease to be valued more highly. Although these criteria have allowed approval of drugs that would have otherwise been denied to NHS patients, all bar one have been cancer drugs [23]. In keeping with this, of the ATMPs, only three of the CAR T-cell therapies (TA559, TA567 and TA677) met the criteria for life-extending end-of-life treatment. Perhaps acknowledging the limited scope and questionable fairness of the end-of-life criteria, NICE is considering replacing these criteria by a modifier for severity of disease [4]. The actual number of technologies that will benefit from a severity modifier depends on how it is defined and applied, but they can be expected to form a broader range of conditions and to reflect more accurately societal values than the previous end-of-life criteria.

NICE has statutory and ethical duties to support innovative technologies, so greater risks may be accepted to enable access to highly innovative technologies, which have valuable benefits for patients and society [24]. Although NICE defines which technologies qualify as innovative [5], there is much room for interpretation, and ambiguity has resulted in significant variability in implementation [25]. This raises concerns as to whether innovation, which is not an independent social value, may be jeopardising the core values of health and equity in the NHS [26]. The possibility of considering innovation as a modifier also does not garner NICE support at present, as there is no evidence that society values health benefits from innovative technology more than equivalent benefits from less innovative technology [27]. In keeping with this, no adjustment to the reference case was warranted for any of the ATMPs, despite all being considered highly innovative.

Cost Effectiveness

Overall, there was substantial uncertainty in estimates of cost effectiveness of ATMPs because of the lack of valid and reliable data to inform the economic models. This was reflected in the disagreements about model parameters between committees, evidence review groups and companies. The poor quality of the clinical studies was further compounded by a lack of data on patient-reported outcomes, particularly health-related quality of life. This meant that utility values specific to the condition of interest were often missing, and extrapolation from the literature introduced further uncertainty to the models (e.g., holoclar, strimvelis, voretigene neparvovec, betibeglogene autotemcel, onasemnogene abeparvovec, OTL-200). In some cases, economic models may have overestimated utility gains of treatment (e.g., talimogene laherparepvec, betibeglogene autotemcel). For CAR T-cell therapies, overestimation of long-term survival and cure points may have exerted a paramount effect on cost effectiveness, as those were key determinants of the economic models. In addition, QALY valuation when effects are predicted to be lifelong remains complex because of concerns about whether different social values apply to evaluations of potential cures and substantial uncertainty in long-term outcomes [28]. Nonetheless, the lifelong nature of the effects of strimvelis, voretigene neparvovec, onasemnogene abeparvovec and OTL-200 was assumed based on biological plausibility, and QALYs were weighted (i.e., granted an increased value) to reflect the added value of large QALY gains throughout life, thus enhancing their cost effectiveness. However, this is controversial as there is no evidence that society places additional value on technologies that are potentially curative, and NICE is not supportive of a specific modifier for potentially curative treatments at present [4].

NICE guidance recommends using a 3.5% discount rate for both costs and benefits, but a 1.5% discount rate is allowed when appraising treatments that “restore people who would otherwise die or have severely impaired life to full or near full health, and when this is sustained over a very long period (normally at least 30 years)” [7]. Although companies claimed that their products were eligible for the 1.5% discount rate (e.g., tisagenlecleucel, holoclar, betibeglogene autotemcel and OTL-200), committees noted that “it is rarely considered appropriate to change the discount rate” and preferred the 3.5% discount rate in most cases. However, for strimvelis and voretigene neparvovec, committees stated that both discount rates would be taken into consideration, acknowledging the uncertainty about whether those therapies would restore normal or near-normal health. For onasemnogene abeparvovec, a 1.5% discount was applied, despite, arguably, similar uncertainty about whether effects would be sustained in the long term and patients would achieve normal or near-normal health. This suggests that a lower threshold for considering the 1.5% discount rate may be applied in highly specialised technology appraisals than in standard technology appraisals. The common disagreement between companies and committees on discounting demonstrates the subjectivity involved in the interpretation of the fulfilment of the eligibility criteria. Furthermore, concerns have been raised that the non-reference case discount rate is rarely applied as it sets a high bar for the technologies it otherwise appears to support by requiring no significant irrecoverable costs and implying that a high degree of certainty is needed [29]. For all these reasons, NICE is considering a proposal to use a reference case discount rate of 1.5% per year for both costs and health effects [4], which is the discount rate for health values recommended by the government [30]. Lowering the reference case discount rate may increase the cost effectiveness of ATMPs, particularly those that have high upfront costs and long-term health benefits.

ATMPs are, in general, very expensive, and patient access schemes have been used to lower the ICER and thus facilitate NICE approval [31]. These are agreements between the Department of Health and pharmaceutical companies that enable companies to offer discounts or rebates that reduce the cost of a drug for the NHS. Simple discount schemes have been preferred over complex schemes, such as provision of free stock, dose caps or payments by results (i.e., performance or outcome-based schemes) [32]. It is arguable, though, that the latter could be appropriate to ATMPs, when there is substantial uncertainty on long-term effects and high upfront costs [33]. National discounts are known to NICE committees, and cost-effectiveness models are based on discounted prices. However, “companies sometimes provide confidential [local] discounts to the NHS, making the real cost of cells difficult to ascertain” (TA477), in which case committees accept models based on the approximate list price for the technology. In addition, the exact details of discounts are commercial in confidence, which, albeit understandable, compromises transparency and may undermine patients’ trust in how decisions are made about rationing of healthcare resources in the NHS [34]. On the other hand, patient access schemes may be disproportionately benefitting certain technologies. CAR T-cell therapies were recommended for inclusion in the CDF, which allows their use in the NHS while further data are collected to support a robust appraisal of their cost effectiveness and a subsequent final recommendation. It is arguable whether similar funding arrangements should be potentially available to all technologies irrespective of the underlying disease (e.g., darvadstrocel could have been approved under such a scheme whilst further data were being collected by a large trial). The 2019 general election featured a promise to replace the CDF with an Innovative Medicines Fund, but this is yet to be implemented.

The four ATMPs evaluated as highly specialised technologies listed children, and specifically very young children, as the primary beneficiaries. In these cases, the higher valuation of QALYs assigned to treatments that have potentially lifelong effects (implemented via a higher cost-effectiveness threshold) captured the increase in benefit that would result from delivering a potentially transformative treatment to children while remaining in line with NICE’s view that a modifier based on age is not appropriate [4]. As ATMPs are expected to target severe genetic diseases that manifest in infancy, the valuation and measurement of QALYs in children presents substantial challenges. NICE recommends using a generic measure with good psychometric performance in the relevant age group and reporting who has completed the questionnaire [5]. Nevertheless, the difficulty of valuing and measuring utility in children introduces additional uncertainty to cost-effectiveness evaluations, and further research is warranted to refine methods for assessing health-related quality of life in children [35].

Subgroup analysis may be especially important for ATMPs, as ICERs are typically high and may vary significantly across subgroups. NICE allows committees to recommend treatment for a selected subgroup of patients irrespective of whether the technology is found to be clinically and cost effective for the whole population, provided that the decision is ethically and methodologically sound [5]. Although ten of the 14 technologies analysed in this review considered whether treatment effects would be different in certain subgroups of patients, only six made specific subgroup recommendations (talimogene laherparepvec, holoclar, ACIs, onasemnogene abeparvovec and betibeglogene autotemcel). The remaining three evaluations considered that data were lacking to determine whether there were meaningful differences in treatment effects across subgroups (tisagenlecleucel, voretigene neparvovec) or that it was clinically difficult to identify those subgroups (tisagenlecleucel). This clearly illustrates the challenges in assessing the credibility and relevance of differences between subgroups, which have also been acknowledged by NICE [4].

The impact on inequalities may be particularly relevant for the evaluation of innovative technologies, such as ATMPs, as there may be a greater risk of creating or exacerbating inequalities [36]. In this review, all ATMPs except strimvelis, ACI and onasemnogene abeparvovec were considered to have a neutral effect on inequalities. The first could reduce inequalities in waiting time for transplant that result from a lack of suitable donors for certain ethnicities in whom the condition for which it is recommended (i.e., adenosine deaminase deficiency–severe combined immunodeficiency) is more common, such as Irish traveller and Somalian family origins. This then leads to increased waiting times and potentially worse outcomes for those individuals. Conversely, ACI could result in inequalities as marketing authorisation excludes those with severe osteoarthritis. However, this was mitigated by allowing clinicians to assess patients’ suitability for ACI based on severity of osteoarthritis. In addition, restricting ACI in TA477 to tertiary referral centres may lead to geographical inequalities as ACI may not be widely available across the country. For onasemnogene abeparvovec, the potential for a delayed diagnosis in disadvantaged children led the committee to allow treatment to be offered to children aged > 6 months in certain circumstances, despite scant evidence on clinical effectiveness beyond 6 months of age. NICE’s commitment to addressing inequalities is illustrated by the fact that it considered adding impact on inequalities as a modifier to future health technology appraisals [4]. This is based on equality legislation [37] and evidence suggesting that the UK population prioritises seeking a fair distribution of health across society and is willing to trade-off less health overall if the health is generated in disadvantaged groups, particularly for socioeconomic disadvantage [38]. However, further work is needed to explore which sources of inequality to include, whether direct and indirect effects would be considered, and how and in which circumstances this could be implemented in health technology evaluations.

Although none of the ATMPs included in this review required major service redesign and staff training, ATMPs are by nature likely to have significant service delivery effects. NICE prefers that the full additional cost of introducing a technology is included in the evaluation. However, it concedes that some of the costs may be apportioned between other technologies, particularly when “high-cost medical devices have other uses beyond the evaluated indication (either now or in the future) or for technologies that need substantial service redesign to introduce them (which may affect other technologies or people not having treatment with a technology)” [4]. This flexibility to allow non-reference case analysis may be especially suitable for ATMPs, as costs may be shared by different stakeholders and technologies. In addition, ATMPs may be associated with additional costs that would not commonly be considered in other technology evaluations, such as costs associated with travelling to specialist centres abroad. For strimvelis, these costs were not incorporated in the economic model, and it was unclear how those costs would be covered in the long term. However, whether this should be included in economic models to fully reflect the cost for the NHS is arguable, especially if, as expected, these issues become increasingly common.

Implications for the National Health Service and Society Overall

Although ATMPs offer potentially important benefits for patients, their families and society overall, their high upfront and often one-off costs can pose challenges related to affordability and implementation in the NHS [39]. Many of the mitigation measures are commercial in nature (e.g., patient access schemes) and not directly in the remit of NICE technology evaluations. However, NICE is accountable to the NHS, the government and the public it serves, and thus the wider implications of health technology evaluations of ATMPs for the NHS and society overall cannot be overlooked.

The first concern raised by the implementation of ATMPs by the NHS relates to the generalisability of research conducted elsewhere. This question, albeit shared with other technologies, is paramount for ATMPs because of the scant evidence available. Committees sought advice from clinical experts and complemented this with data from observational studies in England, whenever possible (e.g., darvadstrocel). Findings were considered generalisable to clinical practice in the NHS for axicabtagene ciloleucel, tisagenlecleucel in TA567 and ACI in TA508. However, in other cases, generalisability was questionable because outcomes for the comparator did not represent contemporary outcomes in the NHS (e.g., relapse rates were much higher for the comparator of darvadstrocel in the single trial available than rates reported in the NHS), the characteristics of the patients in the studies did not match those of typical NHS patients (tisagenlecleucel in TA554, darvadstrocel, holoclar, strimvelis) or heterogeneity in treatment effects was possible depending on individual variability (voretigene neparvovec).

Second, ATMPs are associated with a larger financial risk for the NHS than many other technologies [39]. They often have high upfront and, depending on contractual arrangements, potentially irrecoverable costs, yet the full benefits may take many years to accrue or may not be permanent. Furthermore, the high cost per patient means there is high volatility, which may be challenging to accommodate in annual budgets and increase financial risks if there is an unexpectedly high number of cases. The quintessential question is how much risk the NHS should incur and in which circumstances, as well as whether the risk should be shared by pharmaceutical companies (e.g., by using outcome-based contracts) [40]. NICE considers that greater risks could be accepted for “conditions for which it is recognised that evidence generation is complex and difficult, such as rare diseases, innovative technologies, technologies that provide large benefits”, all of which apply to ATMPs [4]. Nonetheless, whereas voretigene neparvovec, strimvelis and onasemnogene abeparvovec were recommended despite the scant evidence on long-term benefits, betibeglogene autotemcel was not recommended, at least partially because of “the potential to commit the NHS to irrecoverable costs”. This illustrates the difficult compromise between allowing flexibility for non-reference case analysis (e.g., by accepting greater risks) and allocating resources to maximise population health [41].

Third, ATMPs may have wider consequences for patients, their families and carers, the NHS and other governmental sectors and ultimately society at large. Those consequences are often beneficial and hence associated with positive utility (e.g., reduced burden for paid and unpaid carers). However, they can be associated with disutility, particularly when ATMPs extend life expectancy but with significant disability, leading to an increased need for caregiving over a lifetime horizon. For instance, for onasemnogene abeparvovec, caregiver disutility was not included in the model because it was difficult to quantify the disutility for carers and it would increase the ICER, which was considered “counterintuitive”. In other ATMP evaluations, wider benefits contributed qualitatively to the appraisal of the evidence and the final recommendation, as they were hard to quantify or intangible (e.g., OTL-200). This is partly due to the lack of data on these benefits, as no studies had included them as outcomes. However, even if data were available, the reference case does not contemplate wider benefits that may be accrued because of treatment, particularly when these are non-health benefits (e.g., reduced need for social care) or when they fall on someone other than the person receiving treatment (e.g., parents and carers in general). NICE guidance acknowledges that “care delivered by the NHS could have other benefits that are considered socially valuable but are not directly related to health and are not easily captured in a cost per QALY analysis” [7]. Nonetheless, incorporating techniques “that consider the trade-off between health benefits and non-health benefits quantitatively” into decision making was considered unsuitable in 2013, and it did not feature in the case for change of health technology evaluation in 2020 [4]. NICE recommends that non-reference case analysis is used when there are substantial identifiable health benefits not captured by QALYs and emphasises the need to develop methods to formally incorporate qualitative evidence into decision making [4].

Conclusion

NICE evaluation of ATMPs revealed significant challenges, mainly related to large uncertainty about long-term and potentially curative effects, high upfront costs, discounting, innovation, benefits above and beyond QALYs and apportioning of costs. However, these challenges are not unique to ATMPs, so completely different methods may not be required. Adaptations to the conventional decision-making process, such as the use of flexible non-reference case analysis, integration of different sources of evidence, and special funding arrangements, may improve appraisal of ATMPs.

Advanced therapy medicinal products have transformative potential, but the limited evidence available presents significant challenges to economic evaluation.

Evaluation of advanced therapy medicinal products is pervaded with uncertainty because of the lack of robust evidence in many key aspects. However, conventional methods of health technology appraisal employed by the UK National Institute for Health and Care Excellence may still be applied with some adjustments and perhaps greater flexibility than for other technologies.

Considering the methodological uncertainty in cost-effectiveness appraisals and the potential for irrecoverable costs for health systems, decision makers may wish to consider new contractual arrangements that enable access to novel technologies whilst sharing the risks between health systems and pharmaceutical companies.