The Utility of a Rapid Review Evaluation Process to a National HTA Agency.

BACKGROUND AND
OBJECTIVE: In Ireland, similar to other jurisdictions, health technology assessment (HTA) is used to inform the health payer's drug reimbursement decisions. These HTAs are conducted by the National Centre for Pharmacoeconomics (NCPE). In 2009, the NCPE introduced the Rapid Review process to identify drugs that do not require further assessment in the form of the previously established full HTA process.
METHODS: A retrospective analysis of all Rapid Reviews submitted to the NCPE from 2010 to 2019, inclusive, was conducted. Rapid Review recommendation was recorded (i.e. full HTA required or not required). For those submitted from 2012 to 2019, additional data relating to the drug, economic and clinical evidence-related factors were collected. Multivariable logistic regression methods were used to model the relationship between these factors and the likelihood of requiring a full HTA. An exploratory analysis estimated the additional NCPE appraisal time that would have been required to evaluate all drugs, had the Rapid Review process not been established.
RESULTS: Of the 446 Rapid Reviews submitted, approximately half (49.6%) were deemed to require a full HTA. Drugs for cancer indications, drugs designated first-in-class status, and high-cost drugs were positively and significantly associated with the likelihood of requiring a full HTA. No significant association was found for drugs for orphan indications when factors relating to cost and clinical evidence were included in the model. Without the Rapid Review process, an estimated additional 15,631 NCPE appraisal days would have been required to evaluate all drugs submitted over the 10-year period.
CONCLUSIONS: This is the first study to use data uniquely available to the NCPE to evaluate factors associated with the requirement for a full HTA following a Rapid Review. The process has reduced the NCPE appraisal time required to evaluate all submissions over the study period. The NCPE's Rapid Review process allows for appropriate resource prioritisation within a national HTA agency.

Key Points for Decision Makers

Introduction

In Ireland, similar to other jurisdictions, a health technology assessment (HTA)—a multidisciplinary process used to systematically evaluate the costs and outcomes associated with a health technology—is used to inform decisions around drug reimbursement. In line with Irish legislation, clinical effectiveness, cost effectiveness and affordability (amongst other criteria) must be considered when a drug reimbursement decision is made [1]. The National Centre for Pharmacoeconomics (NCPE) has conducted the HTA of drugs in Ireland since its establishment in 1998 [2]. As set out in the European Union Transparency Directive, such assessments to support pricing and reimbursement decisions should be conducted in a timely manner [3]. Since 2006, successive agreements between the Health Service Executive (HSE; the government agency that manages the provision of publicly funded healthcare in Ireland) and the Irish Pharmaceutical Healthcare Association (a body representing the pharmaceutical industry in Ireland) have specified that drugs for which reimbursement is sought must be subject to an assessment by the NCPE [4]. The 2006 agreement stated that assessments were only required for high-cost drugs or those associated with a large budget impact. Since 2009, all new drugs were considered for assessment. In 2009, to meet the demands of both volume and timeliness of assessments, the NCPE introduced the Rapid Review process.

The objective of the Rapid Review is to provide a recommendation to the HSE (the decision maker) on the need for a full HTA and, in some cases, on reimbursement. Each Rapid Review pertains to one drug for one indication. Following the Rapid Review, a full HTA will subsequently be required for those drugs for which additional information and/or analysis is required to inform a reimbursement recommendation. Factors that are evaluated during a Rapid Review and thus inform the requirement for a full HTA include: the cost of the drug relative to potential comparators; comparative clinical effectiveness, and the degree of uncertainty relating to this; anticipated cost effectiveness, and the degree of uncertainty relating to this; and the potential budget impact [5]. In order to facilitate completion of the assessment within a 4-week timeframe, a number of features of the NCPE’s full HTA process are beyond the scope of the Rapid Review process, most notably evidence synthesis analyses and formal cost-effectiveness analyses. Key differences between the Rapid Review and full HTA processes are outlined in Table 1.

Table 1

Summary of key differences between the Rapid Review and full HTA processes

Key components	Rapid review	Full HTA
Evaluation time	Mean: 32.2 days [6]	Mean: 133.3 days, subject to a ‘stop-clock’ process [6]
Elements included in submission	Completed Rapid Review templatea	Completed full HTA template [8]
	Full reference library [7]	Full reference library
		Cost-effectiveness electronic model
		Budget impact electronic model
Pre-submission meetingb	No	Yes
Cost-effectiveness assessment	Indicative cost-effectiveness assessment based on relative efficacy and relative cost of intervention to comparators	Formal cost-effectiveness assessment based on decision analysis methods
Evidence synthesis	No	Yes (where applicable)
Patient organisation submission invited	No	Yes
Formal preliminary review processc	No	Yes
Key opinion leader elicitation	Yes	Yes

HTA health technology assessment, NCPE National Centre for Pharmacoeconomics, SoC standard of care

aProcess currently under review: drug cost calculator expected to be included as part of future updates to submission process

bPre-submission meeting refers to a meeting between the applicant pharmaceutical company and the NCPE prior to submission, where the applicant presents an overview of the key issues in the expected submission

cPreliminary review process refers to opportunity for the NCPE Review Group to request clarification on any outstanding issues in the submission received

There have been attempts to characterise the factors that may be associated with a Rapid Review recommendation; however, these analyses have been subject to limitations. A previous analysis of drugs for orphan indications indicated that such drugs were significantly more likely to require a full HTA following a Rapid Review than drugs for non-orphan indications, but did not account for other factors that may be associated with the likelihood of requiring a full HTA, such as economic or clinical evidence-related considerations [9]. Murphy and Redmond reported that drugs for certain therapeutic areas (including cancer), drugs for orphan indications, and drugs designated ‘first-in-class’ status were more likely to require a full HTA [10]. The analysis was limited by the availability of Irish cost data, and did not account for any factors relating to clinical evidence [11]. The Rapid Review was introduced with the aim of directing the NCPE’s resources to areas where a full HTA would be of greatest value, resulting in an overall reduction in NCPE appraisal time. To date, no empirical analysis of the potential differences in NCPE appraisal time arising as a result of the Rapid Review process has been reported.

This paper provides an overview of the NCPE Rapid Review process, with a specific focus on the perspective of the HTA agency. It evaluates trends in Rapid Review submissions and recommendations over time, assesses factors associated with Rapid Review recommendation using data uniquely available to the NCPE and explores the potential savings in NCPE appraisal time resulting from the process.

Rapid Review Process

The applicant pharmaceutical company (herein ‘the Applicant’) can submit a reimbursement application for the drug under consideration to the HSE at any time once the European Union Committee on Human Medicinal Product’s positive opinion has been granted. Upon receipt of a reimbursement application, the HSE commissions the NCPE to undertake a Rapid Review assessment. The Rapid Review dossier is completed by the Applicant in accordance with the NCPE’s pre-specified template, and encompasses various domains related to the intervention including the licensed indication, the indication for which reimbursement is sought, the target population, clinical efficacy and safety data, any potential comparators, price and budget impact, information on on-going clinical investigations and HTAs published in other jurisdictions [7]. The NCPE review process involves a targeted review of the relevant literature, and an appraisal of the dossier submitted. Depending on the outcome of the Rapid Review appraisal, the NCPE makes a recommendation to the HSE. The recommendation specifies if a further assessment in the form of a full HTA is required; if not, a reimbursement recommendation may be made at this stage. Generally, each Rapid Review is undertaken by a NCPE Review Group assessor, with reporting completed in line with a standardised NCPE format. Following an initial appraisal, the Rapid Review recommendation is made on the basis of a consultation process in accordance with internal protocols. The outcome of the Rapid Review, including information on the timelines, is made publicly available on the NCPE’s website (www.ncpe.ie). At present, the NCPE makes one of five possible recommendations (Table 2). Over time, the recommendations have evolved, primarily to provide stakeholders with more transparency on the decisions made. For the purpose of this analysis, Rapid Review recommendations have been categorised into one of two outcomes: ‘full HTA not required’ and ‘full HTA required’ (Table 2).

Table 2

Rapid review recommendations and interpretation

Recommendation to the HSE	Appraisal interpretation	Categorisation for this analysisa
A full HTA is not recommended. The NCPE recommends that the drug be considered for reimbursementb	The NCPE believe the drug works as well or better than other ways to manage this condition, and that it is value for money. Enough information was provided in the Rapid Review and therefore, a full HTA is not needed	‘Full HTA not required’
A full HTA is not recommended. The NCPE recommends that the drug not be considered for reimbursement at the submitted priceb	The NCPE recommend that the HSE considers not providing this drug unless the HSE can agree a suitable price reduction with the pharmaceutical company. The price of the drug is higher than other ways to manage this condition, and we believe that it is not value for money. Enough information was provided in the Rapid Review and a full HTA is not needed	‘Full HTA not required’
A full HTA is recommended to assess the clinical effectiveness and cost effectiveness of the drug compared with the current standard of care, on the basis of the proposed price relative to currently available therapies	The NCPE believe the drug works as well or better than other ways to manage this condition, but the price of the drug is higher than other treatments used for this condition, and it is not clear that the drug is value for money. A full HTA may not be needed if the HSE can agree a suitable price reduction with the pharmaceutical company	‘Full HTA not required’c
A full HTA is recommended to assess the clinical effectiveness and cost effectiveness of the drug compared with the current standard of care	The NCPE believe that a full HTA is needed to allow us to recommend to the HSE whether to provide this drug. It is not clear that the drug works as well or better than other ways to manage this condition, and/or it is not clear that the drug is value for money	‘Full HTA required’
A full HTA is not recommended until additional efficacy and/or safety data are submitted. On the basis of current evidence, the NCPE recommends that the drug not be considered for reimbursementb	The NCPE believe that a full HTA is needed to allow us to recommend to the HSE whether to provide this drug; however, it should not be done at this time because there is not enough clinical evidence to inform the submission. We will assess this drug when more information is available from the pharmaceutical company. In the meantime, we recommend that the HSE consider not providing this drug	‘Full HTA required’d

HSE Health Service Executive, HTA health technology assessment, NCPE National Centre for Pharmacoeconomics

aNote that, in exceptional circumstances, the NCPE Review Group may make alternative recommendations

bThis recommendation should be considered while also having regard to the criteria specified in the Health (Pricing and Supply of Medical Goods Act) 2013

cOf all Rapid Review submissions that received this recommendation between 2016 and 2019, inclusive, 96% had not submitted a full HTA by April 2021; reimbursement has been agreed for 76% of these drugs following confidential price negotiations with the HSE (note that some negotiations may be ongoing) [source: in-house records]

dOf all Rapid Review submissions that received this recommendation between 2010 and 2019, 75% subsequently went on to have a full HTA submitted (source: in-house records)

Methods

A retrospective analysis of all Rapid Review submissions made to the NCPE from January 2010 to December 2019, inclusive, was conducted. The year 2010 was the first full year where the Rapid Review process was instituted. The cut-off date ensured a final recommendation had been made at the time of data collection. All Rapid Reviews commissioned by the HSE and appraised by the NCPE over this time period were eligible for inclusion. Data were sourced from the NCPE internal records and the European Medicines Agency website. The date the Rapid Review was commissioned by the HSE was considered as the starting date for each of the Rapid Reviews. The number of Rapid Review submissions received annually and over the full 10-year study period was recorded. For each Rapid Review, the Rapid Review recommendation was recorded (i.e. full HTA required or full HTA not required), and the proportion of submissions requiring a full HTA was calculated.

For each Rapid Review, additional data were collected for the variables outlined in Table 3. Because of changes in internal process for the reporting of Rapid Reviews over time, only data pertaining from 2012 onward were collected to ensure completeness of the dataset. Variables are categorised under the following: drug-related factors, economic factors and clinical evidence-related factors. All were hypothesised to be associated with the likelihood of requiring a full HTA. Drugs for orphan indications, drugs for cancer indications and drugs designated first-in-class have previously been found to be more likely to require a full HTA [9, 10]. The year in which the Rapid Review was conducted was also included, as anecdotally it has been proposed that the increasingly complex nature of the clinical evidence supporting submissions may mean a full HTA is more likely. As outlined in the description of the Rapid Review process, clinical evidence-related factors and economic factors are explicitly recognised as being associated with the likelihood of requiring a full HTA. The specific variables collected under these factors were informed by the nature and availability of data routinely reported in the NCPE’s Rapid Review report. Data were collected on whether a patient access scheme (PAS) was currently in place for the drug under assessment (where it was reimbursed at an earlier stage for a different indication), and if in place for the most relevant comparator(s). A PAS is typically implemented as the result of confidential negotiations between the Applicant and HSE. Summary statistics were calculated.

Table 3

Variable definitions

Variable	Definition	Description	Source
Dependent variable
Rapid Review recommendation	Indicates if a full HTA considered to be required following Rapid Review (in line with Table 2)	Binary	NCPE internal records
		Full HTA required = 1
		Full HTA not required = 0
Independent variables
Drug-related factors
Cancer drug	The marketing authorisation under which reimbursement is being sought relates to the treatment of malignancy	Binary	NCPE internal records
		Drug for cancer indication = 1
		Drug for non-cancer indication = 0
Orphan drug	Drug is designated an orphan medicinal product by the EMA for the indication under assessment, at the time of the Rapid Review	Binary	Community Register of Orphan Medicinal Products for Human Use
		Drug for orphan indication = 1
		Drug for non-orphan indication = 0
First-in-class	Drug is designated ‘first-in-class’ by the US FDA for the indication under assessment in the Rapid Review	Binary	FDA Novel Drug Approval reports
		Drug designated first-in-class = 1
		Drug not designated first-in-class = 0
Economic factors
Drug cost	Expected cost per patient per treatment course (excluding VAT)	Continuous (€)	NCPE internal records
Price vs comparator(s)	Indicates if the drug cost per treatment course is higher than most relevant comparator(s)	Binary
		Drug cost higher = 1
		Drug cost equal to or less than = 0
PAS intervention	A PAS identified as being in place for the drug, where already reimbursed for a different indication	Binary
		Existing PAS = 1
		No existing PAS = 0
PAS comparator	A PAS identified as being in place for one or more comparators	Binary
		Existing PAS = 1
		No existing PAS = 0
Clinical evidence-related factors
Final results reported	Variable indicating if final analysis of pivotal trial completed and available at time of undertaking Rapid Review	Binary	NCPE internal records; EMA website; clinicaltrials.gov
		Yes = 1
		No = 0
Trial phase	Variable indicating phase of pivotal trial used to inform marketing authorisation	Categorical
		Phase III
		Phase II
		Othera
Trial concealment	Variable indicating blinding or concealment process used in pivotal trial used to inform marketing authorisation	Categorical
		Double-blinded
		Single-blinded
		Open label
		Othera
Trial design	Variable indicating design of the pivotal trial used to inform marketing authorisation	Categorical
		Active comparator
		Placebo-controlled
		Single arm
		Othera

EMA European Medicines Agency, FDA Food and Drug Administration, HTA health technology assessment, NCPE National Centre for Pharmacoeconomics, PAS patient access scheme

a’Other’ included trials not consistent with other categories. Examples include phase I studies, bioequivalence studies or where a systematic review was presented as the primary source of clinical evidence

https://ec.europa.eu/health/documents/community-register/html/

Multivariable logistic regression models were estimated to evaluate the factors associated with the requirement for a full HTA following a Rapid Review. The dependent variable was the binary variable that indicated if a full HTA was required following a Rapid Review (as categorised in Table 2). A model including all variables relating to drug-specific factors was first specified (Model 1). An additional model (Model 2) was then estimated, which included all variables relating to economic factors and clinical evidence-related factors, in addition to those specified in Model 1. Marginal effects at the means were calculated post-model estimation.

An exploratory analysis was performed to estimate the additional NCPE appraisal time that theoretically would have been required to evaluate all drugs submitted to the NCPE over the study period as full HTAs, had the Rapid Review process not been established. The cumulative NCPE appraisal time for two separate scenarios was estimated, with the difference representing the additional NCPE appraisal time that would otherwise have been required. Timelines for the Rapid Review (mean 32.2 days) and full HTA (mean 133.3 days) processes were sourced from a previous analysis of the NCPE’s HTA timelines [6].1 The first scenario modelled a ‘world with’ the Rapid Review process, where all drugs underwent a Rapid Review and, where recommended, a full HTA. For the ‘world without’, it was assumed that all drugs for which reimbursement was sought over the 10-year study period underwent a full HTA. A number of additional sensitivity analyses were performed to explore uncertainty around the assumptions used in this analysis.

Our base-case analysis was based on a previous study that measured timelines for submissions appraised between 2015 and 2017 [6]; assumptions regarding the generalisability of these timelines to the wider context were examined. Scenarios where mean appraisal timelines both increased and decreased with respect to time were evaluated. Longer appraisal times could occur as a result of increases in both the volume and complexity of submissions received, whereas reductions in timelines might reflect additional recruitment within the NCPE in recent years.

Our base-case analysis assumed that the time taken for a full HTA in the ‘world without’ the Rapid Review process would equal that in the ‘world with’, implying that the Rapid Review does not contribute to the efficiency of the subsequent full HTA process. Timelines were varied by + 20% and − 20% to examine the assumption. The Rapid Review is used to inform pre-submission meetings held between the NCPE and Applicant prior to submission of a full HTA, meaning it is likely that a full HTA in a ‘world without’ would require additional appraisal time.

In our base-case analysis, it is assumed that all drugs for which a full HTA is recommended after a Rapid Review proceed to submit a full HTA dossier. It is the NCPE’s experience that not all Applicants elect to proceed with a full HTA submission, meaning a reimbursement recommendation cannot be made. When testing this assumption, only reductions in the rate of Applicants’ submissions were examined (as it is not possible to ‘over-submit’).

We examined a scenario where, in the ‘world without’ the Rapid Review process a number of Applicants would elect to not proceed with a reimbursement application. It may be that a number of Applicants would not submit a full HTA (owing to the increased workload associated), in the absence of the opportunity to first submit a Rapid Review and potentially receive a reimbursement recommendation at this stage. As for the previous scenario, only reductions were examined.

Data collection and analysis were completed using Microsoft Excel® and Stata® Version 14 (StataCorp)

Results

A total of 446 Rapid Review submissions were assessed by the NCPE from January 2010 to December 2019, inclusive. The number per year increased from 19 in 2010 to 57 in 2019, with a peak of 63 in 2017 (Fig. 1). In terms of Rapid Review recommendations, a full HTA was deemed to be required in 221 (49.6%) of the Rapid Reviews. The proportion requiring a full HTA varied annually, from 68.4% in 2010 to 38.9% in 2014. Of the Rapid Reviews analysed, 390 were conducted between January 2012 and December 2019, inclusive. Summary statistics for additional data collected for these Rapid Reviews are presented in Table 4.

Fig. 1

Annual Rapid Review submissions 2010–19. HTA health technology assessment

Table 4

Summary statistics: Rapid Reviews 2012–19 (N = 390)

Variable		N (%)
Rapid Review recommendation	Full HTA required	192 (49.2%)
	Full HTA not required	198 (50.8%)
Drug-related factors
Cancer drug	Drug for cancer indication	135 (34.6%)
	Drug for non-cancer indication	255 (65.4%)
Orphan drug	Drug for orphan indication	86 (22.1%)
	Drug for non-orphan indication	304 (77.9%)
First-in-class	First-in-class	66 (16.9%)
	Not first-in-class	324 (83.1%)
Economic factors
Drug cost	Mean (SD)	€53,221 (€96,698)
	Median (IQR)	€17,531 (€2240, €62,283)
Price vs comparator(s)	Drug cost higher than comparator	321 (82.3%)
	Drug cost equal to or less than comparator	69 (17.7%)
PAS intervention	Existing PAS	89 (22.8%)
	No existing PAS	301 (77.2%)
PAS comparator	Existing PAS	73 (18.7%)
	No existing PAS	317 (81.3%)
Clinical evidence-related factors
Final results available	Yes	249 (63.8%)
	No	141 (36.2%)
Trial phase	Phase III	326 (83.6%)
	Phase II	42 (10.8%)
	Other a	22 (5.6%)
Trial concealment	Double-blinded	239 (61.3%)
	Single-blinded	8 (2.1%)
	Open label	128 (32.8%)
	Othera	15 (3.8%)
Trial design	Active comparator	184 (47.2%)
	Placebo-controlled	144 (36.9%)
	Single arm	45 (11.5%)
	Othera	17 (4.4%)

HTA health technology assessment, IQR interquartile range, PAS patient access scheme, SD standard deviation

a‘Other’ included trials not consistent with other categories. Examples include phase I studies, bioequivalence studies or where a systematic review was presented as the primary source of clinical evidence

Results of the multivariable logistic regression models estimated are presented in Table 5. Results are presented as marginal effects at means, with a positive coefficient indicating that a variable is associated with an increased likelihood of requiring a full HTA and a negative coefficient indicating a full HTA is less likely following Rapid Review. Based on the results of Model 1, drugs for cancer indications (47.3 percentage points [ppts], p < 0.01), drugs for orphan indications (22.8 ppts, p < 0.01) and drugs designated first-in-class (42.9 ppts, p < 0.01) were all associated with a statistically significant increase in the likelihood of requiring a full HTA. However, when variables relating to cost and clinical evidence were included, the association was no longer significant for drugs for orphan indications (Model 2: 0.1 ppts). As compared to Rapid Reviews completed in 2012, those conducted in 2019 were less likely to require a full HTA (− 37.0 ppts, p < 0.05). Otherwise, no clear trend was observed between the year the Rapid Review was completed and the requirement for a full HTA. Economic factors were significantly associated with a higher likelihood of requiring a full HTA. As compared to drugs priced on par or lower than comparators, drugs that were more expensive than comparators were associated with an 80.2 ppts increase in the likelihood of requiring a full HTA (p < 0.01). For every €1000 increase in the cost per treatment course of the drug, the likelihood of requiring a full HTA increased by 0.3 ppts (p < 0.01). The pre-existence or offer of a PAS for the drug under assessment was associated with a lower likelihood of requiring a full HTA (− 19.0 ppts, p < 0.05); similarly, a lower likelihood was reported where a PAS was known to be in place for a comparator (− 27.0 ppts, p < 0.05).

Table 5

Results of multivariable logistic regression models: Rapid Reviews 2012–19 (n = 390)

Variable	Model 1	Model 2
Drug-related factors
Cancer drug	0.473***	0.452***
Orphan drug	0.228***	0.001
First-in-class year (vs 2012)	0.429***	0.403***
2013	− 0.179	− 0.021
2014	− 0.248**	− 0.169
2015	− 0.184	− 0.128
2016	− 0.108	− 0.056
2017	− 0.150	− 0.007
2018	− 0.141	− 0.159
2019	− 0.265**	− 0.370**
Economic factors
Drug cost (€1000s)		0.003***
Price vs comparator(s)		0.802***
PAS intervention		− 0.190**
PAS comparator		− 0.270***
Clinical evidence-related factors
Final results available		− 0.276***
Trial phase (vs phase III)
Phase II		− 0.300**
Othera		− 0.453***
Trial concealment (vs double-blinded)
Single-blinded		0.130
Open label		− 0.012
Othera		− 0.031
Trial design (vs active comparator)
Placebo-controlled		0.246***
Single arm		− 0.178
Othera		0.294
N	390	390
McFadden’s pseudo-R2	0.2037	0.4391

HTA health technology assessment, PAS patient access scheme, *p < 0.10; **p < 0.05; ***p < 0.01

a’Other’ included trials not consistent with other categories. Examples include phase I studies, bioequivalence studies or where a systematic review was presented as the primary source of clinical evidence

Results presented as marginal effects at the mean. Positive coefficient indicates more likely to be associated with a requirement for full HTA. Negative coefficient indicates less likely to be associated with a requirement for a full HTA

In terms of clinical evidence-related factors, drugs that were supported by clinical trials where results of the final analysis were available at the time of the Rapid Review were less likely to require a full HTA (− 27.6 ppts, p < 0.01). Compared to submissions supported by evidence from a pivotal phase III trial, those that related to phase II pivotal trials (− 30.0 ppts, p < 0.05) or ‘other’ types of trial (− 45.3 ppts, p < 0.01) were less likely to require a full HTA. There was no association found between the concealment used in the pivotal trial and the likelihood of requiring a full HTA. Placebo-controlled pivotal trials were more likely to require a full HTA than those where the comparator involved an active control (24.6 ppts, p < 0.01).

The theoretical additional NCPE appraisal time required, had the Rapid Review process not been instituted, was estimated at 15,631 days, cumulative over the 10-year period assessed. Compared to a world without the Rapid Review process, the time taken in the ‘world with’ represents a 26.3% reduction in overall NCPE appraisal time. Assuming a mean NCPE appraisal time for a full HTA of 133.3 days, the difference in appraisal time represents the equivalent of 117 full HTAs over the 10-year period examined. Results of sensitivity analyses undertaken to examine uncertainty are presented in Table 6. For all the assumptions tested, the Rapid Review process results in a net reduction in NCPE appraisal days (see final column, Table 6).

Table 6

NCPE appraisal time in ‘world with’ and ‘world without’ a Rapid Review process: Rapid Reviews 2010–19 (N = 446)

	‘World with’ (days)	‘World without’ (days)	Reduction in NCPE appraisal time (days)
Base case	43,821	59,452	15,631
Sensitivity analyses
Base-case assumption A: NCPE appraisal timelines for RR and a full HTA constant over time [6]
Option 1: Timelines for a Rapid Review and a full HTA increase over timea	42,576	57,612	15,036
Option 2: Timelines for a Rapid Review and a full HTA decrease over timeb	45,065	61,291	16,226
Base-case assumption B: NCPE appraisal times for a full HTA are the same in both the ‘world with’ and ‘world without’ the Rapid Review [6]
Option 1: NCPE completion time for a full HTA is 20% shorter in the ‘world without’ the Rapid Review	43,821	47,561	3741
Option 2: NCPE completion time for a full HTA is 20% longer in the ‘world without’ the Rapid Review	43,821	71,342	27,522
Base-case assumption C: all drugs for which a full HTA was recommended underwent a full HTA
Option 1: Applicant does not submit a full HTA for 10% of drugs where a full HTA is recommended	40,875	59,452	18,577
Option 2: Applicant does not submit a full HTA for 20% of drugs where a full HTA is recommended	37,929	59,452	21,523
Base-case assumption D: if a full HTA was required for all drugs, all Applicants would still submit
Option 1: Applicant does not submit a full HTA for 10% of drugs in the ‘world without’	43,821	53,507	9686
Option 2: Applicant does not submit a full HTA for 20% of drugs in the ‘world without’	43,821	47,561	3740

HTA health technology assessment, NCPE National Centre for Pharmacoeconomics

aTimelines 2010–14 80% of 2015–17 timelines, and 2017–19 timelines 120% of 2015–17 timelines

bTimelines 2010–14 120% of 2015–17 timelines, and 2017–19 timelines 80% of 2015–17 timelines

For ‘world with’, 221 submissions required a full HTA following a Rapid Review and 225 did not require a full HTA following a Rapid Review. For ‘world without’, 446 required a full HTA. Annual breakdown presented in Fig. 1. Mean time for Rapid Review: 32.2 days; mean time for a full HTA 133.3 days

Discussion

This study describes the Rapid Review process, a system designed and implemented by the NCPE to facilitate prioritisation of resources within a national HTA agency. A total of 446 Rapid Review submissions were received by the NCPE over the 10-year period between 2010 and 2019, with a general year-on-year increase in the number of Rapid Review submissions received. This analysis highlights that a full HTA is deemed to be required in approximately half of drugs for which reimbursement is sought. The analysis of the factors associated with a Rapid Review recommendation is the first to use data uniquely available to the NCPE. As was reported in previous analyses, drugs for cancer indications and drugs designated ‘first-in-class’ were more likely to require a full HTA [10]. In contrast to previous analyses, which did not include factors relating to cost or clinical evidence, drugs for orphan indications were not significantly associated with the likelihood of requiring a full HTA [9, 10]. An exploratory analysis indicated that the Rapid Review process facilitates an appreciable reduction in NCPE appraisal time, with findings robust to a number of sensitivity analyses (Table 6).

Many HTA agencies have highlighted the challenges posed by both an increasing demand for HTA and increasing pressure for ‘rapid’ assessments [12]. In light of these pressures, a key challenge for HTA agencies is determining how a ‘formal, effective and acceptable’ prioritisation process can be selected [12]. Here, we share the experience of a national HTA agency in implementing one such prioritisation strategy. The Rapid Review process has been a core component of the NCPE’s assessment pathway for over 10 years. It facilitates the identification of drugs that require additional assessment in the form of a full HTA, while maintaining a robust appraisal of the relevant clinical and economic (in terms of cost and budget impact) evidence for drugs that do not require further assessment.

The NCPE’s Rapid Review process is unique to the Irish HTA pathway. A number of HTA agencies have developed accelerated HTA processes in order to improve workflow and efficiency, including the abbreviated HTA submission process at the Scottish Medicines Consortium and the ‘fast-track’ appraisal process at the National Institute for Health and Care Excellence [13, 14]. However, only drugs that meet specific criteria are eligible to be evaluated under these systems. In comparison, all drugs under assessment by the NCPE may be subject to a Rapid Review. It is worth recognising that the term ‘Rapid Review’ is used in other instances within HTAs. It is typically used to describe a review that describes the characteristics of the technology under assessment, as well as evaluating safety and effectiveness issues [15]. We do not propose that the ‘rapid’ or accelerated HTA is a novel concept, but highlight how the implementation within the overall HTA framework in Ireland represents a unique and adaptive way to manage resource prioritisation within the NCPE.

A criticism previously made elsewhere of the Rapid Review process was that it results in duplication, particularly for drugs where it is believed there is a high likelihood that a full HTA will be required [10]. The NCPE has developed strategies to mitigate these concerns. For example, the NCPE’s Budget Impact Model template may be used at the Rapid Review stage, and may later be re-submitted along with the electronic model at the full HTA stage if required [16]. Moreover, where drugs do require a full HTA, the Rapid Review offers an important opportunity for the NCPE Review Group to identify key uncertainties in advance of the full HTA. All drugs referred for a full HTA are discussed at a pre-submission meeting between the NCPE Review Group and the Applicant where these issues can be communicated and discussed in advance of the submission. It is not possible to quantify the downstream impact of this process on the full HTA submission. Anecdotally, it is considered a valuable exercise by both the NCPE and pharmaceutical industry representatives.

Limitations

This study is subject to a number of limitations. First, it should be noted that the Rapid Review process only became a part of the HTA process at the NCPE in mid-2009. Therefore, the number of Rapid Reviews recorded for 2010 may underestimate the total number of drugs that underwent assessment with the NCPE at this time.

Additionally, the analysis of the factors associated with the Rapid Review recommendations is limited by the scope of the variables included. It was not possible to include certain variables that were expected to be associated with the requirement for a full HTA. For example, the net drug budget impact is an important consideration in determining the requirement for a full HTA [5]. The Review Group frequently considered the point estimates presented by the Applicant as highly uncertain and it is beyond the scope of the Rapid Review process to revise the Applicant’s budget impact estimates. Therefore, accurate data on the expected net drug budget impact were not routinely available. In addition, when considering the variables selected to represent clinical evidence-related factors, it is important to highlight that the Rapid Review seeks to evaluate how likely it is that the intervention is equally or more effective than the relevant comparator(s), in the target population, with minimal uncertainty. It is unlikely that the clinical evidence-related variables included in the analysis have adequately captured this, as suggested by the unexpected finding that Rapid Reviews supported by phase II trials were less likely to require a full HTA than phase III trials.

Further, while it is likely that the Rapid Review process has resulted in substantial savings in NCPE appraisal time since implementation, estimating the additional NCPE appraisal time that would be required were the Rapid Review process not instituted required numerous assumptions, meaning the uncertainty associated with our findings is unavoidable. A number of one-way sensitivity analyses were performed to evaluate the robustness of our results. Two scenarios resulted in a reduction in NCPE appraisal time that was appreciably lower than our base-case analysis. The first (Base-case assumption B, Option 1) examined a scenario where NCPE completion time for a full HTA is shorter in the ‘world without’ the Rapid Review process. Given the Rapid Review is used to inform pre-submission meetings held between the NCPE and the Applicant prior to the submission of a full HTA, it is unlikely that any change in this parameter would occur in this direction. The second (Base-case assumption D, Option 2) examined a scenario where a proportion of Applicants elect to not submit a full HTA in the ‘world without’ the Rapid Review. Here, we found the Rapid Review process remains time saving at a reduction of 20% of submissions. It is unclear how realistic this scenario is (i.e. how much of a deterrent the full HTA process might be, in the absence of the Rapid Review process). A more general limitation of this analysis is the use of a very specific metric to measure the impact of the Rapid Review process. While NCPE appraisal time remains an informative and intuitive metric, there may be changes in other important outcomes arising from avoiding full HTAs not captured in this analysis.

Moreover, the primary aim of this analysis is to provide insight into the Rapid Review process from the perspective of the HTA agency. However, given the potential for a shorter time to a reimbursement decision depending on the outcome of the Rapid Review, we recognise that the process is of interest to a variety of stakeholders. Previous publications have suggested that reimbursement is likely in cases where a full HTA is not required; however, we highlight that the recommendation that a full HTA is not required does not necessarily indicate a more straightforward path to reimbursement [10, 19]. Reimbursement decisions in Ireland are made by the HSE on the basis of decision-making criteria set out in the Health (Pricing and Supply of Medical Goods) Act 2013, which includes additional criteria to those assessed by the NCPE [1]. Neither reimbursement outcomes, reimbursement timelines, nor any other factors outside the HTA appraisal process that impact on these outcomes have been evaluated in this analysis. As a result, it is not possible to evaluate if there is any association between the Rapid Review process and these outcomes. Further research in this area may provide valuable insight to a broader range of stakeholders.

Conclusions

As the demand for the reimbursement of drugs from finite healthcare budgets continues to grow in Europe and beyond, pressure on reimbursement systems persist. It is important that experiences of HTA agencies are shared as processes of evaluation continue to evolve. This work describes the NCPE’s Rapid Review process in Ireland over the last decade. The process is a pivotal and well-established part of the HTA process in Ireland that allows for appropriate resource prioritisation within a national HTA agency. It is considered to be an efficient way of determining the requirement for a full HTA and targeting resources for those drugs where there is most value in conducting a HTA.

The results of this study demonstrate that a full HTA was not required for approximately half of the Rapid Reviews submitted. Drugs for cancer indications and drugs designated first-in-class were more likely to require a full HTA. In contrast to previous analyses, drugs for orphan indications were not associated with an increased likelihood of requiring a full HTA. A number of variables relating to cost were found to be significantly associated with the likelihood of requiring a full HTA. Determining the relationship between the supporting clinical evidence and the Rapid Review recommendation is challenging because of the limited sensitivity of the variables considered. While it is likely the process has substantially reduced NCPE appraisal time, estimation of such savings is challenging and subject to a number of limitations. The importance of a robust appraisal process is highlighted given that for approximately half of submissions the Rapid Review is the only formal evidence-based assessment. Therefore, there is a need to ensure that the process is continuously evolving to meet the current requirements of the healthcare system.

This work describes the National Centre for Pharmacoeconomics Rapid Review, a pivotal part of the health technology assessment (HTA) process in Ireland that allows for appropriate resource prioritisation within a national HTA agency. It is considered to be an efficient way of determining the requirement for a full HTA and targeting resources for those drugs where there is most value in conducting an HTA.

There has been a general year-on-year increase in the number of submissions received since the process was introduced.

Drugs for cancer indications and first-in-class drugs are more likely to require a full HTA. When economic factors and clinical evidence-related factors were included in the model, drugs for orphan indications were not found to be associated with an increased likelihood of requiring a full HTA. The analysis was limited by the ability of the included variables to capture features such as ‘uncertainty’ in the corresponding clinical data.

The Rapid Review process has resulted in appreciable reductions in NCPE appraisal time over the 10-year study period.