New Medicines in Wales: The All Wales Medicines Strategy Group (AWMSG) Appraisal Process and Outcomes.

BACKGROUND: The All Wales Medicines Strategy Group (AWMSG) develops prescribing advice and is responsible for appraising new medicines for use in Wales. In this article, we examine the medicines appraisal process in Wales, its timeliness and its impact on medicines availability in Wales, and compare its processes and recommendations with the two other UK health technology appraisal bodies [the National Institute for Health and Care Excellence (NICE) and the Scottish Medicines Consortium (SMC)].
METHODS: We reviewed the medicines appraisals conducted by AWMSG between October 2010 and September 2015. The duration of the process and the recommendations made by AWMSG were compared with those of NICE and SMC. Only publicly available data were considered in this review.
RESULTS: AWMSG conducted 171 single technology appraisals for 137 medicines during the study period (34 were for medicines previously appraised by AWMSG but these were for new indications). Of these, 152 appraisals were supported for use in NHS Wales (33 with restrictions) and 19 were not supported. Recommendations broadly concurred with SMC and NICE for the majority of appraisals. Compared with NICE recommendations, the median time advantage gained in Wales for those medicines that received a positive AWMSG recommendation and which were subsequently superseded by NICE advice was 10.6 months (range 3.5-48.3 months; n = 17).
CONCLUSION: This review highlights the work carried out by AWMSG over a 5-year period, and provides evidence to support the effectiveness of the appraisal process in terms of patients in Wales gaining earlier access to medicines and efficiency through reduced duplication with NICE.

Key points for Decision Makers

Background

Since political devolution in 1999, governments across the UK each have control over the level of funding, and the structure and governance, of the National Health Service (NHS) available to their residents. In 2015–2016, the Welsh Government spent £6.7 billion on health [1], representing approximately 12% of gross value added in Wales (2015) [2]. The total spend on prescribed medicines in primary healthcare was £518 million [1]; 7.7% of the NHS expenditure. In the community, Wales dispensed the highest number of prescription items per head per annum among the four devolved countries: 25.7 compared with 21.6 in Northern Ireland, 20.0 in England and 19.0 in Scotland [3].

With a fixed budget [4], each country seeks to achieve best value via established health technology assessment (HTA) bodies which evaluate medicines for their clinical and cost effectiveness. The Scottish Medicines Consortium (SMC) evaluates all authorised medicines submitted to it; in England, the National Institute for Health and Care Excellence (NICE) appraises new medicines referred to it by the Department of Health; and in Wales, the All Wales Medicines Strategy Group (AWMSG) adopts NICE guidance where available and appraises the remaining new medicines not on the NICE work programme. Northern Ireland does not have its own comparable HTA agency so has not been included in this review.

AWMSG was established by the Welsh Government in 2002, as a statutory scientific advisory committee, to provide advice on medicines management and prescribing to the then Minister for Health and Social Services (now the Cabinet Secretary for Health and Social Services). AWMSG consists of NHS clinicians, pharmacists, healthcare professionals, academics, health economists, industry representatives and patient advocates, whose aim is to reach a consensus on the use of medicines within primary and secondary care [5]. The work of AWMSG can be broadly categorised as appraising new medicines, and issuing prescribing and medicines optimisation advice. These activities are the concerns of two subgroups: the New Medicines group (NMG), which considers the clinical and economic evidence on new medicines, and which provides preliminary recommendations to AWMSG on the introduction of new medicines in Wales; and the All Wales Prescribing Advisory Group which advises AWMSG on strategic developments in primary and secondary care prescribing. The latter group assists with monitoring prescribing, advising on prescriber training, and developing prescribing indicators and audits for a national incentive scheme.

In this article, we focus on the AWMSG process for appraising new medicines, review its achievements in terms of appraisal recommendations and compare outcomes with those of the other two HTA bodies in the UK: NICE and SMC.

Appraisals Process

The AWMSG appraisal programme provides recommendations based on the clinical and cost effectiveness of new medicines and new indications via a transparent and evidence-based process. The original remit of the AWMSG appraisal process was focused on high-cost medicines (i.e. those costing > £2000 per patient per year). In April 2007, the AWMSG appraisal process was broadened to also include all new cardiovascular and cancer medicines, in response to clinical service need. Since October 2010, the remit has been further expanded to include all new medicines that are not on the 12-month work programme of NICE [6]. AWMSG therefore considers medicines that are not appraised by NICE and those for which a final NICE appraisal is not expected for a considerable time. Should NICE issue guidance subsequently, however, its final technology appraisal advice supersedes AWMSG recommendations. If a medicine has not been recommended for routine use in Wales, a request for access to the medicine can be made through the individual patient funding process, or through the One Wales Interim Pathways commissioning route [7, 8].

Whereas NICE assesses selected medicines and other health technologies, and operates two appraisal methods (single and multiple technology appraisals), both AWMSG and SMC conduct single technology appraisals of all medicines. AWMSG, unlike SMC, is aligned with NICE in that Welsh health boards have a legal requirement to implement NICE and AWMSG guidance within a given timeframe [9]. This means that NHS Wales benefits from formal, robust evaluation of a wide range of medicines, including treatments such as those for HIV and rare diseases.

AWMSG considers either a full or limited submission from a marketing authorisation holder (MAH): limited submissions are for new formulations, minor license extensions of existing medicines (e.g. use in paediatrics), if the anticipated usage in NHS Wales is considered to be of minimal budget impact, or if the estimated difference in cost compared with relevant comparator(s) is small. There is no explicit threshold for what is considered a minimal budget impact or small difference in cost: these are assessed on a case-by-case basis by the AWMSG secretariat. SMC has a similar approach; full or abbreviated submissions, where the latter is for consideration of new formulations and minor license extensions [10]. In contrast, health technologies referred to NICE for appraisal are prioritised, with only about 40% of all new medicines appraised in 2012 [11]. NICE aims to consider all new significant medicines and indications: new formulations and minor license extensions are not normally appraised by NICE [12, 13].

A summary of the AWMSG appraisal process and schedule are outlined in Fig. 1 [14]. The process used by AWMSG to produce Final Appraisal Recommendations (FARs) for the Welsh Government was awarded NICE accreditation in 2011 for 5 years and was reaccredited in October 2016 for a further 5 years [15]. The secretariat arm of AWMSG, the All Wales Therapeutics and Toxicology Centre (AWTTC), initially suggests the appraisal scope and informs the MAH whether a full or limited submission is required. Following an initial review of the submission for completeness and appropriateness, and clarification of the scope from AWTTC or the MAH, the scope and schedule are confirmed. AWTTC aims to process an application (full or limited) within 6 months of the submission being received.

Fig. 1

Summary of medicines appraisal process in Wales by the All Wales Medicines Strategy Group (AWMSG) [14]. ASAR, AWMSG Secretariat Assessment Report; AWTTC, All Wales Therapeutics and Toxicology Centre; CE, clinical expert; EMA, European Medicines Agency; FAR, Final Appraisal Recommendation; MAH, marketing authorisation holder; NMG, New Medicines Group; PAR, Preliminary (to AWMSG) Appraisal Recommendation

Following confirmation of the scope and schedule, AWTTC produces a report (AWMSG Secretariat Assessment Report [ASAR]). The AWTTC, consisting of pharmacists, clinical pharmacologists, health economists and appraisal scientists, assesses the clinical efficacy, comparative safety, cost effectiveness and likely budget impact of a new medicine. The secretariat summarises and critiques the evidence submitted by the MAH and evidence found in other relevant publicly available material when compiling the ASAR. Evidence from reviews of the literature, conducted by AWTTC, is also considered and incorporated into the ASAR if deemed relevant. The assessment process encompasses a critical evaluation of the clinical evidence, including any quantitative evidence synthesis conducted by the MAH; a detailed interrogation of the economic and budget impact models, with alternative scenarios and analyses conducted if considered appropriate, and comments on the plausibility of alternative scenarios. The AWTTC has about 4 weeks to complete an ASAR, which allows for data to be verified (e.g. checking the economic model sensitivity to changes in key inputs), expert clinical opinion to be elicited and clinical evidence to be summarised and critiqued. Relevant clinical experts and patient organisations are identified and their views solicited in a questionnaire: clinical experts must declare any interests; individual patients and carers can also comment using a questionnaire available on the AWMSG website [16, 17]. The ASAR is sent to the MAH for comment and, on return, AWTTC considers the responses and the ASAR may be amended.

Approximately 3 months after the scope is agreed, NMG members meet in private to discuss the submission and make a Preliminary Appraisal Recommendation (PAR) based on the ASAR, MAHs’ comments on the ASAR, clinical expert opinion, and the views of patients, carers and patient organisations. Approximately 1 month later, this is considered, in turn by AWMSG, which additionally considers equity, broader societal issues and the budget impact of the new medicine in Wales. A FAR is agreed by the AWMSG members and submitted to the Welsh Government for ratification. If the MAH disagrees with the AWMSG recommendation, they may request an independent review to address complaints in relation to process and/or scientific issues: for example, the MAH may feel that insufficient time or opportunity was given for discussion of relevant issues, and/or there has been a misinterpretation of information [18].

AWMSG has always held its meetings in public to ensure the transparency of the decision-making process. The ASAR, the PAR, comments from the MAH on the PAR and a summary of clinical experts’ opinions are posted on the AWMSG website prior to the AWMSG public meeting.

Although the onus for engagement with the AWMSG process lies with the MAH, AWTTC horizon scanning helps to identify medicines expected to receive their marketing authorisation (MA; product licence) within 18 months and will alert the MAH to the appraisal process [19]. Should the MAH fail to engage, and choose not to submit any evidence in support of their medicine, AWMSG will either issue a Statement of Advice confirming the medicine cannot be endorsed for use within NHS Wales, or, when directed to do so by the AWMSG Steering Committee, will appraise the medicine using publicly available information [19].

Appraisal Criteria

The criteria by which NMG and AWMSG appraise medicines are mainly aligned with those of NICE and SMC [20-22]. The determination of clinical effectiveness allows for proportionate consideration of different hierarchies of evidence. This is consistent with a view that diverse forms of evidence serve different purposes [23].

AWMSG judgments on the cost effectiveness of new medicines are made on the basis of the incremental cost per quality-adjusted life-year (QALY) gained. Economic evidence is normally derived from a model which synthesises the available evidence to assess the expected costs and QALY gains over an appropriate time horizon. This is normally a lifetime horizon for treatments for chronic diseases or which impact on survival. Medicines determined to be clinically effective, and whose incremental cost-effectiveness ratios (ICERs) are below £20,000 per QALY, are normally recommended for use; with an ICER between £20,000 and £30,000 per QALY, judgments on their acceptance are based on the level of certainty in the ICER, the innovative nature of the medicine and the nature of the disease being treated. Above an ICER of £30,000 per QALY gained, the case for supporting a medicine on these factors has to be increasingly strong [20].

The end-of-life criteria developed by NICE apply for medicines that are indicated for terminal diseases (life expectancy normally less than 2 years), whose ICERs exceed £30,000 per QALY, and which are expected to generate survival benefits of at least 3 additional months. In such cases, AWMSG is able to apply greater moderation in its assessment of their cost effectiveness. AWMSG assesses the impact of giving more weight to QALYs achieved in the later stages of terminal diseases, and of the magnitude of the additional weight that would need to be assigned to the QALY benefits in this patient group for the cost effectiveness of the medicine to fall within the current threshold range [24]. However, there is no explicit maximum allowable weighting, offering AWMSG members flexibility to enable medicines such as abiraterone (Zytiga®) for the treatment of prostate cancer, which otherwise may not be deemed to be cost effective, to be recommended for use in Wales [25]. SMC uses broader end-of-life criteria; an end-of-life medicine is defined by SMC as one used to treat a condition at a stage that usually leads to death within 3 years with currently available treatments [26].

AWMSG and SMC have specific policies for orphan and ultra-orphan medicines [26-28]. An orphan medicine is defined as a medicine with orphan status as designated by the European Medicines Agency, which includes conditions affecting not more than 5 in 10,000 persons. An ultra-orphan medicine is a medicine that has been designated orphan status by the European Medicines Agency and is used to treat a condition with a prevalence of 1 in 50,000 or less in the UK. The AWMSG [27] and SMC [26] policies enable access to such treatments, which invariably exceed the cost-effective threshold, following a commitment to the non-abandonment of individuals with needs for highly specialised treatments [29]. NICE considers medicines for very rare conditions separately from the HTA process and via their highly specialised technology evaluation process [30].

A further appraisal consideration distinguishing AWMSG from NICE is the assessment of budget and wider societal impacts [20]. Medicines with a large budget impact will be associated with significant opportunity cost if an incorrect appraisal recommendation is made. Thus, AWMSG will need to be more certain of the ICER estimates when the budget impact is considered to be large. SMC also considers budget impact and the wider issues that go beyond the scientific arguments [31].

Methods

All medicines appraised by AWMSG from October 2010 to September 2015 inclusive were reviewed. The dataset was restricted to a recent 5-year period when AWMSG processes and appraisal criteria were stable. Only publically available data from AWMSG [32], NICE [33] and SMC [34] were considered in this review.

The approved therapeutic indication for each medicine at the time of the appraisal was recorded, as well as the medicine’s categorisation according to the British National Formulary (BNF), and whether or not the medicine had ultra-orphan status. All appraised medicines were categorised according to the AWMSG final recommendation: a positive recommendation, a positive recommendation with restrictions (referred to as an optimised recommendation by NICE) or a negative recommendation. A restricted recommendation is given when the use of the medicine is restricted to a sub-population of the full indication under consideration.

The number of independent reviews during this period was recorded, as was the number of positive recommendations based on a resubmission after an initial negative recommendation from AWMSG, and instances when the MA had been withdrawn after the appraisal had been completed. The number of Statements of Advice was recorded for those medicines that were subsequently appraised by AWMSG during the 5-year period.

The duration of the HTA process in Wales was measured as the time from (1) the European Medicines Agency MA to receipt of a MAH submission by AWTTC; (2) AWTTC sending the final appraisal scope to the MAH, to the AWMSG meeting when the recommendation is agreed and announced; (3) MA to the AWMSG meeting; and (4) MA to final decision by the Welsh Government (i.e. ministerial ratification).

The recommendations for the medicines that were also appraised by NICE and/or SMC were reviewed and compared with the AWMSG recommendations, as were the times from MA to advice being published.

The primary aim of this study is to describe the AWMSG HTA process and highlight the recommendations made by the three organisations; therefore, descriptive statistics are reported. Statistical analyses using Wilcoxon matched pairs signed rank tests were carried out to compare (1) the time it takes for AWMSG and NICE or SMC to process an appraisal (MA to recommendation); and (2) the time it takes for patients in Wales (MA to ministerial ratification), England and Scotland (MA to guidance) to have access to the same medicines. AWMSG and SMC time to event data were not normally distributed; therefore, the non-parametric analyses (Wilcoxon) were carried out. In all cases, p < 0.05 was taken as the minimum level of statistical significance.

Results

All Wales Medicines Strategy Group (AWMSG) Appraisal Recommendations

Between October 2010 and September 2015, AWMSG conducted 171 appraisals of 137 medicines; 34 of these 171 appraisals were of new indications for medicines previously appraised by AWMSG. Of the 171 appraisals, 119 (69.6%) received a positive recommendation, 33 (19.3%) received a positive recommendation with restrictions and 19 (11.1%) received a negative recommendation. Of the latter category, six subsequently received a positive recommendation (with or without restriction) based on resubmissions; in the case of one (tacrolimus [Advagraf®]), two negative recommendations were issued prior to a restricted recommendation being made [35]. Additionally, the marketing licence was withdrawn from two products (ferumoxytol [Rienso®], recommended with restrictions; colestilan [BindRen®], not recommended) after the appraisal had been completed [36, 37]. Of the 171 medicines appraised, 56 (32.7%) were previously given a Statement of Advice due to the MAH not engaging in the appraisal process within 3 months of MA; the Welsh Government directed AWMSG to appraise one of these medicines (fampridine [Fampyra®]) using publicly available information [38]. One appraisal was taken to the Independent Review Panel (pazopanib [Votrient®] was not recommended by AWMSG); the MAH had expressed concerns in relation to the interpretation of the evidence. The grounds for review were upheld and AWMSG reappraised the product taking into account the issues raised in the independent review, and subsequently recommended it as an option for use [39].

A breakdown of appraisal outcome by BNF chapter is provided in Table 1. AWMSG most commonly appraised products for the treatment of HIV and made positive recommendations (with or without restrictions) for all 20 treatments. Of the 171 appraisals completed by AWMSG, four were of ultra-orphan medicines, all of which were recommended for use in NHS Wales (one with restrictions).

Table 1

Breakdown of the All Wales Medicines Strategy Group appraisal outcome by British National Formulary chapter and subsection, and orphan status

Orphan status or BNF chapter/subsection	Number of appraisals	Recommended	Recommended with restrictions	Not recommended
All AWMSG appraisals	171	119	33	19
Orphan status
Ultra-orphan medicinesa	4	3	1	0
Non-ultra-orphan medicinesa	167	116	32	19
BNF chapter/subsection
1. Gastrointestinal system	4	2	1	1
2. Cardiovascular system	13	9	2	2
3. Respiratory system	13	12	0	1
4. Central nervous system	16	11	3	2
5. Infections	36	28	8	0
5.3.1. HIV infectionb	20	19	1	0
6. Endocrine system	19	13	3	3
6.1.2. Antidiabetic drugsb	13	9	3	1
7. Obstetrics, gynaecology, and urinary-tract disorders	3	3	0	0
8. Malignant disease and immunosuppression	33	20	6	7
8.1.5. Other antineoplastic drugsb	17	10	4	3
9. Nutrition and blood	11	6	4	1
10. Musculoskeletal and joint diseases	15	10	4	1
11. Eye	2	1	0	1
12. Ear, nose, and oropharynx	0	0	0	0
13. Skin	4	2	2	0
14. Immunological products and vaccines	0	0	0	0
15. Anaesthesia	2	2	0	0

AWMSG All Wales Medicines Strategy Group, BNF British National Formulary

aMedicines classed as either ultra-orphan or non-ultra-orphan by AWMSG

bOnly the BNF subsections where most AWMSG appraisals have been undertaken have been included for comparison

Duration of AWMSG Process

The median time from MA to receipt of an MAH submission by AWTTC was 4.9 months (range 7 days to 56.4 months; n = 132). A total of 39 appraisals had been excluded from analysis. There was no scope date and/or submission date available for 29 of 171, and for nine of 171 the MA was received after submission to AWTTC. One of the 171 was excluded because it underwent an independent review.

The median time from the final scope being sent to the MAH and the AWMSG meeting was 4.2 months (range 2.6–8.3 months; n = 156). Fifteen of the 171 appraisals were excluded from analysis because there was no scope date available.

The median time from MA to AWMSG recommendation was 9.7 months (range 2.7–61.2 months; n =140). A total of 31 of the 171 appraisals were excluded from analysis. One went through independent review and one went through re-appraisal. A further 29 were excluded because no MA date was available.

Comparison with the National Institute for Health and Care Excellence (NICE) and Scottish Medicines Consortium (SMC)

The outcomes from appraisals undertaken by AWMSG and NICE were closely aligned: the two organisations agreed on 87.0% (20/23) of appraisal recommendations. Of 151 appraisals where use of the product was supported by AWMSG (with or without restrictions), NICE did not schedule 120 (79.5%) for appraisal, subsequently recommended 17 (11.3%), did not recommend two (1.3%), suspended one (0.7%), discontinued three (2.0%) and, as of May 2017, were still in the process of appraising eight. Of the 18 products not supported by AWMSG, NICE did not schedule 13, subsequently recommended one, did not recommend three, did not suspend or discontinue any and were in the process of appraising one.

The outcomes from appraisals undertaken by AWMSG and SMC were closely aligned: they agreed on 87.8% (130/148) of appraisal recommendations. Of 169 appraisals undertaken by AWMSG, only 16 (9.5%) had not been scheduled for appraisal by SMC. Of 135 products supported by AWMSG, 119 (88.2%) were recommended by SMC (with or without restrictions) and 16 (11.8%) were not recommended (two due to non-submission). Of the 18 products not supported by AWMSG, five were recommended by SMC (with or without restrictions) and 13 were not recommended (three due to non-submission).

The median time from MA to NICE recommendation (27.5 months; mean 29.3 months; range 9.7–70.9 months) was significantly longer than when the same medicines were appraised by AWMSG (11.5 months; mean 13.9 months, range 4.7–47.9 months; n = 23; p < 0.0001). The median time from MA to NICE recommendation was also significantly longer than when the same medicines were appraised by AWMSG and subsequently ratified by Welsh Government (12.6 months; mean 15.7 months; range 5.4–49.2 months; n = 23; p = 0.0004).

Table 2 shows the time advantage gained for patients in Wales for those medicines that received a positive AWMSG recommendation (with or without restrictions) and that were subsequently superseded by a positive recommendation from NICE. The median time gained in Wales was 10.6 months (mean 17.7 months; range 3.5–48.3 months; n = 17; p = 0.0003).

Table 2

Time advantage gained in Wales for medicines that received a positive All Wales Medicines Strategy Group recommendation which were subsequently superseded by positive National Institute for Health and Care Excellence guidance

Generic name (trade name)	AWMSG appraisal		NICE appraisal		Time differencea (months)
	Outcome	Issue date	Outcome	Issue date
Abiraterone (Zytiga®)	Recommended	08/02/2012	Recommended	27/06/2012	4.31
Enzalutamide (Xtandi®)	Recommended	15/07/2015	Recommended	27/01/2016	4.87
Nalmefene (Selincro®)	Recommended	15/01/2014	Recommended	26/11/2014	8.97
Certolizumab pegol (Cimzia®)	Recommended	08/10/2014	Recommended	01/02/2016	15.09
Tocilizumab (RocActemra®)	Recommended	19/02/2014	Recommended	16/12/2015	19.33
Abatacept (Orencia®)	Recommended	15/01/2014	Recommended	16/12/2015	21.63
Etanercept (Enbrel®)	Recommended	20/11/2013	Recommended	16/12/2015	24.33
Adalimumab (Humira®)	Recommended	17/07/2013	Recommended	16/12/2015	28.07
Adalimumab (Humira®)	Recommended	12/06/2013	Recommended	01/02/2016	30.14
Adalimumab (Humira®)	Recommended	21/03/2012	Recommended	16/12/2015	43.72
Adalimumab (Humira®)	Recommended	22/04/2015	Recommended	16/12/2015	6.28
Epoetin theta (Eporatio®)b	Recommended with restrictions	13/10/2010	Recommended	26/11/2014	48.32
Ledipasvir/sofosbuvir (Harvoni®)	Recommended with restrictions	22/04/2015	Recommended with restrictions	25/11/2015	3.45
Daclatasvir (Daklinza®)	Recommended with restrictions	25/03/2015	Recommended with restrictions	25/11/2015	6.38
Infliximab (Inflectra®)	Recommended with restrictions	17/12/2014	Recommended with restrictions	26/01/2016	10.59
Infliximab (Remsima®)	Recommended with restrictions	17/12/2014	Recommended with restrictions	26/01/2016	10.59
Imatinib (Glivec®)	Recommended with restrictions	11/06/2014	Recommended with restrictions	27/11/2014	4.14

AWMSG All Wales Medicines Strategy Group, NICE National Institute for Health and Care Excellence

aThe time difference was determined by calculating the time difference between ministerial ratification by the Welsh Government of AWMSG recommendation and NICE publishing their advice. All time differences were positive

bAWMSG advice partially superseded by NICE

The median time from MA to SMC recommendation (7.7 months; mean 10.1 months; range 2.5–52.2 months) was significantly shorter than when the same medicines were appraised by AWMSG (10.4 months; mean 14.1 months; range 2.7–61.2 months; n = 127; p < 0.0001) or ratified by the Welsh Government (11.9 months; mean 15.6 months; range 3.9–62.2 months; n = 127; p < 0.0001).

Discussion

Between October 2010 and September 2015, AWMSG provided guidance on 171 indications for 137 medicines, 89% of which were given positive recommendations (either in full or with restrictions). For the majority of these medicines (80%) there would otherwise have been no national guidance in Wales as these were outside the scope of NICE. Four were for ultra-orphan medicines, and while the data are sparse and often involve surrogate outcomes, the AWMSG ultra-orphan medicines policy worked to facilitate patient access to medicines for some of the rarest diseases.

The time from MA to AWMSG recommendation was short compared with NICE (11.5 vs. 27.5 months), and since the Welsh health boards had a legal obligation to implement guidance within 3 (changed in 2017 to 2) months, patients in Wales had quick access to those medicines judged to be clinically and cost effective. Patients in Wales also gained a significant 10-month median advantage (range 3.5–48.3 months) over those in England for medicines which first received a positive AWMSG recommendation and which were subsequently superseded by a positive recommendation from NICE.

Between 2000 and 2017, NICE recommended (either in full, or optimised) 548 (81%) of the technologies it appraised [40]. The higher rate in our sample of AWMSG appraisals (89%) may be explained in part by the exclusion of medicines not imminently considered by NICE. The limited overlap between NICE topic selection and medicines/indications eligible for AWMSG appraisal means agreement rates are not directly comparable.

There was strong agreement in recommendations between AWMSG and NICE (87.0%) or SMC (87.8%) among the comparable samples analysed. It is possible that disagreements are a result of differences in the evidence submitted by the MAH to the organisations. Of the 151 appraisals where use of the medicine was supported by AWMSG, NICE subsequently did not recommend two for routine use within NHS England and Wales. AWMSG’s existing recommendation is normally superseded by NICE guidance and therefore, in these circumstances, the medicine would not be prescribed routinely within NHS Wales. However, patients in Wales who are currently being treated with a medicine, following a positive recommendation from AWMSG, should have the option to continue their therapy until they and their clinicians consider it appropriate to stop. The opportunity costs of patients having access to medicines that are then not deemed cost effective are considered by AWMSG when making their decisions.

The time it took from MA to recommendation for the same medicines to be appraised by AWMSG was significantly shorter than by NICE but longer than by SMC. AWMSG’s relatively short decision time is despite nearly a third of MAHs engaging with the AWMSG process more than 3 months after MA. However, there are confounding factors outside the control of the HTA agency that may affect timelines: for example, the MAH may submit an application to one HTA body and then to another at a later date; or the MAH may submit different evidence to each body. Furthermore, given the avoidance of duplication between AWMSG and NICE appraisal activities, medicines appraised by both organisations will naturally be those for which there is a delayed NICE appraisal, and the time advantage may therefore not be generalised across all appraisals. The timeline of the NICE HTA process, independent from AWMSG, has been considered previously. Single technology appraisals—a process appraising a single product with a single indication—takes NICE approximately 9–12 months to complete [41]. However, there is variability throughout the years; for example, in 2009, the median time to publication was 8 months (range 4–38 months), but in 2010 the median time was 29 months (range 4–30 months) [42]. Lengthier times for NICE guidance may partly be due to measures to provide transparency and the widespread consultation during the NICE process [42].

In this study we restricted the dataset to a 5-year period during which AWMSG processes and appraisal criteria were stable. AWMSG was established in 2002; however, data on pre-2010 and post-2015 appraisals were not included as there were several significant changes to the AWMSG processes and appraisal criteria during these periods. For example, changes were made to the way AWMSG appraises orphan and ultra-orphan medicines and medicines developed for rare diseases in 2016, giving patients and clinicians a stronger voice in AWMSG decision-making. We are unaware of significant analogous changes at NICE or SMC; however, it is possible that changes in their policies may have had an effect on our comparisons.

This is the first systematic description of AWMSG’s decisions and comparison with other UK HTA decisions. However, there are several equivalent studies of the processes and decisions made by NICE [43-45], SMC [46] and the National Centre for Pharmacoeconomics in Ireland [47], as well as other European countries [48]. An analysis of recommendations given by AWMSG reported that 59% of orphan medicines were given a positive recommendation (with or without restrictions) while 73% of ultra-orphan medicines were recommended [49]. A comparative analysis of orphan medicines reimbursement in the UK and other European countries between 2000 and 2016 reported that access was particularly slow in Wales [48]. However, this latter study included the time it takes the MAH to make a submission to AWTTC, and because of the alignment of AWMSG and NICE processes, the MAH often waits for a decision from NICE on whether it will appraise before engaging with the process in Wales. This time-lag due to the MAH submitting first to NICE makes the AWMSG process appear longer when measured from MA to recommendation or MA to ratification. Although the onus is on the MAHs to provide the information needed to appraise the medicine, AWTTC continues to liaise closely with them to encourage them to submit earlier.

Factors which influence the recommendations made by AWMSG, and its subcommittee NMG, have been considered previously [50]. An analysis of appraisal decisions made between 2007 and 2009 revealed a preference for medicines supported by a greater body of clinical evidence, and that consideration of combined uncertainty in economic model parameters had a positive influence on the recommendations of NMG and AWMSG [50]. Furthermore, ICERs for medicines with negative recommendations were, on average, higher than for medicines with positive recommendations; this is consistent with the pursuit of economic efficiency [50]. A stated preference study using a discrete choice experiment of committee members found that the incremental cost effectiveness of new medicines, and the QALY gains they provide, influences decisions to varying degrees depending on whether or not the uncertainty in cost effectiveness has been thoroughly explored, and whether or not the primary impact of the disease is on survival or quality of life [51]. The willingness to trade the cost effectiveness and QALY gains against these other factors indicates that economic efficiency and QALY maximisation are not the only considerations AWMSG takes into account when making recommendations [51].

The findings of the present review highlight the efficiency of the appraisal process in Wales through reduced duplication with NICE, and demonstrate the advantage to patients in earlier access to medicines. Whilst direct evidence of the impact of AWMSG on the health of the population in Wales is unattainable at present, uniform and expedited advice on non-NICE-appraised medicines (and treatments not scheduled to be appraised by NICE) to the NHS in Wales has resulted in more rapid and equitable access to new medicines. The low number of appraisals undertaken by both AWMSG and NICE (n = 23) over the 5-year study period demonstrates a low level of duplication of work from the two organisations (AWMSG has had an agreed Memorandum of Understanding with NICE since 2012, which was renewed in October 2016 for another 5 years).

Evolving challenges have included the growing number of orphan drugs. In response to its appraisal of ivacaftor (Kalydeco®), a treatment for cystic fibrosis, AWMSG changed its orphan drugs policy to enable even greater involvement of patients and clinicians in the decision-making process [27, 52]. Meanwhile, NICE has recently introduced new guidelines for appraising highly specialised technologies [30], the outcome of which are adopted by AWMSG, and a fast-track process for technologies that are considered to offer exceptional value for money [53]. NICE have also introduced a budget impact test to assess the financial impact of a technology over the first 3 years of its use in the NHS [54]. In order to increase access to high-cost treatments, AWMSG has adopted its own Wales Patient Access Schemes, but whereas pricing has been (and will continue to be) set at a UK level, HTA is a matter for the devolved governments, and the relationship between the three HTA organisations and the ‘pricing unit’ has yet to be clarified [55]. A further challenge to AWMSG is the increased need for the opportunity cost to be estimated. It is important to identify what is foregone when new cost-increasing technologies are introduced to understand the effects of HTA decisions on the healthcare system [56, 57].

Conclusion

This article highlights the HTA work carried out by AWMSG over a 5-year period. We demonstrate the efficiency of the appraisal process in Wales and compare it with the analogous HTA processes in England and Scotland.

From October 2010 to September 2015, the All Wales Medicines Strategy Group (AWMSG) conducted 171 appraisals of 137 medicines; a positive recommendation, with or without restrictions, was given in 88.9% of appraisals.

The decisions made by AWMSG and two other UK health technology assessment (HTA) organisations for the same medicines/indications were closely aligned.

Patients in Wales gained a 10-month median advantage (range 3.5–48.3 months) over those in England for medicines that received a positive AWMSG recommendation and which were subsequently superseded by a positive recommendation by the National Institute for Health and Care Excellence (NICE).