Similarities and Differences in Health Technology Assessment Systems and Implications for Coverage Decisions: Evidence from 32 Countries.

Health technology assessment (HTA) systems across countries vary in the way they are set up, according to their role and based on how funding decisions are reached. Our objective was to study the characteristics of these systems and their likely impact on the funding of technologies undergoing HTA. Based on a literature review, we created a conceptual framework that captures key operating features of HTA systems. We used this framework to map current HTA activities across 32 countries in the European Union, the UK, Canada and Australia. Evidence was collected through a systematic search of competent authority websites and grey literature sources. Primary data collection through expert consultation validated our findings and further complemented the analysis. Sixty-three HTA bodies were identified. Most have a national scope (76%), are independent (73%), have an advisory role (52%), evaluate pharmaceuticals predominantly or exclusively (76%), assess health technologies based on their clinical and cost-effectiveness (73%) and involve various stakeholders as members of the HTA committee (94%) and/or through external consultation (76%). The majority of HTA outcomes are not legally binding (81%). Although all study countries implement HTA, the way it fits into decision-making, negotiation processes, and coverage and funding decisions differs significantly across countries. HTA is a dynamic and transformative process and there is a need for transparency to investigate whether evidence-based information influences coverage decisions.

Key Points for Decision Makers

Background

Health technology assessment (HTA) is “the systematic evaluation of properties, effects and/or impacts of health technology” [1]. It aims to improve both quality and value for money [2, 3] and facilitate coverage decisions based on evidence-based information and other socioeconomic factors beyond the clinical and cost-effectiveness of a technology. HTA comprises multiple operational features and practices, the application of which may differ substantially by setting, resulting in different use and application [4, 5]. The context and structure of HTA systems reflect health system priorities and underpin a country’s history, culture, values and preferences. Therefore, HTA is a concept with many facets and may differ in its focus and method, its governance and role, scope and remit, the assessment method employed and its impact on coverage decisions [6-12]. Taking into consideration these variations, it is important to study the different HTA parameters that can influence the way HTA systems are set up, operate and are integrated within national policies. These variations make HTA processes unique, resulting in different levels of use, implementation and impact on the decision-making process and final coverage decision [13]. Whilst some countries directly translate HTA recommendations into coverage decisions, this may not be the case in others where HTA only provides an assessment to be used by healthcare systems when deciding whether health technologies should be included in the reimbursement list or not.

Our main objectives are threefold: first, to understand the multiplicity of approaches employed by different HTA bodies across a wide range of countries; second, to study the role of HTA within the healthcare system and the extent to which it is integrated into or is independent of the healthcare system and what this implies for HTA recommendations; and, third, to identify the link between HTA recommendations and funding decisions.

In this paper we develop and extend a conceptual framework capturing the main operational pillars of HTA. We consider HTA within the broader healthcare system with a view to understanding the links between assessment of new technologies, their appraisal and the implications for coverage and funding. Using this framework, we map HTA activities and analyse HTA systems from an international and comparative perspective, drawing on the operational features of HTA systems from a wide range of countries. To this end, the paper provides a holistic approach to the process of value assessment and its implications for coverage, analyses how different applications of HTA can result in practice variations across settings and discusses how these variations impact HTA recommendations and, possibly, coverage decisions.

Conceptual Framework

Earlier research [6, 14–16] has focused predominantly on studying HTA outcomes of the same technologies among different HTA bodies, while examining the clinical and/or economic evidence submitted. Whilst the submitted evidence can differ due to national and/or regional evidentiary requirements and preferences, there are other important parameters that might shape or influence the way HTA functions [6, 14–16]. In order to understand better why reimbursement decisions differ amongst jurisdictions using HTA and to systematically showcase similarities and differences among HTA systems, we propose a conceptual framework that captures the salient features of HTA systems (Fig. 1). We reviewed and analysed existing frameworks focusing on how HTA is organised and how it operates within healthcare systems [17-26]. To identify relevant literature, we conducted a search through Medline and Scopus and a targeted search on the websites of the European network for Health Technology Assessment (EUnetHTA) and the European Commission. We searched Medline and Scopus using the following keywords: ‘health technology assessment’, ‘value assessment’, ‘comparative assessment’ and ‘framework’. We limited the search to English language and set up the study period for inclusion from 2005 to 2017. The start date of the search timeframe was selected based on the period when independent HTA bodies with refined responsibilities started to be established [17-19]. We screened studies through titles and abstracts and selected studies for inclusion only when the authors had designed a conceptual or analytical framework looking at HTA systems, their operation within countries and their potential role in reimbursement. We excluded studies using existing frameworks created by other authors and studies focusing only on the HTA process and the evaluation of the clinical and economic evidence, as they were outside the scope for our study. For the targeted search, we navigated the websites of EUnetHTA and the European Commission to identify recent studies focusing on European HTA systems, as these studies often draw comparisons across the systems of EU member states. We navigated the websites by using the search tool of both websites and the same keywords used in the literature search.

Fig. 1

Conceptual framework outlining type, scope and nature of HTA activities.

Source: The authors. 1 ‘Other technologies’ refers to public health interventions such as screening programmes, vaccination campaigns, evaluation of surgical and non-surgical interventional procedures, stem cell therapies, innovative cancer vaccines, cell and gene therapies, and other forms of personalised treatments

Based on our findings, existing frameworks [17-26] (a) focus on the HTA process itself in combination with few structural features such as the role of HTA, topic selection, or stakeholder involvement; (b) examine HTA as a reimbursement policy that determines technologies’ availability within markets, specifically focusing on manufacturers’ perspectives; (c) explore decision-making criteria at HTA level; or (d) analyse key components such as level of transparency and scientific rigour, which could influence HTA recommendations. Our framework, in contrast to existing ones, provides a holistic overview capturing the main operational components of HTA together with salient features of HTA systems and their interactions, to help us understand how HTA processes differ across settings and why, how HTA systems function, and whether HTA recommendations are likely to be directly linked to coverage decisions or not.

Governance of Health Technology Assessment (HTA)

HTA bodies may either be independent review bodies operating at arm’s length of governmental structures or may be bodies integrated within governmental structures with decision-making and priority-setting responsibilities [20, 21]. The key differences that distinguish arm’s length from integrated systems lie in (i) the degree of independence in the way HTA bodies operate within the healthcare system and (ii) the transparency of the process. Independent bodies are considered to be more transparent than integrated ones as the former tend to take a broader and more society-focused perspective into consideration [20, 21].

Type of Organisation Performing HTA

Different types of institutions can perform HTA or HTA activities [22]: (i) research institutions include academic bodies with broader research initiatives that could encompass some HTA activities; (ii) HTA-research institutions which have a special department dedicated to HTA activities; (iii) national insurance organisations; (iv) national/regional health organisations, which could be under the supervision of the Ministry of Health focusing on public health and pharmaceutical policy either at national or regional level but usually functioning at arm’s length of the government; (v) national/regional HTA bodies, which perform HTA as their main activity; (vi) governmental organisations that are integrated within the Ministry of Health, and (vii) drug regulators which authorise medicines and/or medical devices with a clear separate HTA function.

Role of HTA

HTA bodies can have an advisory, a regulatory or a coordinating role in the decision-making process, depending on the intent and type of assessment required, the general mission and overall objectives of the review body [20, 23]. Advisory HTA bodies produce coverage recommendations for decision-makers [23], but the latter are not obliged to follow this advice or take it into consideration when negotiating with manufacturers. By contrast, regulatory bodies are directly accountable to the Ministry of Health (MoH) and are responsible for the pricing and reimbursement of new technologies [23]. Hence, regulatory systems have an impact on pricing and coverage decisions compared to advisory systems. Coordination bodies usually conduct independent research on HTA and might be responsible for coordinating HTA activities at national, regional and provider level [24]. HTA recommendations from coordination bodies are rarely considered or accounted for in coverage decisions. This is due to the nature of these bodies and the way they carry the assessment, which in most cases simply evaluates clinical and economic evidence without contextualising the healthcare system’s needs. However, healthcare systems can appoint a coordination body as an advisor and further produce recommendations that are used in decision-making.

Scope and Geographical Coverage of HTA

The structure of a healthcare system and the balance between local autonomy and centralised control influence how HTA systems are organised [21, 25]. Healthcare systems that make pricing and reimbursement decisions centrally, tend to conduct HTA centrally. Healthcare systems with decentralised structures and resource allocation at a regional level can justify HTA activities performed regionally. However, given the unique nature of healthcare systems, there are cases where HTA activities are taking place at both levels.

Remit of HTA

In principle, all types of medical technologies can undergo HTA, including pharmaceuticals, medical devices and ‘other technologies’ [24, 26–28]. The precise remit of HTA showcases which technologies are subject to assessment for listing.

Model of HTA

There are three distinct HTA models which reflect the objectives and priorities of healthcare systems [20]. First, the clinical and cost-effectiveness model uses both economic evidence and comparative clinical benefit to assess health technologies. Second, the comparative clinical benefit assessment model relies on ranking new interventions based on comparative benefit assessment. Under this model, the pricing decision is subject to negotiations between purchasers and manufacturers. Third, the value-based assessment model is directly related to the aforementioned models and further takes into consideration explicitly additional dimensions of value beyond effects and/or costs that are considered important, such as disease severity, burden of disease, treatment innovativeness and equity considerations. It is possible that HTA bodies may adopt more than one HTA model based on certain criteria. For instance, in France, Haute Autorité de Santé (HAS) assesses technologies based on the comparative clinical benefit model. However, since 2013, submission of a cost-effectiveness analysis is mandatory for technologies with a moderate to major improvement in clinical benefit (ASMR I–III) [29].

Assessment Versus Appraisal

Assessment and appraisal are the two different facets of HTA [20, 24, 30]. Assessment refers to a process of collecting, reviewing and synthesising clinical and economic evidence to support funding decisions [8, 16]. Appraisal uses the same clinical and economic evidence but interprets it in the context of the healthcare system in question and takes into account factors that may be of relevance in that context [8, 16]. These contextual factors are known as social value judgements and can be both explicitly recognised, such as the end-of-life criteria in England and severity in France, or implicit, for example, the possible burden on patients’ activities of daily living or the impact on family and carers [31]. The contextualisation of the evidence thus leads to recommendations that reflect the national/regional needs and values.

Stakeholder Involvement in HTA

Consultation of various stakeholders, including healthcare professionals, patients/patient organisations, citizens, health insurers, ethicists and the industry has become an essential part of HTA procedures, contributing to increased transparency, reduced appeals and inclusiveness. To ensure that HTA recommendations are considering preferences, values, judgments, opinions and individual insights, stakeholder participation can occur via two main routes: (i) stakeholders participate as members of an HTA committee, and (ii) stakeholders are engaged through public calls (external engagement). The way stakeholders are engaged and involved in the process varies across HTA systems and the type of stakeholders involved can reflect the inclusiveness of the HTA process and its ability to incorporate values and preferences that matter to different segments of society.

HTA Recommendations and Funding Decisions

HTA recommendations can either be binding or non-binding to the final funding decision [22]. In the non-binding case, a negative recommendation is not necessarily associated with a negative coverage decision. In the binding case, purchasers/commissioners of care are legally obliged to consider the HTA outcome when deciding on coverage.

Methods

Scope and Data Sources

The scope of our analysis captured the 27 European Union Member States (EU MS), the UK (England, Scotland, Wales and Northern Ireland), Australia and Canada and further includes the European network for Health Technology Assessment (EUnetHTA) to account for joint assessments conducted by more than one EU MS. We focused on Europe, the UK, Canada and Australia as they have, for a large majority, well established HTA systems that are used to inform national and regional funding decisions. The refinement of processes that often accompany well established HTA systems allowed us to easily categorise our findings using the conceptual framework and to make comparisons across countries. Therefore, we included countries with explicit HTA systems defined as systems performing HTA routinely and whose existence is enshrined into legislation. We included multiple HTA bodies operating at both national and regional levels from the same country if they existed. We categorised the types of bodies/institutions performing HTA based on their nature and structure, where these institutions lie within the healthcare system and the way they are funded (e.g. research and HTA-research institutions are independent of the government and do not necessarily receive their entire funding from the government). We excluded from our sample informal HTA processes, HTA-like activities (e.g. consideration of pharmacoeconomic studies on an ad-hoc basis only), and mini-HTA activities (mainly assessing ambulatory care medicines or hospital technologies and, therefore, being small scale and not explicit) [17].

Secondary Data Collection

We used both primary and secondary sources to collect relevant data. We collected evidence on the differences and similarities of HTA processes across the study countries through a search of the websites of all relevant competent authorities (including the MoH, national health insurance organisations and the HTA bodies), EUnetHTA, the International Network of Agencies for Health Technology Assessment (INAHTA) and the ISPOR Global Health Care Systems Road Map. When we needed further clarification or additional information, we identified evidence through a literature search of Medline and Scopus, using the search terms ‘Health technology assessment (HTA)’ and the name of the country. We limited the literature search to English language. We conducted the secondary data collection from December 2016 to July 2018, and we updated the information, when applicable, in March 2020. We created a list of all identified institutions undertaking HTA along with their websites.

Primary Data Collection Through Expert Consultation

We contacted 29 stakeholders via email in June 2019 to validate and complement findings from secondary sources and provide further clarification on the nature of HTA activities and operational features in their respective countries. We purposively sampled these stakeholders to ensure the inclusion of leading European health, HTA and pharmaceutical policy experts affiliated with universities and national competent authorities, such as regulatory agencies, institutions responsible for pricing or reimbursement decisions and HTA agencies. In particular, we endeavoured to include experts from countries with less well-established HTA systems, since there was little available information from secondary sources in English. We asked the experts to (a) comment on the design of the conceptual framework; (b) confirm whether we had classified appropriately the key features of HTA systems according to findings from secondary sources; (c) provide additional information about any formal or informal HTA activity taking place in their countries that we had not captured in our search (if applicable); and (d) provide details about how multiple organisations that undertake HTA activities within countries collaborate in relation to assessments and final decision-making (if the respondent was based in a country with more than one HTA agency).

Results

Final Sample of HTA Activities

We identified 63 HTA bodies/institutions undertaking HTA activities across 32 settings. We excluded Northern Ireland as HTA activities are very limited and rely on reviews of the English National Institute for Health and Care Excellence (NICE) decisions [32]. We included EUnetHTA as a supranational organisation that has been created to coordinate joint HTA activities at the EU level [27]. We further acknowledged the proposed regulation of the EU commission regarding HTA cooperation [33] but we did not consider it in this study as it was still under deliberation and consultation at that time.

Out of the 29 contacted experts, 18 experts from 14 countries (Estonia, Ireland, Poland (n = 3), Portugal, Romania, Slovakia, Slovenia, Spain, Sweden (n = 2), Malta, Bulgaria, Austria (n = 2), Belgium and Czech Republic) responded to our call and participated in the consultation round. The results of the primary and secondary data collection are presented according to the attributes of the conceptual framework. Table 1 summarises the key findings on the bodies or institutions undertaking HTA activities across the study countries (see Appendix Table 1 in the electronic supplementary material [ESM] for complete results by country).

Table 1

Summary of HTA systems across EU Member States, the United Kingdom, Canada and Australia

Variable	Summary of evidence	Country examples
Governance of HTA	Arm’s length: 46	Austria, Croatia, Finland, the Netherlands, Canada, England, France, Germany, Australia, Poland
	Integrated: 16	Cyprus, Greece, Malta, Slovakia, Slovenia, Italy, Spain (regional), Canada (regional)
	N/Aa: 1	EU level
Type of organisation performing HTA	Research institution: 6	Austria, Belgium, Denmark, Estonia, Slovakia, England
	HTA research institution: 6	Austria, Finland, Ireland, Lithuania, Spain (national and regional)
	Drug regulator: 6	Czech Republic, Finland, Hungary, Italy, Portugal, Romania
	Governmental institution: 8	Cyprus, Greece, Lithuania, Luxembourg, Malta, Slovakia, Slovenia, Spain
	HTA body: 18	France, Germany, Poland, Scotland, England, Wales, EU level, Canada, Australia
	National/regional healthcare organisation: 14	Bulgaria, Croatia, Finland, Germany, Italy, Latvia, Lithuania, the Netherlands, Spain, Sweden
	National insurance organisation: 5	Austria, Belgium, Croatia, Estonia, Slovenia
Role of HTA	Advisory: 33	Luxembourg, the Netherlands, Portugal, England, Canada, Australia
	Coordination: 2	Finland, UK
	Regulatory: 17	Bulgaria, Croatia, Cyprus, Czech Republic, Italy, Estonia, Germany, Sweden
	Advisory and coordination: 10	Austria, Belgium, Denmark, Ireland, Spain (national and regional), Canada
	N/Aa: 1	EU level
HTA scope	National: 48	Australia, Germany, France, Sweden, Slovakia, Austria, Lithuania, Malta, Luxembourg, the Netherlands
	Regional: 14	Spain (AQuAS–Catalonia, OSTEBA–Basque County, AETSA–Andalusia, SECS–Canary Islands, UETS–Madrid, Avalia-t–Galicia, IACS–Aragon), Italy (AGENAS, CRU–Veneto, ER Salute–Emilia Romagna), Canada (INESSS–Quebec, HQO–Ontario, CED–Ontario, British Columbia)
	N/Aa: 1	EU level
Remit of HTAb	Pharmaceuticals: 48	Austria, Belgium, Bulgaria, Croatia, Cyprus, Finland, France, Germany, Greece, Malta, Scotland, Australia, Canada
	Medical devices: 43	Denmark, Hungary, Ireland, Latvia, Lithuania, Portugal, Germany, Spain, Sweden
	Other technologies: 35	Canada, EU level, England, Wales, Sweden, the Netherlands, Lithuania, Belgium, Croatia
	All: 20	Belgium, Estonia, France, Finland, the Netherlands, Poland, England, EU level
Model of HTA	Comparative clinical benefit assessment: 7	Austria (GÖG and AIHTA), Germany (GBA and IQWiG), Greecec, Slovenia (Health Council), EU level
	Clinical and cost-effectiveness: 46	Belgium, Croatia (both agencies) Cyprus, Spain (national and regional), Malta, Lithuania, Ireland, Finland, Hungary, Denmark, Wales
	Clinical and cost-effectiveness/MCDA: 2	Bulgaria, Canada
	Value-based assessment: 8	France, Slovakia (both HTA bodies), Slovenia (ZZZS), Sweden, England, Scotland, Australia
Assessment versus appraisal	Assessment only: 28	Austria, Belgium, Croatia, Cyprus, Estonia, Denmark, Greece, Ireland, Italy (regional), Spain (regional), EU level, Canada (regional)
	Assessment and appraisal: 35	Austria, Belgium, France, Finland, Croatia, Canada, Australia, Romania, Luxembourg, Portugal, Poland, Slovenia, Sweden, Spain, England, Scotland, Wales
Stakeholder involvement in HTA	Stakeholder participation as members of HTA committee: 59	Croatia, Cyprus, Finland, Greece, the Netherlands, Portugal, Poland, Spain, Sweden, Canada, Australia
	Stakeholders through public calls: 48	Austria, Belgium, Bulgaria, Croatia, Czech Republic, Denmark, Estonia, Finland, France, Ireland, Latvia, Lithuania, Malta, Portugal, Slovakia, Slovenia, Spain, UK, EU level
HTA recommendations and funding decisions	Binding: 12	Bulgaria, Cyprus, Finland, Germany, Italy, Lithuania, Portugal, Sweden
	Non-binding: 51	Austria, Belgium, Croatia, Denmark, Estonia, Finland, France, Greece, Hungary, Ireland, Italy (regional), Poland, the Netherlands, Spain, Wales, Canada, EU level
Publicly available reports	Yes: 48	Australia, Canada, England, Scotland, Wales, Sweden, Spain, the Netherlands, Germany, Belgium
	No: 15	Croatia, Estonia, Greece, Hungary, Luxembourg, Slovakia, Slovenia

Source: The authors, based on primary and secondary data collection

EU European Union, EUnetHTA European network for Health Technology Assessment, HTA health technology assessment, MCDA multiple-criteria decision analysis, N/A not applicable

aEUnetHTA has been categorised as a supranational organisation that has been created and now coordinates the HTA Core Model, which is a methodological framework for collaborative production and sharing of HTA information. Therefore, EUnetHTA does not fall into the classification in which we have placed national HTA bodies

bUnder the remit of HTA, different organisations in each country may perform HTA for different technologies. Examples of agencies that perform HTA for pharmaceuticals only are SUKL in the Czech Republic and DPA in Malta. Examples of agencies that perform HTA for medical devices only are AGENAS in Italy and UETS in Spain. Examples of agencies that perform HTA for other technologies are SBU in Sweden and the Institute of Hygiene in Lithuania

cIn Greece, cost-effectiveness is envisaged in legislation, but is currently not mandatory and no cost-effectiveness threshold has been published

Governance of HTA

The majority (73%, n = 46) of the identified institutions are at arm’s length of government, including regulatory bodies, which are by definition independent of government. Similarly, we categorised as independent organisations the national insurance organisations, which all perform in-house HTA and use cost-effectiveness as one of the criteria for coverage decisions. This is because national insurance organisations are independent entities and do not function as governmental organisations, even though they are a part of national healthcare systems. Institutions or organisations that do not operate within the national or regional MoH are categorised as independent bodies. Integrated agencies to a governmental structure were predominately regional bodies or newly established HTA committees that are responsible for performing HTA within the MoH (e.g. Greece, Cyprus and Malta).

Twenty-eight percent (n = 18) of the identified entities performing HTA had formal HTA agency status, that is, HTA is their predominant activity. Twenty-two percent (n = 14) were national or regional healthcare organisations and 12% (n = 8) were governmental institutions (refer to Table 1 for all the different types of organisations performing HTA). There was variation across the sample as to which types of organisations perform HTA, showcasing that various types of organisations can undertake HTA activities predominantly depending on the structure of the healthcare system and the scope and objectives of HTA.

According to our findings on the governance endpoint and the role of HTA bodies, we created a taxonomy (Fig. 2) to differentiate the included bodies based on their level of integration within the government, as well as their function as advisory, coordination or regulatory entities.

Fig. 2

Taxonomy of HTA bodies.

Source: The authors, based on primary and secondary data collection. *Regulatory body for approval of medicines and/or medical devices with a clear separate HTA function

In 13 countries (Austria, Belgium, Croatia, Estonia, Finland, Germany, Ireland, Lithuania, Slovakia, Slovenia, Sweden, England and Wales), we identified more than one institution with varying roles undertaking HTA activities at a national level. Stakeholders who participated in the expert consultation from Austria, Belgium, Estonia, Slovakia, Slovenia, Sweden and Ireland provided additional information on the responsibilities of multiple national HTA bodies (see Appendix 2 in the ESM for a detailed description on how multiple national HTA bodies are set up in a country’s system, how these bodies interact with each other and their impact on funding decisions). HTA bodies with a regional and provider-level scope were identified in eight countries (Austria, Canada, Germany, Italy, Spain, Denmark, Poland and Sweden). Due to limited access and data, we were able to include regional HTA bodies from Spain, Italy and Canada. In Spain and Italy, organisations performing HTA at a regional level are mainly integrated within the regional government. In Sweden, there are about 15 regional HTA bodies that assess the clinical and cost-effectiveness of procedures and medical devices; they have an advisory role to the county councils and help to inform reimbursement decisions at state level. However, their recommendations are not binding [34]. Due to limited evidence, they were not included in our sample. In Austria, universities such as the University for Health Sciences Medical Informatics and Technology (UMIT), the IAMEV unit in the Medical University of Graz and the Danube University Krems (DUK) perform HTA activities independently by assessing various health technologies. DUK and the IAMEV unit conduct clinical assessments whereas UMIT follows the clinical and cost-effectiveness model [34]. In Poland, hospital-based HTA is evolving and is performed by some university hospitals and institutes to support investment decisions in hospitals. However, their scope and impact on funding decisions are still unknown [34].

In the sample, there is wide variation in the technologies that undergo HTA. From our sample, we identified bodies that assess a specific type of pharmaceuticals only. For instance, the Finnish Medicines Agency (FIMEA) in Finland performs HTA only for in-patient pharmaceuticals. SUKL in the Czech Republic performs HTA only for out-patient pharmaceuticals, while TLV in Sweden assesses mainly out-patient pharmaceuticals, whereas in-patient pharmaceuticals are assessed at county level [34] (see Appendix 3 in the ESM for a detailed list of which technologies undergo HTA assessment by the identified HTA bodies).

All national insurance organisations in Austria, Belgium, Croatia, Estonia, and Slovenia use the clinical and cost-effectiveness model as an additional criterion during the decision-making process on what technologies to include in their reimbursement lists. In Sweden, value-based assessments by TLV always take into consideration explicitly the human dignity and solidarity principles to derive funding decisions [34]. In Slovakia, since the new legislation was implemented in 2011, decisions on resource allocation are based on criteria beyond the clinical effectiveness, safety and the economic benefit of the technologies, notably, disease severity, impact on society and risk of abuse [34]. In France, the award of total therapeutic benefit (SMR) and improvement in therapeutic benefit (ASMR) rests on criteria beyond efficacy; for example, ASMR I is awarded to significant innovations in terms of efficacy improvement in a severe disease setting, in other words, severity is taken explicitly into account; similar criteria inform the SMR rating. Therefore, additional dimensions of value are taken into account during the assessment and appraisal process.

Fifty-six percent (n = 35) of HTA bodies perform appraisals and are not solely collecting and synthesising evidence on the clinical and/or economic effectiveness of technologies. HTA bodies conducting appraisals are mainly national institutions. Approximately 44% (n = 28) of the HTA bodies limit their evaluations in the assessment phase. Regional HTA bodies in Spain, Italy and Canada, research institutions and integrated committees within the MoH responsible for HTA in Malta, Greece, Bulgaria and Cyprus are all performing assessments rather than appraisals.

Stakeholder Involvement in the HTA Process

Involvement of various stakeholders as members of HTA committees was present across almost all the HTA bodies except bodies with a coordination role in Denmark, Finland and England, where assessments are performed by external institutions. The type of stakeholders involved in decision-making varied considerably across countries from representatives of healthcare insurance funds and public health organisations, healthcare experts, ethicists, health economists, healthcare professionals as well as patient and citizen advocates. External expert consultation was not present in 15 HTA bodies, which, in their majority, were regulatory bodies. External consultation was heavily dependent on patients who were able to submit their opinion on the topic selection, the evaluation process or the final recommendations.

Positive HTA recommendations are not always translated into funding decisions regardless of how well HTA systems are developed, their role and what their position is with in the healthcare system. In 81% (n = 51) of our sample, HTA outcomes are non-binding and their impact during reimbursement negotiations is unclear. Nevertheless, HTA outcomes, even if non-binding, weigh heavily on final reimbursement decisions in some countries such as France, England, Scotland, Australia, Poland and Romania [34]. In Poland, the HTA body (AOTMiT) plays a key role in the reimbursement process. Any health technology that is subject to coverage by the public healthcare system has to be assessed by AOTMiT. Both the president of the agency and the Transparency Council (TC), serving as an advisory body to the president, provide a formal position [34]. Non-binding recommendations are submitted to the MoH, where negotiations are taking place between the MoH Economic Commission and the manufacturer. The MoH makes the final decision, taking into consideration the opinions of both the TC and the president as this is one of the reimbursement criteria established by law [34]. In Romania, legislation stipulates that the HTA body (ANMDM) makes a recommendation to the MoH based on a scorecard and a budget impact analysis. Scorecard points are given, taking into account HTA recommendations, in England, Scotland, France and Germany. Additional points are further attributed when the product under evaluation has been granted reimbursement status in EU countries. According to primary evidence, the MoH will always include in the reimbursement list products with a positive recommendation by ANMDM [34]. Moreover, if ANMDM makes a conditional reimbursement recommendation, then the manufacturer must submit a request to the National Health Insurance House to attend price-volume negotiations. The request is analysed by a negotiation commission that decides if contract negotiations will be initiated [34]. In England and Scotland, NICE and SMC have an advisory role and the local NHS must fund all positive HTA recommendations. Technologies receiving a negative recommendation may be subject to negotiations in order to improve their cost-effectiveness and if there is agreement then the NHS will fund the technology; alternatively, if clinical benefit is highly uncertain or is considered inadequate, alternative mechanisms or pathways exist [35, 36]. In Australia, the government or the cabinet should consider all HTA recommendations by PBAC if the medicine is expected to cost more than AU$20 million per year [37].

Discussion

The results in this study point to a number of key features of HTA processes and their relationship to coverage decisions. First, HTA is not a single mechanism but consists of several salient features that can differ substantially across countries and further determine the way it is implemented to inform coverage decisions. HTA operates mainly at national level except in countries with decentralised systems (e.g. Italy, Spain, UK, Sweden) or autonomous regions/provinces (e.g. Canada). As such, HTA infrastructure and activities reflect the structure of the healthcare systems in which they operate. The scope of HTA systems can directly mirror the administrative division of a country (highly regional systems vs centralised ones). In countries where HTA activities are performed both at national and regional level, assessments of clinical benefit for the same technology could amount to duplication of effort. Similarly, when considering comparable HTA structures across Europe, assessment of clinical evidence performed by HTA agencies duplicates effort as manufacturers tend to submit very similar, if not the same, evidence. Therefore, EU HTA cooperation may be able to streamline HTA activities and homogenise methodologies and procedures in assessing health technologies within the European Union [22].

Second, institutions performing HTA at national level are mostly independent from the competent authorities they serve (e.g. Ministries of Health, health insurance organisations, pricing committees), even though their activities may be sometimes supervised by these authorities. Considering that HTA activities can, in general, be grouped into (a) assessment, (b) appraisal, (c) coverage recommendations, and (d) funding negotiations, the remit of independent bodies covers (a), (b) and (c), while integrated bodies cover (a), (b) and (d). Given that HTA recommendations by integrated bodies can result in negotiations, they can play a key role in funding decisions. However, this depends on the role of the HTA and whether recommendations are binding or not. Overall, HTA bodies operating at arm’s length are present in more developed HTA systems and tend to be transparent and independent, avoid conflicts of interest and offer dispassionate advice on the costs and/or effects of assessed technologies adapted for contextual considerations. By contrast, newly founded and less well-developed HTA systems such as those found in Greece, Cyprus and Malta tend to be integrated within existing competent authorities. They may lack transparency as assessments are internalised, and recommendations are not reported in publicly available documents, rendering decision-making and negotiation processes unclear and non-transparent. Nevertheless, it could be argued that integrated HTA functions can be very useful as a starting point in the implementation of HTA activities, particularly in circumstances where there is a lack of capacity for the development of an independent HTA body. Unsurprisingly, most independent bodies make their HTA reports and outcomes publicly available (see Appendix in the ESM), whereas integrated bodies tend to keep their reports confidential. Overall, more transparency improves the extent to which a decision can be controlled by the organisation that has commissioned it in the first place [38].

Third, irrespective of operating at arm’s length or being integrated with competent authorities, the majority of HTA bodies have an advisory role where HTA outcomes act only as recommendations and can be used as supplementary tools or additional criteria during negotiations. HTA bodies with a coordination role may operate in an advisory capacity to competent authorities when asked to assess technologies, however, the extent of their contribution to the final coverage decision is unclear and the consideration of their recommendation is mostly made in a non-systematic manner. Coordination bodies are few and, in our opinion, entities with this role are urgently needed to assist with the coordination and interoperability of HTA activities as well as to generate evidence on how new technologies impact society, including cost–benefit analyses. Coordination bodies can also assist in the transformation of HTA recommendations into clinical guidelines contributing to optimal resource allocation by positioning new technologies along treatment pathways, monitoring their use and assessing the impact they have.

Fourth, the type of HTA evaluation plays an essential role in the way HTA outcomes are translated into funding decisions. During the assessment phase, comparative clinical and/or economic evidence is reviewed, whereas during the appraisal phase, the evidence is assessed and interpreted based on its scientific rigour, the achievement of the endpoints of interest, the design of the included studies, the economic effectiveness, the budget and/or economic model submitted and a other contextual considerations that may be relevant to the setting in question. Value dimensions are examined to investigate the extent to which a new technology is relevant for the healthcare system of interest [39]. Under appraisal these dimensions are always taken into consideration regardless of their nature and the disease context. Therefore, when appraisals are performed, recommendations are context-specific as they take into consideration how a technology can be adopted at national or regional level and what budget impact it will have. Appraisals can inform purchaser–manufacturer negotiations by providing steer, among others, on whether any risk mitigation strategies should be implemented.

Fifth, the vast majority of HTA bodies or HTA-performing institutions have some form of stakeholder involvement or engagement. Whether stakeholders were involved in the HTA process itself or whether they were part of the HTA committee varied, with more developed and well established systems giving patients, carers, citizens and health experts the possibility to act as external stakeholders through public calls. However, engagement of external stakeholders does not always serve the same purpose and ranges across systems from opinions and insights on topic selection and scoping to consultations during HTA assessment or appeals on the final recommendations. In many systems where patient representatives or patient organisations are not involved in HTA, members who are ethicists are bringing the societal perspective into decision-making. However, even though, in theory, stakeholder participation or engagement can result in better uptake of HTA recommendations, there is no evidence establishing a direct link. Undoubtedly, participation of healthcare experts and professionals, experts on ethics, patients, their families and carers in either the HTA process itself or decision-making can ensure transparency, inclusiveness and the reflection of different perspectives in the final recommendations.

Sixth, to assess whether HTA recommendations are translated into funding decisions, we looked at whether HTA outcomes across our sample countries are legally binding. This means that decision-makers are legally bound to respect and follow the final HTA recommendations when making coverage decisions. Based on our findings, the majority of HTA systems issue non-binding recommendations; however, the importance and the weight these recommendations might have on the decision-making process vary across countries. In less-developed HTA systems, such as that of Greece, the role and the impact of HTA recommendations, which are non-binding, is still unknown due to lack of transparency in the decision-making process. Nevertheless, from our sample we identified countries such as Poland, France, England, Scotland, Australia and Romania where recommendations are non-binding but their role during pricing and reimbursement processes at national level is considered crucial.

In order to be able to capture all types of recommendations, we created three alternative scenarios on how HTA outcomes feed into final funding decisions and what their contribution to the final coverage decision of technologies is. Fig. 3 shows three categories of recommendations: (i) binding; (ii) non-binding but as impactful as binding and (iii) non-binding. We created the second category to be able to include the HTA systems of England, Scotland, Australia, France, Poland and Romania, which have an advisory role according to legal statutes, however (positive) recommendations are considered binding for coverage purposes. For instance, the NHS in England is legally obliged to fund technologies recommended by NICE and ensure they are available within 3 months from the date of the NICE recommendation being published [34].

Fig. 3

Positive HTA recommendations and links to final funding decisions.

Source: The authors. The dotted lines show the use of alternative pathways

HTA recommendations that do not fall into the first two categories might jeopardise transparency by creating uncertainty on how evidence-based information is used meaningfully during negotiations or is translated into either price discounts or any other type of Managed Entry Agreements. Importantly, however, non-binding HTAs might inform decision processes further upstream. For instance, in the Greek context, the HTA committee’s recommendations inform the negotiation committee, which has decision-making power over what is reimbursed. In Poland, HTA recommendations play an important role and are extensively used by the MoH, which decides on reimbursement based on negotiations between the Economic Commission of the MoH and the manufacturer [34].

Seventh, while HTA systems across the study countries seem to have well developed processes for the assessment of pharmaceuticals, these do not appear to be in place for medical devices and other technologies, including public health interventions; there is significant need for refinement in the assessment of both medical devices and other technologies [24]. Among other reasons, this is due to the highly fragmented market structure of medical devices, the lack of clear guidance on evidence requirements and the inconsistency in the methods employed in their assessment [24]. Overall, the range of relevant technologies undergoing HTA is determined by budget holders wishing to optimise the available resources. The identification of more than one HTA agency at national level often coincided with the identified HTA bodies having different remits and assessing different health technologies. For instance, in Wales the All Wales Medicines Strategy Group (AWMSG) assesses pharmaceuticals only, while Health Technology Wales (HTW) assesses medical devices and other technologies. This was further validated by experts who participated in the consultation round: according to primary evidence, the majority of countries reporting more than one body/institution performing HTA at national level, except Belgium and Estonia, have different national bodies for the assessment of different technologies.

Finally, there are different avenues for how HTA recommendations can be used depending on the way HTA systems operate, their role, and the technologies undergoing assessment, such as (i) reimbursement and coverage, (ii) price setting, (iii) strategic purchasing and procurement, especially for medical devices, and (iv) to inform clinical guidance. For instance, HTA bodies with a coordination role can rarely impact coverage decisions but their recommendations can be used for the update of clinical guidelines. HTA systems across our sample could be further divided into several categories in terms of the way HTA recommendations are implemented in decision-making. We observed systems where HTA outputs provide a fundamental basis for pricing and reimbursement; for example, in France, HTA recommendations are used both for reimbursement decisions by the national insurance fund and for pricing decisions by the TC. Other systems, such as that of England, operate in a manner whereby the HTA body makes a recommendation that eventually might trigger negotiations if the cost-effectiveness threshold is higher than the acceptable range or when there is considerable clinical uncertainty around the technology under evaluation. In this case, negotiations take place outside the remit of the HTA body between the purchaser/commissioner of care and the manufacturer. Lastly, there are HTA systems such as that of Australia or HTA by health insurance funds that internalise the decision-making process. Under these systems, negotiations, risk-sharing agreements or strategic purchasing and procuring take place within the HTA body based on the HTA outcome of the body itself. Despite how HTA recommendations are implemented, it is important to highlight that the ultimate ‘client’ of the HTA bodies is the healthcare system they operate in.

Our study is not without limitations. (a) Due to unavailability of data and limited access, we were not able to identify all regional HTA bodies across study countries. (b) Reliance on secondary sources has meant that it may not have been possible to capture HTA processes and implementation in detail. (c) Some HTA bodies may consider additional dimensions of value beyond clinical benefit and cost-effectiveness. Nevertheless, it has not always been possible to determine whether these features have an explicit impact on HTA recommendations through literature or expert opinion. (d) Even though we tracked the HTA systems across countries, the actual implementation and uptake of HTA activities during funding decisions were not fully captured. In order to address the latter two limitations, we performed the round of expert consultation to improve our understanding of the role and extent of HTA uptake at the national/regional level.

Conclusion

Based on a conceptual framework and taxonomy, we outlined the main operational pillars of HTA, showcased how HTA systems are set up within countries, how well developed HTA processes are as well as identified the different facets of HTA systems across the EU, the UK, Canada and Australia. Countries may follow similar pathways in the way HTA systems are set up, their role, remit and the way HTA processes are implemented, however, there are variations in the way HTA recommendations are translated into funding decisions. These relate to how well HTA processes are developed and integrated into decision-making and the extent to which purchasers/commissioners of care consider evidence-based information when deciding funding of technologies. While HTA processes are well established for pharmaceuticals across the study countries, there seems to be a need for the development of established HTA processes for medical devices and other technologies. HTA is a dynamic and transformative process which constantly adapts to new types of evidence, innovative technologies, and redefined objectives of healthcare systems. Even though HTA is now present across many settings, there is still an unmet need to make reimbursement and negotiation processes more transparent to better understand how purchasers/commissioners of care use HTA recommendations during negotiations with manufacturers, and to further investigate the extent to which HTA recommendations can influence coverage decisions.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 73 kb)

Health technology assessment (HTA) is an evidence-based tool used to inform funding and coverage decisions by healthcare systems at national and/or regional level. Key features of HTA and their operationalisation within settings can have an impact on whether, and to what extent, HTA recommendations influence funding and coverage decisions.

While there are well-developed HTA processes for the assessment of pharmaceuticals, there is an urgent need for the development of established HTA processes for medical devices and other technologies, such as public health interventions.

Even though HTA is now present across many settings, a lack of transparency in reimbursement and negotiation processes results in a limited understanding of whether or not HTA recommendations are considered in coverage decisions in practice. Because of this, there is a need to make these processes more transparent.