Systematic literature review on the implicit factors influencing the HTA deliberative process in Europe.

Objectives: Deliberative processes in Health Technologies Assessment (HTA) result in recommendations that determine the reimbursement of medicines, diagnostics or devices. These processes are governed by explicit criteria, but are also influenced by implicit factors. The objective of this work was to identify the implicit factors influencing HTA deliberative processes in five European countries (France, Germany, Italy, Spain and the UK).
Methods: A systematic review of literature published between 2009 and 2019 was conducted. The search was performed in Pubmed, The Cochrane Database of Systematic Reviews, Google Scholar and Center for Reviews and Dissemination. The ISPOR database was searched manually.
Results: Out of 100 eligible publications, eight articles were selected for data extraction and analysis. The implicit factors in the HTA deliberative process most frequently mentioned in the identified literature are value judgments, biases, preferences and subjectivity. Five out of the eight articles highlight the need to further improve the transparency of the process, and three provide recommendations on how to address the influence of implicit factors on the HTA deliberative process through a framework.
Conclusion: Even in countries with a long HTA history, evidence on implicit factors is scarce. Some methods have been recommended for addressing these factors. Further research is required to characterize the implicit factors in the HTA deliberative process at a country level and explore potential ways to mitigate the influence of these factors on the HTA deliberative process.

Introduction

In the last decades, substantial efforts have been made to improve decision-making and promote equitable, efficient, and high-quality healthcare systems. As part of such efforts, the process of Health Technology Assessment (HTA) has arisen as one that uses explicit methods and criteria to determine the value of a health technology at different points in its lifecycle to inform decision-making and promote an equitable and high-quality system [1].

With global healthcare expenditure rising from 478.81 US $ per capita in 2000 to 1,015.87 US$ per capita in 2016 [2] the HTA process is becoming increasingly relevant as a tool for assessing the value of new drug therapies in the healthcare system and subsequent decisions on coverage and reimbursement [3].

In an era where healthcare systems strive to deliver innovation and healthcare coverage, the HTA process has come to play a critical role in deliberative decision-making processes, as a means of demonstrating added value of new health interventions (i.e., drugs, medical devices, surgical procedures [4]) beyond efficacy and safety [5] to inform reimbursement decisions [6].

One of the key strategic objectives of the European Network for HTA (EUnetHTA) has been to promote more effective use of resources and increase HTA input into decision-making processes in Europe [7]. The EunetHTA Core model comprises nine dimensions of value, with safety, efficacy and cost-effectiveness being the most commonly assessed factors in Europe [5] and country-specific guidelines have been described elsewhere [8-13].

Beyond the evidence supporting the technology under review, HTA recommendations may be influenced by implicit factors inherent in the decision-making itself [14]. There is evidence showing that discrepancies may occur between recommendations by HTA advisory bodies and their related reimbursement decisions, which could potentially compromise the transparency and fairness of healthcare resource allocation [15]. Such discrepancies may to some extent be explained by variations in decision criteria, and result from contextual or implicit factors [16].

For the purpose of this systematic literature review (SLR), implicit factors can be any factors not explicitly stated in the publication of respective HTA body’s guidelines.

The geographic scope of this research covers Germany, France, UK and Spain. These five countries represent 73% of the European Union population in 2020 and according to Eurostat the health care expenditure of all providers in these countries represented 72% of the EU-28 in 2019 [17]; they all have established HTA agencies and recognised HTA processes [18,19].

According to the HTAi Global Policy Forum 2019 [15] the term deliberation refers to critical assessment and discussion of an issue, and involves the weighing of arguments for and against a measure. Deliberation follows the assessment phase during which evidence is gathered and synthesised, and focuses on the integration of this information through an appraisal process, after which recommendations are made [15,16].

The objective of this systematic literature review (SLR) was to identify and categorise implicit factors involved in the HTA deliberative process of medicines in Germany, France, Italy, UK and Spain.

Materials and methods

Eligibility criteria

A systematic review was conducted using the reporting items for systematic reviews and meta-analyses (PRISMA) guidelines (20,21). Two reviewers (CM and CC) screened the citations (titles, abstracts, and full text from eligible articles), and performed data extraction and analysis. Each methodological step was performed independently by both reviewers. Each reviewer’s results were compared and discussed, and in case of divergence, a consensus was reached. Where consensus could not be reached, a third reviewer (MT) performed the assessment to reach a final decision.

Information sources and search strategy

The literature review was performed in the following selected databases: PubMed, Google Scholar, The Cochrane Database of Systematic Reviews (CDSR), and Center for Reviews and Dissemination (CRD). ISPOR presentations database was searched manually. The reference sections of all identified articles were screened for additional articles that may be relevant for this SLR.

Searches were limited to the period 2009–2019, and to the following languages in the above databases: English, French, German, Italian or Spanish.

The search strategy is detailed in the Supplementary file.

Results were imported into an Excel file, and duplicated articles were removed.

In addition, grey literature searches were conducted using Google to identify documents not indexed in scientific journals, i.e., reports or textbooks, which were added to the bank of references as applicable.

Study selection

Citations were screened following a stepwise approach. In the first step, all titles and abstracts identified in the search strategy were independently reviewed by two reviewers against pre-defined eligibility criteria from the study protocol. Studies were categorised as ‘included’, ‘unsure’ or ‘excluded’.

The second step included a full-text review of all studies categorised as ‘included’ or ‘unsure’ during the first phase. The full-text review was continued until all articles had been categorised as either ‘included’ or ‘excluded’.

Citations’ and full-text screening by eligibility criteria

Inclusion criteria

Articles containing information regarding implicit factors influencingHTA deliberative processes on medicines.

Articles containing information about any of the five target countries together with information about other countries, were eligible for inclusion. However, only information pertaining to the target countries was synthesised.

Exclusion criteria

Languages other than English, French, German, Italian and Spanish

Year of publication: articles published before 2009 and after 2019

Articles describing HTA decision-making processes related to medical devices, diagnostic tests or medical procedures, specific therapeutic areas, and regional or local HTA decision-making processes.

Articles describing shared decision making between patients and healthcare providers

Data extraction and analysis

Data extraction from eligible articles was performed in two steps. The first step was to perform an extraction of implicit factors affecting HTA assessments from each eligible article. This was followed by analysis of the data collected from the first extraction, and a second extraction was performed using a method known as the general inductive method [19]. This method allows the development of general conclusions based on particular facts. Thus, the collected information was classified into categories of implicit factors.

Results

A total of 1,034 references were obtained from the literature search strategy (PubMed: 961; Google Scholar: 45; Cochrane: 5 and CRD: 10) and from the ISPOR Presentations database manual search [13]. Removal of 459 duplicates resulted in 575 references, 100 of which were selected for full text assessment (PubMed: 98; CRD: 2). From this selection, 92 articles were excluded for the following reasons: lack of relevant information on the decision-making process (n = 76); methodology design for HTA (n = 12); HTA decision-making in excluded countries (n = 4). Thus, eight articles were selected for data extraction and analysis (Figure 1).

Figure 1.

PRISMA flow diagram.

A description of the eight articles with the main findings is listed in Tables 1,2. Among the eight articles found in the literature, there were five articles that recognise and explain the occurrence of implicit factors in the HTA deliberative process, highlight the need to further improve the deliberative process, and outline a conceptual approach to address the problem [6,14,22-24] (Table 1). In addition, there were three articles that recognise and elaborate on these implicit factors, and provide recommendations on how to address implicit factors and improve the transparency of the HTA decision-making through a framework [25-27] (Table 2).

Table 1.

Articles proposing a concept as a way to address the research question.

AUTHORS	OBJECTIVE	ISSUES FOUND IN THE HTA DECISION-MAKING PROCESS	RECOMMENDATIONS
Bujar et al. 2019	To identify and investigate decision- making framework utilisation within 24 pharmaceutic al companies and 16 HTA agencies and assess the use of different methodologies s for the quality of decision- making process.The study also investigated the different cognitive biases that occur in decision- making	Unstructured process Presence of biases Decision- making is influenced by biases and emotional judgements. Need to improve the quality of the decision- making process by making it more formal and predictable Divergence in HTA and reimbursement decision- making The majority of organisations do not measure the decision- making even if there is the belief that this assessment is key	There is a need of having more consistent, structured and predictable decision- making processes during the life cycle of medicines. This could be achieved through systematically evaluating the quality of decision making and encouraging utilisation of an international formal decision-making framework within companies and agencies.
Fischer et al. 2013	To analyse influences of transparency, stakeholder participation and the scientific assessment on decision outcomes in the deliberative process.	Impact of transparency on the reasonableness of decisions is contested	Decision outcomes of coverage decisions appeared to be influenced mostly by the use of evidence rather than the degree of transparency or participation.The inclusiveness of the HTA process (thenumber of stakeholders involved) increases the likelihood of apositive reimbursement decision
Phelps et al. 2018	Identify and analyse the value judgements present in the HTA process	Deliberative processes today are often informal and unstructured and often lack transparency. It is unclear what factors have been considered and how thedecisions have been reached.	MCDA models may provide the best opportunity for improvement, but they have not yet been perfected.Greater testing and use of MCDA models are recommended
Kristensen et al.2019	To synthetize HTA good practices to support decision- making to create a basis for education and improved consistency in approaches to HTA informed decision making.	Lack of research on structured approach	Need to focus on developing good practices in using evidence to support decision-making through monitoring of HTA implementation and its inputs to various types of decision-making
Hofmann et al. 2014	Review of existing methods solving aggregation issues through ahealth economics approach.	Value judgments are not explicit in the HTA appraisals and decision-making processes.Acknowledging and explicitly addressing value judgments may improve the accountability of HTA.	Making explicit the value judgments implicit in HTA as well as in theappraisal phase can promote the robustness of thedecision-making. This can be supported by guidelines on the HTA appraisal and decision-making process. More research on identifying and onhow to highlight and integrate value judgments in HTA is needed.

Table 2.

Articles discussing or proposing a framework as a way to address the research question.

AUTHORS	OBJECTIVE	ISSUES FOUND IN THE DECISION-MAKING PROCESS	FRAMEWORK	RECOMMENDATIONS
Donelan etal. 2015	To investigate and identify the issues that Influence Quality decision- Making through semi-structured interviews to decision-makers	Clear evidence of the presence of personal biases of decision-makers, other factors impacting the subjective decision-making approach of individuals, the culture and behaviour of the organisations in decision- making. .	Integrated approach to quality decision-making capturing four stepwise domains: the structure, bias, impact and culture. The next phase following the design of a ‘framework for decision-making’ would be the development of a quantitative tool its psychometric robustness.	Quality decision-making needs a‘structured approach’, education andtraining in decision-makingtechniques, evaluating theimportance of the options anappreciation of the impact of thedecision made. An integratedapproach to quality decision-makingrequires a systematic framework thatconsiders four essential domains, (structure, bias, culture and impact.)
Oortwijn et al. 2017	To assess the level of comprehensiveness of HTA practices around the globe and formulate recommendations to enhance legitimacy and fairness of the decision- makingprocess	Assessment and appraisal processes does not seem aligned in some systems. Regarding the appraisal phase, the monitoring and assessment of the process is not well established across HTA systems.	INTEGRATE-HTA model and the Accountability for Reasonableness framework	-Broad and multidisciplinary stakeholder involvement will be necessary throughout the process.-The HTA should include standardised methods to identify and appraise evidence for clinically important moderators or predictors of treatment effects.-It is important to monitor and review the processes and results to assess their efficiency, consistency, and sustainability over time.- Transparency and robustness of the process will increase accountability and predictability
Baltussen R. et al 2017	Describe the key elements of the frameworkPresents the implications for the conduct of HTA	Presence of ‘vested interests’ in the formal HTA processes	Adopting evidence-informed deliberative processes as a value assessment framework could be an important step forward for HTA agencies in optimising the legitimacy of their priority- setting decisions.	-HTA agencies should take responsibility of organising stakeholder involvement.-Agencies are advised to integrate their assessment and appraisal phases-HTA agencies should subject their decision-making criteria to public scrutiny.-HTA agencies are advised to use a checklist of potentially relevant criteria and to provide argumentation for how each criterion affected the recommendation.-HTA agencies must publish their argumentation and install options for appeal.

The HTA deliberative process is influenced by a number of implicit factors related to the behaviour and personal values of the individuals involved as well as to the context in which this process is performed. These factors have been grouped under categories based on the frequency they have been mentioned in the literature. The categories are ethics, psychology, qualification and experience, politics and society, culture, functional role and disease perception (Table 3).

Table 3.

Implicit factors identified in the literature: frequency of mentions and impact in the HTA deliberative process.

CATEGORY	IMPLICIT FACTORS	FREQUENCY OF MENTIONS	IMPACT ON THE HTA DECISION-MAKING
ETHICS	Value judgments [6,14]	76	Fairness of resources allocation to health technologies may be compromised.May capture particular interests to the detriment of public interests, comoprimising equal access to good quality healthcare
	Bias [24]	34
	Equity [14]	12
	Equality [14]	4
	Intrisic values [14]	2
	Moral values [14]	2
	Interests [23]	2
	Implicit assumed desire [14]	2
	Vested interests [27]	2
	Values of knowledge [14]	1
	Ethical issues [6]	1
	Ethical implications [6]	1
	Fairness [14]	1
PSYCHOLOGY	Preferences [14]	16	Not mentioned
	Subjective [25]	13
	Training [25]	9
	Gut feeling [25]	6
	Opinion [25]	4
	Personal considerations [25]	2
	Personal values [22]	1
	Personality [22]	1
	Overconfidence in own judgment [25]	1
QUALIFICATION AND EXPERIENCE	Qualification [25]	5	Not mentioned
	Precedents for similar previous decisions [25]	1
	Previous decision-making mistakes [25]	1
	Experience in previous decision-making [25]	1
	Experience [14]	1
	Precedents for similar previous decisions [25]	1
	Experience in previous decision-making [25]	1
POLITICS & SOCIETY	Social values [22,27]	4	Not mentioned
	Political processes of the country [35]	1
	Political influences [25]	1
	Political pressures [24]	1
	Societal perceptions [27]	1
CULTURE	Cultural values [22]	1	Not mentioned
	Organizational culture [24]	1
FUNCTIONAL ROLE	Individual responsibility [14]	1	May impact the healthcare resource allocation
	Power of status [22]	1
DISEASE PERCEPTION	Disease severity [22]	1
	Burden of disease [14]	1

The implicit factors under ‘Ethics’ are the most frequently mentioned across the identified articles [6,14,22-24,26,27] (143 times), and this category includes: value judgments, biases, intrinsic values, values of knowledge, ethical issues, ethical implications, moral values, equity, equality, fairness, disparities, vested interests, interests and implicit assumed desire. Value judgments are the most referenced under this category and are constitutive of HTA decision-making. However they may differ from appraisal to appraisal, and they may be considered differently by the stakeholders involved in the appraisal committees since values, principles and considerations differ from person to person. Many of these value judgments are implicit and the objectivity is compromised making the HTA decision-making process biased [14].

Psychology may entail all factors related to the personality and subjective factors such as “preferences, “personal values, personal considerations ‘personality’, ‘opinion’, ‘gut feeling’, ‘overconfidence on own judgment’ [22,25].

Qualification and experience refer to the qualification and previous experience in similar situations of the members providing the recommendations [14,25].

Political influences and societal values or perceptions may also impact the HTA process. This category can include ‘political processes’, ‘political influences’, ‘political pressures’, ‘social values’ and ‘social perceptions’ [22,25-27].

The cultural values of individual HTA members or the organisational culture are also factors that may impact the recommendations. They are referred in the literature as ‘cultural values’ and ‘organizational culture’ [22,24].

As per the functional role, this is related to the roles and responsibilities of the HTA members on the committee such as ‘individual responsibility’ and ‘power of status’ [14,22].

Finally, the disease perception such as the burden of disease and unmet need may differ from person to person and may thus interfere with the recommendations on the technology [14,22].

Regarding the impact these implicit factors may have on the HTA deliberative process, it is argued that factors that fall into the categories of ethics (value judgments, bias and fairness) functional role (individual responsibility),and disease perception (disease severity) may have consequences for resource allocation. As an example, personal biases of HTA members may result in individual interests being favoured over public interests, which has consequences for resource allocation compromising the fairness and legitimacy of these decisions [27] (Table 3).

If deliberative processes are informal, unstructured, and unstandardised, these key issues may be overlooked, making it difficult to address the personal preferences or subjectivity of decision-makers [6,22,24].

As part of the recommendations, there are three articles that propose a framework for addressing the implicit factors, and five articles that provide recommendations without a framework.

Those proposing a framework and guidelines on how to formalise the decision-making process call for a more integrative perspective on HTA, aligning it with evidence-informed deliberative and legitimate processes. Frameworks for improved HTA conduct and processes have been developed aiming for an explicit, transparent, and inclusive process [25-27].

The ‘evidence-informed deliberative processes’ is proposed as a hybrid framework that addresses social values and ethical issues with the goal to enhance legitimacy. This framework entails multicriteria decision analysis and accountability for reasonableness and supports the HTA agencies in their efforts to organise their processes, leveraging the key elements of the framework which are stakeholder involvement in the assessment and deliberation phases and the implications for the HTA process [27].

Another proposed framework captures the integration of elements present in the decision-making which emerged from research into the four key domains of structure, bias, culture, and impact [25].

Following the recommendations, it is suggested that HTA agencies subject their decision-making criteria to public scrutiny and communicate the criteria behind their reimbursement recommendations. Additionally, to make the process more democratic, the importance of HTA agencies having permanent lay members on their appraisal committees, as representatives of the public interest, is highlighted [27].

When it comes to the other group of articles that do not propose a framework, the focus of their recommendations is on assessing and understanding the process and quality of HTA decision-making, as a mean of providing insights into the factors that influence the recommendations and decisions derived from it. Interventions like increasing the degree of participation from stakeholders or getting formal feedback from internal and external stakeholders regarding the decision making could help to improve the quality of the decision-making process itself, and ultimately transparency and consistency in key decisions [14,23,25]. Another approach to formalising the HTA decision-making process and making it more transparent is the Multicriteria Decision Analysis (MCDA) method. This method expands upon implicit factors not captured in the cost-effectiveness analysis such as fairness, equity or disparities [27]. Making explicit the implicit factors in the HTA process could contribute to improve transparency and legitimacy of the decision-making process [22].

Discussion

Like other deliberative decision-making processes, HTA deliberative process is a complex process that is affected by factors that are generally recognised and agreed upon (such as safety and cost-effectiveness), as well as other factors that are not explicitly stated and vary between countries, systems and individuals. The latter factors account for variability in the decision-making criteria [7,27].

This systematic literature review found that implicit factors, being constitutive of the HTA deliberative process, may interfere with the HTA recommendations. It also found a lack of a standardised framework for addressing these factors across the studied countries. For this reason, there is a need to unmask and clarify these factors to increase the transparency, fairness, impartiality and formality of the process [23,26,28].

Some HTA agencies have used frameworks for specific deliberative processes. However, these frameworks do not account for subjective elements such as behavioural and cognitive aspects that may influence the final decisions [29].

Inconsistencies, variability and lack of predictability have been reported in current HTA value frameworks; to address these challenges, methodologies such as MCDA for healthcare decision making have been frequently debated over the last years [30]. This method seems appropriate as a concept for integrating diverse attributes as part of an explicit approach [28]. However, due to practical limitations with respect to its implementation, it is mainly used for experimental or academic purposes [31].

EUnetHA is an example of an HTA framework that is used to assess and appraise new technologies. EUnetHTA proposed its HTA Core Model as a way to harmonise the HTA assessment process across countries [32-34]. However, the HTA core model does not address the implicit factors, except for marginally alluding to them in a section called ‘value judgements’. Nor does this framework assign a dimension to these value judgments and their influence, to allow their impact on the deliberative and HTA processes to be monitored and quantified [34].

Other initiatives that have been developed to guide the HTA deliberative process were highlighted in the Background Paper from the HTAi Global Policy Forum 2020, which proposed a range of questions, principles and actions that could be used to guide the development of deliberative processes within HTA. This includes guidance on the selection of committee members and allocation of roles, as well as recommendations regarding stakeholder involvement and the cognitive aspects present in the process of deliberation. The HTAi Global Policy Forum also recommends using decision-making frameworks to make the process more structured, explicit and formal and minimise the influence of individual stakeholder interests on the decision-making process to safeguard its fairness [15].

Other elements that could be integrated into formal HTA assessments or frameworks include broader dimensions of value, such as cultural and social values that are not commonly addressed in HTA assessments [15,23,31].

Additionally, the patient perspective is an important feature to be considered in these formal assessments [16]. In some areas with high unmet medical needs, such as rare diseases, the role of patients is being increasingly recognised at all stages of evidence development, resulting in active patient involvement in the HTA process. Also, in cases where the evidence is uncertain, increased participation of patient organisations may increase the likelihood of a positive reimbursement decision [18]. From a societal standpoint, decision criteria need to be informed by society's values and preferences, which involves seeking to incorporate the priorities and perspectives of citizens into the decision-making process [33].

The results of this SLR highlight the need for a standardised framework that addresses implicit factors in the HTA deliberative process. Further research is advised to better understand the impact that the identified implicit factors may have on the deliberative process in each country and explore potential ways to mitigate the influence of these factors on the HTA deliberative process.

Strengths and limitations

To the best of our knowledge, this is the first SLR focused on the implicit factors in the HTA deliberative process, which is the main strength of this study. To facilitate the analysis, the identified implicit factors were categorised by type. Acknowledging the scarcity of information on this matter, the research group aimed to have a broad linguistic scope and for this reason, the SLR included publications in English, Spanish, French, Italian and German.

As for the limitations of this study, there is scarce literature published on the research topic with limited authors and HTA agencies involvement. On this line, the evidence on the impact of those implicit factors is scarce.

This SLR comprehends articles published during the period 2009–2019 in a limited number of countries in Europe.

Lastly, as any SLR, in the identified literature it is not always evident to find the information needed to answer all the relevant questions to perform the required analysis. In our case, there was scarce of information regarding the description of many implicit factors and their impact on the HTA decision-making from a general and country level perspective.