Value drivers for pharmaceutical products in health technology assessment (HTA) in Saudi Arabia: Results from a capacity Building, Multi-Stakeholder workshop.

OBJECTIVES: Capacity building exercises are important to increase understanding of healthcare processes by key stakeholders, and to facilitate open discussions to build consensus. This study explored the views of a multi-stakeholder group of local Saudi experts on possible value elements that could be important for health technology assessment (HTA) processes and methods regarding pharmaceutical products in Saudi Arabia ('value drivers').
METHODS: A diversified group of local experts were invited to a two-day capacity building workshop from 18 to 19 December 2019 in Riyadh, Saudi Arabia. Information regarding the participants' demographic and educational/professional background, along with their self-assessed knowledge and experience of HTAs and the concept of value in the pharmaceutical market was collected. For each of 22 value drivers identified during a targeted literature search, participants were asked either to 'opt out' of its consideration for future HTA assessments, or rate it from 1 to 10 (low-high) on feasibility and acceptability.
RESULTS: Efficacy and safety were the highest rated value drivers for acceptability and feasibility. Explicit cost-effectiveness thresholds had the lowest ratings for acceptability and feasibility. Participants highlighted data availability and accuracy as a potential challenge to HTA implementation in Saudi Arabia.
CONCLUSIONS: Participants valued a pharmaceutical product's efficacy and safety alongside the consideration of disease characteristics for HTA processes. Participants also valued a binding HTA recommendation and the use of local real-world evidence, where available, to support HTA submissions.

Introduction

Among the Gulf Cooperation Council (GCC) countries, Saudi Arabia has the fastest growing population, estimated to reach>45.1 million by 2050 (Albert et al., 2018, United Nations Department of Economic and Social Affairs Population Division, 2017). Population growth in Saudi Arabia has been driven by a high birth rate, declining mortality rate among infants and children, and an increase in life expectancy (Albert et al., 2018, Hussein and Ismail, 2017, United Nations Department of Economic and Social Affairs Population Division, 2017). Although the population at present is relatively young (60% of the population is ≤ 35 years old), population growth has increased the demand for healthcare services; and this is only expected to increase as the population continues to age (Albert et al., 2018, Hussein and Ismail, 2017, Khan et al., 2017, Tyrovolas et al., 2020).

In addition, there has been a significant increase in the prevalence of metabolic and lifestyle diseases in the Saudi population, such as obesity, hypertension and diabetes (Albert et al., 2018, Tyrovolas et al., 2020). This recent epidemiological trend, attributable in part to increasing urbanization and disposable income (Albert et al., 2018), has seen chronic disease replace communicable disease as a leading cause of morbidity and mortality in Saudi Arabia (Memish et al., 2014, Tyrovolas et al., 2020). Furthermore, the Saudi healthcare system is experiencing additional burden from an increased prevalence of mental disorders and road traffic injuries (Memish et al., 2014, Tyrovolas et al., 2020).

Healthcare in Saudi Arabia is provided by both the public and private sector, with public spending representing 74.2% of total healthcare expenditure (Albert et al., 2018). Overall access and quality of care has improved over the past decade due in part to significant investment by the government (Albert et al., 2018, Tyrovolas et al., 2020). However, as demographic and epidemiological transitions in Saudi Arabia continue to impose demands on the healthcare system, there is a continued need to identify safe, high quality and cost-effective healthcare treatments to reduce the burden of chronic disease and inequalities in access to healthcare (Alhawassi et al., 2018, Tyrovolas et al., 2020). As such, the public healthcare sector is gradually expanding its role as a regulator to ensure the efficient allocation of resources (Albert et al., 2018).

For many years, there has been a concern that the high pricing of pharmaceuticals and other health technologies creates a barrier to care around the world, posing challenges for healthcare systems in delivering affordable care and value for their patients (Garner et al., 2018). Pharmaceutical spending is high throughout the Middle East and North Africa (MENA) region (Kanavos et al., 2020). In 2018, pharmaceutical expenditure made up 19.4% of the overall healthcare expenditure in Saudi Arabia (Alrasheedy, 2020), and this proportion is continuing to rise (Al-Omar, 2020, Alrasheedy, 2020).

As part of Vision 2030, the Ministry of Health (MoH) in Saudi Arabia is establishing a centralized and unique health technology assessment (HTA) entity that aims to support effective, evidence-based decision-making, specific to its population and healthcare system needs (Al-Omar et al., 2020b). Although there are some similarities in HTA processes and methods between countries, it is often the case that each country has a unique set of circumstances to consider in their implementation of HTA (Akehurst et al., 2017). Although pharmaceutical companies are committed to delivering the high development standards required for an HTA submission, variation in the data required by each HTA entity to demonstrate a drug’s clinical effectiveness, safety, quality, and cost-effectiveness can lead to problematic disparities in data interpretation and valuation between different decision bodies (Akehurst et al., 2017, Al-Omar et al., 2020a, Joint Healthcare Industry Paper, 2011). Therefore, it is important for both pharmaceutical companies and other stakeholders to understand the overall HTA decision-making process and the specific drivers of decision-making that guide each HTA entity with regards to the reimbursement of pharmaceutical products.

As part of a capacity building process to gauge the understanding, and any challenges, of the HTA process in Saudi Arabia with regards to pharmaceutical products, a multi-stakeholder workshop was held with a group of local experts from different sectors. This discussion employed an evaluation tool to explore and quantify which of a set of value drivers were considered acceptable and/or feasible for pharmaceutical products in Saudi Arabia.

Materials and methods

Multi-stakeholder workshop

The two-day workshop was held from 18 to 19 December 2019 in Riyadh, Saudi Arabia, and was organized by the MoH. An invitation was sent to a diverse set of relevant stakeholders in healthcare from government and private sectors (including management, physicians, clinical pharmacists, health economists, regulators, pharmaceutical industry representatives, academics, patient representatives and medical insurance company representatives). On the first day, discussions on HTA impact, valuation models, and value frameworks were discussed. On the second day of the workshop, an additional 10 participants from industry were invited to participate. These individuals were invited based on the price and designation of their pharmaceutical products (i.e. high-impact pharmaceutical products). The polling platform Slido® was used to collect demographic information and comments from the participants regarding their personal or educational/professional experience, background knowledge of HTAs, understanding of the concept of value in the pharmaceutical market, whose perspectives should be considered when defining value, and the perceived challenges for HTA implementation in Saudi Arabia.

A presentation by the workshop organizer provided background information for the workshop participants regarding the present challenges arising from demographic and epidemiological shifts across the world, and the challenges that are particularly relevant to Saudi Arabia. In addition, this presentation described HTA processes and value frameworks used in other countries and detailed the processes involved in implementing a novel HTA value framework.

Value driver selection and voting process

Prior to the workshop, a targeted literature search of established and newly formed HTA entities was used to create a list of the main value criteria and processes used in HTA decision-making around the world. PubMed and Embase were searched for literature published from 1 January 2010–20 October 2019 and available in English. The following search terms were used to identify the main criteria considered in HTA decision-making: value, drivers, factors, criteria, domains, elements, measurement, reimbursement, decision, pharmaceuticals, medications, drugs, methods and processes. Additional value dimensions were identified using related manual searches of the following HTA agency websites: Australia, Canada, England, France, Germany, Italy, Spain and Sweden. From this list, a total of 22 value drivers were selected to be discussed and voted on by the workshop participants (Table 1). These value drivers were chosen to represent important concepts such as public health interest (i.e. disease rarity, communicability, and/or curability), sources of data (i.e. randomized controlled trials [RCTs] and/or real-world evidence [RWE]), the role of manufacturers (i.e. gross domestic product [GDP] and/or local research and development [RnD] contribution) and the HTA decision-making process and outcome (i.e. a binding HTA recommendation and emphasis on collaboration between institutions). In this context, the value drivers represent the factors that may reflect the value of a given pharmaceutical product, as well as potential approaches that may add value to the HTA process in Saudi Arabia. Value drivers were grouped and presented in a survey that was given to participants on the second day of the workshop. Prior to any quantitative evaluation, each value driver was defined by the workshop organizer and discussed amongst the workshop participants.

Table 1

Value driver descriptions.

Value driver	Description
Process and outcomes
Patients	Include patients in the HTA process
Industry	Include industry (or their representative) in the HTA process
Thresholds	Define explicit efficiency ICER thresholds for the acceptance of a new technology
Collaboration	Require collaboration between institutions rather than centralizing the work in one agency
Appraisal Assessment	Separation of the Appraisal and the Assessment, by two independent bodies
Binding	Make HTA evaluations binding for authorities
Evaluation criteria
Efficacy	Clear definition of the marginal benefit of the technology/drug
Safety	Clear definition of the safety profile of the technology/drug
Quality of life (QoL)	Requirement to include QoL data in the submission
End of Life (EoL)	Considering EoL to deserve a differentiated set of evaluation criteria
Special Groups	Definition of specific acceptability rules for special groups
Rare	Define those special groups as having a rare disease
Ultra-Rare	Narrow the rare special group to a subset
Curable	Have specific criteria for those diseases that are curable
Communicable	Have specific criteria for those diseases that are communicable
Innovation	Explicitly recognize innovation as part of the potential benefit
Sources of data
Randomized controlled trial (RCT)	Only look at submissions backed by an RCT
Real-world evidence (RWE)	Include real-world data (RWD) in the HTA submissions to validate the trial data
Epidemiology	Include epidemiological data in the submission
Modelling	Require modelling of the results beyond the clinical trial
Cost/quality-adjusted life year (QALY)	Cost/QALY (and ICER) calculations required in the submission
Gross domestic product (GDP)/research and development (RnD)	Assess and reward the implication of the manufacturer on the GDP and/or local RnD contribution

EoL, end of life; GDP, gross domestic product; HTA, health technology assessment; ICER, incremental cost-effectiveness ratio; RCT, randomized controlled trial; RnD, research and development; RWE/D, real-world evidence/data; QALY, quality-adjusted life-year; QoL, quality of life.

For each value driver, participants were asked by two authors (HA and OSM) to ‘opt out’ if they did not think that it should be considered as a possible value element within the Saudi context. If participants did not ‘opt out’ of a certain value driver, they were asked to score the value driver on acceptability and feasibility (1–10; low–high). Acceptability was a measure of whether the participant felt that the value driver would be acceptable to all stakeholder parties (in this case, acceptability of the value driver was not a personal assessment). Feasibility was defined as whether the relevant data were likely to be available and/or accessible to support a given value driver in the HTA process. A paper-based template was used to collect the votes and capture relevant comments from each participant. All voting was anonymous.

Results for each value driver included the percentage of participants who voted to ‘opt out’, and the mean and standard deviation for the acceptability and feasibility scoring of each value driver.

Results

Workshop participant demographics

A maximum of 43 workshop participants answered a range of questions related to their personal, educational and professional background, the complete results of which can be found in Table 2. More than one third of the workshop participants were female (34.9%) and the highest represented age group was 35–44 years old (42.9%). Half of the workshop participants indicated that their educational background was in pharmacy (50.0%), but participants with a medical (17.5%) or economic (22.5%) background were also highly represented. Over a third of the participants had a degree in health economics or pharmacoeconomics (36.4%). The three current professions with the highest proportion among the workshop participants were clinical pharmacists (28.2%), chair/members of a pharmacy and therapeutic committee (PTC; 25.6%) and academic professors (23.1%). Many of the participants worked within a government setting (56.3%), whereas 25.0% were from the private sector.

Table 2

Demographics and professional history of workshop participants.

Question	n/N (%)
Gender
Male	28/43 (65.1)
Female	15/43 (34.9)
Age (years)
25–34	12/42 (28.6)
35–44	18/42 (42.9)
45–54	11/42 (26.2)
≥55	1/42 (2.4)
What is your educational background?
Pharmacist	20/40 (50.0)
Economist	9/40 (22.5)
Physician	7/40 (17.5)
Finance	1/40 (2.5)
Other	3/40 (7.5)
Do you have a degree in health economics or pharmacoeconomics?
Yes	12/33 (36.4)
No	21/33 (63.6)
What is your current professional background? (you can choose more than one option)
Clinical Pharmacist	11/39 (28.2)
Chair/member of pharmacy and  therapeutic committees (PTCs)	10/39 (25.6)
Academic professor	9/39 (23.1)
Policy maker	6/39 (15.4)
Key opinion leader (in a specific therapy area)	5/39 (12.8)
Payer	3/39 (7.7)
Regulator	3/39 (7.7)
Researcher/member of research agencies	3/39 (7.7)
Clinical guideline expert	2/39 (5.1)
Health authority official	2/39 (5.1)
Physician	2/39 (5.1)
Medical insurance	1/39 (2.6)
Other	16/39 (41.0)
What is the nature of your workplace setting?
Government	18/32 (56.3)
Private	8/32 (25.0)
Authority	3/32 (9.4)
Other	3/32 (9.4)

HTA and value framework polling questions

A high proportion of the participants described their knowledge of HTA as good (32.4%), very good (20.6%) or excellent (11.8%). A smaller proportion of participants described their knowledge of HTA as fair (23.5%) or indicated that this was the first time they had heard of HTAs (11.8%) (Fig. 1A). When asked to assess their level of knowledge about the concept of value in the pharmaceutical domain, most participants indicated good (20.0%), very good (34.3%) or excellent (14.3%) knowledge. A further 17.1% of participants indicated that they had fair knowledge of the concept of value in the pharmaceutical domain (Fig. 1A). A high proportion of participants indicated that for defining value in a Saudi context, perspectives from patients (88.6%), the healthcare system (97.1%) and society (80.0%) should be considered. Just under half (45.7%) of participants indicated that an industry perspective should be considered (Fig. 1B).

Fig. 1

Results from HTA and value framework polling questions. aThis question was populated using participant responses from a previous open-ended question, ‘In your view, what is the major challenge to implementing HTA in Saudi Arabia?’ HTA, health technology assessment.

Of potential challenges for HTA implementation in Saudi Arabia, a high proportion of the participants indicated that availability and accuracy of data (i.e. registries, RWE, quality of life [QoL], and costing) was one of the most important (86.4%). A scattered healthcare system was also considered to be an important challenge for HTA implementation (36.4%). In addition, almost a third of participants highlighted challenges to the HTA process related to lack of acceptance by important stakeholders (31.8%), lack of expertise and manpower (31.8%) and/or lack of credible available data on diseases and their burden (31.8%) (Fig. 1C). Within an HTA context, almost two thirds of participants believed that physicians were not ready to adopt a decision taken by non-physicians (65.4%) (Fig. 1D). Just under half of the participants did not believe that there was enough capacity in Saudi Arabia to implement HTA (47.6%). When asked if an HTA recommendation should be binding, 71.8% of participants agreed (Fig. 1D).

Value drivers

Inclusion/exclusion

Of the 43 workshop attendees, 36 participated in the value driver voting process. None of the participants voted to exclude efficacy, safety, rarity or collaboration from the HTA process (Fig. 2). The highest exclusion rates were observed for the value drivers related to the GDP/RnD contribution of a manufacturer (25.0%) and industry participation in the HTA process (19.4%) (Fig. 2). The requirement that HTA submissions include RCT data was rejected by 13.9% of participants (Fig. 2). In addition, only 13.9% of participants rejected the concepts that an HTA recommendation should be binding and that explicit efficiency incremental cost-effectiveness ratios (ICER) thresholds should be used. Finally, 11.1% of participants rejected both the mandatory inclusion of RWE in an HTA submission to support evidence from RCTs, and End of Life (EoL) considerations (Fig. 2). The full results from the inclusion/exclusion voting on the value drivers are presented in Fig. 2.

Fig. 2

Value drivers that should NOT be included in an HTA process For each value driver, participants were asked to ‘opt out’ if they did not think that it should be considered as a possible value element in Saudi Arabian HTA assessments. GDP, gross domestic product; HTA, health technology assessment; QALY, quality-adjusted life year; RCT, randomized controlled trial; RnD, research and development; RWE, real-world evidence.

Acceptability and feasibility

Efficacy and safety were rated the most acceptable value drivers by the workshop participants with mean values > 9 (on a scale of 1–10, representing low–high) (Fig. 3; Table A.1). Other value drivers that scored highly for acceptability (mean value > 8) included the separation of appraisal and assessment, consideration of special groups, disease curability and communicability, as well as using evidence based on RCTs, RWE and epidemiological data (Fig. 3; Table A.1). The two value drivers that had the lowest values for acceptability (mean value of < 7) were EoL considerations and explicit cost-effectiveness thresholds (Fig. 3; Table A.1).

Fig. 3

Mean acceptability and feasibility for each value driver If participants did not ‘opt out’ of a certain value driver, they were asked to score the value driver on feasibility and acceptability (1–10; low–high). Feasibility was defined as whether the relevant data were likely to be available and/or accessible to support a given value driver in the HTA process. Acceptability was a measure of whether the participant felt that the value driver would be acceptable to all stakeholder parties (in this case, acceptability of the value driver was not a personal assessment). EoL, end of life; GDP, gross domestic product; HTA, health technology assessment; QALY, quality-adjusted life year; QoL, quality of life; RCT, randomized controlled trial; RnD, research and development; RWE, real-world evidence.

Table A1

Value driver acceptability and feasibility voting results.

Value driver	Acceptability		Feasibility
	n	Mean (SD)	n	Mean (SD)
Process and outcomes
Patients	26	7.8 (2.2)	27	5.9 (2.6)
Industry	18	7.1 (2.5)	18	7.8 (2.1)
Thresholds	26	6.1 (2.5)	29	5.0 (2.6)
Collaboration	26	7.7 (2.4)	27	7.2 (2.1)
Appraisal Assessment	26	8.1 (2.1)	26	7.9 (2.1)
Binding	24	7.5 (2.1)	25	7.3 (2.3)
Evaluation criteria
Efficacy	32	9.3 (1.2)	35	9.0 (1.3)
Safety	33	9.4 (1.1)	34	8.7 (1.6)
QoL	30	7.2 (2.5)	28	5.2 (2.8)
EoL	22	6.5 (2.8)	23	5.5 (2.6)
Special groups	27	8.3 (2.0)	28	7.3 (1.9)
Rare	31	7.2 (2.6)	32	6.4 (2.6)
Ultra-rare	30	7.0 (2.9)	31	5.8 (2.9)
Curable	29	8.8 (1.5)	30	7.3 (2.3)
Communicable	26	8.6 (1.7)	27	7.7 (2.0)
Innovation	22	7.7 (2.4)	23	7.0 (2.0)
Sources of data
RCTs	27	8.4 (2.1)	29	6.9 (2.4)
RWE	21	8.7 (1.7)	22	5.8 (2.7)
Epidemiology	31	8.1 (1.9)	31	6.3 (2.5)
Modelling	24	7.6 (1.9)	25	6.6 (2.6)
Cost/QALY	27	7.9 (1.9)	30	6.8 (2.6)
GDP/RnD	13	7.8 (2.2)	14	6.4 (2.1)

If participants did not ‘opt out’ of a certain value driver, they were asked to score the value driver on feasibility and acceptability (1–10; low–high). Feasibility was defined as whether the relevant data were likely to be available and/or accessible to support a given value driver in the HTA process. Acceptability was a measure of whether the participant felt that the value driver would be acceptable to all stakeholder parties (in this case, acceptability of the value driver was not a personal assessment). EoL, end of life; GDP, gross domestic product; QALY, quality-adjusted life year; QoL, quality of life; RCT, randomized controlled trial; RnD, research and development; RWE, real-world evidence; SD, standard deviation.

Efficacy and safety were also rated the most feasible value drivers by the workshop participants with mean values > 8 (Fig. 3; Table A.1). Other value drivers that scored relatively highly for feasibility (mean value > 7) were the inclusion of an industry perspective, collaboration between institutions, separation of appraisal and assessment, a binding HTA recommendation, consideration of special groups, and disease curability and communicability (Fig. 3; Table A.1). There were six value drivers that had a mean feasibility score < 6: the participation of patients in the decision-making process; QoL, EoL, and ultra-rarity evaluation criteria; and the inclusion of RWE and explicit cost-effectiveness thresholds (Fig. 3; Table A.1).

Each value driver (with the exception of the inclusion of an industry perspective in the decision-making process) had a higher mean value for acceptability than feasibility (Fig. 3; Table A.1). Over half of the value drivers had a mean acceptability score that was higher than the mean feasibility score by ≥ 1: inclusion of a patient perspective and explicit cost-effectiveness thresholds; QoL, EoL, special groups, ultra-rarity, and curability evaluation criteria; and evidence from RCTs, RWE, epidemiological data, cost/quality-adjusted life year (QALY), and GDP/RnD. In particular, the largest difference between mean scores for acceptability and feasibility was observed for the inclusion of RWE (mean acceptability = 8.7, mean feasibility = 5.8).

Discussion

HTA review processes consider the clinical, social, economic, organizational and ethical effects and impacts of a health intervention/technology to guide evidence-based healthcare recommendations (World Health Organization (WHO), 2015). However, the relative importance and weight of each of these criteria can vary by country based on the characteristics and values of the local population and their healthcare needs. This exploratory study presents the views that were captured from a capacity building workshop of local Saudi experts, regarding possible value elements deemed to be important for HTA processes within a Saudi context.

Cost-effectiveness analyses are often used by health policy decision-makers to guide recommendations regarding the implementation of effective and safe health interventions while ensuring the efficient allocation of healthcare resources (Bertram et al., 2016). These analyses are used to compare the ‘value for money’ of a new health technology or intervention with the existing standard of care (Cubi-Molla et al., 2020). However, there is significant variation between countries regarding how cost-effectiveness thresholds are established and used in the HTA process. Differences between HTA agencies can include how value is defined, if the cost-effectiveness threshold is published in official guidelines (i.e. explicit), and whether the thresholds are considered ‘hard’ or ‘soft’ (e.g. non-negotiable or used as a reference point for pricing negotiations) (Cameron et al., 2018, Cubi-Molla et al., 2020, Fasseeh et al., 2020). Best practices for how cost-effectiveness thresholds should be calculated and assessed in the HTA decision-making process are widely debated (Cubi-Molla et al., 2020).

Efficacy and safety were the highest rated value drivers by participants for both acceptability and feasibility. The majority of HTA entities use treatment efficacy and safety data from RCTs to compare the clinical benefits of different health interventions and inform the decision-making process (Angelis et al., 2018). A previous study has shown that some Saudi healthcare professionals (HCPs) question the clinical relevance of these RCT data, and would prefer to assess local effectiveness data that reflect a Saudi context (Al-Omar et al., 2020b). A high level of acceptability for the inclusion of RWE in future HTA submissions was also demonstrated by the participants in this workshop. However, participants also highlighted the potential challenges of local data availability in Saudi Arabia; local registries and databases are often not accessible, available and/or lack complete, high-quality data (Al-Omar et al., 2020a, Al-Omar et al., 2020b, Fasseeh et al., 2020). With increasing concern over the generalizability of clinical evidence from RCTs between different countries, local RWE has been increasingly included in HTA submissions (Berger et al., 2017).

Currently, HTA entities primarily consider RWE in the assessment of long-term treatment safety, or for more exploratory analyses regarding epidemiological and treatment trends at a local level (Berger et al., 2017). At present, HTA organizations do not commonly consider RWE in treatment effectiveness decisions, due in part to concerns over the methodology and content validity of studies using real-word data (RWD) (Berger et al., 2017). Policies for RWE use in submissions are not consistent across HTA agencies, and some have argued for improved alignment of policies in order to encourage the consideration of RWE analyses in future HTA decision-making processes (Makady et al., 2017). A recent drive to conduct more clinical trials in Saudi Arabia, coupled with investment in disease registries, may improve the availability and reliability of local Saudi data that can be used in future HTA decision-making processes (Al-Omar et al., 2020a, Al-Omar et al., 2020b).

Although the value of a health intervention may be quantifiable in monetary terms and assessed through a cost-benefit analysis, value is often expressed as a ‘utility’, most commonly in the form of the QALY measure, which captures effects on both quality and quantity of life (Cubi-Molla et al., 2020). For health care treatments, cost-effectiveness thresholds are often calculated through a willingness-to-pay (WTP) model, and expressed as a cost per QALY (Cubi-Molla et al., 2020).

The inclusion of explicit cost-effectiveness thresholds was rejected by 13.9% of participants, and received low scores for acceptability and feasibility. Using explicit cost-effectiveness thresholds can limit the scope of HTA recommendations and/or pricing decisions to take into consideration the local context regarding the need for a given pharmaceutical product, feasibility of implementation, budget impact, value of innovation or other factors that may not explicitly be captured in cost-utility analyses (Bertram et al., 2016, Garner et al., 2018). Explicit cost-effectiveness thresholds do not allow for positive discrimination towards policy or political decisions. As the calculation of cost-effectiveness thresholds can be heavily influenced by the data chosen to include during the analytical process, it is crucial that decision-makers have confidence in the methodology used to determine local cost-effectiveness thresholds (Bertram et al., 2016). Although the use of implicit cost-effectiveness thresholds can provide more flexibility to decision-makers, it can also create an opaque process in which external social and political pressures may influence the final recommendation (Cameron et al., 2018).

Evidence from previous workshops has shown that both local Saudi experts and pharmaceutical representatives value the transparency provided by cost-effectiveness analyses, but that it is important to utilize Saudi-specific data and utility measures (Al-Omar et al., 2020a, Al-Omar et al., 2020b). In addition, these experts expressed concern over the applicability of a single WTP threshold (based on cost/QALY) for all of Saudi Arabia, as existing standards of care can vary across healthcare and regional settings (Al-Omar et al., 2020a, Al-Omar et al., 2020b). Participants in a multi-stakeholder conference on HTA implementation in the MENA region indicated a preference for an explicit soft threshold (51.0%) as compared with implicit (15.7%) or explicit hard thresholds (27.5%) (Fasseeh et al., 2020). A soft approach would provide a range of cost-effectiveness thresholds, rather than a single value, and used to provide flexibility to the decision-making process and allow for context-specific consideration of value elements (Cubi-Molla et al., 2020). More discussions would be required to understand the specific reasoning behind the participants’ perception of explicit cost-effectiveness thresholds that led to overall low scores for acceptability and feasibility in this workshop.

There was a relatively high acceptability amongst participants for HTA evaluation criteria related to disease curability, communicability and special groups. This may suggest that considering alternative methodologies for informing decision-making, such as a multiple criteria decision analysis (MCDA), or augmented or extended cost-effectiveness analysis, are worth consideration (Phelps et al., 2018). A previous conference on HTA in the MENA region found that the majority of participants would consider the use of MCDA methodology in future HTA processes (85.7%) (Fasseeh et al., 2020). The MCDA approach is generally considered valuable due to its consistency and transparency, and as such is increasingly used to guide health policy decisions globally (Hansen and Devlin, 2019). However, there are several challenges with the MCDA approach, including how to select the appropriate methodology to score and weight evaluation criteria, if this methodology should be decision-specific or uniform across similar drug applications, and how to incorporate opportunity cost and uncertainty in the MCDA modelling (Hansen and Devlin, 2019).

There appeared to be general agreement for the inclusion of a binding HTA recommendation (with only 13.9% of participants voting to exclude this value driver), and it also received relatively high acceptability and feasibility scores. This supports the findings of a previous workshop that found binding HTA decisions may be important to stakeholders to create equal access to health care for individuals across Saudi Arabia, regardless of setting (public or private) or region (Al-Omar et al., 2020b). However, it is important to note that for the participants who voted against a binding HTA recommendation, there may be other factors such as education, consensus and inclusiveness that they believe are important to consider alongside the implementation of an HTA recommendation. Further discussions are required to understand the challenges that these participants perceive for a binding HTA recommendation.

A high proportion of the workshop participants voted against including an industry perspective in the HTA process (19.4%). The views of industry representatives have been considered previously in the formation of new HTA entities (e.g. Switzerland) (Joint Healthcare Industry Paper, 2011). Once established, these perspectives can provide input on the challenges and hurdles that industry may encounter due to changing HTA methodologies and processes (Joint Healthcare Industry Paper, 2011). Results from a previous workshop attended by representatives from pharmaceutical companies suggested that the majority of industry participants understood the benefit of HTA processes for the Saudi healthcare system, and expressed a willingness to adapt to meet future HTA submission requirements (Al-Omar et al., 2020a). Incorporating an industry perspective may add value to HTA processes by providing information on specific health care sectors/products, sharing knowledge gained from HTA processes in different countries, and highlighting important factors to consider under ‘innovation’ (Joint Healthcare Industry Paper, 2011). Further discussions will be necessary to understand the specific concerns held by the workshop participants regarding voting against including industry views in the HTA process in Saudi Arabia. Possible avenues to increase general acceptance of an industry perspective in HTA processes include informing stakeholders about the potential benefit of industry views and education for industry representatives to work as non-partisan participants.

Strengths and limitations

The results from this workshop are representative of the different perspectives held by a relatively large multi-stakeholder group. There was variation in the scoring of most of the value drivers amongst participants, and this often prevented a clear consensus. However, reaching a consensus is not necessarily a priority for capacity building exercises, and the diverse backgrounds and experiences of the participants ultimately add to the strength of the findings. A good first step towards building consensus is to have open discussions and understand how peers think about these matters.

Experience with and understanding of HTA processes and each value driver was not consistent across the multi-stakeholder participants at the beginning of the workshop. However, informative presentations and discussion sessions were provided by the authors to ensure a base level of knowledge amongst participants before the value driver voting process. Any potential impact of the presentations and discussions on the responses given, especially among participants with little or no prior knowledge of HTA processes, cannot be discounted. Likewise, prior history and/or knowledge could have influenced the responses of those with more extensive experience in HTA processes. Additionally, the workshop has captured views on only 22 value elements that were selected based on a literature search; it may be that other value elements are also important for pharmaceutical products in Saudi Arabia. In addition, these value elements were not weighted during the voting exercise.

It is important to note that the collected views and perceptions from the capacity building exercise reflected participants’ personal views and do not represent an official position for any participant’s organization or institution. The data collected are likely to reflect diverse views from a range of experts based on their understanding of HTA processes, professional backgrounds and experiences to date. These views should not be considered in a formal process to establish policy. Furthermore, the workshop participants were local experts in Saudi Arabia, and as such the results of the workshop may not be generalizable to other regions/countries.

Conclusions

This study reports on the results from a multi-stakeholder workshop focused on exploring possible important value elements for pharmaceutical products in Saudi Arabia. The workshop was part of a capacity building process which aimed to encourage different stakeholder groups to discuss potential value drivers, and to gauge their understanding (or any challenges) of what the HTA process entails. The majority of participants indicated that future HTA assessments should focus on a pharmaceutical product’s efficacy and safety, in addition to considering special groups and disease characteristics such as communicability and curability. When available, participants specified that it would be valuable to include evidence from local data, such as RWE, to support the HTA recommendation. Future capacity building exercises and open discussions are necessary to further increase the understanding and knowledge of the concept of value for pharmaceutical products in HTA in Saudi Arabia amongst key stakeholder groups.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.