Exploring the value of a Doctor of Philosophy program in Pharmaceutical Outcomes and Policy Research in Saudi Arabia.

BACKGROUND: The need for graduate education in Pharmaceutical Outcomes and Policy Research (POPR) is becoming increasingly apparent worldwide. However, the number of professionals in this field is inadequate in the Middle East. Therefore, this study aimed at gaining insight into the perceived value of a potential Doctor of Philosophy (Ph.D.) program in POPR among different stakeholders in Saudi Arabia.
METHODS: Following the development of a Ph.D. program structure in POPR, a questionnaire was created to explore the perception of its value among decision-makers in different healthcare and governmental institutions. An email with detailed information on the proposed program was sent to 131 identified individuals along with an online link to the questionnaire.
RESULTS: Responses were provided by 107 (81.67%) individuals. The majority of respondents (53.3%) represented large organizations with more than 500 workers; hospitals and academia were the most represented types of institutions. More than 85% of the participants strongly agreed that the program will meet the needs of the healthcare market in Saudi Arabia and that there will be a demand for graduates of the program over the next 5-10 years. However, only 28.04% of the participants declared that they would definitely recommend the program to their colleagues and employees, and 49.53% would consider employing its graduates.
CONCLUSIONS: The obtained results indicate a significant interest among different stakeholders in introducing a Ph.D. program in POPR in Saudi Arabia.

Background

Outcomes research is an interdisciplinary field of studies devoted to the analysis of the relationship between health care interventions and health outcomes (Berger et al., 2003). When applied to pharmaceutical sciences, it includes pharmacoeconomics, pharmacoepidemiology, health economics, public health, patient-centered outcomes, and drug policies (Berger et al., 2003, Slejko et al., 2013). Pharmaceutical outcomes and policy research critically examine the effectiveness of medical services and prescription medications in terms of improving patients’ health status, quality of life, economic cost, and patient-centered outcomes such as access, quality of care, safety, and patient-reported health outcomes (Slejko et al., 2013, Jefford et al., 2005). Pharmacoeconomics involves the assessment of the clinical, economic and humanistic impact of pharmaceutical interventions, with the objective to optimize patient outcomes and allow the most effective allocation of healthcare resources (Berger et al., 2003).

Over the past two decades a growing effort aimed at the incorporation of pharmacoeconomic principles and methods into the educational pharmacy curricula has been witnessed (Rascati et al., 2004). Furthermore, there is an increasing global demand for experts in the field of pharmaceutical outcomes and policy research (Rascati et al., 2004). These trends are driven by several studies which have unequivocally demonstrated the importance of including pharmacoeconomics, pharmacoepidemiology and outcomes research in formulary decision-making, disease management, therapeutic drug monitoring, and the assessment of safety and efficacy of different medications (Reddy et al., 2008, Makhinova and Rascati, 2013, Nwokeji et al., 2007). Pharmacoeconomics also gains importance due to a continuously increasing pressure on the healthcare budgets in most countries (van Oostenbruggen et al., 2005), an issue potentiated in impoverished regions of the world (Desai et al., 2012). The efficient use of limited resources results in a better access to healthcare and pharmaceutical services through the optimization of healthcare costs management (Boncz et al., 2013, Alefan and Rascati, 2017).

Therefore, it is unsurprising that the need for graduate programs in Pharmaceutical Outcomes and Policy Research (POPR) is becoming increasingly apparent. In the United States alone, 32 graduate programs in POPR were active in 2013 (Slejko et al., 2013). However, research and practice in the field of POPR in the Middle East are not of the same caliber of North American or European countries, as evidenced by the shortage of experts in this area (Alefan et al., 2015). One of the primary challenges that prevents the development of POPR in this region is the lack of training and education available to researchers, practitioners, Chief Executive Officers (CEOs) of pharmaceutical companies, and government officials (Alefan et al., 2015). However, the information necessary to propose a path toward achieving these goals is essentially absent. To the best of our knowledge, only one study analyzing data from Egypt has investigated the current pharmacoeconomic education status in the region (Soliman et al., 2013). This research has led to the conclusion that awareness should be raised among the stakeholders in the Middle Eastern countries regarding the need to provide graduate degrees and training in the POPR area. The necessity to train instructors, pharmacists and researchers was recognized as a condition necessary to help Middle Eastern countries establish an integrated scientific community that can start applying pharmacoeconomics to healthcare decision-making (Soliman et al., 2013).

Saudi Arabia is the country with the largest number of pharmacy schools in the Middle East per capita. Over less than two decades, the number of pharmacy colleges has increased dramatically from only one college in 2000 to 27 colleges in 2017 (AlRuthia et al., 2018). However, despite the apparent abundance of pharmacy colleges, a program in the field of POPR is not currently offered to pharmacy students in Saudi Arabia. To overcome this critical deficiency, the availability of graduate programs that encompass subject areas related to POPR, such as pharmacoeconomics, is required. This necessity is further underscored by the fact that the Saudi healthcare system is fragmented and the delivery of quality health services is hindered by many obstacles such as drug shortages, drug waste, inefficient utilization of resources, and frequent lack of competent leadership (Healthcare in the Kingdom of Saudi Arabia, 2018, Alruthia et al., 2018). Establishing a new graduate program that would evaluate different health technologies and propose new policies that address different shortcomings of the current healthcare system may help to achieve the goals of the new economic transformation program. This program, known as the Saudi Vision 2030, aims at improving the quality of healthcare provided to the Saudi citizens and residents as well as ensuring efficient government healthcare spending (Healthcare in the Kingdom of Saudi Arabia, 2018).

Currently, no information is available about the position of stakeholders, such as senior officials in health regulatory bodies and pharmacy colleges, on the merits of establishing a graduate program in POPR in the kingdom. Therefore, the objective of this study was to gain an insight into the perceived value of a potential Doctor of Philosophy (Ph.D.) program in POPR among different stakeholders in Saudi Arabia.

Methods

The proposed Ph.D. Program

The Ph.D. program in Pharmaceutical Outcomes and Policy Research was developed by a committee of 10 faculty members from the Department of Clinical Pharmacy at King Saud University College of Pharmacy in Riyadh, Saudi Arabia. The members of the committee represented diverse backgrounds, corresponding to pharmacoeconomics (four members), pharmacoepidemiology (two members), and pharmacy practice (four members). The applicants for the Ph.D. program will be required to have a Master of Science degree in epidemiology, pharmacoepidemiology, health economics, pharmacoeconomics, or clinical research, granted by a university approved by the Saudi Arabian Ministry of Education. The program consists of six semesters and begins with at least 40 credit hours of courses, including 6 h of elective courses, which cover a wide range of subjects. They include advanced research methods in health outcomes, data analysis and interpretation, applied pharmacoeconomics, pharmacoepidemiology, global pharmaceutical policy, and pharmacovigilance. Thereafter, a comprehensive written exam must be passed before the student can start working solely on his/her dissertation. The completion of six semesters of the program is required for graduation.

Exploring the value of a Ph.D. Program in pharmaceutical outcomes and policy research questionnaire

To explore the perceived value of the proposed Ph.D. program from the perspective of different stakeholders, a questionnaire was developed by five members of the Ph.D. program committee. This subcommittee held several meetings to discuss different themes pertinent to the perceived value of the proposed graduate program in addressing the emerging challenges in the Saudi health sector with regard to efficient utilization of financial resources for health care services, improving patient safety and quality of life, expanding access to health care, and meeting the demand of the labor market in alignment with the kingdom’s economic transformation plan. After four consecutive meetings, the questionnaire was finalized and sent to another subcommittee consisting of five other members of the Ph.D. committee to review the questionnaire and provide any recommendations or suggestions. Only minor changes were suggested and were incorporated in the final version of the questionnaire.

The questionnaire consisted of 16 items. The first six questions cover the characteristics of the participants: gender, education level, institution type, institution size, position held at the institution, and the number of years of relevant experience. The possible choices regarding the education level were Bachelor of Science (B.Sc.), Master of Science (M.Sc.), post-graduate pharmacy residency, and Ph.D. degree. Institution types were categorized as academia, pharmaceutical industry, governmental medical devices and supplies procurement agencies, governmental regulatory bodies, and hospitals. Institution size was determined by the number of employees and included 5–50, 51–100, 101–200, 301–400, 401–500, and more than 500 employees.

The remaining ten questions were designed to determine the value of the potential Ph.D. program in POPR perceived by the various groups of stakeholders. Specifically, the queries were:

Does the program cover diverse domains in POPR?

Will your institution support its employees who wish to join such a graduate program?

Does the proposed program help in creating new tracks within the POPR in Saudi Arabia?

Will the new program help in assessing the different health technologies in Saudi Arabia?

Will the outcomes of the proposed program meet the needs of the Saudi healthcare market?

What is your expectation about the demand for graduates from this proposed program over the next 5–10 years?

Will the program positively contribute to the higher education sector in Saudi Arabia?

Will the outcome of the proposed graduate program result in different policies to address the shortcomings of the current healthcare system in Saudi Arabia?

Would you recommend this program to your colleagues or employees?

Would you consider employing graduates from this proposed program in the foreseeable future?

Study sample

To determine the perceived value of the proposed Ph.D. program, the questionnaire was sent to stakeholders occupying administrative positions in different health and governmental regulatory bodies or having recognized academic credentials in the field POPR. In order to identify stakeholders from different backgrounds, the Saudi Pharmaceutical Society contact database was used. The Saudi Pharmaceutical Society was the only licensed professional society for pharmacists in the kingdom up to 2016 and represents pharmacists from different backgrounds. In addition, the contact database of King Saud University College of Pharmacy Alumni Association was used to identify other potential participants who were not registered in the Saudi Pharmaceutical Society contact database since it was the only college graduating pharmacists in Saudi Arabia up to 2006. The deans and associate deans of the 27 pharmacy colleges in Saudi Arabia were also included in the list. Moreover, the executives of the pharmaceutical industry in Saudi Arabia, who were not registered in the contact databases of the Saudi Pharmaceutical Society or King Saud University College of Pharmacy alumni association were identified. The final list included 131 potential participants from different institutions.

A copy of the program structure, its mission and vision, number of courses and their titles, as well as the proposed admission criteria were sent to all identified persons via email, along with an online link to the questionnaire using Google® forms. If the potential participant did not reply, a reminder was sent one week later. The study took place between November 2016 and January 2017 and was approved by the Institutional Review Board of the College of Medicine at King Saudi University in Riyadh, Saudi Arabia.

Statistical analysis

Descriptive statistics were performed using Chi-square and Fisher exact tests, as appropriate. The results were stratified based on gender, education levels, and institution size. All statistical analyses were conducted using SAS version 9.2 (SAS Institute, Inc., Cary, NC, USA).

Results

Out of 131 individuals who were sent a synopsis of the Ph.D. program and the link to the online questionnaire, 107 (81.67%) provided a response. Their professional characteristics are summarized in Table 1. The majority of respondents were male, and the largest group (43.9%) held a M.Sc. degree. The remaining participants indicated that they hold a B.Sc. (30.8%), Ph.D. (22.4%), or completed post-graduate pharmacy residency (2.8%). Approximately, 25% of respondents had positions in small institutions with 100 or less employees, while the majority (53.3%) represented large organizations with more than 500 workers. Larger groups of participants were associated with hospitals (51.4%), the pharmaceutical industry (28.0%), and academia (13.1%), while 3.7% of respondents worked in government procurement agencies and regulatory bodies. Approximately, 92.5% of the respondents had six to over 20 years of relevant experience. The positions held by the respondents were a reflection of the institutions involved. Over 51% of the participants were directors or assistant directors of hospital pharmacies, 28.0% were chief executive officers (CEOs) or section heads in pharmaceutical companies, and 13.1% were deans or associate deans in academic institutions. The number of the participants who were health economists (5.6%) and heads of drug purchasing departments (1.9%) was small.

Table 1

Professional characteristics of the participants (n = 107).

Characteristic	n (%)
Gender
Male	85 (79.44)
Female	22 (20.56)
Education
Bachelor of Science	33 (30.84)
Master of Science	47 (43.93)
Post-Graduate Pharmacy Residency	3 (2.80)
Doctor of Philosophy (Ph.D.)	24 (22.43)
Institution Size
5–50 Employees	11 (10.28)
51–100 Employees	15 (14.02)
101–200 Employees	10 (9.35)
201–300 Employees	4 (3.74)
301–400 Employees	8 (7.48%)
401–500 Employees	2 (1.87)
>500 Employees	57 (53.27)
Type of Institution
Academia	14 (13.08)
Pharmaceutical Industry	30 (28.04)
Governmental Medical Devices and Supplies Procurement Agencies	4 (3.74)
Governmental Regulatory Bodies	4 (3.74)
Hospitals	55 (51.40)
Years of Experience
1–5	8 (7.48)
6–10	28 (26.17)
11–15	24 (22.43)
16–20	23 (21.50)
>20	24 (22.43)
Position
CEO of a Pharmaceutical Company	3 (2.80)
Section Head in a Pharmaceutical Company	27 (25.23)
Director/Assistant Director of Hospital Pharmacy	55 (51.40)
Dean/Associate Dean	14 (13.08)
Health Economist	6 (5.61)
Head of Medication Purchasing Department	2 (1.87)

A large proportion (82.2%) of the participants agreed with the statement that the proposed Ph.D. program covers a wide range of domains in the field of POPR (Table 2). However, approximately half of them indicated that their institutions would support the employees seeking admission into such graduate programs. Four out of five respondents (79.4%) stated that this Ph.D. program would help in creating different tracks within the field of POPR in Saudi Arabia. The question of whether the proposed program will help in assessing different health technologies indicated the lack of a clear opinion on this subject among the respondents, with 86% of them selecting the “not sure” option. Conversely, there was a general agreement among respondents that the outcomes of the proposed graduate program will meet the needs of the healthcare market in Saudi Arabia, with 86.0% positive responses.

Table 2

Items and responses of the questionnaire.

Item	n (%)
1. Does the program cover diverse domains in POPR?
Yes	88 (82.24)
No	3 (2.80)
Not sure	16 (14.95)
2. Will your institution support its employees who wish to join such a graduate program?
Definitely	30 (28.04)
Probably	29 (27.10)
Possibly	29 (27.10)
Probably No	13 (12.15)
No	6 (5.61)
3. Does the proposed program help in creating new tracks within the POPR in Saudi Arabia?
Yes	85 (79.44)
No	5 (4.67)
Not sure	17 (15.89)
4. Will the new program help in assessing the different health technologies in Saudi Arabia?
Yes	12 (11.21)
No	3 (2.80)
Not sure	92 (85.98)
5. Will the outcomes of the proposed program meet the needs of the Saudi health care market?
Yes	92 (85.98)
No	8 (7.48)
Not sure	7 (6.54)
6. Do you expect that there will be a demand for graduates of this proposed program over the next five to ten years?
Definitely	69 (64.49)
Probably	28 (26.17)
Possibly	8 (7.48)
Probably No	0 (0.00)
No	2 (1.87)
7. Will the program contribute to the higher education sector in Saudi Arabia?
Definitely	92 (85.98)
Probably	6 (5.61)
Possibly	7 (6.54)
Probably No	2 (1.87)
No	0 (0.00)
8. Will the outcome of the proposed graduate program result in different policies to address the shortcomings of the current healthcare system in Saudi Arabia?
Yes	95 (88.79)
No	1 (0.93)
Not sure	11 (10.28)
9. Would you recommend this program to your colleagues or employees?
Definitely	63 (58.88)
Probably	22 (20.56)
Possibly	12 (11.21)
Probably No	6 (5.61)
No	4 (3.74)
10. Would you consider employing graduates from this proposed program in the foreseeable future?
Definitely	53 (49.53)
Probably	28 (26.17)
Possibly	17 (15.89)
Probably No	8 (7.48)
No	1 (0.93)

More than 90% of the participants declared that there would be definitively or probably a demand for graduates of the proposed graduate program over the next 5 to 10 years, with only 2 participants (1.9%) expressing an opposite view (Table 2). Similar responses were obtained to the question of whether the proposed program will have a positive contribution to the higher education sector in Saudi Arabia and whether it will help in proposing different policies to address the shortcomings in the current healthcare system. However, only a minority of those interviewed declared that they would definitely recommend the proposed Ph.D. program to their colleagues and employees (28.04%), and consider employing graduates of this program (49.53%).

The differences in responses to the ten questions regarding the value of the proposed Ph.D. program in POPR based on participants’ gender were also analyzed. As shown in Table 3, the responses of male and female participants were very similar, and no statistically significant difference between genders was found in answers to any of the questions asked. Other variables that did not impact the participants’ responses were the type of the institution represented by the respondents, number of years of experience in the field, and position held in their respective institutions (P > 0.05).

Table 3

Perceived value of the proposed program according to respondent’s gender.

Question and possible answers	Gender		P-value	Total n (%)
	Male n (%)	Female n (%)
1. Does the program cover diverse domains in POPR?
Yes	70 (82.35)	18 (81.82)	0.846	88 (82.24)
No	2 (2.35)	1 (4.55)		3 (2.80)
Not sure	13 (15.29)	3 (13.64)		16 (14.95)
2. Will your institution support its employees who wish to join such a graduate program?
Definitely	22 (25.88)	8 (36.36)	0.441	30 (28.04)
Probably	25 (29.41)	4 (18.18)		29 (27.10)
Possibly	21 (24.71)	8 (36.36)		29 (27.10)
Probably NO	12 (14.12)	1 (4.55)		13 (12.15)
NO	5 (5.88)	1 (4.55)		6 (5.61)
3. Does the proposed program help in creating new tracks within the POPR in Saudi Arabia?
Yes	66 (77.65)	19 (86.36)	0.615	85 (79.44)
No	4 (4.71)	1 (4.55)		5 (4.67)
Not sure	15 (17.65)	2 (9.09)		17 (15.89)
4. Will the new program help in assessing the different health technologies in Saudi Arabia?
Yes	9 (10.59)	3 (13.64)	0.611	12 (11.21)
No	2 (2.35)	1 (4.55)		3 (2.80)
Not sure	74 (87.06)	18 (81.82)		92 (85.98)
5. Will the outcomes of the proposed program meet the needs of the Saudi health care market?
Yes	72 (84.71)	2 0(90.91)	0.778	92 (85.98)
No	7 (8.24)	1 (4.55)		8 (7.48)
Not sure	6 (7.06)	1 (4.55)		7 (6.54)
6. Do you expect that there will be a demand for graduates of this proposed program over the next five to ten years?
Definitely	53 (62.35)	16 (72.73)	0.764	69 (64.49)
Probably	24 (28.24)	4 (18.18)		28 (26.17)
Possibly	6 (7.06)	2 (9.09)		8 (7.48)
Probably No	0 (0.00)	0 (0.00)		0 (0.00)
No	2 (2.35)	0 (0.00)		2 (1.87)
7. Will the program contribute to the higher education sector in Saudi Arabia?
Definitely	74 (87.06)	18 (81.82)	0.534	92 (85.98)
Probably	5 (5.88)	1 (4.55)		6 (5.61)
Possibly	5 (5.88)	2 (9.09)		7 (6.54)
Probably No	1 (1.18)	1 (4.55)		2 (1.87)
No	0 (0.00)	0 (0.00)		0 (0.00)
8. Will the outcome of the proposed graduate program result in different policies to address the shortcomings of the current healthcare system in Saudi Arabia?
Yes	75 (88.24)	20 (90.91)	0.856	95 (88.79)
No	1 (1.18)	0 (0.00)		1 (0.93)
Not sure	9 (10.59)	2 (9.09)		11 (10.28)
9. Would you recommend this program to your colleagues or employees?
Definitely	4 8(56.47)	15(68.18)	0.667	63 (58.88)
Probably	1 9(22.35)	3 (13.64)		22 (20.56)
Possibly	10 (11.76)	2 (9.09)		12 (11.21)
Probably No	4 (4.71)	2 (9.09)		6 (5.61)
No	4 (4.71)	0 (0.00)		4 (3.74)
10. Would you consider employing graduates from this proposed program in the foreseeable future?
Definitely	43 (50.59)	10(45.45)	0.5046	53 (49.53)
Probably	19 (22.35)	9 (40.91)		28 (26.17)
Possibly	15 (17.65)	2 (9.09)		17 (15.89)
Probably No	7 (8.24)	1 (4.55)		8 (7.48)
No	1 (1.18)	0 (0.00)		1 (0.93)

The educational level of the participants affected their decisions to some extent. Respondents with a Ph.D. degree were more likely to agree with the statement that the proposed program will help in assessing different healthcare technologies (P = 0.0102, Table 4). They also tended to answer “definitely” or “yes” more frequently than the other three groups (B.Sc., M.Sc., and post-graduate pharmacy residency), but this difference did not reach statistical significance. The size of the respondents’ institutions also had an impact on their views. This was apparent in answers to the questions of whether the program will help in proposing different policies to address the shortcomings in the current Saudi healthcare system and whether the respondent’s institution would employ graduates of the program. In the latter case, representatives of large companies (>300 employees) were more inclined to provide employment for the recipients of a Ph.D. degree in the POPR field (see Table 5).

Table 4

Value of the proposed program according to respondent’s educational level.

Item	Education				P-value	Total n (%)
	B.Sc. n (%)	M.Sc. n (%)	Residency n (%)	Ph.D. n (%)
1. Does the program cover diverse domains in POPR?
Yes	27 (81.82)	40 (85.11)	2 (66.67)	19 (79.17)	0.2561	88 (82.24)
No	1 (3.03)	0 (0.00)	1 (33.33)	1 (4.17)		3 (2.80)
Not sure	5 (15.15)	7(14.89)	0 (0.00)	4 (16.67)		16 (14.95)
2. Will your institution support its employees who wish to join such a graduate program?
Definitely	11 (33.33)	11 (23.40)	1 (33.33)	7 (66.67)	0.301	30 (28.04)
Probably	10 (30.30)	12 (25.53)	0 (0.00)	7 (66.67)		29 (27.10)
Possibly	8 (24.24)	14 (29.79)	2 (66.67)	5 (20.83)		29 (27.10)
Probably No	4 (12.12)	8 (17.02)	0 (0.00)	1 (4.17)		13 (12.15)
No	0 (0.00)	2 (4.26)	0 (0.00)	4 (16.67)		6 (5.61)
3. Does the proposed program help in creating new tracks within the POPR in Saudi Arabia?
Yes	28 (84.85)	35 (74.47)	2 (66.67)	20 (83.33)	0.2630	85 (79.44)
No	1 (3.03)	2 (4.26)	1 (33.33)	1 (4.17)		5 (4.67)
Not sure	4 (12.12)	10 (21.28)	0 (0.00)	3 (12.50)		17 (15.89)
4. Will the new program help in assessing the different health technologies in Saudi Arabia?
Yes	28 (84.85)	39 (82.98)	2 (66.67)	23 (95.83)	0.012*	92 (85.98)
No	1 (3.03)	0 (0.00)	1 (33.33)	1 (4.17)		3 (2.80)
Not sure	4 (12.12)	8 (17.02)	0 (0.00)	0 (0.00)		12 (11.21)
5. Will the outcomes of the proposed program meet the needs of the Saudi health care market?
Yes	27 (81.82)	42 (89.36)	1 (33.33)	22 (91.67)	0.156
	92 (85.98)
No	4 (12.12)	2 (4.26)	1 (33.33)	1 (4.17)		8 (7.48)
Not sure	2 (6.06)	3 (6.38)	1 (33.33)	1 (4.17)		7 (6.54)
6. Do you expect that there will be a demand for graduates of this proposed program over the next five to ten years?
Definitely	16 (48.48)	35 (74.47)	3 (100.00)	15 (62.50)	0.076	69 (64.49)
Probably	16 (48.48)	7 (14.89)	0 (0.00)	5 (20.83)		28 (26.17)
Possibly	1 (3.03)	4 (8.51)	0 (0.00)	3 (12.50)		8 (7.48)
Probably No	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)		0 (0.00)
No	0 (0.00)	1 (2.13)	0 (0.00)	1 (4.17)		2 (1.87)
7. Will the program contribute to the higher education sector in Saudi Arabia?
Definitely	31 (93.94)	38 (80.85)	2 (66.67)	21 (87.50)	0.103	92 (85.98)
Probably	1 (3.03)	5 (10.64)	0 (0.00)	0 (0.00)		6 (5.61)
Possibly	1 (3.03)	4 (8.51)	0 (0.00)	2 (8.33)		7 (6.54)
Probably No	0 (0.00)	0 (0.00)	1 (33.33)	1 (4.17)		2 (1.87)
No	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)		0 (0.00)
8. Will the outcome of the proposed graduate program result in different policies to address the shortcomings of the current healthcare system in Saudi Arabia?
Yes	29 (87.88)	42 (89.36)	2 (66.67)	22 (91.67)	0.278	95 (88.79)
No	0 (0.00)	0 (0.00)	0 (0.00)	1 (4.17)		1 (0.93)
Not sure	4 (12.12)	5 (10.64)	1 (33.33)	1 (4.17)		11 (10.28)
9. Would you recommend this program to your colleagues or employees?
Definitely	21 (63.64)	27 (57.45)	1 (33.33)	14 (58.33)	0.416	63 (58.88)
Probably	5 (15.15)	11 (23.40)	1 (33.33)	5 (20.83)		22 (20.56)
Possibly	6 (18.18)	5 (10.64)	0 (0.00)	1 (4.17)		12 (11.21)
Probably No	0 (0.00)	3 (6.38)	1 (33.33)	2 (8.33)		6 (5.61)
No	1 (3.03)	1 (2.13)	0 (0.00)	2 (8.33)		4 (3.74)
10. Would you consider employing graduates from this proposed program in the foreseeable future?
Definitely	18 (54.55)	2 1(44.68)	2 (66.67)	12 (50.00)	0.373	53 (49.53)
Probably	7 (21.21)	13 (27.66)	0 (0.00)	8 (33.33)		28 (26.17)
Possibly	4 (12.12)	10 (21.28)	0 (0.00)	3 (12.50)		17 (15.89)
Probably No	4 (12.12)	3 (6.38)	1 (33.33)	0 (0.00)		8 (7.48)
No	0 (0.00)	0 (0.00)	0 (0.00)	1 (4.17)		1 (0.93)

Indicates statistical significance, P < 0.05.

Table 5

Value of the proposed program according to the size of respondent’s institution.

Item	Institution Size							P-value	Total n (%)
	5–50 Employees n (%)	51–100 Employees n (%)	101–200 Employees n (%)	201–300 Employees n (%)	301–400 Employees n (%)	401–500 Employees n (%)	>500 Employees n (%)
1. Does the program cover diverse domains in POPR?
Yes	10 (90.91)	10 (66.67)	10 (100.0)	3 (75.00)	7 (87.50)	2 (100.00)	46 (80.70)	0.099	88 (82.24)
No	1 (9.09)	0 (0.00)	0 (0.00)	1 (25.00)	0 (0.00)	0 (0.00)	1 (1.75)		3 (2.80)
Not sure	0 (0.00)	5 (33.33)	0 (0.00)	0 (0.00)	1 (12.50)	0 (0.00)	10 (17.54)		16 (14.95)
2. Will your institution support its employees who wish to join such a graduate program?
Definitely	1 (9.09)	5 (33.33)	2 (20.00)	1 (25.00)	6 (75.00)	0 (0.00)	15 (26.32)	0.158	30 (28.04)
Probably	3 (27.27)	6 (40.00)	4 (40.00)	0 (0.00)	2 (25.00)	1 (50.00)	13 (22.81)		29 (27.10)
Possibly	4 (36.36)	3 (20.00)	1 (10.00)	1 (25.00)	0 (0.00)	1 (50.00)	19 (33.33)		29 (27.10)
Probably No	3 (27.27)	1 (6.67)	3 (30.00)	1 (25.00)	0 (0.00)	0 (0.00)	5 (8.77)		13 (12.15)
No	0 (0.00)	0 (0.00)	0 (0.00)	1 (25.00)	0 (0.00)	0 (0.00)	5 (8.77)		6 (5.61)
3. Does the proposed program help in creating new tracks within the POPR in Saudi Arabia?
Yes	10 (90.91)	12 (80.00)	7 (70.00)	2 (50.00)	8 (100.00)	2 (100.00)	44 (77.19)	0.513	85 (79.44)
No	0 (0.00)	0 (0.00)	0 (0.00)	1 (25.00)	0 (0.00)	0 (0.00)	4 (7.02)		5 (4.67)
Not sure	1 (9.09)	3 (20.00)	3 (30.00)	1 (25.00)	0 (0.00)	0 (0.00)	9 (15.79)		17 (15.89)
4. Will the new program help in assessing the different health technologies in Saudi Arabia?
Yes	10 (90.91)	13 (86.67)	9 (90.00)	2 (50.00)	8 (100.00)	2 (100.00)	48 (84.21)	0.504	92 (85.98)
No	0 (0.00)	0 (0.00)	0 (0.00)	1 (25.00)	0 (0.00)	0 (0.00)	2 (3.51)		3 (2.80)
Not sure	1 (9.09)	2 (13.33)	1 (10.00)	1 (25.00)	0 (0.00)	0 (0.00)	7 (12.28)		12 (11.21)
5. Will the outcomes of the proposed program meet the needs of the Saudi health care market?
Yes	10 (90.91)	10 (66.67)	10 (100.0)	3 (75.00)	8 (100.00)	2 (100.00)	49 (85.96)	0.484	92 (85.98)
No	1 (9.09)	2 (13.33)	0 (0.00)	1 (25.00)	0 (0.00)	0 (0.00)	4 (7.02)		8(7.48)
Not sure	0 (0.00)	3 (20.00)	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)	4 (7.02)		7 (6.54)
6. Do you expect that there will be a demand for graduates of this proposed program over the next five to ten years?
Definitely	6 (54.55)	8 (53.33)	9 (90.00)	3 (75.00)	5 (62.50)	0 (0.00)	38 (66.67)	0.068	69 (64.49)
Probably	5 (45.45)	6 (40.00)	1 (10.00)	0 (0.00)	2 (25.00)	2 (100.00)	12 (21.05)		28 (26.17)
Possibly	0 (0.00)	1 (6.67)	0 (0.00)	0 (0.00)	1 (12.50)	0 (0.00)	6 (10.53)		8 (7.48)
Probably No	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)		0 (0.00)
No	0 (0.00)	0 (0.00)	0 (0.00)	1 (25.00)	0 (0.00)	0 (0.00)	1 (1.75)		2 (1.87)
7. Will the program contribute to the higher education sector in Saudi Arabia?
Definitely	11 (100.0)	15 (100.0)	9 (90.00)	3 (75.00)	6 (75.00)	2 (100.00)	46 (80.70)	0.284	92 (85.98)
Probably	0 (0.00)	0 (0.00)	1 (10.00)	0(0.00)	1 (12.50)	0 (0.00)	4 (7.02)		6 (5.61)
Possibly	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)	1 (12.50)	0 (0.00)	6 (10.53)		7 (6.54)
Probably No	0 (0.00)	0 (0.00)	0 (0.00)	1 (25.00)	0 (0.00)	0 (0.00)	1 (1.75)		2 (1.87)
No	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)		0 (0.00)
8. Will the outcome of the proposed graduate program result in different policies to address the shortcomings of the current healthcare system in Saudi Arabia?
Yes	10 (90.91)	13 (86.67)	10 (100.0)	3 (75.00)	7 (87.50)	2 (100.00)	50 (87.72)	0.005*	95 (88.79)
No	0 (0.00)	0 (0.00)	0 (0.00)	1 (25.00)	0 (0.00)	0 (0.00)	0 (0.00)		1 (0.93)
Not sure	1 (9.09)	2 (13.33)	0 (0.00)	0 (0.00)	1 (12.50)	0 (0.00)	7 (12.28)		11 (10.28)
9. Would you recommend this program to your colleagues or employees?
Definitely	6 (54.55)	8 (53.33)	5 (50.00)	1 (25.00)	8 (100.00)	2 (100.00)	33 (57.89)	0.482	63 (58.88)
Probably	1 (9.09)	6 (40.00)	3 (30.00)	1 (25.00)	0 (0.00)	0 (0.00)	11 (19.30)		22 (20.56)
Possibly	2 (18.18)	1 (6.67)	2 (20.00)	1 (25.00)	0 (0.00)	0 (0.00)	6 (10.53)		12 (11.21)
Probably No	1 (9.09)	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)	5 (8.77)		6 (5.61)
No	1 (9.09)	0 (0.00)	0 (0.00)	1 (25.00)	0 (0.00)	0 (0.00)	2 (3.51)		4 (3.74)
10. Would you consider employing graduates from this proposed program in the foreseeable future?
Definitely	3 (27.27)	6 (40.00)	3 (30.00)	1 (25.00)	5 (62.50)	2 (100.00)	33 (57.89)	0.004*	53 (49.53)
Probably	4 (36.36)	5 (33.33)	3 (30.00)	1 (25.00)	3 (37.50)	0 (0.00)	12 (21.05)		28 (26.17)
Possibly	2 (18.18)	1 (6.67)	4 (40.00)	1 (25.00)	0 (0.00)	0 (0.00)	9 (15.79)		17 (15.89)
Probably No	2 (18.18)	3 (20.00)	0 (0.00)	0 (0.00)	0 (0.00)	0 (0.00)	3 (5.26)		8 (7.48)
No	0 (0.00)	0 (0.00)	0 (0.00)	1 (25.00)	0 (0.00)	0 (0.00)	0 (0.00)		1 (0.93)

Indicates statistical significance, P < 0.05.

Discussion

Despite the growing worldwide trend of increased demand for professionals skilled in the field of POPR (Rascati et al., 2004, van Oostenbruggen et al., 2005), programs graduating experts in this area are generally absent in the Middle Eastern countries, and particularly in Saudi Arabia. The present study represents an effort to evaluate whether such a graduate program is favorably perceived by different stakeholders in the Saudi healthcare system. The results obtained using an internet-based questionnaire indicate a significant interest among the decision-makers in the pharmaceutical field. There were three questions for which an overwhelmingly positive response was given, with more than 85% of the participants reporting a positive response. These questions concerned the quintessence of the expected benefit of the proposed Ph.D. program. They were related to the program’s potential to meet the needs of the healthcare market, to positively contribute to the higher education sector, and to address the shortcomings of the healthcare system in Saudi Arabia. Despite the strong expression of appreciation for the significance of the proposed graduate program, only one out of four participants (28%) believed that their institutions would “definitely” support their employees who are interested in joining the proposed Ph.D. program. Also, only half of the respondents were confident that they would like to employ graduates of this program in their institutions. Thus, while the significance of this proposed graduate program was clearly recognized, there was a prevailing hesitation toward utilizing its potential benefits. The reasons for these apparently contradictory sentiments cannot be identified by the available data. However, the most probable causes are limited sources of funding, lack of information about the field of POPR among some respondents, or preference for graduates from an international program instead of a domestic one. This reluctance is clearly a problem since in the entire Mediterranean region the demand for experts in pharmacoeconomics markedly exceeds the supply of qualified graduates, and the current education infrastructure suggests that this situation will not improve in the foreseeable future (Alefan et al., 2015).

Pharmacoeconomics is taught as a course in many pharmacy colleges in the Middle East, but the students’ exposure to this subject is minimal and ranges from 12 to 20 classroom hours (Alefan et al., 2015, Nwokeji and Rascati, 2005). It is hoped that the present study will promote discussion on the future of education in this field. Since POPR is unquestionably critical for the evaluation of healthcare outcomes, further research in this area is needed. The relevant questions to be answered are, among others, what is the number of pharmacoeconomists and health outcomes researchers that is actually needed in Saudi Arabia, and what is the best balance of practical versus theoretical training, and whether the Saudi universities have the capacity to run such a program.

One of the indicators of growing interest in the POPR field among decision makers is the high response rate to the distributed survey, which markedly exceeded the rates noted in previous studies on different aspects of pharmacoeconomics education (Alefan et al., 2015, Nwokeji and Rascati, 2005). Although the collected information unequivocally indicates a high level of enthusiasm for the development of a Ph.D. program in the PORP field, several limitations of this study have to be acknowledged. First, no definitions of the technical terms used in the description of the suggested program were given because basic knowledge of outcomes research, pharmacoeconomics and pharmaceutical policy among respondents was assumed. Additionally, no effort was made to determine the level of understanding of pharmacoeconomics by the participants. Furthermore, the survey collected insufficient background information about the participants’ educational exposure such as courses taken, or seminars attended in subject areas relevant to POPR. Moreover, due to the exploratory nature of the study, there was a lack of extensive validity and reliability testing of the content of the questionnaire. Finally, the study sample did not include stakeholders in private healthcare institutions. However, the participation of private enterprises in the Saudi healthcare market is still limited compared to the governmental sector, and their absence should not significantly affect the overall outcome of the present study.

Conclusions

An unmet need for professionals trained in the area of POPR is currently present in Saudi Arabia. The findings of this study indicate a high level of interest among different stakeholders in the Saudi healthcare system in having an academic program that would provide post-graduate education in the area of POPR. While the enthusiasm of the respondents for the proposed graduate program was very strong, the willingness to support employees interested in joining such a program, or to employ the graduates of the program was less evident. Finally, efforts to promote the development of different academic programs to ensure training of a sufficient number of experts in the field of POPR in Saudi Arabia should be encouraged.

Declarations

Ethics approval and consent to participate

This research has been reviewed and approved by the research ethics committee of the college of medicine at King Saud University in Riyadh, Saudi Arabia. The approval ID number is (E-18-2857). Responders can only access the online survey after reading the invitation letter that was sent via email and clicking on an online link to accept the invitation to participate in the study.

Consent to publish

Participants were informed of the possibility that the anonymous data gathered in the online survey will be published in future research. By choosing to complete the survey, participants gave their consent to the possible publication of the gathered data.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Declaration of Competing Interest

The authors declare that they have no competing interests.