State of health economic evaluation research in Saudi Arabia: a review.

BACKGROUND: If evaluation of economic evidence is to be used increasingly in Saudi Arabia, a review of the published literature would be useful to inform policy decision-makers of the current state of research and plan future research agendas. The purpose of this paper is to provide a critical review of the state of health economic evaluation research within the Saudi context with regard to the number, characteristics, and quality of published articles.
METHODS: A literature search was conducted on May 8, 2011 to identify health economic articles pertaining to Saudi Arabia in the PubMed, Embase, and EconLit databases, using the following terms alone or in combination: "cost*", "economics", "health economics", "cost-effectiveness", "cost-benefit", "cost minimization", "cost utility analysis", and "Saudi". Reference lists of the articles identified were also searched for further articles. The tables of contents of the Saudi Pharmaceutical Journal and the Saudi Medical Journal were reviewed for the previous 5 years.
RESULTS: The search identified 535 citations. Based on a reading of abstracts and titles, 477 papers were excluded. Upon reviewing the full text of the remaining 58 papers, 43 were excluded. Fifteen papers were included. Ten were categorized as full economic evaluations and five as partial economic evaluations. These articles were published between 1997 and 2010. The majority of the studies identified did not clearly state the perspective of their evaluation. There are many concerns about the methods used to collect outcome and costs data. Only one study used some sort of sensitivity analysis to assess the effects of uncertainty on the robustness of its conclusions.
CONCLUSION: This review highlights major flaws in the design, analysis, and reporting of the identified economic analyses. Such deficiencies mean that the local economic evidence available to decision-makers is not very useful. Thus, building research capability in health economics is warranted.

Introduction

In Saudi Arabia, health care services are provided primarily by the Ministry of Health through a network of 2037 health care centers and a broad base of 244 general and specialist hospitals.1 Other governmental agencies, such as the Ministry of Defense and Aviation, the Ministry of the Interior, the Saudi Arabian National Guard, and the Saudi Arabian Oil Company, finance and deliver primary, secondary, and tertiary care to a defined population, usually employees and their dependants. The Saudi government also finances and provides care on a referral basis in specialized tertiary care hospitals, such as King Faisal Specialist Hospital and Research Center. Services offered by public hospitals are free of charge for all eligible citizens. The private sector also contributes to the delivery of health care services, especially in cities and large towns, with a total of 125 hospitals (11,833 beds), 2218 dispensaries, and clinics.1 In addition, Saudi working for the private sector and expatriates are eligible for a comprehensive package of health insurance benefits. The Council of Cooperative Health Insurance, an independent government body, regulates and supervises a health insurance strategy for the Saudi health care market.2

The Ministry of Health is responsible for managing, planning, and formulating health policies, and supervising health programs, as well as monitoring health services in the private sector. The Ministry of Health was historically the regulatory authority responsible for licensing pharmaceutical products, medical devices, and manufacturing facilities, but the Saudi Food and Drug Authority took over this function in July 2009.

In 2012, the government has allocated SAR 86 billion towards health and social affairs, a 26% increase on the previous year’s budget.3 Despite that, the steady increase in health care costs because of technological advances, the growing number of people with chronic diseases, and high demand resulting from free services, means that Saudi decision-makers will struggle to make choices concerning allocation of health care resources.

Economic evaluation is a technique that has been developed by economists to assist decision-making when choices have to be made between several courses of action. By definition, economic evaluation is a comparative analysis of alternative courses of action in terms of their costs and consequences.4 It addresses the question of whether something is worth doing when compared with other possible uses of the same resources to ensure that efficiency has been attained or approached. Inefficiency exists when resources could be reallocated in a way that would increase the health outcomes produced.

Many countries have started to use economic evidence to support decisions on licensing, pricing, reimbursement, or addition to the formulary.5 In Saudi Arabia, it is not mandatory to submit evidence of economic evaluation to support licensing decisions; however, data will be considered if submitted. The new Saudi Food and Drug Authority guidelines for pricing of pharmaceuticals indicate that pharmacoeconomic evidence will be utilized to supplement pricing decisions.6 In a survey of 48 Saudi Pharmacy and Therapeutics Committee members, two thirds of the respondents stated that they used pharmacoeconomic evaluation in their formulary decision-making processes and 80% of respondents stated that pharmacoeconomics should be used as a decision-making tool, as in the rest of the world.7

If evaluation of economic evidence is to be used increasingly in Saudi Arabia, a review of the published literature would be useful to inform policy decision-makers on the current state of research and to plan future research agendas. The purpose of this paper is to provide a critical review of the state of health economic evaluation within the Saudi context with regard to the number, characteristics, and quality of published articles.

Materials and methods

Literature search strategy

A literature search was undertaken on May 8, 2011 to identify papers on health economics pertaining to Saudi Arabia in the PubMed, Embase, EconLit databases, using the following terms alone and in combination: “cost*”, “economics”, “health economics”, “cost-effectiveness”, “cost-benefit”, “cost minimization”, “cost utility analysis”, and “Saudi”. The databases were searched without language restriction or publication year limits (ie, from the start of the databases). The tables of contents for the Saudi Pharmaceutical Journal and the Saudi Medical Journal from May 2007 to May 2011 were reviewed by the author. The search was restricted to the last 5 years because the task is time-consuming. Google scholar was also searched. Reference lists of the articles were also searched for additional articles.

Literature selection criteria

Articles were excluded if there was no statement or word in the title, abstract, or keywords that indicated that an economic (including cost) analysis was conducted. Articles were also excluded if they were not original economic evaluations (eg, if the paper was a narrative review on cost-effectiveness), not pertaining to Saudi Arabia, not published in a fully peer-reviewed journal (eg, conference proceeding abstracts), or did not address a health-related topic. Titles and abstracts were screened by a group of three PharmD students following a 15-week course of pharmacoeconomics at the College of Pharmacy, King Saud University. The author explained the inclusion and exclusion criteria to the students. Two students independently screened the titles and abstracts of identified citations for potential eligibility using a standardized screening guide. A random selection of title and abstract was independently reviewed by the author to ensure the accuracy of the inclusion and exclusion process. The citations judged potentially eligible by at least one student were retrieved in full text. The author then read the full texts of potential papers to confirm that they satisfied the inclusion criteria.

Synthesis and reporting

Depending on whether both costs and consequences had been considered and whether a comparison with alternative treatment was made, the studies included were classified by the author into two categories, ie, partial economic evaluations (cost outcome description, cost comparison) and full economic evaluations (cost-effective analysis, cost-benefit analysis, cost-utility analysis, cost-minimization analysis). Data were recorded about the author, year of the study, sample, methods, sample size, study focus, and main findings. The methodological quality was assessed against published criteria.4

Results

In total, the comprehensive search identified 535 citations. Based on a reading of abstracts and titles, 477 papers were excluded. The remaining 58 articles were retrieved in full text and reviewed by the author. Upon reviewing the 58 articles, 43 were excluded (Figure 1). No additional references were identified during searching of bibliographies. Searching the tables of contents for the Saudi Pharmaceutical Journal and the Saudi Medical Journal and a Google Scholar search did not identify any additional citations. Ten studies8–17 described cost and outcomes for two interventions or more and were categorized as full economic evaluations. Five studies18–22 were considered to be partial economic evaluations. The earliest study was published in 199722 and the latest was published in 2010.10 Four studies were published in Saudi journals and 11 were published in non-Saudi journals. Only one study was published in a specialized economic journal.10

Figure 1

Flow diagram of literature selection for systematic review.

Characteristics of full economic evaluations

A description of the main characteristics for each of the papers included according to year of publication is provided in Table 1. In two papers,8,15 the aim did not contain any reference to measurement of cost, cost-effectiveness, or cost-benefit. In the remaining papers, the aim was to assess cost-effectiveness (n = 4),9,13,14,16 cost-benefit (n = 1),12 or compare costs with outcomes (n = 3).10,11,17 Eight studies compared two alternatives and one compared three alternatives. Four compared intervention using a “do nothing” strategy. The articles included had addressed a wide number of intervention areas. Seven studies were on pharmaceuticals, two were on surgery, one was on a mixture of interventions for intensive care patients, and one was on diagnostic procedures.

Table 1

Characteristics of full economic evaluations (n = 10)

Reference	Population (n)	Interventions	Effectiveness measure	Source of effectiveness	Cost categories
Al-Tawfiq and Abed8	ICU patients	IHI care measure for patients on ventilation versus no IHI measures	VAP rate	Pre and post trial	Length of hospital stay
Sabry et al9	ESRD hemodialysis patients (23)	UFH versus tinzaparin during dialysis	Clotting rate	Prospective crossover trial	Medications
Ali et al10	Type 2 diabetes patients, hypothetical cohort	Biphasic insulin aspart 30 versus human insulin	QALY	Prospective controlled trial	Medications, screening, complications management
Mammo et al11	Thrombophilia (2111) blood samples	Light cycler versus PCR diagnostic test	Test conformity and reproducibility	Prospective trial	Reagents, consumables, equipment, personal, patient
Abbas et al12	Workers from industry (2400)	Influenza vaccination versus no vaccination	Influenza-like illnesses	Prospective trial	Vaccine, personal, productivity loss
Pejaver et al13	Neonatal with RDS (145)	Surfactant replacement therapy versus no surfactant	Survival rate	Prospective trial	Surfactant, personal, equipment, length of hospital stay
Qari et al14	Thyrotoxicosis (100)	Medical versus surgery versus radioactive iodine	Remission rate	Retrospective	Not clear
Jawad et al15	Cholelithiasis in children (11)	Laparoscopic cholecystectomy versus open cholecystectomy	Mortality and major postoperative complications	Pre- and post-trial	Surgery, length of hospital stay
Al Umran and Yaseen16	Neonates with HMD (83)	Surfactant replacement therapy versus no surfactant	Survival rate	Pre- and post-trial	Length of hospital stay
Kubeyinje17	Herpes zoster	Oral acyclovir	Zoster-associated pain and medication complications	Prospective trial	Medications

Abbreviations: IHI, Institute for Healthcare Improvement bundle; ICU, intensive care unit; VAP, ventilator-associated pneumonia; ESRF, end-stage renal failure; UFH, unfractionated heparin; QALY, quality-adjusted life years; PCR, polymerase chain reaction; RDS, respiratory distress syndrome; HMD, hyaline membrane disease.

As Table 1 shows, six of the included studies collected effectiveness measures using a prospective design,9–13,17 three using pre-intervention and post-intervention design,8,15,16 and one using a retrospective design.14 Only one study calculated a sample size.12 The study periods were one year or less (n = 5),8,9,11,12,17 two years (n = 2),15,16 six years (n = 1),13 and eight years (n = 1).14 One study projected outcomes and costs over 40 years using a Markov modeling technique.10 The time horizon of the study or the follow-up period was not clear in some studies, but appeared to be duration of hospital stay. Most of the studies measured outcomes using natural units. The main outcome measures reported were intermediate disease-specific outcomes (n = 6), survival rate (n = 3), and quality-adjusted life years (n = 1).

The costs of pharmaceuticals (n = 5), length of stay (n = 4), and instruments and consumables (n = 2), along with personal (n = 3) and productivity costs (n = 1) were considered. The currency used for cost valuation included US dollars (n = 6 )8,9,11,12,16,17 and local currency (n = 4).10,13–15 In seven papers, one intervention was dominant,8,10–14,16 ie, cost less and achieved better outcomes, in two studies one intervention was cheaper with equivalent outcomes,15,17 and one study found one intervention to be more expensive but with a better outcome.9

Characteristics of partial economic evaluations

Two studies were medication utilization reviews and reported the associated total costs of the medications concerned18,19 (Table 2). One study presented costs and consequences for a surgical intervention.20 One study presented costs savings associated with reuse of a dialyzer.21 Another study reported the usefulness of surveys in screening for varicella immunity and presented the cost savings achieved by vaccination of individuals at risk.22

Table 2

Characteristics of partial economic evaluations (n = 5)

Author	Population, sample size	Interventions	Design	Costs categories
Alsultan et al18	Patients using PPI (225)	Intravenous PPI	Retrospective	Intravenous PPI
Alangari et al19	Patients using IVIG (305)	IVIG	Retrospective	IVIG
Jawadi and Abdul-Samad20	Femoral fracture in children (178)	Intramedullary K wire	Retrospective	K wire
Almuneef et al22	Health care workers (2047)	Varicella zoster virus immunity surveillance	Survey	Antibody screening tests and vaccine
Mitwali et al21	Hemodialysis patients (10)	Reuse of hemodialyzer	Prospective	dialyzer, consumables

Abbreviations: PPI, proton pump inhibitors; IVIG, intravenous immunoglobulin; K wire, Kirschner wire.

As shown in Table 2, the studies collected costing data using a retrospective study design (n = 3)18–20 or a prospective design (n = 1),21 and in one study the design was not clear.22 Four studies collected data over a period of one year or less,18–20,22 and one study collected data for more than one year.20 The costs of medication (n = 3) and instruments and consumables (n = 2) were considered.

Quality of full economic evaluations

Table 3 shows the extent to which the studies included meet the recommendations for good reporting of economic evaluations. The perspective taken was specified explicitly in only one paper,10 but it appears to have been that of the hospital for most of the included studies. However, there were some studies which also included cost components, like the costs borne by patients themselves11 and loss of productivity costs.12

Table 3

Quality of full economic evaluations (n = 10)

	Al-Tawfiq and Abed8	Sabry et al9	Ali et al10	Mammo et al11	Abbas et al12	Qari et al14	Pejaver et al13	Jawad et al15	Al Umran and Yaseen16	Kubeyinje17
Perspective clearly stated	No	No	Yes	No	No	No	No	No	No	No
Time horizon clearly stated	NC	Yes	Yes	Yes	Yes	NC	NC	Yes	Yes	Yes
Analytical technique(s) clearly stated	No	Yes	Yes	No	Yes	Yes	Yes	No	Yes	No
Competitive alternatives clearly described	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes
Sources of outcome data explained	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Primary outcome measure(s) stated	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Main outcomes presented with statistical measures of dispersion	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes	Yes
Methods for estimation of resource quantities described	Yes	No	Yes	No	Yes	No	Yes	No	Yes	No
Resources used reported in physical units	Yes	No	Yes	No	Yes	No	Yes	No	Yes	No
Quantities of resources presented separately from prices	Yes	No	Yes	No	Yes	No	Yes	No	Yes	No
Total costs reported with their statistical measures of dispersion	No	Yes	Yes	No	Yes	No	No	Yes	No	No
Sensitivity analysis performed	No	No	Yes	No	No	No	No	No	No	No
Incremental analysis reported	No	No	Yes	No	No	No	No	No	No	No

Abbreviation: NC, not clear.

In all the papers identified, data on effectiveness came from a single clinical study. In many papers, the calculation of cost components is not clearly described. Only two papers described methods for estimation of resource use. The sources of unit prices was only reported by three studies, ie, from the literature,8 expert opinion,10 and hospital data.16 Four studies reported quantities of resources (unit costs) separately from prices. Two papers presented cost estimates in their discussion section.14,16 One study stated clearly the base year of the cost data. Discounted costs were used in one study by Ali et al, who reported costs and outcomes using a 3% discount rate and calculated the incremental cost-effectiveness ratio by calculating the incremental cost per quality-adjusted life years.10 This study used utility scores for quality-adjusted life years from other countries and did not attempt to estimate these in the Saudi context. Only one study applied sensitivity analyses, specifically one-way sensitivity analysis.10

Quality of partial economic evaluation

Again, the perspective taken was not explicitly specified in any of the reviewed papers, although it appears to have been that of the hospital. As with full economic evaluation, the quantities of resources used, prices (unit costs), and sources of prices were not reported by most of the studies. The costs of medications (n = 3), instruments and consumables (n = 2) were considered. Two papers presented cost estimates in their discussion section.20,22

Discussion

Characteristics of economic evaluation in Saudi Arabia

Compared with the number of economic evaluations found for other developing countries, such as Bangladesh23 (n = 12), Nigeria24 (n = 44), Zimbabwe25 (n = 26), Thailand26 (n = 41), and Korea27 (n = 45), Saudi seems to be lagging behind in conducting health-related economic evaluations. An important observation is that the economic evaluations identified by this research are extremely heterogeneous and assess an intervention after its diffusion into practice. This suggests that health economics assessment is used on an ad hoc basis rather than as a systematic approach to compare alternatives and make decisions that maximize efficiency.

Quality of economic evaluation

There were many flaws in the design, analysis, and reporting of the economic evaluations identified. The perspective of an economic evaluation is an important issue dictating which outcomes and costs should be measured, but the perspective taken was reported by only one study.10 Consequently, the author could not assess if all important and relevant outcomes and costs for each alternative were identified.

A serious and common methodological pitfall was found, ie, poor quality of effectiveness data. None of the studies based evidence of effectiveness on the “gold standard” of randomized, clinical trials. Instead, the effectiveness data came mainly from small, single, retrospective, or non-randomized prospective studies. This may introduce bias which weakens the conclusions of any economic evaluation. Also, many papers used intermediate disease-specific measures of benefit. Intermediate outputs are appropriate only to the extent that a valid link can be established between these and a final health output.5 Furthermore, these measures do not allow for meaningful comparison of cost-effectiveness across disease areas. Only one study used quality-adjusted life years as an outcome measure. The use of quality-adjusted life years is recommended not only because it enables cross- disease comparisons to be made, but also because it captures the impact of disease on a patient’s quality of life.

Another shortcoming of the existing studies is failure to describe clearly the methods used to collect and evaluate data on use of resources. The reason for inclusion or exclusion of specific resources was not explained. Moreover, details on type and quantities of resources used were not provided, which undermines the external relevance of the results of these evaluations. Unit prices were not presented separately from quantities of resources, which limits the possibility of replicating the analysis in other settings. The date to which the prices referred was not reported, and again this limits the reproducibility of the results.

Another serious methodological pitfall was inadequate handling of uncertainty around benefits and cost estimates by sensitivity analysis. In sensitivity analysis, uncertain key parameters are varied to assess the impact of uncertainty on the robustness of any conclusions. Given the poor quality of evidence for estimating clinical effects, this could seriously undermine confidence in the findings of these evaluations and their ability to inform decisions concerning allocation of health care resources.

Poor-quality economic evaluation studies are not unique to Saudi Arabia. Previous studies have reported that developing countries23–27 lack sound economic evaluation data. Also, reviews of published economic evaluations from Europe and the US28–32 point to a number of shortcomings in the published literature. Formal and informal health economic guidelines have been issued in many countries to standardize and improve the quality of economic evaluation in health care.33

Future recommendations

The findings of this review show that building an economic evaluation research capacity in Saudi Arabia is warranted. The author makes several recommendations. First, there is a need to establish a national agency or research institute to provide the infrastructure required to support and nurture health economic evaluation research. It could, for example, advise a process and criteria for priority setting of future economic evaluations in Saudi Arabia. It could also commission funds and facilitate procedures for synthesizing and disseminating the results of economic evaluation. This agency needs to build on the organizational and methodological experience of established health technology assessment agencies, such as the Australian Pharmaceutical Benefits Advisory Committee, the Canadian Agency for Drugs and Technologies in Health, and the National Institute for Health and Clinical Excellence in England and Wales. It would also be necessary to educate potential users about methods of economic evaluation, interpretation of economic evidence, and appraisal of cost-effectiveness claims. Efforts towards this end include expanding the availability of short-term and long-term courses on economic evaluation for health care professionals and policy-makers. Investment in teaching of economic evaluation in health care curricula is also required.

Limitations

This study may suffer from some limitations. It is possible that some published studies were inadvertently missed or omitted. The review only includes studies incorporated in the databases searched. This may mean that unpublished data, such as in government reports, pharmaceutical company reports, and academic theses, were not identified by the literature search. Furthermore, the inclusion of only published articles may have introduced publication bias, because studies with positive results are more likely to be published than studies with negative findings. In addition, the methods sections of many studies did not clearly describe what was done, making it difficult to categorize them, and other readers may categorize them differently.

Conclusion

This review indicates that there are major flaws in the design, analysis, and reporting of economic analyses performed in the Saudi health care setting. Such deficiencies mean that the findings of this evaluation may not be very useful in informing decisions on health care resource allocation. Thus, building research capacity in health economics is warranted in Saudi Arabia.

Table 4

Quality of partial economic evaluations (n = 5)

	Alsultan et al18	Alangari et al19	Jawadi and Abdul-Samad20	Almuneef et al22	Mitwali et al21
Perspective clearly stated	No	No	Yes	No	No
Time horizon clearly stated	NC	NC	NC	NC	Yes
Methods for the estimation of resources quantities described	Yes	Yes	Yes	No	Yes
Resources used reported in physical units	No	Yes	Yes	Yes	No
Quantities of resources presented separately from prices	No	No	Yes	Yes	No
Total costs reported with their statistical measures of dispersion	No	No	No	No	No
Sensitivity analysis performed	No	No	No	No	No

Abbreviation: NC, not clear.