Foreword: Health Economic Evaluations in Low- and Middle-income Countries: Methodological Issues and Challenges for Priority Setting.

Background

In the last two decades, both the production and use of economic evaluations for priority setting in low‐income and middle‐income countries (LMICs) have steadily increased (Sixty‐Seventh World Health Assembly, 2014, 28th Pan American Sanitary Conference, 2012, Resolution of the WHO Regional Committee for South‐East Asia, 2013). More than 230 economic evaluations set in LMICs are now published each year (Pitt et al., 2016). The dissemination of the Bill and Melinda Gates Foundation reference case (NICE International, 2015) is helping to harmonise standards for economic evaluations, particularly as increasing numbers of research groups enter the field of global health. As highlighted by the theme of this year's Prince Mahidol Award Conference in Bangkok, “Priority setting for universal health coverage” (UHC), ensuring that the world's population receives the effective health services they need without financial hardship requires explicit priority setting processes. The publication of this supplement therefore provides an opportune moment to reflect on the current state of the art of economic evaluation in LMICs and to propose a future agenda for research and action.

None of the issues raised in this supplement are unique to LMICs. Rather, the articles reflect challenges for the design, conduct and use of economic evaluations, which may be more acute in LMICs, but which may also resonate with researchers and policy makers in high‐income countries (HICs). A central challenge for those conducting and using economic evaluations in many LMICs is the scarcity, quality and accessibility of data, often attributable to the absence of routine cost accounting systems and limited patient‐information systems. Additional challenges stem from other health‐system constraints, including substantial human resource shortages, financing arrangements which result in substantial out‐of‐pocket expenditure, and fragmented and sometimes weak governance. The complex burden of disease, including both a substantial continuing burden of infectious diseases and an equally large and growing burden of non‐communicable diseases, necessitates sophisticated analytical techniques and models; however, health economic research capacity and funding are very limited in most LMICs. Finally, complex and informal funding processes compound the challenges for researchers seeking to increase the use of economic evaluations in priority setting LMICs.

With 62 authors based in 32 institutions in 13 countries, the 12 research articles presented in this special issue have emerged from a collaborative process reflecting a substantial depth and breadth of experience of conducting and using economic evaluations for policy in LMICs. Here, we provide an overview of the supplement and consider the future of economic evaluations in LMICs.

The Special Issue Articles

We begin the supplement with two articles comparing recent economic evaluations across low‐income, middle‐income and high‐income countries. Pitt and colleagues drew on 14 literature databases to create a single, comprehensive database comprising 2844 full health economic evaluations published over a recent 28‐month period (Pitt et al., 2016). The authors characterise this literature in a bibliometric analysis and find vastly more economic evaluations set in HICs than in LMICs, and even wider disparities in authorship. They also show that nearly half of studies of low‐income and lower‐middle‐income countries failed to include an author based in those income groups and that the distribution of economic evaluations across health areas correlates more closely with disease burden in HICs than in LMICs. Griffiths and colleagues use the same database to examine differences across country income groups in 10 methodological areas (Griffiths et al., 2016). They find that while context drives some of the differences identified, such as the types of models used, other differences appear less justified. They recommend greater collaboration between researchers across settings to improve methods in all settings. In particular, in HICs, they recommend greater scrutiny of the widespread use of reference costs and the adoption of a government health sector perspective. In LMICs, they highlight the need for further promotion of routine use of probabilistic sensitivity analysis, modelling where necessary, and statistical analysis of cost data, as well as further research into quality‐of‐life measurement.

Where routine cost data are unavailable, economic evaluations in LMICs require extensive primary cost data collection; three articles therefore focus on costing methods. Sweeney and colleagues examine methods to estimate the costs incurred by patients. While a number of poverty impact metrics exist, the authors identify methodological challenges in collecting data for intervention studies, and address these challenges by developing a framework for planning and reporting data collection as part of economic evaluations (Sweeney et al., 2016). Cunnama and colleagues compare approaches to estimate provider costs. They find that top‐down and bottom‐up approaches produce substantially different estimates of the incremental costs of alternative tuberculosis diagnostic methods in South Africa. They recommend that researchers clearly delineate their use of these two methods, rather than combine them, and examine existing capacity when estimating the additional costs an intervention may require (Cunnama et al., 2016). Economic evaluations commonly also assume that unit costs are constant, which is inconsistent with economic theory. Lépine and colleagues provide an example of estimating cost functions in LMICs. They examine a non‐governmental public health programme to prevent HIV using data on 138 non‐governmental organisations over 4 years in India. Their findings of substantial economies of scale and a number of other determinants of unit costs support the case for using cost functions when conducting economic evaluations of interventions that need to be scaled up (Lépine et al., 2016).

The choice of outcome metric is also critically important in economic evaluation. Disability‐adjusted life years (DALYs) and natural units remain the most common outcome metrics in low‐income countries. Quality‐adjusted life years (QALYs) are more frequently used than DALYs in upper‐middle‐income countries; however, nearly half of UMIC studies measure outcomes in natural units (Pitt et al., 2016). When assessing the outcomes of public health interventions, however, all of these conventional measures of health may be too narrow, omit important dimensions of programme effects and therefore undervalue the interventions (Greco et al., 2016). Two broader measures of quality of life, capabilities and subjective well‐being, have received widespread interest but have not been widely used to date. Greco and colleagues discuss the methodological challenges in applying these more holistic outcome measures in economic evaluations in LMICs using examples from Malawi and Thailand.

Three articles address specific analytical challenges in economic evaluation, notably health system constraints, predicting real‐world use of new interventions, and accounting for the transmission dynamics of infectious diseases. Vassall and colleagues use case studies of malaria and tuberculosis diagnostics to inform a conceptual framework of the influence of demand and supply on the cost‐effectiveness of new technologies and recommend that supply and demand constraints be considered from the earliest stage in economic evaluations of a new technology (Vassall et al., 2016). The use of discrete choice experiments in economic evaluations is demonstrated by Terris‐Prestholt and colleagues, who apply this technique to improve the parameterization of uptake and substitution in early economic evaluations of new interventions (Terris‐Prestholt et al., 2016). Drake and colleagues review methods used in economic evaluations of infectious disease interventions in LMICs which use dynamic transmission models, present a summary of the state of the art of this crossover discipline spanning mathematical modelling and economic evaluation, and recommend specific improvements in analysis and reporting of economic evaluations employing dynamic transmission models (Drake et al., 2016).

Finally, three articles consider the use of economic evaluations in priority setting. Wiseman and colleagues present a systematic review of methodological frameworks for priority setting, which incorporate economic evaluation evidence in low‐income and lower‐middle‐income countries (Wiseman et al., 2016). They find a number of frameworks were used, most commonly multi‐criteria decision analysis and generalised cost‐effectiveness analysis. The authors argue that frameworks could be enhanced by identifying areas for the redeployment of resources, rather than only investment of new resources; reflecting health system constraints realistically; and developing local capacity to set priorities. They also advocate for greater transparency about priority setting approaches in global institutions, whose decisions determine substantial proportions of expenditure in many low‐income and lower‐middle‐income countries. The experience of setting priorities in Thailand, an upper‐middle‐income country where formal stakeholder consultations and health technology assessments have been institutionalised in decision‐making processes, is described by Teerawattannon and colleagues. The authors examine the case of defining a population‐based health screening package for 12 leading health problems as part of the country's pursuit of universal health coverage (Teerawattananon et al., 2016). Theirs is an important example of how to incorporate local stakeholders’ preferences into a process that recommends both disinvestment and investment and impacts decision‐making. Kaló and colleagues close the supplement by describing the variation in models and maturity of health technology assessment (HTA) in countries of Central and Eastern Europe. They develop a scorecard for national stakeholders to identify explicitly the present and desired future status of HTA in their country. In doing so, they highlight the importance of addressing both the supply of HTA through research capacity development and the demand for HTA in the decision‐making process (Kaló et al., 2016).

The Future of Economic Evaluations in LMICs

While the work in this supplement touches on a wide range of current issues in economic evaluations in LMICs, the field is rapidly evolving. To date, developments in economic evaluation methodology in HICs and LMICs have sometimes evolved in parallel, with research communities and methods focusing on challenges assumed to be specific to each setting (Griffiths et al., 2016). However, increased collaboration and mutual learning between HICs and LMICs offer benefits for all. For example, researchers working in HIC settings may find approaches developed in LMICs to incorporate health system capacity constraints increasingly relevant as greater demands are placed on HIC health systems. The extensive experience and methods developed to estimate costs in LMICs may also be useful in supplementing and improving cost estimates derived through administrative databases. The challenges of promoting equity; integrating health and social policy, underpinned by a growing understanding of the importance of social determinants of health, and integrating evaluations across human and animal health and food systems are developing areas of mutual interest to those working in LMICs and HICs. In addition, the growing burden of non‐communicable diseases in LMICs will require greater focus on morbidity outcomes and Markov models (Griffiths et al., 2016), and LMIC researchers can draw on the extensive HIC experience in these areas. Finally, recent publications by both HIC and LMIC research communities have highlighted the common interest in developing appropriate cost‐effectiveness thresholds, which accurately reflect budget constraints and opportunity costs.

Nonetheless, economic evaluation in LMICs will continue to present several distinctive challenges. First, increased interest from global funders who are concerned with assessing value for money across a range of countries is driving demand for model‐based, multi‐country evaluations, which may encompass dozens of countries. Novel methods are required to integrate heterogeneity in epidemiology, demography, and unit costs, and supply and demand constraints across diverse settings in an expedient and informative way (Vassall et al., 2016). Second, the issue of data scarcity will remain for some time ahead. Considerable investment in large‐scale costing studies and further analysis of cost functions will be required to inform assessments of the costs of scaling up interventions and the generalizability of unit costs across settings. Yet, large‐scale cost surveys remain expensive, so further work is required to improve the efficiency and accuracy of cost‐estimation methods.

Further methodological development and investment are also required to support economic evaluations to inform national‐level decision‐making in LMICs. In addition to efforts to develop routine health management information systems, including both cost and health outcomes data, further research is required into the generation and use of evidence for economic evaluations, in particular guidance on data collection alongside trials and in “real‐world” implementation. The application and further development of methods to assess the transferability of economic evaluation findings from other countries and to adapt them to the local context could make a substantial contribution to increase the use of economic evaluation results.

This methodological effort needs to be accompanied by a substantial expansion in the capacity to produce economic evaluations in LMICs. To conduct a sufficient number of economic evaluations to inform priorities meaningfully across the range of new interventions, far more investment in training of researchers is needed, particularly at Master's degree level, in study design, data collection and analysis for economic evaluations with a focus on the challenges most relevant to LMICs. On‐the‐job training and mentoring of economic evaluation researchers is also very important and should be an explicit and fully funded component of applied and methodological research. This can be challenging in contexts of highly constrained research funding, where epidemiologists often lead trials and may not prioritise economics. To improve the quality of evaluations, funders have a role to play not only in providing methodological funding streams but also in requiring economic evaluations to be included in major trials and other evaluations, in ensuring that these activities are adequately funded and receive due emphasis within the evaluation, and in creating separate funding streams specifically for conducting economic evaluations. Those interested in promoting economic evaluation should also draw on researchers and the research literature on approaches to capacity development and how to evaluate it (Chootipongchaivat et al., 2015). Capacity development should be understood as a collaborative process in which all participants develop their individual capacity and so contribute to institutional and global capacity development.

Any focus on improving economic evaluation methodology has to be balanced with investment in the institutionalisation of economic evaluation and health technology assessment in LMICs. Despite recent efforts to organise comprehensive health sector planning processes, fragmented domestic and external priority setting remains the norm in many settings. However, as countries move towards UHC, there will be increasing pressure for health systems to adopt strategic purchasing approaches that explicitly assess interventions for inclusion in benefit or entitlement packages. Economic growth is also likely to increase individuals' expectations of health services and the need for civil society to be involved in the priority setting process. Similarly, political instability and economic constraints require strong priority setting processes to avoid increases in healthcare inequality. Institutionalising priority setting through HTA is already proceeding in some middle‐income countries through national agencies, such as HITAP in Thailand. In low‐income settings where capacity is more constrained, intermediate steps might include fostering opportunities for assessing transferability of evidence and use of evidence produced regionally. Working with national, regional and global policy makers – despite their often short tenure – to strengthen their skills in assessing economic evaluation evidence and in using it to inform decision‐making is a particular priority.

Drawing together this collection of papers has highlighted some of the rich methodological contributions emerging from the experience of conducting economic evaluations in LMICs, spurred by the acute challenges facing researchers in LMICs. The articles in this supplement clearly show, however, that no simple dichotomy exists between LMICs and HICs. Most countries have a very long way to go before economic evaluations are produced and used effectively to inform priority setting. As Thailand has demonstrated, however, tremendous progress can be achieved in a relatively short period. We hope the unifying, global drive towards UHC will be accompanied by greater collaboration amongst researchers across all settings to address the shared challenges we face and that this supplement plays some part in this effort.