Malaria and Economic Evaluation Methods: Challenges and Opportunities.

There is a growing evidence base on the cost effectiveness of malaria interventions. However, certain characteristics of malaria decision problems present a challenge to the application of healthcare economic evaluation methods. This paper identifies five such challenges. The complexities of (i) declining incidence and cost effectiveness in the context of an elimination campaign; (ii) international aid and its effect on resource constraints; and (iii) supranational priority setting, all affect how health economists might use a cost-effectiveness threshold. Consensus and guidance on how to determine and interpret cost-effectiveness thresholds in the context of internationally financed elimination campaigns is greatly needed. (iv) Malaria interventions are often complimentary and evaluations may need to construct intervention bundles to represent relevant policy positions as sets of mutually exclusive alternatives. (v) Geographic targeting is a key aspect of malaria policy making that is only beginning to be addressed in economic evaluations. An approach to budget-based geographic resource allocation is described in an accompanying paper in this issue and addresses some of these methodological challenges.

Key Points for Decision Makers

Introduction

Total annual spending on malaria has grown to over US$ 2.6 billion globally [1]. The current availability of resources represents an arguably unprecedented opportunity to reduce the global public health burden of malaria and perhaps eradicate the disease altogether. Promising gains have been made, with global malaria-associated mortality decreasing by 60% between 2000 and 2015 (down to an estimated 438,000 deaths) [1]. It is important to continue to make the most of the currently available resources by spending in a way that maximises impact.

Healthcare economic evaluation can offer useful information to support this goal by appraising the balance of costs and consequences of technologies or services in different contexts. A recent review by Gray and Wilkinson describes the development of economic evaluation methods [2] including two key initiatives for priority setting in low and middle income countries (LMICs) (i) the World Health Organisation’s (WHO) Generalised Cost Effectiveness Analysis (GCEA) [3] and (ii) the more recent reference case, an initiative to support methodological standardisation backed by the Bill and Melinda Gates Foundation [4].

In most endemic countries in Asia and Latin America, and increasingly in Africa, malaria decision making takes place within the context of a disease elimination goal, though the best path to achieving this goal is far from clear. Key questions facing malaria decision makers are whether, where and when to deploy several interventions and programmes, including:

Scale up vector control (typically through long-lasting insecticide treated bed nets).

Scale up access to malaria diagnosis and treatment.

Forms of mass drug administration (including mass screening and treatment).

Active case follow-up.

Deployment of an emergent malaria vaccine.

The malaria economic evaluation evidence base to guide policy makers is substantial and growing. In 2011 a systematic review of the cost and cost effectiveness of malaria control interventions identified 43 economic evaluations published between 2000 and 2010 [5]. A broader review of economic evaluation in LMICs published in 2015 found that malaria comprised 20% (n = 41) of the published literature between 2000 and 2013 [6]. The most recent and broadest review identifies a further 29 malaria economic evaluations published between 2012 and May 2014, representing a 13% share of the literature across all disease and income settings [7]. Since different time periods are covered by the 2011 and 2016 reviews, there are at least 72 malaria economic evaluations published to date, 40% of which have been published since 2012. However, there are some important idiosyncrasies; this paper identifies some common characteristics of malaria decision problems that affect both the application of economic evaluation methods and interpretation of results.

Elimination: Intervention Cost-Effectiveness Decreases with Incidence (Or Appears To)

In the context of disease elimination it is possible to draw a distinction between two types of economic evaluation; (i) evaluation of the policy of elimination (or eradication [8]); and (ii) evaluation of the component interventions required to achieve elimination.

Economic evaluation of malaria or other elimination campaigns are relatively uncommon; a systematic review in 2015 identified 43 economic analyses, though many are not economic evaluations1 [10]. Such evaluations are methodologically challenging due to the complexity of elimination campaigns and expected impacts. In malaria the expected benefits of elimination are not only to population health but various other sectors including (though evidence is mixed) education [11, 12], tourism [13], and economic productivity [14, 15]. Cost benefit or cost consequence analysis [16] can be used to appraise the multi-sectoral impacts of elimination, though concrete results may remain elusive. To capture the full costs and benefits of an elimination campaign, the evaluation time horizon must extend some years beyond the expected date of elimination. Such analysis inherently entails a great degree of uncertainty, in particular due to influence of secular trends that can play a decisive role in driving transmission. It is very difficult to anticipate the impact of economic development, deforestation or climate change on malaria, and such factors could play important roles over the course of 10 or 20 years. There is also a risk of failure, either through never reaching elimination [17] or from a post-elimination resurgence of transmission [18].

Routine economic evaluations of malaria interventions typically take shorter time horizons, evaluating the differences in direct health impact or reductions in transmission over the short term or up to the point of elimination [19]. Evaluations comparing specific interventions rarely, if ever, include post-elimination benefits. This is important because cost effectiveness can appear to decline as malaria transmission falls on the path to elimination. Malaria intervention costs are wholly or partially fixed with respect to malaria incidence. The costs of prevention activities such as bed net distribution are unaffected by malaria incidence, as is the cost of mass drug administration or improved diagnosis. Even for case management focused interventions, the commodity costs that would be variable with incidence comprise a minor proportion of total programme costs [20]. In other words, a non-negligible portion of intervention costs is fixed with respect to malaria incidence. Any decrease in cost due to lower incidence will be proportionally less than the change in health impact and the cost-effectiveness ratio will rise. This is illustrated in Fig. 1 with two arbitrary interventions, one with fixed costs with respect to incidence and one with partially fixed costs.

Fig. 1

General relationship between intervention cost effectiveness and declining disease. (i) Intervention with fixed costs with respect to incidence, including prevention activities such as vector control or vaccination; (ii) Intervention costs are partially variable with incidence, including diagnosis and treatment based interventions. Notation: . A time horizon that excludes post-elimination benefits is assumed

If the campaign is successful and incidence declines, so too (almost paradoxically) does the apparent cost effectiveness of the interventions. At some point on the decline in transmission the relevant incremental cost-effectiveness ratios (ICERs) will no longer fall under the appropriate cost-effectiveness threshold (CET) (see Box 1 and Box 2). Prima facie, it is not possible to achieve malaria elimination without investing in cost ineffective interventions because the benefits of investing in the late phases of malaria elimination lie in the difficult-to-define post-elimination period.

An opportunity to move past this challenge perhaps lies in the elimination decision itself. Whether or not it is based on quantitative analysis of the costs and benefits of elimination, a political decision to aim for elimination implies that spending on this goal is perceived to be cost effective compared with alternative spending in the health sector. If the elimination goal is a genuine commitment, it perhaps implies a shift in the objective of routine malaria economic evaluation from allocative efficiency across the health sector in general, to what could be considered technical efficiency within the elimination campaign. That is, for decisions on the allocation of malaria funds to specific interventions the CET may be discarded as the representative of resource constraints in favour of a budget, which for malaria is often known and ring-fenced [21]. In this case the appropriate budget size for the malaria elimination campaign remains an open question; one with diverse stakeholders and further complications to the application of a CET that reflects health-sector constraints (Sect. 4).

An elimination policy does not mean that routine economic evaluation has reached the end of the road and that all spending to reach this goal can be considered value for money. Economic evaluation remains a valuable source of information that can support planning to achieve maximum impact on health, but consensus and clearer guidance is needed for analysts and users in the application and interpretation of economic evaluations in the context of disease elimination.

International Aid: Available Resources Exceed Local Funding Constraints

In the past decade, international aid has transformed malaria control and elimination efforts. In 2014, recipient countries reported receiving almost US$2 billion in financial aid for malaria control and elimination, comprising 76% of all malaria spending (Fig. 2) [1]. Putting to one side the complications of using CETs in the context of disease elimination, how does international aid affect economic evaluation decision rules?

Fig. 2

Sources of funding for malaria control and elimination in 2014

The purpose of international aid is to overcome the severe resource constraints faced by LMICs. While there is a longstanding debate surrounding so-called vertical aid [27, 28], earmarked disease-specific funding continues to facilitate the implementation of malaria programmes that otherwise would not have been affordable. As outlined in Box 2, a CET is used to represent the resource constraints of the society or healthcare provider. Regardless of the CET chosen, on introducing a non-negligible sum of aid the CET would, in theory, need to be raised to reflect the increase in healthcare services now affordable to this society. It may be that across the health sector total aid inflows are not significant; however, in many countries there is substantial additional financing directed specifically towards malaria control and elimination. In these contexts a CET that accurately reflects governmental or societal constraints may be inaccurate in reflecting resource constraints in malaria planning.

Probabilistic analysis and cost-effectiveness acceptability curves enable analysts to consider a range of possible CETs. Nevertheless, again, consensus is needed on whether and how to adjust an expected health-sector CET where aid affects affordability of malaria control and elimination programmes.

International Aid: Supranational Priority Setting

Economic evaluation typically aims to address allocative efficiency within a nation state. However, before international aid arrives in-country to be allocated to malaria interventions, a series of decisions have already been made regarding which disease areas and which countries to prioritise. No formal economic analysis of supranational priority setting of malaria funds has been published to our knowledge.

There are several important factors that affect the interpretation of cost-effectiveness evidence for supranational priority setting. As outlined in Box 2, standard decision rules (including GDP indexed CETs) are intended to reflect the resource constraints of a particular country. Therefore, conclusions about whether an intervention is cost effective (or not) to a large degree reflect the affordability of the intervention in that context as much as the efficiency with which investment may be converted to health gains. For example, a multi-country study of pandemic preparedness found that, in general, stockpiling antivirals is not cost-effective for LMICs but may be cost effective in high-income countries [29]. As the authors note, the use of GDP based CETs means this conclusion reflects local affordability and does not reflect where investment would yield the most health gains. The reverse would likely be true [30]. Decision makers and analysts should therefore not use GDP-based CETs for between-country priority setting. Notably, recent work on decision rules for health system strengthening in the context of supranational priority setting focuses on budget allocation rather than threshold analysis [31]. To an extent, GDP-based CETs may remain relevant to supranational priority setting in terms of allocative efficiency within the health sector that aid is delivered into. However regional malaria elimination or global eradication is a weakest link global public good, requiring effort from all affected countries for success [17]. GDP-based CETs are a decision framework for optimal allocation of healthcare resources at the national level yet regional malaria elimination policy must consider more than simply the efficient allocation of resources from a country perspective.

Supranational priority setting perhaps implies a role for a common or global CET [32], matching the perspective of the funder. That is, health gains are valued equally in all countries and funds directed to where impact is greatest. A global CET might reflect supply-side2 constraints of international aid budgets or demand-side norms regarding the extent to which poorer countries should be supported to provide healthcare services that would otherwise be unaffordable. GiveWell, an advisory organisation for charity donors, is effectively applying a global CET of US$5000 per life saved in their assessments of philanthropic causes and organisations [33].

Ultimately, if malaria elimination is cost effective (and political commitments to achieve elimination suggests it is considered to be in the countries that have made them) then other factors such as the rate at which local health systems can efficiently absorb additional financing, the period for which this can be sustained, equity considerations and rises in domestic financing may drive international aid allocations.

Choosing Between Compatible Interventions

In economic evaluation methodology, a decision problem is typically structured as a set of mutually exclusive alternatives (see Box 1). Some malaria decision problems such as the choice of first-line therapy or diagnostic test fit this framework well [34-36]. In other cases, the decision problem entails choosing between interventions that are not mutually exclusive but compatible or even complimentary, such as bed nets and community health workers. In this case, it is not necessary to choose one or the other, it is entirely possible to deliver both. This complicates the rationale for constructing a cost-effectiveness frontier and calculating incremental cost-effectiveness ratios (since these reflect the value of replacing one intervention with another). There are two options available when addressing complimentary interventions.

Use a base-case comparator scenario for all interventions.

Construct intervention bundles defined as mutually exclusive alternatives.

In the first approach, the costs, effects and cost-effectiveness ratios of intervention options are all expressed in comparison to a common baseline, most likely a null or “no additional intervention” scenario, which is similar in this respect to the generalised cost effectiveness analysis (GCEA) framework [3]. These generalised cost-effectiveness ratios allow the reader to make a judgement about which interventions, in isolation, yield the greatest health gains per unit of investment. The cost-effectiveness rank could be applied to determining the order in which interventions are added to a package of services but may be less applicable when choosing one intervention instead of another (when an incremental cost-effectiveness ratio based on the difference in costs and effects between the two interventions is required). An additional limitation is that intervention interactions are not accounted for. This approach assumes that the costs and, in particular, effects of intervention A are unchanged by the presence of intervention B, which is rarely a tenable assumption for interventions addressing the same disease.

To incorporate both intervention compatibility and interactions, several malaria economic evaluations have constructed priority setting landscapes based on discrete packages of interventions [37-39]. Intervention bundles can then be treated as mutually exclusive alternatives and incremental analysis applied to assess the value of switching from one package to another. A limitation of this approach is that the intervention combinations are defined a priori and may restrict the decision space since it may not be possible to include all possible combinations. Input from decision makers into the design of intervention packages will improve the relevance of the evaluation to policy decisions. Evidence users should take care not to interpret the package with the lowest cost-effectiveness ratio as the optimal choice. This applies to all economic evaluation results but can be particularly tempting with evaluations of intervention bundles.

Geographic Targeting

The epidemiology of malaria varies considerably between different geographical areas. A core aspect of priority setting in malaria control and elimination is to determine which interventions, or combinations of interventions, are provided where.

There are several options to address geographic heterogeneity in economic evaluation. Stratification of results, sensitivity analysis or scenario analysis can be used to present the cost effectiveness of healthcare interventions in contexts with certain characteristics. Readers are then free to match their context of interest to the closest example given. In this approach the reader still has a degree of work to do in translating from illustrative examples to applied policy, particularly if targeting resources across a large number of geographical units such as districts or townships. Moreover, any distinction between heterogeneity and uncertainty in results may be lost. Calculation of costs, effects and cost effectiveness in all geographic units of interest would provide a decision maker with a fuller picture of the decision landscape. Decision rules or constraints can then be applied to yield a recommended geographic allocation of interventions according to a CET, or, given the limitations discussed here, direct allocation of a relevant budget. Two recent studies address the geographic allocation of resources in malaria planning [26, 37] but such examples are rare and methods are varied.

Summary

We identify five areas where the realities of malaria decision problems affect the application of economic evaluation methods. The complexities of (i) declining incidence and cost effectiveness in the context of elimination; (ii) international aid and resource constraints; and (iii) supranational priority setting, all affect the way in which health economists might use a CET. Guidance and consensus regarding best practice on when and how to use a CET for malaria economic evaluation, and the selection or calculation of this threshold, is greatly needed. Alternatively, a policy of malaria elimination may allow economic evaluation to focus on direct budget allocation within the elimination campaign. (iv) In order to properly inform priority setting in malaria policy, economic evaluations must be able to assess combinations of compatible interventions, rather than sets of mutually exclusive alternatives. Assessment of exhaustive sets of intervention combinations is not possible and communication with policy makers will be essential to construct relevant evaluation questions. (v) Geographic priority setting is a core element of planning in both the control and elimination of malaria. Economic evaluation can do more to support this priority setting by incorporating geographic heterogeneity into analyses and making a clear distinction between heterogeneity and uncertainty.

In an accompanying paper in this issue [40] we outline an approach to geographic resource allocation in malaria planning that addresses several of the challenges outlined in this paper.

The use of cost-effectiveness thresholds is complicated by the disease elimination objective and international aid.

To be relevant to key policy questions, economic evaluations may need to evaluate compatible (rather than mutually exclusive) interventions and provide information to support geographic targeting.

Box 1

Cost-effectiveness priority setting: a brief summary

Comprehensive descriptions of economic evaluation methods can be found elsewhere [9, 21, 22]. The most common frameworks, cost-effectiveness or cost utility analysis, typically include the following steps:• Define a decision problem in terms of a set of alternative healthcare technologies or services (interventions)• Measure or model intervention costs and effects• Express the differences between competing interventions on the cost effectiveness frontiera as incremental cost effectiveness ratios (ICERs)b • Use a cost-effectiveness threshold (CET) to define the intervention with the highest ICER below the CET as the optimal choice

aAll interventions that are not dominated. Interventions A, B, D and F in the example

bThe difference in costs between two alternatives divided by the difference in effects, reflecting the value in replacing one intervention with another

Box 2

Cost-effectiveness thresholds

The cost-effectiveness threshold (CET) is considered to represent the willingness-and-ability-to-pay of the Ministry of Health or society in general, and would ideally be set at such a level that it reasonably reflects real budget constraints. In theory, the CET is equal to the opportunity cost of alternative public healthcare spending. Defining an accurate CET is a challenge and it is often indexed to the national gross domestic product (GDP) per capita. In low and middle income countries (LMICs), thresholds of 1× and 3× GDP per capita are commonly used [23], though a recent study suggests a lower threshold may be more appropriate [24]. An earlier review by Shillcutt et al. summarises the methodological debate in defining CETs in LMICs [25]