Editorial.

The proposal from Catherine Pitt and colleagues from the London School of Hygiene and Tropical Medicine (LSHTM) for a supplement on the methodological challenges faced by lower and middle income countries (LMICs) when applying the methodology of health economic evaluation could not have been more timely. In 2014, NICE International had just completed coordinating an international effort, funded by the Bill & Melinda Gates Foundation (BMGF), to define the first version of the International Decision Support Initiative (iDSI) Reference Case for health economic evaluation that could be applied to work conducted in an LMIC context (NICE, 2014). The concept of this iDSI Reference Case was founded on similar reference cases, such as that proposed by the US Panel on Cost‐Effectiveness (Gold et al., 1996), and NICE's own version in the UK (NICE, 2013). The iDSI Reference Case was launched in the UK House of Lords in June 2014 (NICE 2016), and publication in the peer‐review literature is imminent (Wilkinson et al., [Link]n.d). The reason this supplement is so timely is that this first version focused on the principles of the evaluative approach without being prescriptive about the methodology – in explicit recognition that the methodological challenges faced in the context of LMIC may need further development.

What is clear from the collective papers in this supplement, and the collective experience of authors across many countries, is that the differences between health economic evaluation in LMIC and those in higher income countries (HIC) are chiefly down to the context, which drives such major considerations as perspective, type of intervention, study question, data availability, transferability and so on. In that sense, the iDSI Reference Case may have been correct to focus in on the principles relating to what reflects an appropriately conceived evaluation. But beyond general guidance, what specific help does this offer the analyst, who, while attempting to conform to the principles, nevertheless has to choose and implement the methods? This supplement takes on that challenge and has successfully started a discussion for applying health economic evaluation principles to the LMIC context. The logistical challenges alone of assembling a diverse set of authors and getting full manuscript submissions in a short period of time are huge. But to do so in a way that results in a cohesive set of papers that covers the issues across such a broad topic area shows true dedication and determination. As independent editors of this supplement, not only have we had the pleasure of seeing the supplement come together under tight timescales, but we have also seen firsthand the enthusiasm, drive and resolve of the team from LSHTM to deliver a quality supplement. Catherine Pitt in particular has led this initiative, and the success of this supplement is in large part down to her hard work and dedication.

What is apparent from many of the papers in the supplement is that the conceptual differences between LMIC and HIC are relatively few. Many of the challenges of applying evaluative methods in LMIC are the same as those in HIC, just that some of the contextual challenges are exacerbated in the LMIC setting. For example, the need for transmission dynamic modelling to adequately address the complexities of infectious disease are the same in both jurisdictions, but infectious disease is more of an issue in LMICs, while the epidemiological data needed to model transmission are more often lacking in those settings. Similarly, the transferability of results from one jurisdiction to another remains a challenge in HICs, but is exacerbated in LMICs by the global nature of many funding streams for intervention and the desire to use single study results across many LMICs. Evidentiary requirements are always a challenge in HICs, but are hampered in the LMIC setting by lack of infrastructure for data capture. The paper by Griffiths and colleagues (Griffiths et al., 2016) details the methodological differences between evaluations conducted in HICs and those in LMICs. Principally, they found that many of the methodological differences followed the contextual differences, with HICs more often using a health service or payer perspective and relying on reference costs. Evaluations conducted in LMIC have a greater focus on infectious disease, more commonly use DALYs as an outcome measure and are less frequent in their use of modelling methods and sensitivity analysis.

Despite the careful nature of the Griffiths review, their reliance on existing checklists – particularly those that were developed in HIC settings – such as the CHEERS (Husereau et al., 2013) and Drummond (Drummond et al., 2015) checklists may have had the effect of obfuscating some of the differences that exist. Indeed, while the supplement itself represents an important and substantial body of work, and an excellent first attempt at laying out some of the issues, it is a surprise that some key issues for methods relating to evaluation in LMIC did not get more attention. We see three key areas that we feel would benefit from more in depth discussion.

The first of these relates to outcome measures. While the debate of QALYs versus DALYs is touched upon, the main focus in the supplement seems to be the potential for a totally new capability measure in the paper by Greco and colleagues (Greco et al., 2016). Although QALYs and DALYs are sometimes presented as being two sides of the same coin in that QALYs measure health gain and DALYs health loss, careful analysis shows that there are some fundamental differences between the approaches (Anand and Hanson, 1997; Airoldi and Morton, 2009). They can produce quite different results depending on assumptions relating to when a disease manifests and how long it lasts and the extent to which age weights are included. The importance for methods is that DALYs and QALYs have developed from different schools of thought and represent perhaps the most fundamental difference between the approaches employed between the HIC and LMIC settings. Two main reasons explain the prevailing use of DALYs in LMIC settings: the role of the funder in requiring the estimation of DALYs (e.g. BMGF mandates the use of the DALY in their funded evaluations) and that DALYs were designed to be estimated at the system level, meaning they can more easily be employed with aggregate health systems data and can then be used for comparing outcomes between health systems – something that global funders will inevitably take an interest in.

The second issue is one of equity. Although it is noted that the lack of Universal Healthcare Coverage (UHC) is a major contextual factor that differentiates HIC and LMIC evaluations, the critique of Griffiths and colleagues seems to relate mainly to perspective of the evaluation. They note that the consequences of not adopting a societal perspective in favour of the narrower health service or payer perspective will be less consequential in a health system with UHC. Yet more recently, other analysts have developed new techniques for ‘extended’ cost‐effectiveness that highlight the equity implications of universal public finance treatment options (Verguet et al., 2015). Whether this truly is a new methodology in a welfare economic framework is perhaps open to debate, but nevertheless, it seems like a new initiative with something important to say about equity that might have usefully been included in the discussion this supplement has started.

Finally, it is clear that most of the evaluations conducted in HICs are static in that they explore how cost‐effective interventions would be if employed, but have little to say about how those interventions should be implemented. Griffiths and colleagues identified the tendency for LMICs to focus on complex health system interventions (Griffiths et al., 2016) – broadly defined to include the need to evaluate bundles/packages of interventions and alternative delivery modalities. Given the contextual differences with fewer institutional infrastructure resources in most LMICs, the lack of implementation methods is perhaps another area that could be explored in future work. Indeed, in the context of health impact assessment, we have already seen much development in novel methods of improving implementation of complex interventions. For example, a team at LSHTM has already demonstrated how the UK Medical Research Council (MRC) complex interventions guidance (Craig et al., 2008) could be adapted and given a theoretical basis within a theories of change framework (De Silva et al., 2014). It would be a small, but important step, to consider how the health economic evaluation of interventions, many of which would be considered ‘complex’ by the MRC definition, could be enhanced by considering not only the evaluation impact but also the pathway to implementation, perhaps using a theories of change or similar framework.

Catherine and her colleagues who conceived and scheduled this supplement, together with all of the authors who contributed, are to be applauded for the enormous achievement of starting a real debate on the methods of evaluation and challenges for priority setting in LMICs. The completion and production of this supplement in time for the 2016 Prince Mahidol Award Conference with a theme of health priority setting should represent the end of the beginning of this important debate in a way that will maximize its impact. Certainly, the journal Health Economics would be interested in hearing from other groups of researchers working in the area of LMIC economic evaluation, who might have different perspectives to the authors covered in this package of papers in an effort to further broaden the debate this supplement has started. We sincerely hope that researchers and funders in both HICs and LMICs will take up the challenges posed by local resourcing of health economists, development of appropriate methods for the context of the evaluation and use of economic evaluation results for priority setting that will ensure the use of better evaluations to support better decisions that will ultimately lead to better population health.