How to assess for the full economic value of vaccines? From past to present, drawing lessons for the future.

Background:Cost-effectiveness analysis (CEA) is the economic analysis method most commonly applied today in the context of replacing one treatment with a new one in a developed healthcare system to improve efficiency. CEA is often requested by local healthcare decision-makers to grant reimbursement. New preventative interventions, such as new vaccines, may however have much wider benefits inside and outside healthcare, when compared with treatment. These additional benefits include externalities on indirect clinical impact, reallocation of specific healthcare resources, improved quality of care, better productivity, better disease control, better fiscal revenues, and others. But these effects are sometimes difficult to integrate into a meaningful CEA result. They may appear as specific benefits for specific stakeholders, other than the stakeholders in healthcare. Objective: Based on a historical view about the application of economic assessments for vaccines our objective has been to make the inventory of who was/is interested in knowing the economic value of vaccines, in what those different stakeholders are likely to see the benefit from their perspective and how  were/are we able to measure those benefits and to report them well.
Results: The historical view disclosed a limited interest in the economic assessment of vaccines at start, more than 50 years ago, that was comparable to the assessment of looking for more efficiency in new industries through optimization exercises. Today, we are exposed to a very rich panoply of different stakeholders (n= 16). They have their specific interest in many different facets of the vaccine benefit of which some are well known in the conventional economic analysis (n=9), but most outcomes are hidden and not enough evaluated and reported (n=26). Meanwhile we discovered that many different methods of evaluation have been explored to facilitate the measurement and reporting of the benefits (n=18).
Conclusion: Our recommendation for future economic evaluations of new vaccines is therefore to find the right combination among the three entities of stakeholder type selection, outcome measure of interest for each stakeholder, and the right method to apply. We present at the end examples that illustrate how successful this approach can be.

Introduction

Most health economists of today have been educated or trained in applying methods of cost-effectiveness analysis (CEA) to assess the economic value of new medical interventions, using examples of new treatments introduced in well-established healthcare systems [1,2]. CEA conventionally calculates the incremental cost-effectiveness ratio (ICER) that assesses whether extra payment for extra health benefit obtained with the new intervention is good value for money, mainly from the perspective of the healthcare payer. This is expressed through a maximum threshold value that should ideally not be exceeded. The extra money should come from displaced lower-valued activities within the defined healthcare budget if feasible [3]. The final objective is to improve efficiency of the healthcare system, gaining more quality health at a reasonable price to pay.

New treatments are given to patients suffering from diseases when under direct medical attention. In contrast, disease prevention through vaccination may happen in very different environments without unprompted medical attention [4,5]. Disease prevention may have far-reaching externalities beyond healthcare, which may completely remodel the impact of disease on healthcare and society, potentially spanning different domains of variable relevance to various stakeholders. Under such circumstances, conventional CEA may not be as suitable for measuring the full economic value of prevention as when comparing different treatments that can be substituted without radical system changes. Vaccines may have value to stakeholders beyond healthcare decision-makers and consequently, the value measurement must be broader than the clinical gain. A range of outcome measures linked to additional analysis methods are needed to account for the variation in economic and healthcare development with the introduction of vaccines [6,7]. Vaccines are meaningfully evaluated in terms of avoiding or reducing risk for losing some quality-adjusted life-years (QALY), using plans and programs that target specific populations to control infection spread [8,9]. The optimal prevention strategy may vary depending on the disease epidemiology, available budget, setting, and the public health goal (control, elimination, and/or eradication) [10,11]. This needs a broader evaluation beyond the clinical focus and has promoted the search for and the development of multiple methods of economic assessment, organising task forces with experts and representatives of supranational organisations [12,13].

This article first summarises the historical setting of economic evaluation of vaccines, as that may indicate the different stakeholders who were/are interested and helps to understand the current context of evaluation. It then reviews the stakeholder list with the focus on the specific benefits sought by each [7,14-17]. This is followed by a range of analysis methods for measuring those gains, where we briefly describe how we came to our current findings [18]. In the discussion section, we give examples of the use of a combination of different methods to inform different decision-makers about the full economic value of vaccines. The objective of this work is to draw lessons from past to present about the methods used, and to raise questions and possible answers about the needs of future assessment.

A historical perspective on the economic evaluation of vaccines

The first economic assessment of interventions for the control of communicable diseases reported in the literature dates from the 1970s. It was proposed at that time to use a type of constrained (called controlled) optimisation (CO) and cost-benefit analysis (CBA) [19-21]. CO was a technique frequently applied during that period in new industries searching for better performance, and healthcare was considered an industry where efficiency could also be enhanced using the same technique. CBA was the standard method of economic evaluation for programs that were publicly funded [22]. The first vaccines introduced, such as those against polio, typhoid fever, yellow fever, measles, pertussis and smallpox, were vaccines against major public health problems in childhood worldwide. Their benefits were large and their costs low. Economic assessments of those vaccines in the 1960s and early 1970s were not formally requested by any decision-maker. If economic analyses were presented, the cost-benefit evaluations were very simple using nomograms (rough approximations of the size of potential gains) [21]. The economic issue at that time was not about setting the price of the new vaccine introduced. The dynamic market forces that normally determine the price of goods were absent for vaccines because there was no competition between different producers in a private setting. During that period, the vaccine debate was about whether public or private institutions should ultimately develop and produce vaccines; and this remained a subject of discussion for a long period in many countries in Europe until recently [23].

The first publications applying CEA to vaccines appeared in 1980 for the classical vaccines against diphtheria, pertussis and tetanus in developing countries with restricted healthcare budgets [24], and for the polysaccharide pneumococcal vaccine in high-income countries (HIC) [25]. The latter publication was focused on reimbursement of prevention and vaccination by the authorities in the US, and the objective was to identify the most efficient vaccination strategy, rather than defining an acceptable price. The recombinant vaccines against hepatitis A and B had a similar economic story in the late 1980s and early 1990s, although their initial price was higher compared with the vaccines introduced earlier [26,27]. The focus of these economic publications was on defining precise target groups such as travellers, specific child clinics and prisoners, among others, who were at a higher risk of exposure to infection and subsequently had greater benefits of protection through vaccination. The still-present phenomenon of price erosion of vaccines within a short period of time, through regular tender processes involving different vaccine producers, was a feature first described during that period for those vaccines [28]. Later, in the early 2000s, with the introduction of the new conjugate vaccines against pneumococcal infection (pneumococcal conjugate vaccines [PCV]), models on the natural history of the disease were first developed, although those model types were already well known and customised [29]. Superimposed on those models were simulations of the vaccine effect demonstrating its direct and indirect impact (also called the herd effect) [30-32]. There were two reasons for this evolution in the economic assessment. First, the newer vaccines addressed diseases with high severity but lower incidence rates. Invasive pneumococcal disease prevalence in children was at least 100 times lower than the infections tackled with previous vaccines, thus requiring the entirety of the benefit (direct and indirect) of vaccination to be assessed to evaluate the price. Second, the prices of the new vaccines introduced after 2000, such as PCV, vaccines against human papillomavirus (HPV) and against rotavirus, were dramatically higher compared with the prices of the first vaccines introduced 20 to 30 years earlier. The reasons for these higher prices were the more complex processes for developing and producing the new vaccines, which were often conjugated and/or adjuvanted. There was also the effect of using CEA in the treatment drug world to define the price of new medical interventions as requested by local authorities, which was then also applied to vaccines [1].

In summary (see Figure 1), before the year 2000 the application of CEA to vaccines was not driven by searching for a price setting but rather by using static models and subgroup identification to find the most cost-efficient vaccine strategy. Later, with the introduction of more expensive vaccines and drugs, the emphasis on the additional benefits of vaccination through indirect effects was initiated using advanced dynamic models. This has been applied to the vaccines against HPV, rotavirus, influenza, pneumococcal disease, varicella, measles, and the new vaccines against meningococcal infections, all searching for the herd effect, and reporting CEA results to local decision-makers [31,33-36].

Figure 1.

Historical trends on the health economic evaluation methods of vaccines

New developments in the economic assessment of vaccines emerged some 10 years later around 2010, driven by several factors. More than 7 new vaccines were introduced in a short period of 15 years, and health authorities were not well prepared for the perceived high level of investment required to fund all of them. The infections tackled by the new vaccines (such as rotaviral diarrhoea in the high-income countries) were not considered to be as severe or life-threatening as those addressed by the previous vaccines, or for some new vaccines, there were long delays in the appearance of the expected benefit (e.g., it takes several decades for the benefit of HPV vaccination on reducing cervical cancer to become evident). Therefore, the short-term economic value of these new vaccines appeared questionable without a broader assessment of their value within healthcare. But it was only possible to evaluate most additional benefits after the vaccine was on the market with a price already set. These benefits include indirect or herd effect (which is difficult to demonstrate in randomised clinical trials), and the improvement in quality of care (where the hospital manager responsible for bed occupancy emerged as a new stakeholder). In addition, broader vaccine benefits beyond healthcare were not directly of interest to healthcare decision-makers. Other stakeholders outside healthcare had to be considered as potential beneficiaries of vaccination, such as the employer, the worker, the community and the government, which could benefit from indirect gains in productivity and tax receipts obtained by improving vaccine coverage by co-payment, demonstrated through model simulations. It was the period where international groups such as the World Health Organization (WHO), the Global Alliance for Vaccines and Immunization (GAVI), The Bill and Melinda Gates Foundation, and also the International Society for Pharmaceoconomics and Outcomes Research (ISPOR) and Health Technology Assessment International (HTAi) developed initiatives to investigate and assess the full value of vaccines in different contexts [7,12,15,37-39].

The advent of many new vaccines within a short period also encouraged the reappearance of CO models evaluating the best combination of interventions including vaccines to optimise health benefit within and across specific diseases under specific constraints [40-43]. CO could also help decision-makers to prioritise the introduction of vaccination programs [44]. One of the most recent areas of new research involved the use of fiscal health models (FHM) and social accounting matrix (SAM) modelling to better identify the indirect costs saved with vaccination [45,46]. Since indirect costs had not previously been explored in detail because there were no known tools available to investigate the issue, SAM methods may now help to demonstrate the wider benefit of vaccines and vaccination [47].

Shifts in the current economic evaluation of preventative interventions

This short history of evaluations raises the question of whether CEA is still the most appropriate economic evaluation technique for vaccines. The overall shortcomings of CEA were first formulated by Garber and Phelps in 1997 [48]. Additional shortcomings of CEA, specifically on evaluating vaccines, were expressed by Beutels et al. in 2008 [49], Standaert et al. in 2014 [50], and more recently by Standaert and Rappuoli in 2017 [9]. For instance, differential discounting (between costs and health outcomes) produced a paradoxical result whereby increasingly favourable CEA outcomes were obtained the longer the introduction of a preventative intervention was delayed [51,52]. Other paradoxical results include the justification of higher vaccine prices in low-income countries (LICs) compared with HICs because the disease burden is much higher in LIC as a result of lower overall healthcare investment [53-55]. Slow accrual of health benefit over time (e.g., HPV benefits in cervical cancer reduction) could make the impact of vaccination uncertain and difficult to observe directly, unless specific monitoring programs are set up to capture subtle and long-term health benefits [56]. Models can be used to estimate long-term gains but need to be built in a manner that is transparent, easily accessible, and subject to precise validation processes. To evaluate the full benefit of vaccination requires a comparator group of unvaccinated subjects, which is often not possible when the vaccine is introduced as universal mass vaccination. Therefore, an artificially constructed control group is required for the comparison [17,57], introducing uncertainty into the assessment and requiring use of impact measures instead of effectiveness. Vaccines need to be assessed at the level of populations instead of individual subjects and therefore require use of measures that make sense at the population level. Different thresholds may need to be applied to preventative interventions, reflecting the additional benefits beyond healthcare. Willingness-to-pay (WTP) studies should therefore be initiated at the population level to assess the full societal benefit of preventative interventions [12,58]. Pricing models for vaccines that consider the country’s development and income status, known as tiered pricing systems, have been introduced across the globe [59].

Poor health conditions are intimately connected with poverty traps in LICs, with devastating long-term consequences for families in populations living in survival-type economic mode with fragile human capital and relatively high out-of-pocket payments. Preventative interventions in such environments with high coverage bring much broader societal benefits to the population (improved productivity, quality of care, better education, and jobs) than any treatment could achieve [60]. These issues make an impressive list of items to consider in the current economic value assessment of vaccines.

Table 1 shows, in the upper part, the known classical items normally assessed in a CEA related to medical and non-medical aspects. The lower part indicates other domains that should be evaluated to show the full benefit of prevention but for which there may be difficulties in obtaining data. Additional evaluation methods beyond CEA may also be needed to demonstrate the meaning of these other gains, for example, by facilitating the analysis and reporting of budget handling and money flows.

Table 1.

List of items measures to be assessed in an economic evaluation of vaccines, categorised by known/conventional versus new/exploratory and by medical versus non-medical impact

BCR, Benefit-cost ratio; DALY, disability-adjusted life-year; GDP, gross domestic product; GNI, gross national income; HALE, health-adjusted life expectancy; HALY, health-adjusted life-year; HDI, human development index; ICER, incremental cost-effectiveness ratio; IRR, internal rate of return; NMB, Net Monetary Benefit; NNV, number needed to vaccinate; NPV, net present value; NWBI, national well-being index; PRO, patient-reported outcome measure; QALY, quality-adjusted life-year; ROI, return on investment; SRR, social rate of return; SWF, social welfare function; VPDI, vaccine-preventable disease incidence; VSL, value of a statistical life; VARR, vaccine-attributable rate reduction.

Effects (Blue items in Table 1)

The two measures that calculate the effect of a vaccine (efficacy and effectiveness) use a ratio of events with the case (vaccinated) arm as the numerator and the control (or unvaccinated) arm as the denominator [61]. This could be problematic if the control arm is not isolated from the vaccinated arm, because for transmissible diseases the control arm is then not operationally independent of the case arm. It is then important to consider critically how and over what period the outcome value has been measured. A more neutral approach is to work with impact measures and/or measures such as vaccine-preventable disease incidence (VPDI) or also known as the vaccine-attributable rate reduction (VARR) [62]. The latter may better reflect the vaccine effect in the real world because it compares with a condition of no exposure with the vaccine (complete isolation or historical data). However, impact measures are influenced by many other items such as herd effect, vaccine coverage, compliance and completion, and it is difficult to disentangle their specific contribution to the summary outcome measure [63]. Any single value for vaccine effect should be considered as the result of a dynamic process influenced by many different factors. The real value of the vaccine effect can only be obtained after reaching a new steady-state level of infection spread in the community, which can sometimes take a period of several years but is infection-specific. Vaccine effect may also vary substantially by time, geography, societal composition, disease epidemiology, and mutation risk of the pathogen.

Outcome measures (Purple items in Table 1)

Most measures presented in Table 1 can be considered as outcome measures that may help interpret the economic value of the vaccine effect. Some can only be considered in relation to other external measures as a reference and are not shown in Table 1, such as the threshold value for the ICER [64]. Others are constructions that need the outcome of other entities in the list to make the number meaningful, such as the benefit–cost ratio (BCR) which requires the cost results and the benefit results expressed in money terms [11].

Ideally, all outcome measures would be translated into monetary units to facilitate comparisons between goods and services delivered today and over time. Unfortunately, in health evaluations that seems difficult to achieve. Outcome measures are therefore often split into two categories: health–focused, expressed in natural units such as time and health quality; and resource-focused, expressed in cost. Cost measures may vary between societal units, from the basic unit of the individual, to the household or family unit, the community, the region or the national level, depending on where decisions are made and who holds the budgets.

The impact of preventative interventions in avoiding disease events can involve many different items within and beyond the scope of healthcare, resulting in the long list of outcome measures such as those listed in Table 1 (all purple items) [17,65]. Decision-makers generally prefer to work with summary measures, but that can often be difficult because there are multiple perspectives to consider with potential conflicts of interest: a payment from one perspective may be an income from another. For example, a vaccine that avoids severe disease events is a direct gain for individuals who avoided the disease, but for a hospital manager, this may represent a loss in income as fewer people require hospital care. It is therefore rare to see in healthcare a condition of Pareto-improvement with a win-win result for all those involved [66]. Finally, it should be considered that vaccination is a form of active medical insurance avoiding the risk of specific infections. Because the individual, once vaccinated, does not change his behaviour, vaccination could be considered as an improved insurance gain that is not under pressure of moral hazard.

As with the effect measurement, caution should be applied when using fixed values in a world that is very dynamic in consumption, risk exposure, and developments in healthcare with a social security network. As such, a result will never be an isolated established value but will be influenced by many different factors, such as vaccine coverage rate, compliance and completion rate, discount rates, period of evaluation, duration of protection, vaccine waning, and whether a new steady-state level of infection spread has been reached.

Government (Green items in Table 1)

Governments, including Ministries of Finance and Planning, may be interested in understanding the economic value of vaccine prevention. Avoiding disease and illnesses may improve societal productivity, economies, and tax returns, which may be highly beneficial to governments [46]. However, to convince those authorities, research needs to recognise their vocabulary, their interests and the outcome measures they use to assess value. The green items in Table 1 should be investigated and reported to explain the gain from vaccines to government authorities.

Societal (Orange items in Table 1)

Many different entities may be interested in the economic value of preventative vaccines, each with their own perspective on the benefit from a specific intervention (see also next section). Is it possible to obtain an overall measure of the benefit to society as a whole? Overall measures have been developed for education, but similar measures have not been applied to health and healthcare. Such a societal measure of protection could be valuable to classify regions or states in relation to their level of immune protection against common vaccine-preventable infectious diseases. Analysis of vaccines that present results for those measures is not yet available, but should be a subject of research as they might shed light on the economic consequences of reaching a high or a low score on social welfare function (SWF) and social rates of return (SRR) through immunisation.

In summary, considering the broader context for the economic evaluation of prevention requires the consideration of critical aspects of impact, including sustainability (finance), quality improvement (better and maintained health gain), and measurable societal benefits (externalities) expressed as welfare functions [67].

Stakeholder types and their search for benefit

During the last 15 years, research has identified different stakeholders interested in vaccines and their value assessment [6]. This is reflected in the assessment of factors such as a need for acceptance by the target population (e.g., for HPV vaccination against cervical cancer), or the recognition that much of the benefit of some of the newer vaccines could be seen by non-medical stakeholders including employers, employees or financial authorities. In the context of price setting, a search for co-payment was considered among different stakeholders when it was discovered where hidden benefit was measured in specific target groups. Vaccine producers were also interested to see how payers would evaluate the economic benefit of their vaccines. Finally, it was also important to understand how prescribers of vaccines were motivated in understanding the risk-benefit and total value of the new vaccines. Figure 2 illustrates all known stakeholder types who could be interested in the economic value assessment of vaccines. We identified 16 types, grouped into 6 categories called the 6 Ps: population, prescriber, producer, payer, provider, and politician. The largest group are the payers who are most interested in understanding the value for money, but others should also be considered as they are potentially interested in components of the total value health gain and/or cost savings arising from the use of vaccines.

Figure 2.

Stakeholder types to be considered for valuation of vaccines: The 6Ps

Available methods and measures

Since a systematic review to collect different methods used for the economic assessment of vaccines requires searching for a predefined list of methods, for this review we sought to explore the range of methods available using a different approach.

The following three elements have helped us in identifying the current list of available methods. First, our internal evaluation comparing treatment with vaccine prevention identified 24 items that could be regrouped into four domains (population focus, societal perspective, budget need, and different role functions of the vaccine over time [6]). This was followed by further investigation on ‘budget need’ evaluations independent of CEA, such as budget impact analysis (BIA), CBA, CO, and Cobb-Douglas evaluation functions. Second, our own research investigating different sources of finance for vaccines, especially for rotavirus vaccination, identified areas of taxation assessment and money flows between different agents [45]. Finally, a grey literature search using Google on different types of CEA for vaccines identified incremental, decremental, distributional, and extended CEA approaches. An initial list of the different options was presented in a previous publication in 2017 [18].

Eighteen evaluation methods have been identified for the economic assessment of vaccines. Different stakeholders may have different objectives resulting in different information needs, and therefore different evaluation methods may be more suited to some stakeholders than to others, reflecting their different demands. The available methods can be categorised into three groups depending on whether the primary focus is on outcome, budget or finance (Table 2).

Table 2.

Summary of different methods developed and used for the economic assessment of vaccines

BIA, budget impact analysis; CBA, cost-benefit analysis; CEA, cost-effectiveness analysis; CO, constrained optimisation; FHM, fiscal health modelling; MCDA, multi-criteria decision-making analysis; RSA, risk-sharing agreements; SAM, social accounting matrix SMAA, Stochastic Multi-criteria Acceptability Analysis.

Outcome-focused methods

The listed outcome-focused methods are mainly variations on the classical incremental CEA. The incremental analysis of the cost difference divided by the extra health benefit (often expressed in QALYs), has been for years, and still is today, the basic economic assessment in healthcare when introducing new interventions. It evaluates whether the gain from a new intervention in relation to a reference comparator, is worth the extra cost within a local context (often the healthcare system), using a specific threshold to accept/reject the new intervention as ‘cost-effective’ or not. CEA is not a comprehensive economic evaluation method for evaluating mass vaccination programs because those programs require a substantial initial investment budget, and budget assessment is only indirectly implied in CEA by the selected cost-effectiveness threshold.

Decremental CEA (DCEA) considers the question in the opposite direction: how much loss in health effect is acceptable for a reduction in cost? It has seldom been applied because that situation is rare and the concept of paying less for less health gain is unfamiliar [68]. The development of the CEA concept to distributional CEA and Extended CEA (ECEA) was to focus attention on health inequality in society related to income inequality and methods to address that situation [69,70]. When a new intervention is introduced, there is often a trade-off between total health gain obtained versus a reduction in health inequality among different social groups. Vaccines may often provide both if the coverage rate is high.

Extended CEA accounts for equity considerations such as protection from financial risk due to out-of-pocket payments. It combines economic and social indicators to prioritise new interventions at the household level. It allows for more rational selection of interventions, with stronger advocacy for developing countries with limited resources and competing priorities. The analysis shows an evidence-based articulation of effectiveness and benefits at the household level beyond health outcomes.

Multi-criteria decision-making analysis (MCDA) uses a composite outcome measure that is a weighted combination of different variables assessed through the elicitation of preferences of the various stakeholders affected by the decision [71,72]. Cost remains the same as for CEA. The additive value function is based on preference criteria. No specific threshold is defined to accept or reject a new intervention. It allows for a more robust, inclusive and holistic decision-making process, which can account for linkages between various dimensions of immunisation programs and beyond.

Stochastic Multicriteria Acceptability Analysis (SMAA) is similar to MCDA but includes ways to cope with uncertainty, imprecise information, or partially missing data in the inputs and preference weights [73]. The additive value function uses rank acceptability indices for priority selection. SMAA can be considered as a type of MCDA, taking more account of uncertainty around the value assessment during the evaluation. It can be considered as the next step after MCDA.

Cost-consequence uses the same incremental analysis as CEA but instead of a specific health effect, it can consider any outcome measure of interest, such as hospital stay or medical visit, assessing the extra cost to avoid one hospitalisation or medical visit [74]. The problem is that there is no reference threshold available to assess whether the gain is worth the extra cost, which is determined by the opinion of the decision-maker.

Budget-focused methods

BIA directly considers the impact of a preventative intervention on budgets and is a valuable addition to CEA. However, it includes only the money to be spent and any cost offsets, excludes non-monetary benefit [75], and tends to have a short evaluation period as it is focused on short-term budget spending. BIA therefore cannot be considered appropriate for a full economic assessment of new interventions where important cost-offsets will only appear many years after introduction, such as HPV and hepatitis B vaccines (HBV) [76]. Additional tools are needed for a complete evaluation.

CO evaluates the maximum benefit obtainable under specific budget constraints. This reflects the decision-making reality for many payers, including finance and other economic stakeholders such as insurance companies and corporations. In CO, different interventions such as vaccination and screening, or vaccination and vector control, are evaluated in combination rather than against each other as in CEA [40-42]. Optimisation calculations link the constrained budget and the outcome to be maximised into one analysis [43,77,78]. CO can also set public health goals to be reached within a certain time frame at the lowest possible budget. CO for vaccines can thus be transparent about maximising different outcomes, such as averted deaths, hospitalisation days, cases or QALY loss, under specific constraints. Constraints other than budget can be added, including infrastructure/logistic limitations, maximum vaccine coverage rates and maximum vaccine impact, among others. CO offers greater transparency than CEA, as instead of a threshold it uses an actual budget constraint that restricts the acceptance of intervention types. It provides richer information than CEA as it directly links budget and outcome, can generate budget plans for reaching health goals, and clarifies the connection between budget and impacts, helping to prioritise more clearly than CEA combined with BIA [44]. The method has a long history of application outside healthcare, with a proven record of benefit in domains such as fishery, forestry, agriculture and industry, among others [79]. It may be surprising that the widely available and long-established technique of CO has seldom been used and has not been more widely applied in healthcare. A possible explanation could be that the decision-makers when health economics was first applied to healthcare could recommend proposed new interventions but were not always the budget holders, or perhaps it was not intended ultimately to be interchangeable with evaluations in other areas of expenditure such as defence. Early health economic evaluation made a simple direct comparison between two interventions and was not concerned with trying to optimise a combination of different interventions to reach specific health goals under a set of constraints. However, a CO analysis conducted to maximise the QALY gain will still not capture benefits of vaccination outside the healthcare system.

CBA methods such as the Cauliflower toolbox [6], permit evaluation of vaccine benefits in domains other than healthcare. The different benefits are expressed in money terms where possible. For benefits that are difficult to express in money terms, WTP assessments (the demand perspective) are applied.

Portfolio assessment assigns a priority to the introduction of specific vaccines when budgets are limited, while seeking to optimise outcomes such as QALYs gained, hospitalisations avoided, or direct medical cost reduction [44].

The Cobb-Douglas function has recently been applied to assess quality of care (QoC) improvement in hospitals after introducing rotavirus vaccination, choosing between investing in treatment or prevention to maintain the same QoC [60].

Finance-focused tools

Any healthcare intervention requires a source of funding. In healthcare, this is most often the government, or the Ministry of Finance/Treasury that collects taxes. FHM measures the economic value of a new intervention seen from this perspective. It can be applied following a cohort approach or following a cross-sectional annual population approach using the SAM method [80,81].

FHM evaluates the impact of disease episodes on tax payments, allowing decision-makers to consider the potential return on investment resulting from disease prevention [82]. This return may be two-fold: a reduction in healthcare expenditure resulting from disease prevention; and higher tax receipts due to a healthier and more productive population. Assessing taxes may be complex because there are different inter-related sources of tax (e.g., income taxes, corporate taxes, value-added and consumption taxes). However, an individual with a disease episode will tend to make a lower tax contribution (income, corporate or consumption) compared with a healthy individual, while government expenditure tends to be higher (transfer cost into healthcare to treat those who seek medical attention and possibly social security support payments). A disease episode requiring treatment therefore produces a two-fold loss for the government (decreased income and increased payment) and a two-fold loss for the affected individual (less income and consequently less spending). FHM tools for evaluation of vaccines may be highly relevant to governments and other important budget holders, such as insurance companies, who also apply return on investment (ROI) in their evaluations of new interventions [83]. For private insurance companies, their income from the premiums paid by members is analogous to the income governments receive from tax receipts, so the same techniques used by governments to identify the best return on income can also be used by insurance companies to evaluate whether paying for prevention offers a better return than paying for treatment. FHM is typically conducted for a single country or a region within a country, whereas macroeconomic analysis would include price and consumption/production behaviour change and may be used to compare between countries.

The difference between budget and finance is that a budget analysis defines a lump sum, whereas a finance analysis explores the money impact of a certain condition and its effect on money flows. Evaluation methods may focus on specific infectious diseases and/or vaccine types, and on specific target groups by age (e.g., paediatric, adults or adolescents), vulnerability (e.g., maternal, tourists), risk status (e.g., immunocompromised) or specific disease areas (e.g., diabetes or cancer).

The evolution of this range of methods demonstrates considerable creativity. However, the optimum application of these methods remains a challenge, as many may be unfamiliar to researchers conducting and stakeholders using economic assessments. Education and training is important to get better acceptance and real implementation.

Summary

Figure 3 summarises the links between the 6Ps of Figure 2 with the outcomes described in Table 1 and the methods in Table 2. The range of outcomes and methods makes it possible to conduct economic evaluations from different perspectives. Some outcome measures are common to different types of methods, whereas others are preferentially linked to a specific method (e.g., ROI and net present value [NPV] are linked to FHM). The most common outcome is cost, although the different methods may measure different aspects of cost and may therefore produce different results.

Figure 3.

Making the links between the stakeholder types (Figure 2), the effect/outcome measured (Table 1) and the analysis methods (Table 2)

Figure 3 indicates that information could be explored on many more outcomes that could be generated by vaccines to the benefit of different stakeholders, which are not currently systematically considered (right column in effect/outcome [new]). The societal measures of SWF and SRR have not yet been applied to the assessment of vaccination programs. They are mentioned here as further options to explore as potential ways to capture the social benefit of interventions, as they have been used for evaluation in other domains such as education [84,85].

Discussion

This paper argues that for preventative interventions, particularly vaccines, a complete economic value assessment needs an evaluation conducted in a broader context than healthcare. Recent moves in this direction have been instigated because (1) many new vaccines were introduced in a short time period aimed at infections other than the widespread childhood diseases, (2) these new vaccines had higher prices compared with earlier vaccines, (3) there was uncertainty among decision-makers about their immediate value, (4) it was recognised that much vaccine benefit was generated outside the healthcare system, (5) and more comprehensive datasets for evaluation became accessible. A process of assessment of the full economic value of new vaccines suggests following an approach with several steps, as outlined below.

After obtaining a precise picture of the total disease burden caused by a vaccine-preventable infection, and modelling the potential vaccine impact over time, a first step should be to identify potential stakeholders with interests in the economic value measurement of the new vaccine. The disease burden and simulated vaccine impact should identify the likely major beneficiaries, and this should be tested through sensitivity analysis because of the level of uncertainty in the assessments. Subsequently, the most relevant outcome measure for each stakeholder should be identified, and the most appropriate analysis method selected. The availability of many different methods for the economic evaluation of vaccines allows presentation of the most relevant outcome results to specific stakeholders [12]. For instance, CEA may be appropriate for healthcare stakeholders familiar with the technique, while financial decision-makers paying for health care may be more accustomed to ROI. A range of different analysis methods can take account of different situations, such as variations in the degree of economic and healthcare development between HICs and LICs. It is important to retain a broad view, considering multiple potential impacts of a new vaccine on the primary disease under study, and also on others with an indirect impact. For instance, vaccination against pneumococcal disease in children has an indirect impact on the health of aging adults via effects on respiratory and cardiovascular disease and possibly also on mental health conditions [86]. The specific impact of vaccination, being a preventative intervention, extends beyond any treatment impact. This is the consequence of avoiding ill health events instead of mitigating those events.

We have developed a series of different evaluation options reporting different types of outcome measures, with which we have approached many different stakeholders. At international level, these include bodies such as WHO, Pan-American Health Organization (PAHO), GAVI, The United Nations, USAID, the Department For International Development (DFID, UK), the European commission, The Bill and Melinda Gates Foundation, ISPOR and HTAi. More local examples across the world include Ministries of Health, Ministries of Finance, members of parliaments, treasuries, politicians, National Immunisation Technical Advisory Groups (NITAGs), healthcare providers (specialists and general practitioners), hospital managers, patient groups, working mothers, employers, parents, and individual subjects.

We have been able to provide overviews of full immunization programs developed using portfolio models. Examples include: showing the cost offsets and health gains of different dosing schemes using different vaccines for the same indication under a constrained budget (rotavirus vaccination) [41]; prioritising different intervention types to manage paediatric infections, and malaria [42,44]; demonstrating the benefit of child vaccination for working mothers with evidence of reduced work absenteeism [87]; improving QoC with vaccination and translating this into cost gains while maintaining QoC [50,60]; estimating the best combination of vaccination and screening to maximise cervical cancer reduction under a fixed budget [40]; and showing that extra budget does not guarantee substantial health gain in aging adults from PCV and/or influenza vaccination [43].

To illustrate all these effects, we have used CO models [40-44], Cobb-Douglas production functions [60], FHM and SAM modelling [45,46], macro-economic computable general equilibrium models (CGE) [88] and impact instead of effectiveness measures [89]. Others have used ECEA [69], distributional CEA [70], BIA [75] and MCDA [72].

Our recent experience in a Central European country convinced us that the combination of different tools such as CO portfolio with CEA was much more informative for the local NITAG [90]. The CO evaluation provided an overview of the immunisation benefits for different childhood diseases. Moreover, the method allowed priority ranking of different vaccines depending on the outcome local decision-makers chose to optimise (e.g., deaths, medical visits, hospitalisations, direct medical costs, or QALYs). Ultimately, the NITAG recommended a CEA for the two highest-ranking vaccines. This process of vaccine assessment and selection was additionally supplemented by a budget plan and an ROI evaluation. Systematic implementation of these methods can require a substantial effort but is a rewarding leap towards transparent value-based decision-making in healthcare.

Many countries have improved their organisation of immunisation programs by setting up NITAGs and independent advisory boards as proposed by the WHO. Such local organisations need to be provided with adequate data so that they can consider and make sound recommendations. Presenting infectious disease data with the vaccination impact supports them in judging their decisions and recommendations, and it is important to provide the broad picture from different angles. Recently updated guidelines from WHO intend to support NITAGs and decision-makers on the approach for the economic evaluation of vaccines [91]. The guidelines recommend the use of CGE models as a preferred method over CEA in a societal perspective when analysing diseases with economy-wide impacts that exceed infected individuals, their contacts, their employers and the health sector.

The availability of different methods of evaluation offers a greater range of information to assess the value of vaccines more accurately. It helps to obtain a more balanced picture of disease impact and the effect of prevention at different levels of society. For example, if a new vaccine is considered cost-effective but requires a net investment that is not available, identifying potential alternative budget sources (such as higher taxation receipts due to better productivity of a healthier population) would be useful information. Similarly, a high healthcare cost could be offset by a reduction in work absenteeism resulting in larger gains in social welfare. Social preferences could be considered in the assessment of a vaccine that prevents a rare but very severe disease causing permanent sequelae and deaths in affected patients. The balance between different perspectives involved will be an enrichment for any decision-maker if information is provided in a transparent overview. Using a combination of evaluation approaches enlarges the vision of the usefulness and the economic benefits of vaccination. This will help to define priorities for investment to augment social wealth through disease prevention.

Different factors in each method may influence the results. For example, the results of a portfolio model and a fiscal model on vaccine ranking can be different. Having comparative tools on budget and public health impact available is important, but they are not always sufficient to capture all the insights needed to implement a sustainable immunisation programme. Countries may benefit from combining different approaches involving different stakeholders to support the decision process. For example, NITAGs are now often composed of multi-disciplinary teams in need of a broad evaluation of vaccination impact, not confined to medical doctors.

Our proposed broad approach may have limitations. It is not practical to explore every outcome measure with every method in every analysis for every stakeholder. It will be necessary to identify when to move to the next level of investigation, and therefore might require early engagement with decision-makers on the most relevant approach. Although CEA is positioned in this paper as one of the many options to assess the economic value of a new intervention, in most cases it will be appropriate to report the CEA in the first instance because most researchers trained in health economics are familiar with the concept and the demand for CEA information remains high. Immediately presenting new outcome measures and new evaluation methods while ignoring methods that have been established for a long time may be difficult for decision-makers unfamiliar with the new method, even if it provides a fuller picture of the vaccine benefits. This may be less problematic for CO as some decision-makers may already know it from other contexts. An evaluation combining different means of disease management including prevention with specific constraints that may limit the introduction of new interventions is highly informative for a decision-maker. However, this concept of broader perspective still lacks a single summary measure to quantify the contribution of vaccination in general or for each vaccine separately to societal and population gain. The proposed summary measures of SRR and SWF have been applied to education and road security measures, but have not yet been developed and conceptualised for healthcare. This aspect should be considered in depth in future research, to help provide a better picture of the full economic value of vaccine prevention for society.

We recommend enriching the range of tools, perspectives, and outcome measures used for the full economic evaluation of vaccines. Current established techniques have shown their limits in measuring the wider benefits of vaccination. A broader range of evaluation methods, providing fuller information for interested stakeholders, would provide a stronger position for vaccine prevention in a world currently dominated by treatment. Additionally, it will be important to enrich the training curriculum of new health economists with these new approaches making the close links between health, human capital, labour market and the fiscal space of public finance.