Principles of Economic Evaluation in a Pandemic Setting: An Expert Panel Discussion on Value Assessment During the Coronavirus Disease 2019 Pandemic.

As the coronavirus disease 2019 (COVID-19) pandemic continues to generate significant morbidity and mortality as well as economic and societal impacts, the landscape of potential treatments has slowly begun to broaden. In the case of a novel disease with widespread consequences, society is more likely to place significant value on interventions that reduce the outsized economic burden of COVID-19. Treatments for severe disease will have a different value profile to that of large-scale vaccines because of their application in targeted and potentially small subsets of those with symptomatic disease vs broad deployment as a preventative measure. Where vaccines reduce transmissibility of COVID-19, use of therapeutics will target symptoms, up to and including death for infected individuals. This paper describes discussions from a virtual expert panel that met to attempt a consensus on how existing principles of economic evaluation should be applied to therapeutics that emerge in a pandemic setting, with specific focus on severe hospitalised cases of COVID-19. The panel concluded that the core principles of economic evaluation do not need to be drastically overhauled to meet the challenges of a pandemic, but that there are several additional elements of value such as equity, disease severity, insurance value, and scientific and family spillover effects that should be considered when presenting results to decision makers. The panel also highlighted the persistent challenges on how society should value novel therapies, such as the appropriate cost-effectiveness threshold to apply, which are particularly salient during a pandemic.

Key Points for Decision Makers

Introduction

The coronavirus disease 2019 (COVID-19) pandemic has, as of July 2021, infected over 180 million individuals and caused almost four million deaths worldwide, with over 30 million cases in the USA and India, with 600,000 deaths in the USA and 400,000 in India [1]. Alongside significant health impacts, countries around the world have endured heavy and wide socioeconomic consequences of the virus—in the USA alone, the economic toll of COVID-19 is estimated at over $16 trillion, which is considered a conservative estimate [2].

The majority of patients with COVID-19 have no or mild symptoms and recover from the disease. Up to 22% of patients experience severe infection and require hospitalisation [3]. These patients may experience severe consequences such as acute respiratory distress syndrome, necessitating intubation/mechanical ventilation or extracorporeal membrane oxygenation in intensive care units, and COVID-19, sometimes independent of an intensive care unit stay, can lead to long-lasting symptoms post-discharge. Through improvements in clinical management, mortality rates have dropped since the start of the pandemic; however, 8–20% of patients hospitalised with COVID-19 in the USA still die [4, 5]. Mortality rates adjusted for population size have been particularly high in South American countries such as Brazil, Peru and Argentina, as well as European countries such as the UK and Italy [6]. Patients at increased risk of severe COVID-19 include those of an older age and/or with underlying medical conditions including obesity [7].

A key issue from a health economics perspective is whether and in what situations standard utilitarian approaches and frameworks in a conventional cost-effectiveness analysis—incremental benefits as measured by quality-adjusted life-years (QALYs)—should be expanded to incorporate other elements of value that are specifically relevant to a pandemic situation. These may include additional value based on impact to non-health sectors, equity impact, or interventions that are life saving or drastically improve lives in the face of imminent death. Methods currently exist to incorporate these impacts by, for example, weighting of QALYs [8, 9]. There is, however, no indication that these methods should be specifically invoked in times of a pandemic. Indeed, if cost-effectiveness frameworks are modified for pandemics, results cannot be directly compared to pre-pandemic therapies. None of the issues raised below is exclusively relevant for a pandemic, but some issues were brought into sharp relief during the COVID-19 pandemic. In addition to the methodological considerations pertinent to economic evaluations during a pandemic, how we interpret the results and assign appropriate value to interventions during a pandemic is another key issue.

Therapeutics for COVID-19 in the Context of Vaccines

There are 20 COVID-19 vaccines that have gained regulatory approval in at least one country as of July 2021 [10], and government-sponsored vaccination campaigns are now underway. Uncertainty surrounding the rate of uptake, a full picture of the vaccine safety profile, and the length of conferred immunity and effectiveness against emerging variant strains of the virus in real-world settings are still to be clarified. While vaccines will reduce population-level infection spread, any of these factors may mean a continued reliance on therapeutics to reduce mortality, symptom severity, duration, or complications among infected patients. We therefore limit the scope of this paper to the therapeutics class in patients with COVID-19 requiring treatment, where decision problems around resource allocation are more likely to arise. We define the therapeutics class as any active treatment for symptomatic infection. Discussions on this topic took place in a 1-week virtually convened panel in October 2020, using an interactive platform that allowed the six authors to answer questions and post replies directly to each other.

Over 460 trials of COVID-19 therapeutics have been evaluated by the US Food and Drug Administration with 11 Emergency Use Authorizations being issued [11]. Three Emergency Use Authorizations cover therapy options for patients requiring hospitalisation, two of which include remdesivir [12-14]. Another Food and Drug Administration-approved drug, dexamethasone, was repurposed for patients with COVID-19 on supplemental oxygen or those who are intubated [15]. The Institute for Clinical and Economic Review’s assessment of remdesivir monotherapy [16] was met with praise for its rapidity and transparency, but also with some criticism over its lack of consideration of non-health effects and a societal perspective, lack of an open-source model, as well as the use of a more stringent cost-effectiveness threshold than the organisation typically considers [17, 18]. The remdesivir evaluation underscores debates at the heart of valuing therapeutics during the COVID-19 pandemic: compared with vaccines, there is less clarity on the degree of impact for therapeutics on broader social and economic consequences of the pandemic. There was also consideration of the appropriate willingness-to-pay threshold to be used, in the face of potentially a large number of patients requiring therapy. As financing for therapeutics research and development is concentrated in the USA, this paper uses the US healthcare system for most examples. However, the larger context of pandemic-based value assessment may be applied globally keeping in mind that countries will generally vary in their assessment given differences in epidemiology, healthcare delivery and economic conditions. This paper starts with a presentation of different elements of value currently identified in existing frameworks and how they relate to economic evaluations for pandemic interventions, followed by a discussion on how these considerations impact on a willingness-to-pay threshold for value assessments and price setting.

Existing Value Frameworks and Elements Relevant to a Pandemic

The existing value frameworks [9, 19, 20] and methods are, by and large, generalisable—with rewards driven by health gain and medical cost offsets—but some different emphases are warranted as a result of a pandemic, with specific implications for including or excluding them from a value assessment. Alternative or additional elements of value have been discussed in a number of different forums, with the International Society for Pharmacoeconomics and Outcomes Research Special Task Force on Value Assessment Frameworks’ “value flower” [9] having summarised many elements not incorporated in traditional value assessments. In the following sections, we discuss a selection of value elements of particular importance to novel therapeutics in the pandemic context.

Equity

COVID-19 disproportionately affects racial and ethnic minorities and low-income patients in terms of case rates, severity of disease and mortality [21, 22]. Existing inequalities in the US healthcare system persist in the face of a pandemic and may be exacerbated. “Essential workers”, disproportionately low income and minority, are more likely to be unable to work from home, more often in service occupations, childcare, factory, farm work, or custodial work, and have borne the brunt of infection risk since the spread of COVID-19 in the USA [23]. Even in countries with a homogenous ethnic population, such as Japan, disparities in outcomes were found among those with differing socioeconomic status [24], suggesting that equity considerations will have widespread application globally. In low-income and middle-income countries (LMICs), equity may need to be considered slightly differently. It may not only be accounting for differing outcomes among sub-populations, but which outcomes are being considered. For example, the emphasis on mortality as the most readily available metric may ignore the morbidity impact on the younger generations who have a lower risk of dying from COVID-19, but who make up a larger proportion of the population in LMICs and who may form the essential core of the country’s economy [25].

When valuing therapeutics for a pandemic that has a disproportionate impact on particular subsets of the population, it is advisable to consider methods that can account for any differential impacts. The distributional form of cost-effectiveness analysis (DCEA) evaluates interventions with differential impacts on socioeconomic subgroups and compares the magnitude in each group [26, 27]. In lieu of a full DCEA, which would require trial data or a decision-analytic model to empirically inform the distribution of effects, Love-Koh et al. describe an aggregate DCEA for health technologies that rolls up existing data alongside a measure of inequality [28]. Use of the DCEA model can help value therapies based on reductions in health inequities among specified at-risk subgroups. As an alternative, the Generalized Risk-Adjusted Cost-Effectiveness (GRACE) approach [29] could also provide a means of conducting economic evaluations that allow incorporation of variables such as severity of illness for those who experience a disproportionate burden of the same disease. Both methods challenge the implicit assumption that a gain in health utility is equally valued by everyone.

Severity of Disease

Hospitalised patients with COVID-19 may experience complications, including bacterial infections, acute kidney injury, respirator-induced lung injury, malnutrition and generalised weakness [21]. Moreover, patients with COVID-19 of any severity may experience ongoing fatigue, the inability to concentrate and/or other symptoms, collectively dubbed post-acute sequelae of SARS-CoV-2 infection [30]. Willingness to pay for therapies may be increased from both the patient and societal perspective in reducing potential long-term hospitalisation and post-hospitalisation outcomes even if average QALY gains across the entire population are similar to those for a non-severe condition. In theory, therapeutics that prevent severe COVID-19 cases would be rewarded for clinical improvement among patients with the most burdensome symptoms. Given that only a minority of COVID-19 cases require hospitalisation, as well as the dearth of long-term clinical data post-discharge to compare against other severe conditions, the additional value for therapeutics in preventing severe COVID-19 outcomes remains an issue of debate. However, some sub-populations are at high risk of severe complications of COVID-19 including prolonged hospital stay or death [31], potentially even after vaccination, either due to breakthrough infection or waning immunity [32, 33]. Adjusting for life expectancy and the impact of existing co-morbidities, a recent study has shown that the burden of QALYs lost due to such deaths can be substantial in many countries [34]; international variation in this burden is driven by age distribution at the time of death, which implicitly accounts for differing levels of co-morbidities, as well as total number of deaths within each country.

What further confounds these evaluations is the uncertainty of results in current studies. Uncertainty around the long-term impacts of COVID-19 is especially large and to be expected; however, evaluations of acute outcomes are also subject to uncertainty because of several factors. Trials are being designed in a rapid adaptive manner, in a disease area that continues to evolve, with therapy guidelines that are constantly being updated. These are all leading, at times, to contradictory results on acute outcomes. An example of this was the extent to which remdesivir could prevent mortality, estimates of which initially relied on the ACTT-1 trial [35] and were later contradicted by the SOLIDARITY trial [36]. This highlights the difficulty of quantifying reductions in severity, morbidity and mortality from treatment when evidence is emerging at a rapid pace, as well as the importance of conducting thorough sensitivity analyses to account for all plausible outcomes.

Fear of Contagion/Insurance Value

Reducing the “fear of contagion” has been suggested as an externality that is often overlooked in economic evaluations [9]. Though a single therapeutic may not have a significant or oversized effect on fear of contagion reduction, its benefits may be accrued as one component of a complementary successful intervention landscape (including vaccines and other nonpharmaceutical interventions). The economic consequences of fear of contagion have been called “fearonomic effects” and may include business health impact and social life losses [37]. According to one study, “fearonomic effects” in China totalled $275 billion, or 1.9% of GDP, during the Lunar New Year week alone. Ma et al. have published a checklist for measuring the economic impact due to fear of contagion, quantifying both near and long-term deficits during a pandemic [37].

Insurance value places an additional value on the personal health risk reduction from being infected as well as the financial risk protection of insuring the population [9]. As with fear of contagion, any single therapeutic is unlikely to have significant benefit in illness or financial risk reduction, but rather, incorporates some share of benefit from the landscape of COVID-19 treatment options. In diseases with severe or long-term sequelae, as with COVID-19, therapeutics that address and/or add certainty in the reduction of ill-desired health consequences have an associated value benefit. Lakdawalla et al. describe the methodology for reflecting insurance value through consumer utility maximisation in an economic value assessment [9]. Consideration of either of these value elements is unlikely to have a significant impact on economic evaluations of current therapeutics; these elements would be considered more important when evaluating a therapeutic with significant effectiveness, for example, one that lowers viral load and therefore leads to decreased transmission along with symptom alleviation, or when evaluating the entire therapeutic class as a strategy.

Scientific and Family Spillover Effects

Scientific spillovers, a type of economic externality, can add value to a therapy that creates a significant advancement in the scientific knowledge base for a novel or unknown disease. Therapies with novel mechanisms of action, or therapies that treat previously untreatable diseases, “light the path” for additional innovation and can be rewarded to encourage additional knowledge production in the therapy area [9, 38]. Therapies developed for COVID-19 have the potential to provide this spillover effect, or indeed to benefit from prior spillovers (e.g. remdesivir and dexamethasone were originally developed for other diseases).

The impact of severe disease and its associated therapies on family members, particularly those fulfilling an informal caregiving role, has been well documented, especially in neurological diseases such as Alzheimer’s disease and Parkinson’s disease [39], though systematic incorporation of such an impact into economic evaluations remains low [40]. This may be particularly true for LMICs, where data to quantify the impact on non-patients are difficult to generate. Imposed self-isolation for patients with severe COVID-19 whether within a home or hospital will undoubtedly affect the entire household (i.e. “family spillover” effects), including those who must give informal care to the patient from a distance. Matters are further complicated by the potential economic impact on the family, if the patient is also the main source of income. There is some measure of value associated with improved treatment outcomes at the family/household level.

Incorporation of Additional Perspectives

For the purposes of an economic evaluation, including setting the price of a single therapeutic, the healthcare sector tends to be the main focus as payers do not generally consider non-health effects when underwriting premiums or setting reimbursement levels [41]. However, the Second Panel on Cost-Effectiveness in Health and Medicine recommended that economic evaluations consider both healthcare sector and societal perspectives [9, 42] as the reference case for all appraisals. The case for presenting both perspectives is especially relevant in a pandemic setting, where the value of a vaccine or therapeutic that addresses social, economic and health system upheaval necessitates consideration of societal benefits relevant to health systems and policymakers. Rewards for improvement in constrained hospital system capacity and return to work/school are not appreciated by a traditional value assessment [43]. Non-health impacts are essential components of a therapeutic’s value assessment from a societal perspective, for example, given significant unemployment during COVID-19. The Second Panel recommends including productivity effects in added costs in the numerator rather than as a utility adjustment in the denominator along with the “Impact Inventory” that lists the affected non-health sectors, including lost productivity, consumption and social services [19]. However, a lack of systematically produced non-health estimates hampers the synthesis of evidence from a valuation and price-setting perspective. Variation in the methods employed to incorporate these non-healthcare perspectives can also be a challenge [44], with results that lead to different conclusions. As better data become available, estimation of non-health effects will improve. The challenge is especially compounded in LMICs, where reliance on informal family care is high but data to quantify it are low [45]. In the meantime, societal and non-societal costs can be included in evaluations to be presented as scenario analyses alongside a base case as is already permissible in many countries’ pharmacoeconomic guidance, such as Spain, Italy and Australia [46-48].

Opportunity costs, however, could be more readily incorporated into an economic evaluation of COVID-19 therapies. Opportunity costs for COVID-19 vaccines/therapeutics vs other interventions can be explicitly measured to support decisions from a public health and healthcare system capacity perspective. Burdens on the healthcare system in general and intensive care beds in particular have meant that patients with other acute illnesses and preventative health services have in some settings been de-prioritised [49]. Hospital capacity has also been used as an explicit metric by which to trigger other non-pharmaceutical interventions that have large effects on society and the economy. Sandmann and colleagues have explored several techniques for estimating the value of foregone bed-days in a scarcity setting [50], which could be incorporated into an economic evaluation of COVID-19 therapies that affect time in critical care or the hospital generally. The inclusion of both health sector and societal perspectives is recommended in evaluations of any pandemic treatments, whether vaccine or therapeutic.

Willingness-To-Pay/Cost-Effectiveness Thresholds

In addition to the consideration of expanding the scope of economic evaluations for COVID-19 therapeutics with these elements of value, the appropriate cost-effectiveness threshold to apply has also been called into question. In the Institute for Clinical and Economic Review’s remdesivir assessment in 2020, the cost-effectiveness threshold was set at $50,000 per QALY in a departure from the organisation’s standard $100,000–$150,000 per QALY range, with the reasoning that thresholds should be reduced during pandemics with a sizable patient population uptake [16]. It is also possible to argue that inclusion of added value elements when valuing therapeutics should reduce the cost-effectiveness threshold further, at least from the healthcare sector perspective, as the additional elements of value take into consideration non-health benefits. It could be argued that part of the payment for a fuller capture of value should be borne by sectors other than healthcare, and the threshold used to set a price for the healthcare sector should therefore be lower.

However, a higher threshold may be justified given the urgent need to restore functioning economies. Effective therapies may not only have direct health benefits for the patient, but also have non-health effects around productivity and the opening of society by relieving pressure on the healthcare system. There may even be a case for the suspension, or abbreviation, of formal health technology assessment approaches during the pandemic in lieu of implicit judgements, and to conduct value assessments at a later point. Therapeutics that treat human immunodeficiency virus, for example, have only in the past year been selected for a full technology appraisal by the National Institute for Health and Care Excellence in the UK, resulting from the 2019 Voluntary Pricing and Access Scheme [51]. The National Institute for Health and Care Excellence has also issued rapid guidelines over the course of the pandemic specifically recommending the use of such therapeutics as tocilizumab without a formal technology appraisal [52, 53]. Several other countries are also conducting abbreviated forms of their standard appraisals, sometimes foregoing economic analyses altogether. This can be seen in such examples as the modified approach used to develop COVID-19 therapy evidence reviews in Canada [54], or in Australia through the formation of a new taskforce dedicated to evaluating evidence for COVID-19 clinical care. In countries where health technology assessments and economic evaluations are not as well established, such as India and other LMICs, it is even more likely that there has not been any explicit consideration of a cost-effectiveness threshold or willingness to pay specifically for COVID-19.

It is important to remember, however, that even during, or perhaps especially during pandemic times, healthcare budgets are finite. Considering a higher cost-effectiveness threshold, or indeed suspending health technology assessment altogether for a set period of time, which essentially signals an unlimited cost-effectiveness threshold, will need to balance the opportunity costs of such a suspension. Claxton et al. have described methods for quantifying opportunity costs of investing in new technologies in the UK National Health Service and incorporating those costs in the National Institute for Health and Care Excellence cost-effectiveness thresholds [55]. Incorporation of such opportunity costs would undoubtedly lead to a lower threshold given the impact on the ability of the healthcare system to offer services for other acute illnesses and preventive health services by funding COVID-19-specific therapies. These considerations may be particularly acute in LMICs, where the trade-off between COVID-19-specific interventions and existing preventative health measures is likely to have a larger impact [56, 57]. There may also have to be consideration of other COVID-19-specific, but non-therapeutic health measures, such as vaccines and screening programs. These are complicated not only by the estimation of opportunity costs, but also by the interactions between these efforts. A public health strategy including multiple measures may have different outcomes and value associated with it compared to individual strategies considered separately.

The consequences of altering the cost-effectiveness threshold should also be considered. Depending on the reasoning behind the alteration, this can send different signals to manufacturers about what innovations are desirable. Dedicating resources to vaccines at the expense of therapeutics may disincentivize innovation for future treatments, which has implications for future pandemics where vaccines may not be developed with as much speed as has been the case for COVID-19, leading to more dependence on effective therapeutics.

When making decisions around the adoption of new technologies and the appropriate price, decision makers should be aware of these arguments, as well as the implications of altering an existing threshold specifically in response to a pandemic. In particular, there should also be awareness of the implications for post-pandemic and non-COVID19 diseases in the future to maintain a sustainable healthcare system. Indeed, such implications further complicate the policy environment and could hamper the implementation of any health strategies that are considered a good use of resources. It requires a multi-stakeholder discussion, across several sectors, of whose resources we are trying to judiciously allocate.

Conclusions

Economic evaluation principles remain unchanged within the context of COVID-19. However, given significant morbidity and mortality, in addition to the widespread economic and public health impact of the COVID-19 pandemic, any treatment developed should be evaluated with a view toward the broader benefits of restoring economic and social activity; although quantifying some of these value elements, especially as the knowledge base evolves rapidly, is difficult. From a pragmatic perspective, the traditional health sector valuation of therapeutics should remain as a reference case. Modelling strategies allow us to evaluate both the traditional and potential larger societal value of COVID-19 therapeutics. In the future, as the pandemic evolves into different stages, the elements of value discussed above may also shift in their relative importance. Where strong evidence of a therapeutic’s effect on the wider pandemic situation exists, the appropriate additional elements of value could be incorporated in a scenario analysis to highlight potential therapeutic value and to present a full picture to decision makers. As the external environment is evolving rapidly, it is important to remain reactive to the potential non-health benefits of an intervention, but also to retain a core reference case for comparison.

Although usual economic evaluation principles remain unchanged within the context of the coronavirus disease 2019 pandemic, treatment evaluation should consider the broader benefits of restoring economic and social activity.

Several additional elements of value should be considered, including the disproportionate impact on certain patient groups, long-term effects of the disease, insurance value and impact on families, and the evaluation should adapt as the relative importance of these elements change as the pandemic evolves.

Finally, careful consideration should be given to the most appropriate cost-effectiveness threshold to apply in a pandemic setting.