What are the benefits and risks of using return on investment to defend public health programs?

Return on investment (ROI) is an economic measure used to indicate how much economic benefit is derived from a program in relation to its costs. Interest in the use of ROI in public health has grown substantially over recent years. Given its potential influence on resource allocation, it is crucial to understand the benefits and the risks of using ROI to defend public health programs. In this paper, we explore those benefits and risks. We present two recent examples of ROI use in public health in the United States and Canada and conclude with a series of proposals to minimize the risks associated with using ROI to defend public health interventions.

Introduction

Return on investment (ROI) is a ratio that has come into increasing use over recent years. ROI indicates how much economic benefit is derived from a program in relation to its costs. This ratio, calculated to demonstrate how relevant investments are, has the esthetic quality of being remarkably synthetic. Both private sector and non-profit firms use it to stimulate co-investments. Foundations, for example, seek to attract more funds for their activities by demonstrating to their funding partners that money is wisely invested. In recent years, there has been growing research interest in ROI use in the public health sector,(Baxter et al., 2014, APHA, 2013, CPHA, 2013) mainly to demonstrate the economic value of public health programs in a context of rationed public spending. In this paper, we explore the benefits and risks of using ROI to defend public health programs. The issue is salient because ROI-based public decisions will necessarily influence the public good and people's well-being and may have an impact on resource allocation and use. To orient our discussion, we present two recent examples of ROI use in public health and explore the context within which its use is expanding in public health economic evaluation. We then discuss the benefits and risks of using ROI to defend public health programs. Lastly, we offer proposals for using ROI information while acknowledging the full range of impacts of programs.

ROI in public health

Both the American Public Health Association (APHA) and the Canadian Public Health Association (CPHA) recently launched videos showing returns on investment for various programs, advocating that “investing in public health is smart” (APHA, 2013, CPHA, 2013). In these videos (see screenshots in Fig. 1), programs are compared in terms of their different ROI ratios. Some of the key CPHA messages are:

Fig. 1

Screenshots from Public Health: A Return on Investment (left, CPHA) and Public Health is ROI (right, APHA).

Every $1 spent on early childhood health and development saves up to $9 in future spending on health, social and justice services. Return on investment: 800%.

Every $1 invested in tobacco prevention programs saves up to $20 in future health care costs. Return on investment: 1900%.

The CPHA video concludes by saying, “Public health is a return on investment… Make investing in public health a priority.” The APHA video ends with the same message: “Public health is a return on investment: prioritize public health funding.”

Clearly, using ROI in the public sector is a way to defend public health programs in a context where funding is threatened, budgets are controlled, and the relevance of public health interventions still needs to be justified (Stine and Chokshi, 2012, Ifanti et al., 2013, Potvin, 2014). While this particular trend is new, comparing costs to economics benefits has a long-standing history in the field of economic evaluation. Cost-offset and ROI studies have been around for years (Kelly, 2005, Neumann et al., 2008, Chattopadhyay and Carande-Kulis, 2004, Drummond et al., 2008, Brent, 2003).

More often than not, public health interventions are complex undertakings (Shiell et al., 2008). A significant proportion of them target effects produced over the long term whose impacts are diffuse and may affect a variety of sectors (e.g. health, education, employment) (Alayli-Goebbels et al., 2013). Their adverse effects, which do exist, are also difficult to document (Drummond et al., 2008, Goldsmith et al., 2004, Brousselle and Lessard, 2011, Coast, 2004, Weatherly, 2007, Killoran et al., 2009). Many of these interventions also have externalities, in that they affect people not directly targeted by the intervention. Finally, the nature of these interventions makes it difficult to use experimental designs, and, when measuring the effects, researchers must contend with both contextual influences and those of concurrent interventions. These characteristics render causal relationships very complex, such that it is not often easy to attribute observed effects to a given intervention (Shiell et al., 2008). Identifying effects, measuring them, and attributing value to them are challenges in themselves (Alayli-Goebbels et al., 2013, Weatherly et al., 2009, Chalkidou et al., 2008). All these characteristics may explain why economic evaluations of public health programs and interventions have traditionally been under-represented in published economic evaluations (Drummond et al., 2008, Goldsmith et al., 2004, Weatherly, 2007, Killoran et al., 2009). Most economic evaluation studies deal with new technologies or drugs, while fewer than 10% address upstream interventions such as public health interventions,(Rush et al., 2002) with the majority (55%) of those targeting clinical prevention (Goldsmith et al., 2004).

In recent years, an emerging movement has provoked in-depth reflection on what economic evaluation methods are best suited to public health programs and interventions. The publication of the Wanless report (Allin et al., 2005) and the creation of the NICE Centre for Public Health Excellence in the United Kingdom have prompted considerable questioning and redeployment of economic evaluation methods for public health programs and interventions (Kelly, 2005). These reflections found concrete expression in the NICE report Supporting Investments in Public Health, published in 2011 (NICE, 2011). In Canada, a similar reflection occurred, (PHAC, 2009) resulting in a report that proposed a way to grasp the complexity of public health programs using economic evaluation methods (Husereau et al., 2014). It is important to note that all these published reports give new prominence to the valuation of impacts. They explicitly suggest including complementary information on ROIs (NICE, 2011) or “some estimate (or valuation) of the monetary value of added improvement in health outcomes” (Husereau et al., 2014). These publications are examples of the new trend of using ROIs when conducting economic evaluations of public health programs and interventions. They have also helped to institutionalize this approach, with the consequence that ROIs are now being used not only in advocacy efforts, but also, and more importantly, in decision-making on resource allocation. This new prominence given to ROI in public health raises the need for thoughtful reflection on the benefits and risks of this orientation.

Benefits and risks

Benefits

In 2008, Neumann et al. observed that:

“The value of governmental public health systems may seem obvious in light of progress in public health over the past century. The reality of chronic underfunding of these systems suggests that the general public is unaware of public health's value” (p. 2173)(Neumann et al., 2008).

Funding for public health has always been difficult to secure (Freedman et al., 2013, CPHA, 2010, Robert Wood Johnson Foundation, 2013). Even though public health programs and interventions may lead to important population health gains, in our society, medical care still receives the larger portion of funding (Robert Wood Johnson Foundation, 2013). In recent years, in Canada and in the United States, public health expenditures represented 5.3% and 3% of total health spending, respectively (CIHI, 2013, Harvard and Chan, 2012, Institute of Medicine, 2012). In contexts where public expenditures are being reduced, there is a real risk that the proportion of resources devoted to public health will shrink. Public health has historically contributed to changing the epidemiological profile of societies. Today's public health challenges, which include chronic diseases, life course epidemiology, and environmental health issues, call for multisectoral interventions.

ROI is a timely tool with which to advocate for public health interventions that have long term implications and require substantial investments. Assessing the economic value of public health programs and interventions using ROIs, cost-offsets, and profitability threshold analysis could provide robust arguments in their defense. First, it would turn the discourse around, such that the public health budget would no longer be seen as an expense, but rather as an investment that is often profitable over the short, medium, or long terms. Second, in a context of controlling public expenditures, such arguments might influence public opinion and ultimately help safeguard funding for public health programs and interventions. Third, for public health interventions with demonstrated savings, the issue would no longer be their worthiness as financial investments, but rather their affordability, as the question then becomes: even if we know the benefits outweigh the costs, can we afford to invest today in a public health intervention whose effects will only manifest over the longer term? Affordability might present the next serious obstacle to funding, and budget impact analysis may be necessary as complementary decisional information.

Risks

Even if there are known benefits to using ROIs in relation to public health interventions, there are also several risks. First, using ROIs inevitably engenders comparisons. For instance, using the two previously mentioned videos, we ranked the ROIs for various public health programs (Table 1). A cursory look at the results prompts questions such as this one: given that the ROI of water fluoridation is twice that of early education programs, should water fluoridation have priority over early education? If allocation decisions were based only on ROI, would it mean interventions with the lowest ROIs should not be funded?

Table 1

Examples of public health intervention ROIs, ranked highest to lowest.

Intervention	ROI	Source
Child safety seat	3900%	CPHA and APHA
Water fluoridation	3700%	CPHA and APHA
Mental health and addiction	3600%	CPHA
Tobacco prevention	1900%	CPHA
Vaccination	1500%	CPHA
Early education	1300%	APHA
Biking and walking opportunities	1200%	APHA
Food and nutrition	1000%	APHA
Childhood health and development	800%	CPHA
Workplace safety	500%	CPHA and APHA
Cleaner vehicles	300%	CPHA
Tobacco cessation	125%	APHA

Sources: APHA, 2013, CPHA, 2013.

Taking ROIs into account in allocation decisions may considerably change which interventions are funded and which are not. Decision-makers may wish to stop funding interventions that do not show a positive ROI. What would these interventions be? Who would be the losers? And what would the social impacts be of reallocation decisions? Finally, how would innovation and experimentation be funded if showing a positive ROI became a condition of funding?

Second, there are important concerns related to the way ROIs are calculated. As discussed earlier, public health interventions are complex, with effects that are sometimes scattered and intangible and with externalities that are neither easily quantifiable nor easily convertible into dollars (Shiell et al., 2008, Goldsmith et al., 2004, Payne et al., 2013). ROI calculation readily accommodates medical costs averted with improved health (e.g. reduced hospitalizations related to asthma, shortness of breath, lung cancer treatment), but assigning dollar values to life-years saved, to intangible effects such as increased well-being or empowerment, and to externalities is not so easy. While some evaluators do not hesitate to attribute an economic value to life-years saved, this is still very much subject to ethical debate (Drummond et al., 2005, Pinkerton et al., 2002, Robinson, 1993, Kelman, 1981, Mooney, 1980, Weinstein and Fineberg, 1980, Mishan, 1971). On the other hand, not including these effects because of a conviction, on principle, that they should not be converted into dollars will inevitably favor interventions with more acute and easily measurable impacts. Finally, whereas public policies and organizations advocate for the importance of integrating equity considerations into public health evaluations (World Health Organization, 1986). ROI methodology has no way of taking into account equity benefits, such that these consequences do not figure in the calculation.

Discussion

The mission of public health is to prevent diseases and traumas and to protect and promote health (World Health Organization). In austerity contexts, ROIs can offer reassurance on the economic value of public investments in this sector (Stuckler & Basu, 2014). However, quantifying the economic benefits of public health programs can also lead to people being seen as inputs into the overall economic system, with no intrinsic value other than the economic advantages their health and activities can provide.

Public health has a value in itself, as it has the potential to positively impact the health of humans. Nevertheless, every public investment has an opportunity cost: when we decide to invest in one program, our capacity to invest in another is reduced. There is no reason why public health should be exempt from this predicament. How, then, are we to be guided in these considerations? Based on the above discussion, we propose the following guidelines to minimize risks associated with using ROIs to defend public health interventions:

Defend public health programs for what they do. ROIs should not be central to the discussion, but rather should be seen as complementary information to be considered after effectiveness indicators. Furthermore, we must always keep in mind that cost and benefit data are eminently contextual and that, as such, ROIs cannot be exported to other contexts than those used for the study.

ROIs should only be used for equivalent alternatives and not to compare interventions that are different in their objectives.

To avoid ethical questions, it would be judicious and sufficient simply to assess whether benefits outweigh costs, without attempting to identify, calculate and translate into dollars all the effects of public health interventions and programs (Tchouaket et al., 2013).

For programs that may be threatened by ROI comparison, it could be useful to anticipate what the impacts of stopping their funding would be.

The effectiveness of public health programs should be systematically documented. There is a dearth of reliable data on public health programs; evaluative efforts should be intensified to adapt methodological approaches and compile more evidence.

Finally, greater use should be made of logic models. Based on evaluation methodologies, the logic model shows the resources invested in an intervention, as well as the activities and the chain of effects, including direct effects (e.g. morbidity, mortality), indirect effects (e.g. reduced utilization of health care and of professional resources in schools, fewer work days lost) and externalities (protection of other individuals) (Funnell & Rogers, 2011). Logic models are a powerful tool to represent all the effects, both those included in ROI calculations and those that are not.

Conclusion

ROIs are increasingly being calculated to demonstrate the value of investments and ultimately to reinforce funding (Gargani, 2014, King, 2014). Consequently, careful reflection is needed on how their use influences allocation decisions. ROIs of public health interventions are helpful to introduce economic information into the political debate and are used mainly as an advocacy tool in the political arena. It is therefore crucial to understand the basics of how ROIs are calculated and to know their limitations and risks, rather than blindly accepting black-box numbers.

Conflict of interest statement

The authors declare that there is no conflict of interest.