Evaluating one health: Are we demonstrating effectiveness?

The perceived benefits of a One Health approach are largely hinged on increasing public health efficiency and cost effectiveness through a better understanding of disease risk-through shared control and detection efforts, and results that benefit human, animal and ecosystem health. However, there have been few efforts to identify and systematize One Health metrics to assess these perceived efficiencies. Though emphasis on the evaluation of One Health has increased, widely cited benefits of One Health approaches have mainly been based on modeled projections, rather than outcomes of implemented interventions. We conducted a review of One Health literature to determine the current status of One Health frameworks and case studies reporting One Health metrics. Of 1839 unique papers, only 7 reported quantitative outcomes; these assessments did not follow shared methodology and several reviewed only intermediate outcomes. For others, the effectiveness of One Health approaches was often assumed without supporting evidence or determined subjectively. The absence of a standardized framework to capture metrics across disciplines, even in a generic format, may hinder the more widespread adoption of One Health among stakeholders. We review possible outcome metrics suitable for the future evaluation of One Health, noting the relevance of cost outcomes to the three main disciplines associated with One Health.

Introduction

One Health refers to the health connections among people, animals and our shared ecosystems [1]. Over time this definition has expanded to incorporate food security, poverty, gender equity, and health systems strengthening [1], [2], [3]. Incorporating a One Health approach into public health policy is widely expected to increase efficiency and cost-effectiveness by reducing overlap among public health, animal health and ecosystem health sectors. Based on these anticipated benefits, One Health initiatives have been established among intergovernmental organizations [4], [5], [6], [7], national agencies in the USA [8], and internationally (e.g. the World Bank's Global Program for Avian Influenza) [9]. This has been supported by new societies [10], [11], [12], journals [13], [14], and other private sector initiatives [15], [16].

These initiatives promote integrated research, surveillance, and control programs and policy frameworks. Given the transboundary nature of people, pathogens, and ecosystems, One Health collaborative partnerships have been set up internationally, e.g. the East African Infectious Disease Surveillance network [17], One Health Alliance of South Asia (OHASA) [18], the South East Asian One Health Universities Network [19] and Mekong Basin Disease Surveillance consortium [20]. One Health curricula have been incorporated into public health and veterinary degree programs [21] (e.g. the One Health post-graduate program through the Royal Veterinary College) and One Health research centers and institutes have been formed, such as the One Health Institute at the University of California Davis and the Center for One Health Research at the University of Washington.

Despite these initiatives, there has been little focus on measuring the efficacy, cost-savings, or reduced duplication of effort within One Health programs, and it has been proposed that this hinders more widespread political interest in the approach [22]. Evaluative metrics of interventions in the One Health realm (e.g. rabies, brucellosis, pandemic prevention) such as cost-benefit analysis, cost-effectiveness analysis, or Disability-Adjusted Life Years (DALYs) averted are often derived from models rather than One Health interventions in practice. For example, models from the World Bank project rates of return upwards of 71%— an expected benefit of US$30 billion per year from the prevention of a pandemic—if the international community were to invest up to US$3.4 billion per year in veterinary and human health service capacities [23]. Analysis of rabies in Africa and Asia project that the cost-effectiveness of mass dog rabies vaccination would be US$837 per averted human exposure [24]. A similar analysis of brucellosis in a scenario of 52% reduction through livestock vaccination demonstrated that a total of 49,027 DALYs would be averted with a net present value of US$18.3 million. Whereas contribution between sectors would give a cost-effectiveness of US$19.1 per DALY averted [25]. Another study showed that mitigation is a more cost-effective policy than adaptation programs, saving between US$344.07 and $360.3 billion over the next 100 years if implemented today [26]. While informative, global figures may be too abstract to motivate stakeholder investment on a regional or national scale, and without demonstrated outcomes, it is unclear whether approaches perform to modeled expectations.

Furthermore, a lack of standardized One Health metrics means that there is limited objective evidence on the potential benefits of these programs [27]. In the current paper, we assess a wide scope of One Health literature to capture metrics reported across all outcomes and to identify and analyze new programs that may not have been reviewed by previous authors. We then consider policy recommendations for a more systematic evaluation of One Health across disciplines in an effort to strengthen its integration into the decision-making process.

Materials and methods

We conducted a literature review using Scopus, PubMed, and ISI Web of Science searching the term ‘One Health’, restricting publication date from the formal introduction of the term in the literature (2003) [28] until May 26, 2015, when the literature review was first initiated. References were extracted, their abstracts and articles separated into ‘Topical’ (One Health referred to as a concept—i.e. the linkages between animals, humans and ecosystems) or ‘Non-Topical’ (One Health not referred to as a concept). Non-English articles were not reviewed. Articles without an abstract were categorized by title.

‘Topical’ references were included in a full text review if their abstracts referenced specific One Health research, action (e.g. collaboration, surveillance, zoonotic disease control program integrated across animal-human-ecosystem interface) or case studies. Articles for which full text could not be extracted were excluded. All articles that passed screening were examined by topic, sectors involved, metrics used, policy and regulations implemented, challenges posed, and best practices suggested. Articles that discussed a specific One Health intervention were then categorized based on whether an assessment of their intervention was or was not conducted. Evaluations that were conducted were then identified to be either quantitative or qualitative, and whether demonstrated metrics were intermediate or distinct outcome-based (Fig. 1).

Fig. 1

Flow chart of review of One Health literature.

Results

A total of 3858 articles were identified: 1333 in Pub Med, 1172 from Web of Science, and 1353 in Scopus. After removing 2019 duplicate papers, 1839 unique papers were included for a primary screening of title and abstract. Of these, 1025 were determined to be ‘Non-Topical’, seven were printed in a language other than English, and 807 were identified as ‘Topical’. Of the 73 ‘Topical’ articles included for full text review, 39 detailed a specific One Health action or intervention. The approach used was evaluated in 15 of these articles, with seven using quantitative metrics to report on a One Health program. Examples demonstrating quantitative cases are given in Table 1.

Table 1

Summary of One Health interventions and reported quantitative metrics found in the search of “One Health” literature.

	Country	Disease	Intervention	Outcome metric(s)	Outcome	Reference
Intermediate Indicators	Mexico	Chagas	Window installation program	Cost, number of windows installed	• Increase in average cost per household spent on Chagas control from an average of $US32 to $US35 (with insect screen installation) • Increase of 822 windows installed into 1606 homes	[29]
	Tanzania	Bovine Tuberculosis, Brucellosis, Extrapulmonary Tuberculosis	Testing wildlife, livestock and water sources for zoonotic pathogens; monitoring water quality and use; evaluating livestock and human disease impact on pastoral livelihoods; new diagnostic techniques; zoonoses training for Tanzanians; new health and environmental policy interventions	Identification of pathogens and local perceptions of disease transmission	• Identified BTB and Brucellosis in livestock and wildlife • More than 2/3 of participating pastoral households do not believe that illness can be contracted from livestock and 1/2 believe the same of wildlife	[31]⁎
	Ghana	NA	Field epidemiology and lab training	Number of disease outbreak and disease surveillance investigations	• 23 disease outbreak investigations were conducted by GFELTP residents between 2007 and 2011 • 31 evaluations of various disease surveillance systems were conducted between 2008 and 2011.	[32]
Outcome indicators	Chad	Trypanosomiasis	Insecticide footbath	Vector prevalence	• Reduction of 80% in total tsetse catches by end of 6-month footbath treatment	[33]
	India	Rabies	Vaccination and post-exposure prophylaxis	Incidence in animal bite/ exposure	• 30% reduction in animal bite and exposure cases reported	[35]
	Sri Lanka	Rabies	Vaccination and dog sterilization campaign	DALYs, social impact, cost, case load	• 738 DALYs averted • Increased acceptance of dogs roaming in society (5.68 mean acceptance score) • Increase in net cost to society of US$1.03 million • Caseload decreased from an average of 43 per year to 2 in first six months	[30]
	Thailand	Opisthorchiasis	Community education curriculum, praziqantel treatment	Reduction in vector prevalence	• Reduction in liver fluke infection from 67% to 16% • 9 schools certified as liver fluke-free • Fish species shows less that 1% prevalence compared to 70%	[34]

Paper did not formally evaluate program.

Programs reporting intermediate inputs were separated from those reporting targeted outcome metrics. Quantitative outcome metrics included data from economic, epidemiological and social assessments. Cost was defined both as direct monetary expenditures for the implementation of control activities (i.e. surveillance, window installation) [29], education programs, treatment costs, epidemiological investigations (i.e. disease outbreak investigations) and indirect losses (i.e. loss of income due to absence from work) [30]. Intermediate epidemiological parameters included number of wildlife sampled, number of water sources sampled monthly [31], and number of disease outbreak and surveillance investigations conducted by residents [32]. Outcome epidemiological parameters ranged from vector prevalence [33], [34] and human exposure incidence [30], [35] before and after intervention. In the instance that DALYs averted were used, this was based on death and psychological burden [30]. Other examples of outcome metrics included animal welfare scores to demonstrate impact of control activities on animal welfare and social acceptance scores regarding changes in society due to intervention [30].

Our findings suggest One Health programs have rarely been formally evaluated using quantitative information: 4.83% of ‘Topical’ articles detailed a specific One Health intervention and less than 17.9% of articles detailing a specific One Health intervention used quantitative metrics in their evaluation. Few papers included comparative costing data (n = 2) [30], [31]. Of the 7 articles reporting quantitative outcomes, three took the form of prevalence rates before and after the intervention [27], [29], [31]. Within the 15 articles that conducted an evaluation, in some instances efficacy was assumed or determined subjectively, with the perceptions of stakeholders or the number of partnerships as proxy for effective collaboration (i.e. [36], [37], [38], [39], [40], [41]). In some cases, metrics were siloed to a single discipline (e.g. human case load, livestock productivity, impact on ecosystem services) (i.e [40], [41]) or utilized a certain type of analysis (i.e. epidemiological or economic)(i.e. [42], [43], [44]). This finding is consistent with previous studies, particularly those that found environmental drivers, data and disciplines often excluded from One Health implementation [45].

While we did not conduct a formal analysis of qualitative outcomes, we collected examples showing how One Health was articulated in the literature. In some cases where authors advocated for a One Health approach, there was not a specific rationale for a program to be considered under the auspices of One Health. For instance, this may be due to authors not identifying specific implementation actions spanning the linkages across animal, human and ecosystem health (i.e. focusing on human disease prevalence or animal productivity) (i.e. [46], [47]). In other cases, effectiveness was mentioned but quantitative outcomes were not reported (e.g., that One Health approaches lead to “faster diagnosis” of disease or contribution to disease control without formal mention of how this was evaluated nor reported outcome metrics (e.g. DALYs, duration of outbreak)) (i.e. [48], [49]).

Discussion

While using a broader search term than past analyses, our findings further indicate a disparity between the suggested benefits of One Health and the lack of quantitative metrics demonstrating these purported benefits [23], [34].

The lack of reported systematic evaluation combined with the sporadic collection and presentation of One Health metrics limits a full understanding of outcomes. A standardized framework for systematic evaluation of One Health would be useful to identify how much value can be gained by fusing efforts across health sectors. Such a framework would need to include measures relevant to each sector. For example, DALYs (Disability Adjusted Life Years) are meaningful for public health in that they measure the overall human disease burden, but they convey no information about ecosystem impacts or poverty. Similarly, disease incidence and prevalence in either animals or people do not indicate the severity and distribution of a disease. Table 2 highlights possible outcome metrics representative of each sector and the benefits and limitations of each to adequately capture the One Health spectrum, thus underscoring the need for an integrated and standardized framework that different sectors can endorse and find value in.

Table 2

Potential outcome metrics used to evaluate One Health interventions.

Outcome Metric	Definition	Sectors Represented	Advantages	Disadvantages
Disability-adjusted life years (DALY)	Number of years lost to morbidity, disability or premature death	Human health	Measures overall human disease burden and accounts for long-term/chronic effects	Does not capture effects on animal health, ecosystem health, or impact on poverty levels in a population
Pre- and post- prevalence rate	Proportion of population that has particular disease at a specified point in time or over a specified period of time	Animal healthHuman health	Conveys magnitude of disease spread and transmission patterns of individual populations	Does not demonstrate relative severity or distribution of disease
Pre- and post-mortality rate	Number of deaths in a given period or area, or from a particular disease	Animal healthHuman health	Measures direct severity of disease	Does not convey secondary effects (i.e. impact on governance, poverty levels, effects of diseases with high morbidity rates that are not fatal)
Outbreak duration	Time span between initial disease occurrence and end in a population	Animal healthHuman health	Comparable across interventions and diseases	Does not represent severity, or lack there of, of disease spread (i.e. a long duration that has few cases)
Cost	Monetary price associated with intervention efforts or lack there of (i.e. vaccination campaign, loss of ecosystem services, loss in animal productivity)	Animal health Human health Ecosystems	Easy to compare across sectors and interventions	Does not account for a more cost-effective option in relation to disease mitigation that appears more expensive
Monetary unit (cost-benefit analysis)	Assigns cost to all monetary and non-monetary outcomes (i.e. treatment costs, social impact, livelihood) to compare scenarios	Animal health Human health	Broad scope of application, allowing information to extend to different sectors of the economy. This is particularly attractive when outcomes have a value to society that is not necessarily equal across locations and populations; for instance, ecosystem services that hold differential worth.	Sometimes difficult to monetize biological and environmental effects (i.e. differing values across societies and cultures, ethical concerns regarding the monetization of ecosystem values)
Monetary unit (cost-effectiveness analysis)	Identifies most cost-effective option, expressed in terms of monetary cost per unit (i.e. cost per DALY gained)	Animal healthHuman health Ecosystems	Overcomes difficulty of attributing monetary units to health and environmental effects as seen in CEA	Requires initial set of standards (i.e. budget constraints, assigned threshold) and alternatives to which to be compared, problematic with lack of agreement on alternatives or control measures, difficulty comparing interventions that do not use same units of evaluation
Productivity	Effectiveness of goods and services production of animal or environmental sectors	Animal health Human healthEcosystems	May indicate changes in health status of animals or ecosystem, represents in-direct, secondary effects of outbreak on poverty or biodiversity	Not easily comparable across interventions, sectors or regions
Perception	Qualitative measure of whether expectations of intervention were met or general reaction towards outbreak response by policy and non-policy stakeholders	Animal health Human healthEcosystems	May highlight transmission pathways through human behavior, attitudes towards intervention or disease, areas of positive and negative externalities	Subjective measure that is not directly comparable across individuals

Economic analyses may be more applicable to One Health because they often quantify outcomes across sectors. For example, similar cost-benefit analyses are used in evaluating the allocation of resources in public health [50], animal welfare and production, and ecosystem services [51]. Decision- making may be streamlined when the allocation of resources across different outcomes of economic sectors are translated into monetary terms. Additionally, costing data would be critical for identifying the often-perceived efficiency and reduction of redundancy in One Health programs. Cost effectiveness analyses may be particularly useful for One Health. These identify the most cost-effective option in comparison to a set of alternatives, expressed in terms of monetary cost per unit of outcome (e.g. cost per DALY gained). However, they are limited by requiring an initial set of standards (i.e. budget constraints, assigned thresholds) and alternatives to compare and assess the value of a program against, which due to differing interests or priorities may be difficult to agree upon [30], [51].

Economic data may oversimplify the severity and distribution of disease. For instance, economic shocks often exist beyond the cost of control, agriculture loss, income loss, and declining agricultural productivity. Metrics that account for these wider societal implications could also be used in evaluating One Health– such as the extent to which livelihoods are affected, poverty is exacerbated, or account for specific populations that may face a disproportionate burden of disease. For example, in a 2010 study of avian influenza in four Sub-Saharan countries, an outbreak not only increased household vulnerability for small-scale poultry farmers, but increased vulnerability where livestock was used as a risk-coping mechanism—influencing savings, food security, and gender equality [52]. Similarly, stakeholder behavior may be critical to understanding the efficacy of a One Health intervention. In response to empirical studies determining the effectiveness of insecticide footbaths in Trypanosomiasis prevalence, a 2011 study in Burkina Faso analyzed the adoption rate of footbaths by farmers. While 78% considered the footbath useful against tick and tsetse flies only 60% were ready to invest money to build new footbaths [53].

Current evaluative tools (i.e. cost analysis) may not easily conform to One Health principles, as reported by Hasler and colleagues when trying to apply cost-analysis to assess effects on animal health [30]. Given the participation required from multiple sectors, it may appear overwhelming to have one framework representing multiple types of interventions, diseases, and social and geographic contexts. While analytical tools cannot individually account for the scope of One Health or uniqueness of specific interventions, a combination of assessments (e.g. epidemiological assessments, environmental impact assessments, socio-economic assessments) can be manipulated for a variety of circumstances and still produce comparable, targeted data. This may be easiest by identifying outcome indicators (i.e. cost, avoided cost, DALYs, outbreak duration, mortality, morbidity, productivity) and intermediate indicators as means to calculate targeted outcomes (i.e. number of questionnaires disseminated, number of wildlife tests, number of water sources sampled, number of workshops, vaccination rate). While intermediate activities may be siloed to a specific discipline, they may be part of reaching a cross-sectoral outcome. For instance, intermediate activities such as sampling wildlife and testing water sources contribute to the outcome target of assessing wildlife and livestock for zoonotic pathogens in addition to ecological monitoring of water quality and availability [31]. Further, cross-sectoral collaboration towards a systematic set of metrics may be instituted by one sector, but informed by many sectors.

Ideally, a plan for assessing the effectiveness of One Health interventions will be considered even before program implementation. This has been seen with an ICONZ (Integrated Control of Neglected Zoonoses) project for Brucellosis and Bovine Tuberculosis in Nigeria in which pre and post intervention data (i.e. disease burden, socioeconomic indicators, monetary and non-monetary loss), is integrated into the designing and planning of the intervention itself [54]. Further, some approaches have identified the different components of evaluation in terms of the required intermediate inputs to achieve desired outputs. For instance, the HALI project's multilevel framework separates the program's “Objectives”, such as “evaluation of livestock and human disease impacts of livelihoods of pastoralists”, and the required “activities” aimed to capture those desired outcomes, in this case “159 household surveys” [31].

It is possible that the single search term used in our review limits inclusion of programs that could be considered One Health but did not specifically name the approach (e.g. ‘Ecohealth’ programs). However, the programs we examined were self-identified as One Health, and therefore, by implication, purported to demonstrate outcomes from combining cross-sectoral approaches.

While our study highlights a lack of objective evaluation of the One Health approach, there have been some significant efforts to overcome this. Hasler et al. [30]. evaluated a rabies control program using epidemiological, economic, social, animal welfare and ethical assessments. Results showed that over four years, One Health measures increased costs by approximately US$1.03 million more than previous program, reported 738 DALYs averted, and decreased caseload from 43 per year to 2 in first six months. The 2008 strategic One Health framework created by WHO, OIE, FAO, UNICEF, and Word Bank [55] identified elements essential to One Health decision-making and resource allocation. Narrod et al. [56]. evaluated societal costs of zoonotic diseases through an integrated economic and epidemiological framework. Coker et al. [42]. put forth a framework to identify One Health research questions and generate policy. De La Rocque et al. [57]. proposed a framework to respond to RVF (Rift Valley Fever) outbreaks by outlining responsibilities to assigned committees composed of different stakeholders. These efforts are often hampered by the nature of collaborative partnerships, which may make it difficult to determine who has jurisdiction over disease control and reporting.

Given the participation required from multiple sectors, designing one framework to represent multiple types of interventions, diseases, and social and geographic contexts will be challenging. To deal with this, more generic frameworks may allow comparability among sectors. For example, a strategic framework developed by WHO to manage Ebola virus and Marburg virus was adjusted for RVF (Rift Valley Fever) control as the outbreak followed a similar chronology of events [57], [58]. Current initiatives include the Network for Evaluation of One Health, which brings together researchers across Europe to develop a protocol for systematic evaluation, compile case studies and conduct meta-analyses, and form policy recommendations [59], as well as the Checklist for One Health Epidemiological Reporting of Evidence (COHERE) [60], which seeks to develop reporting guidelines and promote integration of information across the three main One Health sectors. If designed in coordination with decision makers, at scales from single disease programs to global intergovernmental agency collaboration, they may help build an evidence base around the value of One Health to move forward more integrated policies.

Conflict of interest

None.