Economic Evaluations of Public Health Surveillance Systems: a Systematic Review.

Introduction

‘Public Health surveillance is the ongoing systematic collection, analysis and interpretation of health data, closely integrated with the timely dissemination of these data both to those providing the data and to those who can apply the data to control and prevention programs’. Public health surveillance is therefore, closely linked to action as it provides accurate and validated information to local and national health authorities in order that these same authorities can implement appropriate prevention and control measures and health promotion strategies. This definition was set up several decades ago. Since then, the adoption of the revised International Health Regulations in 2005 has implemented new regulations such as national obligations to guarantee a set of core surveillance and response capacities to prevent international spread of disease irrespective of its origin (biological but also chemical or radio-nuclear sources). Furthermore, new diseases have emerged (H1N1pdm09, Middle East Coronavirus), some others have spread dramatically such Ebola virus disease in West Africa. In the context of globalisation, the concept of public health surveillance has evolved to include public health security. , Politicians and stakeholders rely upon their public health agencies to be informed in a timely fashion and to be able to respond promptly to all potential health treats that may arise as populations need protection whatever the risk. For public health surveillance, this implies the need to monitor a very large spectrum of all hazards health threats. At the same time, as many public agencies are currently facing budgetary constraints, they must allocate theirs resources as efficiently as possible, as surveillance systems cannot be implemented for all known health threats. The priorities for diseases surveillance need to be reviewed regularly in order to guarantee that the most topical public health issues are consistently tackled and to ensure quality and effectiveness of surveillance. Furthermore, best practice recommends periodic evaluation of public health surveillance systems (PHSS). Like other national public health agencies, the French National Institute of Public Health (InVS) has implemented several prioritization exercises to guide its own surveillance strategy. It has also implemented an external and formal evaluation process of its surveillance systems, based on a generic evaluation protocol drawn up in accordance with the United States Centers for Disease Control and Prevention (CDC) guidelines, and using a design developed by the European Center for Diseases Prevention and Control (protocol provided by Dr A. Ammon, personal communication). Four surveillance systems for infectious diseases were assessed between 2009 and 2013. Given the current economic context, this assessment process was less than comprehensive as it did not consider the increasingly important question of the costs and the benefits of surveillance systems. The CDC guidelines indicate that costs should not be estimated alone but be judged relative to benefits or at least relative to the effects of the surveillance system on prevention, care, research or decision-making. The guidelines state also that ‘a more realistic approach would be to judge costs on the objectives and usefulness of the system’. However, the CDC guidelines do not clearly define methods to do so. A WHO report issued in 2005 underlined that the evaluation of the costs and the benefits of national surveillance and response systems was an area of great importance. In the report, the authors indicated that little related work has been carried out to that point and highlighted a number of methodological difficulties in evaluating benefits of a surveillance and response system such as implementing the appropriate study design, defining the scope of the surveillance or assigning a monetary value to health outcomes averted.

To assess the current development of economic evaluations in the field of public health surveillance, we performed a systematic review of the literature. In this review, we also assessed the methods used in the selected studies to evaluate the benefits of public health surveillance.

Methods

For this systematic review, we followed PRISMA methodology PRISMA checklist for this review is available as supplementary material on line. The search was performed in two major databases for health interventions Medline and Scopus. In each database, two search strategies were performed. The first search strategy used in each of the database was to employ a structured thesaurus in order to focus the search. The second search strategy in each database use text words (in the title and in the abstract for Medline and in the title for Scopus) in order to capture articles not yet or incorrectly indexed. We focused on two major topics: public health surveillance and full economic evaluations. Public Health surveillance is included in the MESH term ‘population surveillance’ and full economic evaluations [cost-benefit analysis (CB), cost-effectiveness analysis (CE) and cost-utility analysis (CU)] were included in the MESH term ‘cost-benefit analysis’. The four search strategies are displayed in Appendix 1.

Additionally, a search using terms ‘Surveillance’ AND (‘CB analysis’ or ‘CE analysis’ or ‘CU analysis’) in any field was conducted in the CRD’s NHS Economic Evaluation Database (NHS EED).

Inclusion criteria

Only studies published in peer-reviewed literature which performed a full economic evaluation of the PHSS in question were included in the present review. A full economic evaluation is defined as ‘the comparative analysis of alternative course of actions in terms of both their costs and consequences’. Three main types of full economic evaluation were considered eligible: cost-effectiveness analyses, cost-utility analyses and cost-benefit analyses. The research was conducted on relevant articles published in English or French written articles until December 2013, and no time period was specified.

Exclusion criteria

Cost-minimization analyses were excluded as the consequences or the benefits of compared surveillance systems were assumed to be equivalent and only relative costs were compared.

PHSS evaluations on technical attributes other than costs and benefits (sensitivity, timeliness, data quality…), cost evaluations of a PHSS, comparison of costs between two types of PHSS, comparison of performance indicators between two types of PHSS were excluded of the analysis as they do not provide any evaluation of the benefits of the surveillance. Besides, in order to get insights in a reproducible methodology for public health surveillance, evaluations of surveillance systems in the field of veterinary surveillance were excluded of this review. Finally, articles written in a language other than French or English were excluded.

A first review on the title and the abstract was performed to identify and exclude articles written in a language other than English or in French, and those related to veterinary surveillance. The remaining retrieved papers were eligible for a full text review. The first author selected the studies which met the inclusion criteria. When there was any doubt about whether to include or not the article, the latter was reviewed by the second author and discussed with all the authors to reach a decision.

Results

Studies selection

The two search strategies on Medline provided 407 articles of which 5 were duplicates leading finally to 402 papers selected in this first database. The research on the second database (Scopus database) identified 556 articles. Overall, the searches in the two databases produced 958 articles, of which 343 were duplicates and therefore excluded. Finally, the four search strategies in the two databases led to 615 original articles ( figure 1 ).

Figure 1

Flow diagram of studies selection

After a first review on the title and abstract of the 615 articles, 21 papers written in a language other than English or French and 27 other articles related to veterinary surveillance were excluded. Finally, 567 papers were eligible for a full text review.

Among the 567 papers selected, 461 were not full economic evaluations and were therefore excluded while 106 included full economic evaluations. The majority of the latter assessed medical screening treatments (including vaccination strategies) ( n = 80). Among the 26 remaining articles, 17 addressed public health interventions and finally only 9 full economic evaluations of a surveillance system were retrieved and therefore selected according our review’s eligibility criteria. Among these nine selected studies, eight were identified in the two databases and one was only identified in the Scopus database. The references of the selected studies were also checked and no additional study was found. Two of the nine studies , referenced one of the other seven selected studies. ,

The query performed in the NHS EED identified three articles already identified from the two above-mentioned databases.

Summary of the selected studies

Most of the items listed in the CHEERS checklist were used to assess the method and the results of each of these studies looking at target population, setting and location, study perspective, time horizon, discount rate, choice of model, choice of health outcomes, estimation of costs and resources, currency and conversion and incremental costs and outcomes ( tables 1 and 2 ).

Table 1

Cost-benefit analyses

	Country Year of publication	Methods	Benefits	Costs	Main Results
Hinds 17	USA-1985	Active versus Passive surveillance for hepatitis A	Costs of averted HVA cases among contacts of reported cases through the active surveillance	No perspective mentioned No discount rate applied * costs linked to surveillance and tracing contacts: personnel, travel and cost linked to prophylaxis (Personnel, IG treatment)	A 2.8-fold increase in reported HVA cases observed with the active surveillance compared to the passive one. The benefit: cost ratio active versus passive surveillance is estimated at 2.5:1
Roberts 19	UK- 1989	Early intervention for an outbreak of salmonellosis versus no intervention	Costs avoided by the intervention (secondary prevention) Costs potentially avoidable if the outbreak had been prevented (primary prevention)	Societal perspective No discount rate applied * Costs linked to surveillance: personnel and laboratory Costs linked to health care: primary care and hospitalization Costs to family and society: loss of productivity, pain, loss of life, recalled and destroyed product	The benefit: cost ratio of the intervention implemented to investigate and to limit the outbreak is estimated at 3.5:1 for the public sector and at 23.3:1 for the society
Payne 20	UK-1992	Comparison of working hours spent in data management between a computerized system and manual entry data	Evaluation of the time saved after the introduction of the computerized system	No perspective mentioned No discount rate applied * Capitals costs, costs of operating the system, staff training	The time saved in managing data was estimated at 400 h per year. Other benefits were difficult to quantify and refer to a higher data quality (accuracy, reactivity) or personnel satisfaction
Elbasha 22	USA-2000	Estimation of threshold number of cases, a subtype-specific system for identifying EC O157/H7 outbreaks need to avert for the costs to be equal to the benefits under two scenarios: a constant number of case averted every year and a given number the first year and no cases in subsequent years	Estimation of an average social cost of EC 0157/H7 according to nine severity categories. Costs included were medical costs, productivity losses and lost lifetime earnings	Societal perspective Discount rate applied Costs of installing and operating the surveillance system: equipment, personnel, laboratory and costs of investigation for an outbreak (laboratory, personnel, telephone, data management, meetings)	Under the first scenario, the costs of the system are equal to the benefits if five cases are averted each year during 5 years. Under the second scenario, 14 cases averted the first year will be enough for the costs to be equal to the benefits. As comparison, 90 EC cases were reported in year 1998 in the state where the study was conducted
Ebi 17	USA-2005	Estimation by multiple linear regression of excess mortality during 45 days of heat waves which led to a warning from an early alert system and to the implementation of actions to reduce mortality risk	Each warning issued by the system saved 2.6 lives assuming no mortality displacement. A monetary value was assigned for each life saved	No perspective mentioned No discount rate applied Costs linked to extra-wages (costs link to personnel mobilized during the alert)	The warning was issued 45 days during 3 years period leading to an estimation of 117 live saved Assuming that the 117 lives saved occurred on people aged 65 years of older and giving monetary value of 4$million for one live, the system allows to save $468 millions

* Retrospective costs analysis.

Table 2

Cost-effectiveness analyses

First Author (Ref. number)	Country-Date of publication	Methods	Effectiveness	Costs	Main results (Base case scenario)
Morris 21	UK-1996	Estimation of costs and benefits of HIV/AIDS surveillance and prevention program	Estimation of the number of lives prevented by the HIV surveillance and program prevention	Public health perspective Discount rate applied Capital costs, salary costs and costs of consumed goods for six components of the HIV and AID surveillance program	Reference cost-effectiveness (C/E) ratio defined by the authors equals 1994 £14 721 To be cost effective, the HIV surveillance and prevention program must prevent at least 333 new cases each year In 1993–94, 2514 newly HIV infected cases were reported. The number of cases to be prevented is equivalent to 0.4% of the number of new infections reported
Wild 23	USA-2005	Markov model to compare annual surveillance and passive case finding for isocyanate asthma among a cohort of 100 000 exposed workers	Number of Qaly saved	Societal perspective and employer perspective Discount rate applied Costs linked to screening and diagnosis Cost of productivity loss and absenteeism Costs linked to health care: primary care and hospitals	For 100 000 workers exposed during 10 years, the incremental C/E ratio for surveillance was estimated at $24000/Qaly saved Surveillance was cost-saving from the employer perspective
Brathwaite 24	USA-2006	Cohort model to simulate the effects of aerosolize anthrax spores among 100 or 100 000 persons-single exposure. Comparison of different mitigation strategies including an emergency surveillance and response system (ESR)	Number of live saved	No perspective No discount rate applied Costs of installing and operating the system Costs of drugs and vaccination Costs of outpatient and inpatients medical care for inhalational anthrax	If the attack is certain ( P = 1.0), the cost-effectiveness (C/E) of an emergency and surveillance system is equal to $73/life saved for 100 000 person -exposure and increases up to $1 680 000/life saved for 100-person exposure If the attack is uncertain ( P = 0.001), C/E = $142 000/life saved per 100 000-person exposure.; C/E > $1 billion/life saved for 100-person exposure
Somda 25	Burkina Faso-2010	Estimation of the impact of an integrated disease surveillance and response program for meningitis. Surveillance data were compared before and after the implementation of the program	Number of case prevented Number of death prevented Number of sequelae prevented	Public health care perspective Discount rate applied Costs for each activity of the program (personnel, transportation items, office consumable goods, public awareness campaigns, laboratory, response materials as supplies, capital items) Medical costs (local and regional level)	The C/E ratio was estimated at $23/per case averted and at $98/meningitis-related death averted Both C/E ratios were cost-saving

The nine selected studies accounted for only 1.5% of the 567 references initially identified. They were published between 1986 and 2010. Five of the nine topics studied infectious diseases surveillance (HIV infection, salmonella outbreak, meningitis, hepatitis A and Escherichia coli (EC O157:H7) infection). ,,,, Three addressed environmental exposure (heat waves, professional asthma and anthrax exposure) , and one described the benefits of the implementation of an electronic surveillance system. These evaluations were conducted in the USA ( n = 5), in UK ( n = 3) and in the Burkina Faso ( n = 1). There were three cost-effectiveness, ,, one cost-utility and five cost-benefit analyses. ,

Target populations varied among the studies from a whole country or a federal state , to a single county or city. , Two studies were performed on theoretical exposed populations from 100 to 100 000 persons exposures. , Time period was mentioned for four studies and costs were adjusted to the study period in all four. Three of the studies were based on retrospective events lasting from 22 weeks to 3 years. The two remaining studies were based on one year surveillance periods. ,

With respect to the four cost-effectiveness studies, two cost-effectiveness and the one cost-utility analysis, compared two surveillance strategies (active case finding versus passive surveillance or surveillance and response versus no surveillance). A Markov model was used for two of these studies. , An experimental design was set up for the third one where the authors used the surveillance data from the previous 7 years’ data for 5 years after the implementation of the surveillance and response program in order to assess the net costs for each health outcome prevented. Health outcomes were expressed in number of cases or death prevented for two studies and in number of Qalys saved for one. These three studies have estimated an incremental cost-effectiveness (or cost-utility) ratio for the more efficient strategy. The fourth cost-effectiveness study focused on HIV surveillance. The authors of that study were not able to estimate the number of cases prevented by the HIV surveillance activities alone (which constituted only a part of the entire HIV prevention program). To overcome this methodological issue, the authors defined a level of cost-effectiveness based on a median value of ratios of 116 cost-effectiveness published studies. Based on this ratio, they estimated the number of HIV infections the entire HIV prevention program would need to prevent in order for it to be considered cost-effective.

With respect to the five cost-benefit studies, two were based on a comparison of two surveillance systems (active versus passive and electronic reporting versus manual reporting). Benefits were estimated as the costs saved for the cases prevented for one study and by the time gained in employing electronically captured surveillance data for the other. , A third study described the surveillance and response activities implemented to control an outbreak of salmonella. Benefits were estimated by the costs avoided of resulting potential cases and associated deaths if the outbreak had not been controlled by surveillance and response. In the fourth study, the authors estimated the threshold number of cases of Escherichia Coli O157:H7 (EC O157:H7) that the system would need to prevent in order for this system to be considered cost-beneficial. The fifth and final study modeled, over a 3-year period, the number of deaths prevented among persons aged 65 years and over during the days when a heat wave surveillance system had issued a warning.

The type of costs included was heterogeneous between the nine studies and depended on the scope of surveillance and response activities considered and the perspective used. The perspective used for the cost analysis was mentioned in five of them. ,,,, The four remaining in which the perspective was not stated, have used a government-funded public health perspective as they included only the costs linked to the surveillance system and response. ,,, Accordingly, some authors included only the direct costs for operating the surveillance system (capital costs, salaries or training costs, recurrent costs) , while others also included indirect costs linked to treatment and care or those linked to biological testing. ,,, Some authors also included indirect costs related to loss of productivity or pain and suffering as they use a societal perspective. , In one of the five cost-benefit studies and in three of the four cost-effectiveness studies, ,, a discount rate was applied. Among the five modelling studies, four have performed sensitivity analyses.

Among the five cost-benefit studies, four concluded that a positive benefit existed for active surveillance versus passive or versus no surveillance in term of money or time spent. In the remaining study, the authors considered that the surveillance system was cost-beneficial if five infections were averted per year. This estimate is much lower to than the 90 cases of EC O157 reported to the surveillance system the year of the study.

Among the four cost-effectiveness and cost-utility studies, two have considered that the implemented surveillance was cost-saving. , The third one found different cost-effectiveness ratios under different scenarios of anthrax exposure (probability and number of persons exposed). In one over the four scenarios, the surveillance and emergency response was cost-effective. For the fourth study focusing on HIV surveillance, the entire HIV prevention program was estimated to be cost-effective if at least 333 new cases of infections were averted each year which represents 13% of the new diagnosis reported the year of the study.

Discussion

Although the efficiency of public health surveillance has been under the scrutiny by governments for many years, this systematic review clearly indicates that, to date, very few economic evaluations of PHSS have been performed worldwide. This observation was already pointed out in 2005 in the above mentioned WHO report.

Another systematic review on surveillance system evaluations published in 2012 found only three articles which aimed at evaluating cost-effectiveness of PHSS. Only of the three was selected in this review. The other two articles compared costs or costs and performance indicators, respectively among two surveillance systems and therefore did not evaluate surveillance benefit. , For this reason, these two studies did not meet our eligibility criteria. The current review did not find any published studies on this subject from 2010 onwards.

Three main approaches, -cost-effectiveness, cost-utility and cost-benefit analyses- were used for economics evaluations of PHSS in the studies selected in this review. Studies focusing on surveillance system’s cost-effectiveness or cost-utility provide valuable information for comparing several surveillance strategies and ranking these same strategies among other medical or public health interventions. However, the benefit measured in these studies was limited to the number of health outcomes prevented by one surveillance strategy compared with another one.

The cost-benefit analysis studies reviewed here, all evaluated the benefits of surveillance by assessing the number of cases or deaths the particular system prevented, assigning a monetary value per case prevented or life saved. Classically, if the expenditures for installing and operating the surveillance system are equal to or less than those for treating patients then the system is considered cost-beneficial. This approach implies not only knowledge of the costs of the disease in both its acute phase and for chronic disease over the long term, but also a consideration of all clinical presentations from the mildest to most severe. Besides, assigning a monetary value to a live saved implies obtaining this information from the literature or implementing an ad hoc study.

Estimating the costs of a health intervention requires a standardized approach and implies the need to follow certain methodological steps such as defining the perspective, including direct and indirect costs, discounting future costs and health outcomes, describing all the resources used and assigning a monetary value to these resources. In this review, discrepancies were found between the different studies examined with respects to the type of costs included and with the definition of scope of the surveillance and/or response activities they considered. This highlights the practical obstacles to making standardized costs estimations of PHSS. The above mentioned WHO report suggests including both the costs of surveillance and response as the ultimate purpose of surveillance information is to inform public health decision-making and to guide response and control activities. A conceptual framework of public health surveillance and action comprising six surveillance core activities such as detection, confirmation or analyses and two public health actions (acute and planned responses) and four support activities such as communication and training has recently been developed. This framework aims at assessing the performances of PHSS and could also be useful for conducting standardized cost analysis for PHSS.

In this review, several of the studies examined included surveillance and response activities where the authors estimated the benefits of surveillance through the number of cases or deaths prevented by response activities. Nevertheless, this approach cannot be applied when the primary objective of the surveillance activities is not linked to immediate response as illustrated in the study of HIV cost-effectiveness. Although information coming from surveillance is crucial to guide prevention activities and to allocate funds, in that study the authors underlined their difficulties in quantifying the surveillance-specific benefits of the surveillance from all the benefits provided by the a wide range of prevention activities implemented in the specific public health HIV program.

With respect to the CDC guidelines mentioned above for evaluating PHSS, costs and benefits of a surveillance system should be estimated in the light of many other criteria including the objectives and the utility as well as several technical performance attributes. The methods used in the selected studies did not take into account all these parameters and could therefore have led to an underestimation or an overestimation of the benefits of the surveillance if, for example, the objectives were not been reached or if some technical attributes such as timeliness did not meet the quality standards. Finally, assuming that benefits can be defined solely in term of the costs of the health outcomes prevented may constitute a limited approach as cost-benefit analysis of PHSS includes also an option value and the need to take into account positive non- monetary effects like the perception of health security where the general public is aware of the fact that health threats are regularly monitored and are controlled in a timely and efficiently manner irrespective of the number of cases avoided. There are also economic benefits in preventing diseases or large outbreaks in that people to continue to work and to be productive.

These methodological drawbacks and difficulties listed above may to some extend explain why so little work has been carried out to date in the field of economic evaluation of surveillance systems. In the light of the limitations of commonly used approaches, other methods of economic approaches may be useful. Among these, conjoint analysis (CA) could overcome to some extent the limitations and methodological issues discussed above. The theory behind this approach is that any product or good or service can be described by a set of characteristics and that the extent to which an individual places value on a product is determined by the level of these characteristics. CA asks individuals to state a preference by presenting competing scenarios with both desirable and undesirable characteristics of a product or a service. CA has been applied to health care and public health interventions such as HIV vaccine and cancer control strategies. , More recently, CA was performed to prioritize zoonotic diseases of public health concern. CA has been also used to derive utility weight for QALY or to estimate willingness to pay for CB analysis. To our knowledge, the method has not been used for a PHSS and may be worth exploring. Indeed, this type of economic evaluation could help policy makers and health professionals elicit preferences and help estimate their willingness to pay for a surveillance system according to different system characteristics and performance. This technique may also be useful for estimating the non-use value of a surveillance program. This information could be useful to public health agencies and could help them to prioritize surveillance in a context of limited resources.

This review has several of its own limitations. First, because ‘Public Health surveillance’ is a specific term in Medline, we had to use the generic MESH term ‘Population surveillance’. Consequently, many articles were related to medical interventions rather than to public health surveillance and the selected studies accounted for a very small percentage of the identified articles. Conversely, had we used a query with the term ‘Public health surveillance’ we would have identified only seven references, none of which being one of those finally selected. Additionally, focusing on public health surveillance evaluation and displaying the different types of studies (economic versus non-economic) led to a much higher number of articles (over 1200 articles) and did not retrieve the selected studies. However, the fact that this review is extensive and that the majority of the selected studies (8/9) were identified in both databases proves that few economic evaluations of a PHSS have been performed to date and that the risk of having missed an article is most probably very low.

Second, the line between public health surveillance and medical intervention or control intervention is often a very thin one and initially some of the studies finally selected were the subject of discussion between the authors of this present review in order to decide whether or not they did in fact deal with surveillance. Third, this article did not aim to critically review the quality of the selected studies. Certain methodological issues are pointed out in this article to underline how difficult it is sometimes to perform a full economic analysis in the field of public health surveillance. Finally, our primary objective was to look at the methods used in the economic evaluation of a PHSS. We did not focus on the results of these economic analyses despite the fact that in most of these studies, the authors considered the surveillance system in question either cost-effective or cost-beneficial.

In conclusion, this review demonstrates that few studies have been performed to evaluate the benefits, a PHSS can bring. The most frequent approach employed to assess these benefits is to value the health outcomes prevented by the surveillance and response activities or to estimate the costs per number of health outcomes avoided by comparing two surveillance strategies. By using these approaches, other dimensions are overlooked. As an alternative, using other economic methods such as conjoint analysis could show promise for future development.

Click here for additional data file.