Triggering interventions for influenza: the ALERT algorithm.

BACKGROUND: Early, accurate predictions of the onset of influenza season enable targeted implementation of control efforts. Our objective was to develop a tool to assist public health practitioners, researchers, and clinicians in defining the community-level onset of seasonal influenza epidemics.
METHODS: Using recent surveillance data on virologically confirmed infections of influenza, we developed the Above Local Elevated Respiratory Illness Threshold (ALERT) algorithm, a method to identify the period of highest seasonal influenza activity. We used data from 2 large hospitals that serve Baltimore, Maryland and Denver, Colorado, and the surrounding geographic areas. The data used by ALERT are routinely collected surveillance data: weekly case counts of laboratory-confirmed influenza A virus. The main outcome is the percentage of prospective seasonal influenza cases identified by the ALERT algorithm.
RESULTS: When ALERT thresholds designed to capture 90% of all cases were applied prospectively to the 2011-2012 and 2012-2013 influenza seasons in both hospitals, 71%-91% of all reported cases fell within the ALERT period.
CONCLUSIONS: The ALERT algorithm provides a simple, robust, and accurate metric for determining the onset of elevated influenza activity at the community level. This new algorithm provides valuable information that can impact infection prevention recommendations, public health practice, and healthcare delivery.

Influenza, a leading cause of death in the United States [1], causes approximately 40 000 deaths each year, more than motor vehicle accidents [2]. Despite the infections and hospitalizations triggered by influenza annually [3-5], the ability to predict the timing of these outbreaks remains elusive. Public health practitioners and clinicians lack a reliable, simple method for determining the onset of a period of elevated influenza incidence in a community. Early, accurate predictions that influenza transmission is rising would enable a proactive and fast response to increased transmission and outbreaks.

As one example of how these predictions may be used, many hospitals establish a time period of enhanced precautions for healthcare workers, family members, and visitors during the peak winter influenza season [6]. Karanfil et al implemented a similar strategy to identify an influx of infectious children with respiratory illness and decreased nosocomial transmission by almost 50% [7]. The Centers for Disease Control and Prevention (CDC) and the American Academy of Pediatrics recommend a number of strategies to be used to control influenza during periods of increased risk, including minimizing elective visits of individuals with suspected influenza and setting up triage stations [8, 9]. Identification of periods of increased influenza incidence could also help target screening for influenza antiviral use. However, little guidance is available for facilities to identify periods of increased influenza incidence.

In recent years, researchers have attempted to predict the course of seasonal influenza epidemics [10-15]. Some introduce important methodological advances in disease forecasting. Yet, facilities face multiple challenges in implementing these methods to define a period of enhanced precautions. Some methods have shown promise for influenza prediction but require extensive methodological training to implement [15]. Another prominent example, Google Flu Trends, a publicly available and easily accessible tool, can provide timely insight into overall trends of cases in a given region [10, 16]. However, Google Flu Trends has shown poor performance in estimating the burden of influenza [17], correlates better with influenza-like illness than with laboratory-confirmed influenza [18], cannot accommodate data from healthcare settings, and is considered a supplementary rather than authoritative source for influenza surveillance [19-21].

There has been a need, therefore, for a translational system that uses surveillance data from a particular setting to inform a simple rule to identify periods of high influenza incidence and guide the initiation of enhanced precautions. By identifying outbreaks early enough to implement public health measures, many potential cases could be prevented. Ideally, such a tool is both sensitive and specific in identifying increasing activity. The costs of activating too late are clear: preventable morbidity and mortality. On the other hand, there are real costs to intervening too early, leading to waste of resources and, potentially, increased community member fatigue.

In the context of a large-scale multisite clinical trial, the Respiratory Protection Effectiveness Clinical Trial (ResPECT), we developed the Above Local Elevated Respiratory Illness Threshold (ALERT) algorithm, a method to trigger the study intervention period so that it overlapped with the period of highest seasonal influenza activity [1, 6, 22]. This simple, easy-to-implement, and data-driven algorithm can help public health practitioners, researchers, and clinicians define the onset of seasonal influenza epidemics in a community, such as a city or a hospital, that systematically collects surveillance data on influenza.

METHODS

Settings

The Johns Hopkins Hospital (JHH) is a 900-bed tertiary care center in Baltimore, Maryland, with a 200-bed children's hospital. During the period for which we have data, surveillance for respiratory viruses was a mix of active surveillance (for a limited time in pediatric units) and passive surveillance, as previously described [7, 23]. Children's Hospital of Colorado (CHCO) is a 414-bed hospital serving Denver, Colorado, and surrounding areas. A passive surveillance system is in place at CHCO, where children with respiratory symptoms are tested for common respiratory viruses [24]. In both centers, testing included culture, antigen testing, and polymerase chain reaction.

Data Sources

We obtained weekly case counts of laboratory-confirmed influenza A virus from surveillance systems at both institutions between 2001 through 2013. We excluded data from the summer of 2009, due to anomalous H1N1 disease transmission associated with the pandemic.

The ALERT Algorithm

The ALERT algorithm works by defining an “ALERT period”—a window of time in a given influenza season when elevated incidence is expected. In brief, historical data from the same surveillance system are used to establish a number of cases or threshold (eg, 5 cases). When the observed number of cases in a given week is greater than or equal to the chosen threshold, the ALERT period begins. In a given season, real-time case data determine the start of the ALERT period.

However, choosing the right threshold poses a challenge. To guide the user to an evidence-based decision, the ALERT algorithm summarizes data from previous years as if each of several thresholds had been applied. For each threshold considered, the ALERT algorithm calculates and reports a set of metrics across all years of historical data. A complete list of these metrics is provided in Table 1.

Table 1.

ALERT Algorithm Metrics

The median ALERT period duration.

The percentage of all influenza cases in an entire season contained within the ALERT period (median, minimum, maximum).

The fraction of seasons in which the ALERT period contained the peak week.

The fraction of seasons in which the ALERT period contained the peak week ± k weeks (where k is specified by the user and defaults to 2 weeks).

The mean number of weeks included in the ALERT period with counts less than the threshold.

The mean difference between, for each season, the duration of the ALERT period and the duration of the shortest period needed to capture P percent of cases for that season.

Abbreviation: ALERT, Above Local Elevated Respiratory Illness Threshold.

Given these summary metrics for different thresholds, the user may choose the threshold that fits their needs. This threshold may then be applied to future surveillance data in real time. In the applications shown below, we chose the largest thresholds that had captured at least 90% of cases in half of the previous influenza seasons.

The ALERT algorithm calculations may be implemented by using a Web applet [25], the ALERT package for the R statistical programming language [26, 27], or an Excel spreadsheet provided as a supplement. The spreadsheet provides sufficient calculations to use the ALERT algorithm, but computes a subset of the metrics that the web applet and the R package provide. Detailed instructions on how to use the ALERT algorithm are found in the Supplementary Data.

Data analysis for this project was approved by the institutional review boards at the University of Massachusetts, Johns Hopkins University, University of Colorado, and the CDC.

Validation

To estimate the performance of the ALERT algorithm in prospective use, we performed validation by leaving 1 year out of the training data set, fitting the model, and assessing the fit to the held-out data. We performed this “leave-one-season-out” cross-validation for each year, in turn (Supplementary Data).

RESULTS

Using reported influenza A cases from both hospitals between August 2001 and August 2011 (Figure 1), we prospectively applied the ALERT algorithm to compute historical performance metrics for a range of possible thresholds. Table 2 presents reformatted output from the ALERT R package that shows metrics summarizing the performance for multiple possible thresholds.

Figure 1.

Historical influenza surveillance data from Johns Hopkins Hospital and Children's Hospital Colorado.

Table 2.

Summary of ALERT Performance Across Different Thresholds for Johns Hopkins Hospital and Children's Hospital of Colorado

Site	Threshold	Median Duration	Cases Captured, %			Peaks Captured		Mean Weeks Below Threshold	Mean Duration Difference
			Median	Minimum	Maximum	%	% ±2
CHCO	1	19	98.9	72.6	99.4	100	90	2.3	7.1
	2	15.5	97.3	69.9	98.3	100	90	1.2	3.5
	3	13	93.9	68.9	97.8	100	90	1.2	1.3
	4	12.5	91.9	68.9	96.6	100	70	1.3	0.6
	5	12	89.9	68.9	96.1	100	70	1.1	0
	6	11.5	89.9	68.9	96.1	100	70	1.3	−0.5
	7	9.5	84.1	60.7	96.1	100	60	1.4	−1.6
	8	8.5	81.9	60.7	96.1	100	50	1.6	−2.6
	9	8.5	80.7	60.7	96.1	100	50	2	−2.8
JHH	1	18	96.9	0	99.8	70	70	3.7	3.7
	2	18	97.4	61.3	99.4	90	90	2.9	5.1
	3	15.5	95.6	58	98.7	90	90	1.8	1.8
	4	14.5	94	57.3	98.7	90	80	2	0.8
	5	13	91.8	57.3	96.3	90	80	2.2	−0.3
	6	12.5	90.3	57.3	96.2	90	80	2.6	−0.9
	7	12	86.4	47.7	94.9	80	80	2.7	−1.4
	8	11	82.6	47.7	94.9	80	80	3	−2.1
	9	10.5	82.6	0	94.9	80	60	2.2	−2.44
	10	9	76.8	0	94.9	70	60	2.2	−3.11

See Table 1 for a complete description of all the metrics. The “Peaks Captured” columns provide the percentage of seasons in which the ALERT period contained the peak of the flu season (the “%” column) and the peak of the flu season ±2 weeks (the “% ±2” column). The “Mean Duration Difference” column displays the average difference in duration between the ALERT period and the shortest number of consecutive weeks needed to capture 90% of cases across all seasons.

Abbreviations: ALERT, Above Local Elevated Respiratory Illness Threshold; CHCO, Children's Hospital of Colorado; JHH, Johns Hopkins Hospital.

For each location, we chose the highest threshold that had captured at least 90% of cases ≥5 times in the past 10 years. For CHCO, this resulted in a threshold of 4 cases per week. For JHH, this led to a threshold of 6 cases.

The ALERT thresholds were applied to the 2011–2012 and 2012–2013 influenza seasons in both hospitals. Figure 2 depicts the timeline of reported cases in the 2012–2013 season. At JHH, 71% and 91% of all reported cases fell in the ALERT period in the 2 seasons, respectively. At CHCO, the ALERT period captured 77% and 89% of all influenza cases during the 2 seasons.

Figure 2.

Examples of real-time implementation of the Above Local Elevated Respiratory Illness Threshold (ALERT) algorithm, using the 2012–2013 data from Johns Hopkins Hospital and Children's Hospital Colorado. The shaded region indicates the ALERT period.

To determine the robustness of the ALERT algorithm, we cross-validated our metrics for our decision rule (“highest threshold capturing >90% at least half the time”) and others, by calculating the “prospective” performance by leaving 1 season of data out at a time. The historical performance metrics were similar to the prospective performance metrics (Supplementary Table 2), demonstrating that the simple historical calculations may be sufficient for estimating prospective performance.

DISCUSSION

We have developed a simple strategy to prospectively determine the start and end to a period of elevated influenza incidence in a community. We demonstrated the utility, flexibility, and robustness of the ALERT algorithm, which can serve as a valuable tool for communities, schools, hospitals, and other institutions looking for a method to objectively define a period when, for example, enhanced patient contact precautions, empiric therapy, or other prevention measures should be implemented. Accurate and robust thresholds could aid in preventing morbidity and mortality while also reducing costs associated with excessive precautions.

The ALERT algorithm can be easily operationalized. It is not meant to provide detailed predictions of week-to-week incident counts. In recent years, some models have made large strides predicting influenza [10, 11, 13, 14]. However, we are not aware of any open-source, publicly available tool for defining the influenza season that (1) is currently in use by public health practitioners or (2) can be utilized by someone who does not have substantial graduate-level training in a quantitative, statistical, or computational science. The ALERT algorithm may prove to be especially useful for entities that need to define a programmatic “elevated respiratory illness season” (eg, a health system, hospital, or a clinical trial). While the ALERT algorithm could also be used in nonclinical settings (such as schools or other community milieus), we have not provided a validation of its performance in settings without laboratory-confirmed cases. Furthermore, in these situations it would be especially important to understand the correspondence between unconfirmed case counts and laboratory-confirmed cases.

Our results suggest that the ALERT algorithm performs well at predicting the beginning and end of a seasonal period of increased influenza incidence. However, we expect variation in the results when applying this algorithm. Indeed, we see with the results from JHH that in 1 year, 91% of cases were captured, whereas in the other year 71% of cases were captured. In both years the same threshold was used. The lower-than-expected number resulted from an influenza season that was unlike others in the dataset (only 1 other season in our data had a lower seasonal total). However, in future years of implementation, these data could be utilized as historical data and therefore incorporated into the threshold calculations.

A strength of the ALERT algorithm is that its results are tailored to a specific location. Without involvement of a third party, laboratory-confirmed cases from a facility can be used in the algorithm. In comparison, a system such as Google Flu Trends may also provide useful information for hospital administrators, but ultimately it only provides noisy proxy measures of influenza incidence in a particular setting [19-21]. There is nothing preventing the ALERT algorithm from being used with data from a larger community (eg, city- or state-level surveillance data) or with other pathogens that follow seasonal trends (eg, respiratory syncytial virus, norovirus), although we have not evaluated its performance in these settings. It could also provide guidance to municipal public health authorities about when to implement behavioral messaging campaigns to a population.

Picking an ALERT threshold is a task that will depend on the goals of the user. For example, one institution might put a high cost on having ALERT periods that last for >12 weeks. Another might want to capture 95% (or only 75%) of the cases. Each of these specifications might lead to a different way to evaluate the ALERT algorithm output. However, because the ALERT algorithm presents a series of options, it makes it easy for the users to apply their particular priorities. As seen in Table 2, lower thresholds have longer durations and a larger percentages of cases captured. If long durations are not cost-prohibitive, then having the threshold be more sensitive (ie, the threshold is a lower number) may be desired, although this should be weighed against the increased likelihood of an early false alarm.

We have used only influenza A case data in the work presented here because the explicit goal in the ResPECT study (our motivating example) was to maximize capture of influenza A. However, influenza A and B case data could be combined to compute a combined ALERT threshold.

Ultimately, the power of the ALERT algorithm lies in its simplicity, flexibility, and generalizability. However, some technological extensions to the algorithm as it stands now could enhance its utility and empower public health and medical practitioners. As those of us in the public health community, both in and out of healthcare, struggle with planning and managing annual viral epidemics, we must develop tools that will allow us to respond nimbly yet reliably. The ALERT algorithm is an additional tool that does not require sophisticated modeling and is accessible to the public health community. As with all these efforts, additional research and refinement will enhance our ability to be ready for such epidemics.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online (http://cid.oxfordjournals.org). Supplementary materials consist of data provided by the author that are published to benefit the reader. The posted materials are not copyedited. The contents of all supplementary data are the sole responsibility of the authors. Questions or messages regarding errors should be addressed to the author.