Dynamical Malaria Forecasts Are Skillful at Regional and Local Scales in Uganda up to 4 Months Ahead.

Malaria forecasts from dynamical systems have never been attempted at the health district or local clinic catchment scale, and so their usefulness for public health preparedness and response at the local level is fundamentally unknown. A pilot preoperational forecasting system is introduced in which the European Centre for Medium Range Weather Forecasts ensemble prediction system and seasonal climate forecasts of temperature and rainfall are used to drive the uncalibrated dynamical malaria model VECTRI to predict anomalies in transmission intensity 4 months ahead. It is demonstrated that the system has statistically significant skill at a number of sentinel sites in Uganda with high-quality data. Skill is also found at approximately 50% of the Ugandan health districts despite inherent uncertainties of unconfirmed health reports. A cost-loss economic analysis at three example sentinel sites indicates that the forecast system can have a positive economic benefit across a broad range of intermediate cost-loss ratios and frequency of transmission anomalies. We argue that such an analysis is a necessary first step in the attempt to translate climate-driven malaria information to policy-relevant decisions. ©2019. The Authors.

Introduction

Timely availability of information is key for effective decision making in any sector. The ability to reliably predict the transmission of malaria for the season ahead sufficiently far in advance would be of significant benefit to health planners in Africa (Hay et al., 2001; Thomson & Connor, 2001; Thomson et al., 2005; Cox & Abeku, 2007). The lag between the rainy season and the peak of malaria transmission implies that monitoring weather conditions can provide warning of anomalous malaria transmission 1 to 2 months in advance, resulting in calls to improve climate monitoring capacity in Africa for health applications (Thomson et al., 2014).

Numerous studies have investigated the potential of climate surveillance (Abeku et al., 2004; Ceccato et al., 2007; Grover‐Kopec et al., 2005; Ototo et al., 2011; Worrall et al., 2008) although to date there has been limited progress in sustainably operationalizing such approaches into health systems (Thomson et al., 2014). This is partly due to the challenge of effectively integrating climate into existing health planning procedures. While referring to longer timescales of climate change, many of the issues highlighted by Campbell‐Lendrum et al. (2015) are also valid for shorter seasonal planning timescales, notably the lack of high‐quality, multidecade health data sets with which to develop and evaluate climate‐driven planning systems, the potential mismatch of scales between climate information and the local decision process, the lack of relative effective decision support tools to marry climate‐driven information with the decision process and a general lack of a framework to effectively communicate model system uncertainties to end users (see also Tompkins & Thomson, 2018). Moreover, the 1‐month advance warning provided by the monitoring of climate may be too short, limiting the scope to implement or modify cost‐effective, preemptive actions (Checchi et al., 2006) such as sending out indoor residual spraying teams to identified vulnerable areas. DaSilva et al. (2004) emphasize that this latter shortcoming could be addressed by incorporating information from reliable seasonal climate forecasts that provide predictions of climate variables (such as temperature and precipitation) from 6 to 13 months ahead. If proven skillful, these could be used to increase the advance warning time for impending outbreaks. The present work limits the scope of the predictions to an advance warning (lead time) of 4 months, since previous research has shown that this is likely the upper limit for skillful prediction for the present generation of seasonal prediction systems (particularly for precipitation, e.g., Ogutu et al., 2017; Shukla et al., 2016), while noting that this time frame may still be inadequate for some operational decisions.

The uptake of seasonal climate predictions in malaria‐related health planning has been hampered by several knowledge bottlenecks. Historically, prediction at seasonal timescales was reliant on statistical models of seasonal anomalies based on regional sea surface temperature anomalies (Mason et al., 1996; Mutai et al., 1998), and forecast skill of rainfall and temperature in the tropics from dynamical systems was limited to the very short range of at most a few days (Vitart, 2014), thus preventing longer range, skillful, climate‐based prediction of outbreaks. At the same time, spatial disease modeling systems that accounted for climate were statistical in nature and relied on accurate and long‐term health data records. The quality, consistency, and availability of subnational level health data for malaria was often inadequate for the training and subsequent evaluation of disease prediction systems (Thomson et al., 2014). Paper‐based surveillance systems and limited confirmation of suspected cases through microscopy or diagnostic test kits lead to large health data uncertainties that compound those deriving from the use of imperfect climate and malaria modeling systems. The first demonstration of climate forecasts for malaria prediction used dynamical climate forecast models to drive a statistical model for national malaria cases in Botswana (Thomson et al., 2005). To date, there has been no demonstration of malaria early warning from dynamical modeling systems at the subnational, health district scale.

Scientific and surveillance advances are reducing the barriers to the development of dynamical disease prediction systems. Case confirmation through rapid diagnostic test kits, while still not universal, has increased substantially (Zhao et al., 2012), while many countries in Africa now have digital‐based health surveillance systems in place (Chaulagai et al., 2005). In tandem, weather forecasting techniques and order‐of‐magnitude increases in the use of satellite monitoring of the atmosphere have improved climate forecasting skill (Bauer et al., 2015). Mathematical dynamical disease modeling systems that account for climate have been developed and can be applied to model seasonal and interannual spatial changes in malaria hazard (Hoshen & Morse, 2004; Laneri et al., 2010; Lunde et al., 2013; Tompkins & Ermert, 2013). Recent theoretical studies have used climate forecast information to drive such dynamical disease models to demonstrate the potential predictability of malaria over regional scales (Jones & Morse, 2010, 2012; Tompkins & Di Giuseppe, 2015), with the caveat that no validation was made for the presented systems with actual health data. One example of an early warning system evaluated with monthly case data aggregated at the national scale for Botswana has been presented by MacLeod et al. (2015), consisting of a dynamical malaria model driven by monthly and seasonal forecasts. The forecasts were statistically skillful, while highlighting the considerable number of outbreak false alarms (MacLeod et al., 2015).

Here we present the first subnational evaluation of a spatially distributed, climate‐driven, malaria early warning system built (MEWS) with dynamical modeling systems. The system is evaluated using high‐quality sentinel site and regional health district data in Uganda, and it is demonstrated that the system can predict which seasons may have enhanced transmission up to 4 months in advance. Our key finding is that, despite the use of an imperfect forecast system verified with imperfect health data, a threshold has now been superseded whereby the system skill may translate into positive economic value for decision making.

Methods

Technical details of the MEWS are given in Tompkins and Di Giuseppe (2015) and in supporting information S1–S3. Briefly, the early warning system consists of dynamical malaria model that models the parasite sexual reproduction in the human host in a classic compartmental susceptible‐exposed‐infected‐recovered model, coupled to a model for the vector life cycle and the parasite sporogonic cycle, which account for climate variations. The climate conditions are provided by monthly weather and seasonal climate forecasts with one forecast ensemble made per month for the years 1994 to 2013. The ensemble of five forecasts sample chaotic uncertainty of the atmosphere and account for weather forecast model and analysis system uncertainties.

This version of the MEWS predicts the entomological inoculation rate (the number of infectious bites received per person per unit time), the human bite rate, the detectable parasite ratio, and the vector density, and the entomological inoculation rate is used to generate a proxy for total cases off‐line. To focus on the skill at predicted interannual variability, all series are detrended prior to analysis and monthly standardized anomalies of both the model proxy and the observed cases (detailed next) are calculated. The skill is then classified using the Spearman's rank correlation score between the modeled and observed series of standardized anomalies, with the forecasts classed as skillful if the correlation value for the period for which data are available is significant at the 95% level.

In Uganda, high‐quality, laboratory total confirmed case data from six sentinel sites distributed throughout across the country are available for the period 2006–2013 (Sserwanga et al., 2011). Of these, Kabale has an altitude of 2,000 m and has mean temperatures below the 18 ° C threshold required for sustained transmission in the model, while the other five sites (Apac, Jinja, Kanungu, Mubende, and Tororo) range between 1,000 and 1,320 m. We mainly focus on Jinja, Kanungu, and Mubende as the three highest altitude sites above 1,100 m where climate variability is expected to be more important for driving interannual variability in malaria transmission (Alonso et al., 2011; Haque et al., 2010; Ototo et al., 2011; Zhou et al., 2004). In addition to the sentinel sites, monthly totals of suspected malaria cases obtained from the Ministry of Health of Uganda and aggregated at the administrative district level are used for a period of approximately 10 years. For details of the forecast system, observations, and evaluation method, refer to the supporting information S1.

Results

The predictions of transmission anomaly compared to the actual measured anomaly in malaria cases is shown for the three focus sites of Jinja, Kanungu, and Mubende (Figure 1). The panels show information that would be available to a decision maker 1 month (lead 1) and 4 months (lead 4) in advance. The shaded region shows the span of the five‐member forecast ensemble, an indication of the forecasts uncertainty related to the climate forecasts but does not include the uncertainty of the malaria model or the initial conditions (see discussion in the supporting information and Tompkins & Thomson, 2018). The uncertainty due to climate increases with lead time; hence, its range is much larger for the 4‐month lead time predictions. This is also emphasized in Table 1, which shows the rank correlation skill score for all five sites as a function of the forecast lead time. The system is skillful at most forecast lead times for all five sites, but the skill does not decrease monotonically as a function of lead time. This is an artifact of the relatively short validation period, constrained by the availability of health data.

Figure 1

Standardized anomaly of the detrended observed total malaria cases (C ) at the Jinja, Kanungu, and Mubende Sentinel sites (red line) compared to the standardized anomaly of the detrended cases proxy (C ) from the forecast ensemble, with the black line showing the ensemble mean and the gray shading the range (minimum and maximum) of the five‐member forecast ensemble members. In the left‐hand column, each point of the forecast time series is a forecast started 1 month prior to the observation and thus give an early warning of 1 month (lead 1), while the right‐hand column shows a forecast started 4 months ahead (lead 4). EIR = entomological inoculation rate.

Table 1

Spearman Rank Correlation Coefficient at Jinja, Kanungu, and Mubende Sentinel Sites as a Function of Forecast Lead Time From 1 to 4 Months

Sentinel site	Lead 1 m	Lead 2 m	Lead 3 m	Lead 4 m
Apac	0.56	0.29	0.55	0.39
Jinja	0.75	0.66	0.80	0.78
Kanungu	0.87	0.14	0.69	0.63
Mubende	0.39	0.57	0.51	0.62
Tororo	0.73	0.33	0.63	0.46

The comparison shows that the MEWS is able to predict the seasonal trends in case anomalies at the sentinel sites, with the 2010 identified as a year of anomalous transmission, likely associated with the occurrence of a medium strength El Ninõ event, which would produce anomalously warm temperatures and can sometimes increase rainfall if associated with warmer seas in the Indian Ocean (Lindblade et al., 1999). From the two high altitude sentinel sites (i.e., Kanungu and Mubende), the model performance is superior in Kanungu, with the higher transmission in 2012 at Mubende underestimated. The early warning system is able to predict the secondary maximum that occurs in 2012 in Mubende and Kanungu in the highlands and yet also predicts the single period of enhanced cases at the lower‐altitude Jinja site. This indicates that the system has the potential to predict differences occurring at a subnational scale, likely driven by rainfall spatial heterogeneity.

The forecasts are also validated using the suspected malaria case surveillance time series obtained from the Ministry of Health of Uganda and aggregated at the district level, and it is found that the system is skillful in approximately half of the districts (Figure 2). While the districts that are skillful include many of the higher‐altitude locations in the east and south west, there is no obvious characteristic that determines the skillful districts, which also include districts where malaria is highly endemic.

Figure 2

Map of Uganda showing the Spearman's rank correlations for districts for which the model forecast system has statistically significant skill relative to the suspected malaria district data obtained from the Ministry of Health.

The cost‐loss analysis of relative economic value V of the MEWS is conducted using the monthly anomalies for the three focus sentinel sites (Figure 3) and shows that the system has positive value for a range of intermediate cost‐loss ratios bounded by 0 or negative value at very low or high cost‐loss ratios. As the event threshold becomes rarer, the system has value at lower cost‐loss ratios, seen in the diagonal slant of the area of positive economic value, and for a decreasing range of cost‐loss ranges. Kanungu has a wider range of decision entry points and cost‐loss ratios for which the system has positive economic benefit relative to Mubende, for which the system only has benefit for thresholds below approximately 0.85, corresponding to a 1 in 7‐month event, and for cost‐loss ratios of 0.15 to 0.3. At Jinja, the range of economic value is limited. Note that an upper quintile event as calculated here is not precisely equivalent to the definition of an epidemic by the World Health Organization field handbook on malaria epidemiology (Hook, 2004), since the latter is defined in terms of a seasonal anomaly, whereas the present calculation is based on monthly anomalies.

Figure 3

Relative economic value V of using the forecast system at a 4‐month lead time at Jinja, Kanungu, and Mubende, using a range of cost‐loss (C/L) ratios (x axis) and percentile threshold of the monthly standardized transmission anomaly (y axis). For example, a percentile fraction of 0.66 corresponds to higher transmission anomaly that is expected to occur 1 in 3 months, while 0.8 refers to a 1 in 5 standardized anomaly. Refer to supporting information S1 for full outline of analysis method.

Discussion

The forecasting system has demonstrable skill in predicting temporal variations in malaria cases, although the analysis is necessarily limited by the temporal length of data available. This is the first time that a dynamical early warning system for malaria has been shown to have demonstrable skill at the subnational scale using monthly real time series of epidemiological surveillance data. For context, previous studies have quantitatively compared their malaria forecasts against national annually averaged malaria indices (MacLeod et al., 2015) or qualitatively against subnational annual Plasmodiumfalciparum prevalence data (Lauderdale et al., 2014).

The skill was demonstrated for high‐quality confirmed sentinel site cases as well as for approximately half of health districts. It is emphasized that the district malaria data are affected by many factors in addition to climate variability, including population mobility, changes in land use, and systematic increases in preventative interventions. In addition, we suspect that a key factor is that district data, especially in the earlier period of the database, is subject to large uncertainties due to the lack of systematic confirmation of cases at the time the data were collected (Yeka et al., 2012). As the spatial scale of seasonal climate anomalies will exceed the district scale, in particular for temperature, one would expect a positive spatial autocorrelation of the normalized seasonal malaria transmission anomaly. Instead, the health district data show a Moran's spatial autocorrelation (Figure 4) close to 0 and lower than the climate‐driven malaria model. Note that heterogeneities in urban‐rural settings, population densities, and land cover would not greatly affect autocorrelation as these factors do not change rapidly on interannual timescales. Recognizing the limitations of the district data, it is encouraging that the malaria predictions are nevertheless skillful in approximately half of the districts in Uganda, 4 months in advance.

Figure 4

Moran spatial autocorrelation for each month calculated for the Uganda district data (circles) and the lead 1 district malaria predictions (squares). The whiskers indicate the interannual standard deviation.

Apart from the need to reduce uncertainties and errors in the climate forecasting and malaria modeling systems themselves, the next step is to ascertain how to best incorporate such as system effectively into a national or regional decision‐making process concerning health planning and interventions. The cost‐loss economic analysis represents a potential framework for this as it allows a decision maker to assess whether using the forecast system to decide when to apply a particular invention makes economic sense. For example, in the analysis of the relative economic value V, it was seen that the system never has value at low cost‐loss ratios C/L. This is straightforward to interpret since at low cost‐loss ratios, the system has to be very accurate to have positive economic value as a single miss will prove more costly than simply intervening every year. In these cases, the decision maker would err on the side of caution and simply intervene even with a highly accurate forecast system available for guidance. The diagonal slant of the area of positive economic value in Figure 3 means that for rarer events, the threshold for using the early warning system moves to lower cost‐loss ratios. This also makes sense, since as anomalies that one is attempting to mitigate become rarer, the economic wastage of always intervening when not required starts to outweigh the cost of losses due to a forecast miss.

At high cost‐loss ratios, the cost of forecast false alarms is instead the issue, since the high cost of invention in these cases outweighs the alternative strategy of never intervening. It should be said that these cases highlight a weakness of the cost‐loss analysis, in that intervention decisions are political as well as economical, and a decision maker may decide to intervene even if it is not the optimal strategy to avoid the appearance of inaction in the face of an unexpected disease outbreak. Another drawback of the cost‐loss approach is that mapping actual policy decisions and outcomes to economic values of intervention costs and economic losses is extremely challenging. For example, an economic loss may refer to loss of productivity due to worker absence or the direct cost of treating clinical cases, with estimates made from analyses of previous outbreaks in an epidemic transmission setting. On the other hand, the action cost could refer to implementing interventions in the area. Economic analyses have been carried out previously to evaluate and compare the economic value of interventions in various transmission settings (Morel et al., 2005; Wiseman et al., 2003; Worrall et al., 2004, 2008; Yukich et al., 2008) but necessarily involve simplifications. Despite these challenges, such an economic framework is important to emphasize the fact that a forecast may well have statistical skill at predicting an outbreak but that this may not necessarily translate into actionable information for a specific decision.

Another aspect of health early warning systems that requires further attention regards the initialization of the forecasts. Weather forecasts have been operational now for over six decades, and in that time, the World Meteorological Organization has worked with national weather centers to build up a wide‐reaching global telecommunications system to transmit satellite and in situ measurements to collecting centers in near real time for their use in assimilation systems to provide the initial conditions for weather forecasts. No such system for monitoring epidemiological or entomological conditions exists. While case data are now collected digitally in many countries using Health Management Information Systems (Chaulagai et al., 2005), these are usually country based and, for obvious reasons of privacy, rarely open access or shared with neighboring countries. Entomological conditions such as vector density are sporadically monitored, if at all. At best, satellite information can be used to monitor surface water availability to attempt to infer vector densities indirectly (Franke et al., 2015). The lag between climate and malaria means that the forecast skill in the first month derives almost entirely from the correct initialization of the epidemiological and entomological conditions (Tompkins & Di Giuseppe, 2015). Thus, the development of improved health early warning systems will be hampered unless the issue of data availability for initialization can be addressed. A first step would be for a coordinated international action to collect past digital health records, processed to adequately address privacy concerns, to be released in digital format publicly as a resource for research to develop and evaluate the potential of health early warning systems. The ensuing development of pilot systems with improved assimilation techniques for system initialization would in turn encourage further efforts to open up data resources publicly and make investments toward the collection of health information in near real time to support operational use for intervention planning (Tompkins et al., 2018). In addition to improving and expanding broadband networks that facilitate the implementation of Health Management Information Systems, low‐cost technology solutions now exist for long‐distance wireless networks (Zennaro et al., 2010) and low‐volume data transmission via mobile networks to reach isolated, rural locations (Aranda‐Jan et al., 2014) that would allow relevant epidemiological, as well as supporting entomological and in situ environmental information to be transmitted and gathered efficiently in near real time.

Conclusions

We have shown that a dynamical malaria system driven by a combination of monthly and seasonal climate forecasts is able to produce forecasts of the normalized force of infection that reproduce some of the temporal variability in the malaria cases, both in terms of district unconfirmed cases and high‐quality data from several sentinel sites in Uganda. This is the first such demonstration on a subnational scale. Further confidence in the system could be gained by its evaluation in alternative geographical areas or by using longer data series, highlighting the need for continuous, subnational malaria data across large areas. An economical analysis demonstrates that the system has positive benefit for a range of intervention‐benefit ratios at some sites, possibly indicating a method for incorporating such a system into the decision‐making process.

Conflict of Interest

The authors declare no conflicts of interest relevant to this study.

Supporting Information S1

Click here for additional data file.