Coverage and system efficiencies of insecticide-treated nets in Africa from 2000 to 2017.

Insecticide-treated nets (ITNs) for malaria control are widespread but coverage remains inadequate. We developed a Bayesian model using data from 102 national surveys, triangulated against delivery data and distribution reports, to generate year-by-year estimates of four ITN coverage indicators. We explored the impact of two potential 'inefficiencies': uneven net distribution among households and rapid rates of net loss from households. We estimated that, in 2013, 21% (17%-26%) of ITNs were over-allocated and this has worsened over time as overall net provision has increased. We estimated that rates of ITN loss from households are more rapid than previously thought, with 50% lost after 23 (20-28) months. We predict that the current estimate of 920 million additional ITNs required to achieve universal coverage would in reality yield a lower level of coverage (77% population access). By improving efficiency, however, the 920 million ITNs could yield population access as high as 95%.

Introduction

Insecticide-treated nets (ITNs), which comprise conventional (cITNs) and long-lasting insecticidal nets (LLINs), are the single most widely used intervention for malaria control in Africa, proven to significantly reduce morbidity and mortality via direct protection and community-wide reductions in transmission (Lim et al., 2011; Lengeler and Lengeler, 2004; Eisele et al., 2010; Killeen et al., 2007). The World Health Organization (WHO) promotes a target of universal coverage for all populations at risk with either ITNs or indoor residual spraying (IRS), with the former representing the primary vector control tool in nearly all endemic African countries (WHO, 2013a). The international community has invested billions of dollars in the provision of at least 700 million LLINs since 2004 (WHO, 2013a). While these investments have led to enormous scale up in population access to ITNs (Noor et al., 2009; Monasch et al., 2004), the target of universal coverage remains distant and millions of African households at risk remain unprotected (WHO, 2013a).

Bridging this gap is a key component of future strategies to reduce further the burden of malaria in Africa (WHO, 2014), and will require sustained commitment from donors, policy makers and national programmes. Central to these efforts is the capacity to monitor reliably current levels of ITN coverage in populations at risk and evaluate the systems that give rise to this coverage. This, in turn, enables progress towards international goals to be tracked and opportunities for efficiency gains to be identified. Such information is essential for evaluating the existing commodity and financing shortfalls and assessing future requirements if the target of universal coverage is to be achieved.

Modelling coverage

To facilitate standardised and comparable monitoring of ITN coverage through time, WHO and the Roll Back Malaria Monitoring and Evaluation Reference Group (RBM-MERG) has over the past decade defined a series of indicators to capture two different aspects of ITN coverage: access and use. Gold standard measurements of these indicators are provided by nationally representative household surveys such as Demographic and Health Surveys (DHS) (Measure, 2014), Multiple Indicator Cluster Surveys (MICS) (UNICEF, 2012), and Malaria Indicator Surveys (MIS) (RBM, 2014a). These surveys are carried out relatively infrequently, however, meaning they cannot be used directly for evaluating year-on-year coverage trends or for generating timely estimates of continent-wide coverage levels. In contrast, programmatic data such as the number of ITNs delivered and distributed within countries, while not describing coverage directly, are available for most countries and years (WHO, 2013a ). In a 2009 study, Flaxman and colleagues (Flaxman et al., 2010) used a compartmental modelling approach to link these programmatic and survey data, generating annual estimates of the two ITN indicators recommended at that time on access (% households with at least one ITN) and use (% children < 5 years old who slept under an ITN the previous night).

Since that study, there has been increasing recognition that a richer set of indictors is required to identify the complex nature of ITN coverage (Kilian et al., 2013). An intra-household 'ownership gap' may exist whereby many households with some nets may not have enough for one net between two occupants (the recommended minimum level of protection (WHO, 2013b). Similarly, a 'usage gap' may exist whereby individuals with access to a net do not sleep under it. In response, the measurement of two additional indicators was recommended: % households with at least one ITN for every two people and % population with access to an ITN within their household (assuming each net was used by two people) (RBM, UNICEF, WHO, 2013; RBM, 2011). In addition, the indicator on usage was extended to include the entire population rather than only children under 5 years old. This updated set of four indicators, used individually and in combination, has the potential to provide a nuanced picture of ITN access and use patterns that can directly guide operational decision making (Kilian et al., 2013). To achieve this, there is a need to develop modelling frameworks to allow all four to be tracked through time.

Evaluating efficiency

Countries have an ongoing struggle to maintain high LLIN coverage in the face of continuous loss of nets from households due to damage, repurposing, or movement away from target areas. In response, systems need to be responsive to emerging coverage gaps by ensuring nets are distributed to households that need them and avoiding over-allocation (i.e. distribution of nets to those that already have them). Together, the rate of net loss and the degree of over-allocation of new nets play a key role in determining how efficiently delivery to countries will translate into household coverage levels. These factors are not currently well understood but triangulation of survey and programmatic data allows new insights into both.

Estimating future needs

The WHO define universal access to ITNs on the basis that two people can share one net. Using the working assumptions of a 3-year ITN lifespan and a 1.8 person-per-net ratio (one-between-two but allowing for odd-numbered households), a simple calculation yields an indicative estimate of 150 million new nets required each year to provide universal coverage to an African population at risk of around 810 million (WHO, 2013a). To support country planning and donor application processes (RBM-HWG, 2014), a more elaborate needs assessment approach has been developed by the RBM Harmonization Working Group (RBM-HWG) and implemented by 41 of the 47 endemic African countries (RBM, 2014; Paintain et al., 2013). The tool takes into account the size and structure of national target populations, a 1.8 person-per-net ratio for mass campaigns, additional routine distribution mechanisms employed by countries, and volumes of previously distributed nets and their likely rates of loss through time. Countries have used these inputs to calculate requirements for new nets to achieve national coverage targets, leading to an estimated continent-wide need for 920 million ITNs over the 2014–2017 period (approximately 230 million per year) (RBM, 2014). This tool provides a transparent, intuitive and standardised mechanism for comparing forecasted needs against current financing levels and identifying likely shortfalls. However, calculated needs are sensitive to assumptions about how a given volume of new nets will translate into population coverage, and inefficiencies in the system such as such as over-allocation and rate of net loss are not accounted for explicitly in the current needs assessment exercise.

The purpose of this study is to define a new dynamic modelling approach, triangulating all available data on ITN delivery, distribution and coverage in sub-Saharan Africa in order to (i) provide validated and data-driven time-series estimates for all four internationally recommended ITN indicators; (ii) explore and quantify different aspects of system efficiency and how these contribute to reduced coverage levels; and (iii) estimate future LLIN needs to achieve universal access by 2017 under different efficiency scenarios and how these compare to existing needs assessment estimates.

Results

Net stock estimates

Figure 1A summarises the main inputs to and outputs from the stock-and-flow model for LLINs when aggregated at the continental level. Some 718 million LLINs have been delivered across the 40 endemic countries since their introduction in 2004. As is well documented (WHO, 2013a), annual LLIN deliveries increased year-on-year from 2004 to 2010, reaching 145 million in that year, but then declined dramatically in 2011 and 2012 to less than half that amount before rising again to 143 million in 2013 (green line). Taking into account rates of loss in households, these LLIN deliveries led to a continental net crop shown by the red line. We estimate that there were 252 million LLINs in sub-Saharan households by the end of 2013, with that net crop growing approximately linearly from 2004, with the exception of a slow-down resulting from the reduced supply of nets in 2011–2012. Figure 1B shows equivalent distribution and resulting net crop estimates for cITNs, which constituted nearly all ITNs prior to 2005 but diminished rapidly in importance following the introduction of LLINs thereafter.

Figure 1.

Time series of ITN delivery, distribution, and estimated net crop in sub-Saharan households 2000–2013 for (A) LLINs and (B) cITNs. 

Manufacturer data on deliveries were available for LLINs only. cITNs, conventional insecticide-treated nets; HHs, households; ITNs, insecticide-treated nets; LLINs, long-lasting insecticidal nets; NMCP, National Malaria Control Programme.

DOI: http://dx.doi.org/10.7554/eLife.09672.003

Coverage estimates

Figure 2 shows continent-level time-series estimates of the four internationally recommended ITN indicators, along with the 'access gap' indicator. All four indicators show a similar temporal trend: very low coverage levels and modest year-on-year increases for the first 5 years from 2000, with a marked inflexion point in 2005 and much more rapid gains thereafter. Importantly, however, all four indicators show that the pace of increase has, overall, slowed since 2005. By the end of 2013, we estimate that around two-thirds (66%, 95% CI 62%–71%) of households at risk owned at least one ITN. However, less than one-third (31%, 29%–34%) owned enough for one ITN between two people. This much lower level of adequate ownership is reflected in the levels of access and use, with 48% (45%–51%) of people at risk having access to an ITN within their household (on a one-between-two basis) and 43% (39%–46%) sleeping under an ITN the previous night. Comparison of Figure 2A,B demonstrates that many households that own some ITNs do not own enough for one-between-two, and this is captured in the time-series for the 'ownership gap' (Figure 2E). Encouragingly, this gap has been narrowed from 77% (76%–78%) of net-owning households having insufficient nets in 2000 to 56% (54%–57%) in 2013. Analysis of the 'use gap' suggested a large majority (89%, 84%–93%) of those with access to an ITN in the household slept under it the previous night, and we found no evidence of significant change in this proportion through time.

Figure 2.

Continental-level time series of estimated ITN coverage indicators for the years 2000–2013.

(A) % households with at least one ITN; (B) % households with at least one ITN for every two people; (C) % population with access to an ITN within their household; (D) % population who slept under an ITN the previous night; (E) 'ownership gap', the % of ITN-owning households with insufficient ITNs for one-between-two. Black circles are the annual estimates; pink envelopes denote the 95% posterior credible interval. ITNs, insecticide-treated nets.

DOI: http://dx.doi.org/10.7554/eLife.09672.004

The relatively smooth temporal trends seen at continental level obscure a great deal of complexity in the patterns of ITN scale-up occurring at national level (Figure 3). Nearly all countries began with very low coverage levels in 2000 and display a marked inflection point towards the middle of the decade, although there was considerable variation in the timing of onset of concerted scale-up activities. Importantly, the monotonic increases in coverage seen at the aggregated continental level are often replaced at national level with pronounced periods of rise and fall, and in many cases, 2013 does not represent the peak year. Variation in contemporary levels of coverage remains stark. The population with access to ITNs within the household, for example, was at or below 15% in seven countries in 2013, while above 70% for the top four.

Figure 3.

Country-level time series of estimated ITN coverage indicators 2000–2013.

Each plot shows the four ITN coverage indicators: % households with at least one ITN (black); % households with at least one ITN for every two people (red); % population with access to an ITN within their household (green); % population who slept under an ITN the previous night (blue). CAR = Central African Republic; DRC = Democratic Republic of Congo; ITNs, insecticide-treated nets; HH = household.

DOI: http://dx.doi.org/10.7554/eLife.09672.005

Over-allocation

Over the 14-year period since 2000, on average 15% (12%–18%) of all ITNs distributed to households were over-allocated (owned by households already owning sufficient nets for one-between-two). Figure 4 illustrates how these over-allocation rates have changed through time. Around 7% (6%–9%) of ITNs were over-allocated in 2000, and this has risen steadily to 27% (22%–32%) in 2013. The year-on-year increase in over-allocation is to some extent an expected consequence of the overall growth in ITN provision: we found that over-allocation increased approximately 15 percentage points for each one-ITN-per-capita increase in net crop. Over-allocation also varied substantially between countries, for example ranging in 2013 from 50% (36%–65%) in the Republic of the Congo to 11% (9%–15%) in Côte D’Ivoire.

Figure 4.

Time series of over-allocation for the combined set of 40 sub-Saharan endemic countries, 2000–2013.

Over-allocation refers to insecticide-treated nets distributed to households already owning enough nets for one-between-two, measured as the percentage of over-allocated nets among all nets in households.

DOI: http://dx.doi.org/10.7554/eLife.09672.006

Net loss

Averaged over all years and all countries, we found the median retention time for LLINs in households was 23 (20–28) months. We found no statistically significant evidence of continent-wide temporal trends in retention times, but substantial between-country variation. Figure 5 plots the LLIN loss function representing the most recent three years (2011–2013) for each country individually (blue lines), along with the aggregated continental-level curve (red line). For reference, we also overlay on Figure 5 some alternative loss functions that have been proposed. Flaxman et al. (orange line) fitted very small annual loss rates (5%) for years 1, 2 and 3 - with all LLINs then assumed lost after 3 years (Flaxman et al., 2010). The RBM-HWG proposed rate of loss (green line) is 8, 20 and 50% of LLINs to remain after 1, 2 and 3 years, respectively, with all nets being lost thereafter (Networks, 2014). As can be seen, we found rates of loss for the first 3 years to be greater than both these alternatives for all countries. Both alternatives impose a three-year maximum retention time and our decision not to do so meant that we modelled a small proportion of LLINs lasting some years beyond that point.

Figure 5.

Insecticide treated netretention.

Estimated long-lasting insecticidal net retention curves for each country individually (blue lines) and combined (red line), in both cases relating to the average of the most recent 3 years, 2011–2013. Also shown for reference are the rate of loss recommended in the Roll Back Malaria Harmonization Working Group needs assessment exercise (green line) and the loss rate fitted by Flaxman et al. (orange line).

DOI: http://dx.doi.org/10.7554/eLife.09672.007

ITN requirements to achieve universal coverage

Figure 6 shows the projected levels of coverage that we estimate would be achieved by the end of 2017 with LLIN deliveries for the 2014–2017 period varying from zero to 2.5 billion and under a range of different efficiency scenarios. The most important characteristic of our results is the pronounced shallowing of the delivery-coverage curves: proportionately smaller gains are made in coverage as more LLINs are delivered in an archetypal 'law of diminishing marginal returns'. This means that under a business-as-usual scenario, where current levels of over-allocation and LLIN loss persist, very large increases in LLIN delivery would be required to achieve high coverage. Under this scenario, we estimate that 1 billion LLINs (i.e. an average of 250 million per year) would be required to achieve 80% of the population with access to an LLIN in the household by the end of 2017, although this would only translate into 70% population use.

Figure 6.

Projected 2017 coverage for sub-Saharan Africa in relation to number of LLINs delivered over 2014–2017 period.

(A) % households owning at least one ITN; (B) % households owning enough ITN for one between two; (C) % population with access to ITN within the household; (D) % population sleeping under an ITN the previous night; (E) 'ownership gap', the % of ITN-owning households with insufficient ITNs for one-between-two. For each indicator, we project likely coverage under four scenarios: current levels of over-allocation and net loss (i.e. 'business as usual'); with minimised over-allocation; with longer average net retention (3-yr median); and with both minimised over-allocation and longer net retention. The vertical dashed lines indicate the number of LLINs calculated as required over the period under the country programmatic needs assessment supported by Roll Back Malaria Harmonization Working Group. LLINs, long-lasting insecticidal nets; ITNs, insecticide-treated nets.

DOI: http://dx.doi.org/10.7554/eLife.09672.008

The extent to which coverage gains diminish as deliveries increase is mitigated substantially when over-allocation and ITN loss rate are reduced. In a scenario with minimised over-allocation (where over allocation is set to zero), 80% population access in 2017 would be achievable with just 700 million nets (175 million per year). Reducing ITN loss rate to a 3-year median retention time would have a broadly similar impact, acting in isolation, to minimising over-allocation. If these two hypothetical efficiency gains were combined, however, 80% access could be reached in 2017 with around 560 million nets (140 million per year). We found that the relative importance of the over-allocation and LLIN loss rates changed as more LLINs were introduced. Increasing LLIN retention times was the most important factor at low levels of net delivery, but as more and more nets were provided, over-allocation became progressively more important. This is intuitive since it becomes increasingly difficult to avoid over-allocation as more households obtain adequate numbers of nets.

For reference, we also plot on Figure 6 the 920 million additional LLINs calculated by countries as required for universal coverage of targeted populations by 2017 under the RBM-HWG needs assessment exercise. Under current levels of over-allocation and net loss, we estimate that by the end of 2017 this quantity of new LLINs would translate into 77% access (among those populations targeted by countries for ITN coverage) and, assuming current behaviour patterns continue, 68% sleeping under an ITN. Under the combined efficiency scenario with minimised over-allocation and 3-yr median ITN retention time, however, the 920 million nets would approach universal access (slightly over 95%).

Discussion

By linking manufacturer, programme and national survey data using a conceptually simple model framework, the intention has been to provide a transparent and intuitive mechanism for tracking net crops and resulting household coverage that reflects the input data while simultaneously providing a range of insights about the system itself. In doing so we have been able to (i) provide a new approach for estimating past trends and contemporary levels of ITN coverage; (ii) explore the effects of uneven net distribution between households and the rates of net loss once in households; and (iii) use these insights to estimate how many LLINs are likely to be required to achieve different coverage targets in sub-Saharan Africa.

We have, for the first time, extended dynamic model-derived estimation of ITN coverage to all four internationally recognized indicators, along with the two 'gap' metrics. Our results reinforce a simple message: while gains in ITN coverage have been impressive, there remains an enormous challenge if the goal of universal access is to be achieved and sustained. The importance of the new expanded suite of indicators is also exemplified: while an encouraging two-thirds of households now own at least one ITN, less than half of these have enough to protect everyone who lives there. This ownership gap is narrowing but the disparity remains evident across nearly all countries. Conversely, there is little evidence that non-use of available nets contributes substantially to low coverage levels. We therefore reinforce earlier studies that suggest the overwhelming barrier to not sleeping under an ITN is lack of access rather than lack of use WHO, 2013a; Eisele et al., 2009; 2011; Koenker and Kilian, 2014; Koenker et al., 2014). Of course, non-use may be important in certain local contexts, and finer-scale analysis can support identification of areas where behaviour change communication interventions may be appropriate to reduce it (Kilian et al., 2013).

We found substantial over-allocation of nets to households already owning a sufficient quantity, and that this became more pronounced as overall ownership levels increased through time. Mass distribution campaigns can, in principle, be designed to minimise over-allocation and maximise evenness of nets allocated to households strictly on the basis of households members and pre-existing nets. As other studies have highlighted, however, any possible commodity savings achieved by such strategies must be compared against the operational cost of these more complex distribution mechanisms (Yukich et al., 2013). What is certain is that over-allocation becomes a major barrier to achieving universal coverage when levels of ITN provision are high because most new incoming nets are simply leading to surpluses in many households, while elsewhere there remains a shortfall. This may have a disproportionately high public health impact if those surplus nets are concentrated in households at lowest risk. Wealthier, better educated and more urban households may be better placed to obtain available nets but are often located in regions of lower transmission (Steketee and Eisele, 2009; Webster et al., 2005). While beyond the scope of the present study, the approaches we have developed here could be extended to consider these issues of equity in coverage versus risk in more detail.

One of the most important observations in our study is that LLINs may be lost from households at a substantially faster rate than is currently assumed. Importantly, we assess loss by comparing total inputs to countries (from deliveries) to total numbers in households (net crop), and so we measure real losses rather than, for example, reallocation of nets between relatives (Koenker et al., 2014). Longer retention times of the sort observed in some local studies are not supported by the body of evidence we have provided by triangulating large-scale net distributions and household survey data. This more rapid loss rate has potentially important implications for existing guidelines. Current RBM guidance is for mass ITN campaigns to be conducted every 3 years, complemented by continuous distribution of nets via routine channels in order to maintain coverage levels between those campaigns. However, whatever levels of coverage are achieved by a given campaign, we estimate that one-half of the campaign nets distributed, on average, will not be present in households just 2 years later. Our coverage time-series for many countries suggest that routine distribution channels are not yet compensating fully for this rate of loss, often displaying pronounced dips in coverage levels between mass campaigns. Maintaining higher continuous coverage therefore clearly requires some combination of more frequent campaigns, greater ongoing distribution between campaigns, or more durable nets and improved care behaviour by users that lead to longer overall retention times.

We considered nets in households as simply present or absent, with no allowance for their condition. In reality, of course, nets may be retained by households (and thus 'present' in our calculations) even when they are badly torn, or have diminished insecticidal properties. As such, our estimates of 'coverage' would be revised downwards if additional measures of net efficacy were included. Our model is able to provide an estimate for every country and every year of the age-profile of ITNs in households. This raises the possibility of extending the predictions to incorporate modelled or observational data on average rates of net degradation in different contexts (Briët et al., 2012) to explore measures of entomologically effective coverage.

Tools developed to assist countries to calculate LLIN requirements, have tended to define need using a simple ratio to populations at risk (such as 1.8 people per net), and have made allowances for net loss from households using pre-defined rates of loss. We have been able to show that true LLIN requirements are likely to be considerably larger when the more rapid rates of loss are taken into account, along with the additional effect of likely over-allocation patterns. This more realistic framework not only provides the basis for more accurate needs assessments but identifies the relative importance of these different factors in determining the coverage that can be achieved for a given delivery level. Our analysis of future LLIN needs from the present time to 2017 demonstrates how these factors lead to a pronounced law of diminishing returns: as more nets are introduced to a population, proportional increases in coverage diminish, with over-allocation a particular problem at high net provision levels.

Under business-as-usual, the number of nets required to approach full coverage is prohibitively large. Clearly, however, reducing current system inefficiencies and increasing net retention are not straightforward and already the subject of much attention by countries and international partners. Over-allocation is the complex result of different distribution strategies and varying levels of population access to services, and any solution comes with its own cost. Net retention can doubtless be increased by improved LLIN technology coupled with behaviour-change communication efforts, although it is also feasible that retention times may reduce when overall net provision increases (with new nets displacing older ones). Additionally, we look only at the RBM definition of use and ignore the effectiveness of nets in repelling mosquitoes once they are being used. This is potentially an important confounder when considering retention times. While not aiming to provide solutions to these complex challenges, the results we present here provide an analytical framework in which the impact of theoretical efficiency gains can be assessed and this could be extended to include formal cost–effectiveness analysis.

In conclusion, our results provide evidence that LLIN requirements to achieve universal coverage have been underestimated. If obtaining higher coverage remains an accepted goal of the international community, then larger LLIN volumes must be considered and planned for at national and international levels. We emphasise, however, that this would be best achieved in parallel with a renewed focus on maximising the efficiency of coverage achieved for each new net financed. Given that the pattern of diminishing coverage returns for each dollar spent is likely to be unavoidable, the cost–effectiveness of pursuing universal coverage rather than a lower operational target must ultimately be weighed against alternative malaria control investments.

Materials and methods

Overview

Two important preceding studies have sought to model national-level ITN delivery, distribution, and coverage: the Flaxman et al. study (Flaxman et al., 2010) and the work of Albert Killian culminating in the NetCALC tool (Networks, 2014) and a series of related publications (Paintain et al., 2013; Yukich et al., 2013). Although very different in implementation, both approached the problem in a similar two-stage process. First, a mechanism was defined for estimating net crop — the total number of ITNs in households in a country at a given point in time—taking into account inputs to the system (e.g. deliveries of ITNs to a country) and outputs (e.g. the discard of worn ITNs from households). Second, empirical modelling was used to translate estimated net crops into resulting levels of coverage (e.g. access within households). We have adopted a similar analytical outline, but the models we have developed for each stage differ structurally and conceptually from these earlier efforts. Our underlying principle has been to represent the ITN system in a simple and intuitive way and to parameterise that system using a data-driven approach that minimises the reliance on assumptions or small external datasets. In this Methods section, we describe: (i) the main data sources used; (ii) a new compartmental model for estimating net crop that also offers insights into rates of ITN loss from households; (iii) a new coverage model linking net crop to household net access and use that also assesses the efficiency of between-household distribution (i.e. the extent of over-allocation); and (iv) the use of our models to predict future ITN requirements to meet the goal of universal access. A schematic overview of our analytical framework is provided in Figure 7, and additional methodological detail is provided in the Supplementary Information.

Figure 7.

Schematic showing overall analytical framework linking data, model components, and outputs.

HH = household; ITN, insecticide-treated net; NMCP = National Malaria Control Programme.

DOI: http://dx.doi.org/10.7554/eLife.09672.009

Data

We used three principal sources of data to fit our models. These are described briefly below and in more detail in Supplementary Information.

LLINs delivered to countries: data provided to WHO by Milliner Global Associates on the number of LLINs delivered by approved manufacturers to each country each year (WHO, 2013a; AMP, 2014). These were complete for each country from 2000 to 2013 inclusive.

ITNs distributed within countries: data provided to WHO by National Malaria Control Programmes (NMCPs) on the number of cITN and LLINs distributed annually within each country (WHO, 2013a). Data were available for 365 of the 560 country-years addressed in the study. We treated these data as only partial records of distribution activities because the extent to which NMCP reporting captures distribution by non-government agencies is not known for all countries.

Nationally representative household surveys. We assembled 99 national surveys from 39 sub-Saharan African countries from 2001–2013, covering 18% of all possible country-years since 2000 (Figure 8). More recent surveys provided household-level data on the number of cITNs, LLINs, people within each dwelling, and people sleeping under nets the previous night. RBM-MERG guidelines detail the conversion of these data into the standardised ITN indicators (RBM, UNICEF, WHO, 2013) and, in combination with national population data (UNPD, 2012), they can yield an estimate of national net crop (see Supplementary Information). Older surveys had less information: providing data on use but not ownership, for example, or for cITNs but not LLINs (see Supplementary Information). For most surveys (95/99), we were able to access the underlying data, while for the remaining four we used only the survey report.

Figure 8.

Distribution of national survey data on ITN net crop and household ownership, access and use used in this study, by country and year.

The different types of survey are shown in the key: CAR = Central African Republic; DHS = Demographic and Health Survey; DRC = Democratic Republic of Congo; ITN, insecticide-treated net; MICS = Multiple Indicator Cluster Survey; MIS = Malaria Indicator Survey; STP = São Tomé and Príncipe.

DOI: http://dx.doi.org/10.7554/eLife.09672.010

Countries and populations at risk

Our main analysis covered 40 of the 47 (WHO, 2013a) malaria endemic countries of sub-Saharan Africa. We excluded six endemic countries on the basis that ITNs do not form an important part of their vector control programme, as reported by the respective NMCPs to the African Leaders Malaria Alliance, ALMA (M. Renshaw, pers. comm. 3rd August 2014). These were Botswana, Cape Verde, Namibia, São Tomé and Príncipe, South Africa and Swaziland. We also excluded the small island nation of Mayotte, for which no ITN delivery or distribution data were available. We limited all analyses to those populations categorized as being at risk by NMCPs (WHO, 2013a). When interpreting NMCP distribution and household ownership data, we made the simplifying assumption that all reported ITNs were distributed among, and owned within, households situated in malaria endemic regions (Burgert et al., 2012). Additionally, we used data from African Leaders Malaria Alliance (ALMA) on the proportion of populations at risk targeted for ITNs versus IRS, and downscaled targeted populations at risk accordingly. It should be noted that restricting the distribution of ITNs to populations at risk makes the assumption that no ITNs are distributed to populations not at risk.

Estimating national net crops through time

Like Flaxman et al.(Flaxman et al., 2010), we represented national ITN systems using a discrete time stock-and-flow model. In this structure, a series of compartments were defined that contained a given number of nets at each time-step, with possible movement of nets from one compartment to another between time-steps (see Supplementary Information). Nets delivered to a country by manufacturers were modelled as first entering a 'country stock' compartment (stored in-country but not yet distributed to households). Nets were then available from this stock for distribution to households by the NMCP or other distribution channels. Years where NMCP distributions were smaller than available country stock represented potential ‘under-distribution’, with nets left to stockpile rather than reaching households. However, because of the uncertainty associated with NMCP distribution data, these discrepancies could simply reflect under-reporting of distribution levels. To accommodate this uncertainty, we specified the number of nets distributed in a given year as a range, with all available country stock as one extreme (the maximum nets that could be delivered) and the NMCP-reported value (the assumed minimum distribution level) as the other.

New nets reaching households joined older nets remaining from earlier time-steps to constitute the total household net crop, with the duration of net retention by households described by a loss function. In this representation, the net crop simply reflected the differences over time between inputs to and outputs from households. This meant that distribution, net crop, and the loss function together formed a closed system: the three must triangulate exactly and knowledge of any two components allowed the third to be calculated directly. Flaxman et al. (Flaxman et al., 2010) assembled data from six studies on ITN durability and rates of loss. Using a loss function fitted to these data, however, they found that the three components tended not to triangulate: net crops observed in surveys were too small, given the data on nets distributed to households and their modelled rate of loss. Their interpretation was that the number of ITNs distributed each year may be systematically over-reported by NMCPs, and a 'bias parameter' was included in the model, adjusting downward the volume of nets entering households in each country compared with reported levels. As described above, we took a different approach: with no a priori expectation that NMCP distribution reports exaggerate distribution levels. Rather than fitting the loss function to a small external dataset, we fitted this function directly to the distribution and net crop data within the stock-and-flow model itself. Conceptually, this reflected the view that the 560 country-years of distribution data triangulated against the 102 survey-derived national net crop values represented a more impartial and data-driven way of inferring rates of loss than using limited data from local ITN retention studies. Loss functions were fitted on a country-by-country basis, allowed to vary through time, and defined separately for cITNs and LLINs. We compared these fitted loss functions to existing assumptions about rates of net loss from households. The stock-and-flow model was fitted using Bayesian inference and Markov chain Monte Carlo (MCMC), providing time-series estimates of national household net crop for cITNs and LLINs in each country along with evaluation of under-distribution, all with posterior credible intervals. A complete technical description is provided in the Supplementary Information.

Estimating national ITN access and use indicators from net crop

Levels of ITN access within households depend not only on the total number of ITNs in a country (i.e. net crop), but on how those nets are distributed between households. In simple terms, a more even distribution yields a greater proportion of households owning nets than if those same nets are concentrated in fewer households. Many recent national surveys report the number of ITNs observed in each surveyed household. This allows, a histogram to be generated that summarises the net ownership pattern (i.e. the proportion of households with zero nets, one net, two nets and so on). By analysing such data from multiple surveys, previous studies have demonstrated that histograms for different countries vary in a broadly predictable way according to national net crop (Flaxman et al., 2010; Yukich et al., 2013). By representing these histograms using a formal statistical distribution (such as the negative binomial), and linking its parameters to net crop, predicted histograms can be generated for any country-year for which a net crop estimate is available (Flaxman et al., 2010; Yukich et al., 2013). These histograms, in turn, allow direct calculation of the first access coverage indicator (% households owning one or more ITN). We took the view that this approach—linking net crop to a statistical distribution, and using the distribution to calculate access indicators—is preferable to the alternative of regressing the access indicators against net crop directly. The latter approach, used in the NetCalc tool (Networks, 2014), is simpler but provides less direct insight into the patterns of between-household ITN distribution that ultimately link net crops to access levels.

One aspect that is known to strongly influence the relationship between net crop and household ownership distribution is the size of households found in different countries (Networks, 2014; Yukich et al., 2013), which varies greatly across sub-Saharan Africa (Swaziland, for example has an average household size of around three members, while in Senegal the average is nearly ten). Household size also, of course, determines whether a given number of owned nets will be sufficient to provide access to all residents. We extended earlier analyses (Flaxman et al., 2010; Yukich et al., 2013) to explicitly account for household size: using a bivariate (i.e. two- rather than one-dimensional) histogram model to link net crop to ownership distributions for each household size stratum (see Supplementary Information). We replaced the negative binomial distribution with a 2-d zero-truncated Poisson distribution and, for each household size stratum, fitted the distribution using two parameters: (i) the proportion of households with zero ITNs and (ii) the mean number of ITNs per ITN-owning household. Using the household-level data from 83 national surveys, we found that both parameters were strongly related to national net crop, allowing bivariate histograms to be generated for every country-year that were closely representative of the true ITN ownership distribution.

Stratifying our analysis by household size had three important advantages over earlier approaches. First, the distribution of net ownership tended to vary substantially between households of different sizes within a given country and this variation would be missed if all households were considered together. Accounting for this enabled better fits to the data. This makes sense: all else being equal, larger households would be expected to own more nets than smaller ones and so distribution patterns would differ systematically. Second, the bivariate ownership histograms predicted for each country-year could be used to directly calculate all three indicators of household access. While a simple univariate histogram allows calculation of % households with at least one ITN, a bivariate histogram means the number of both ITNs and people in every household can be triangulated which, in turn, allows direct calculation of the two additional indicators: % households with at least one ITN for every two people and % population with access to an ITN within their household, along with the 'ownership gap' (see Supplementary Information). Linking these bivariate histograms to our annual net crop estimates for each country meant we could predict time-series of the access indicators at the national level from 2000–2013, with all parameters fitted in a Bayesian framework providing posterior credible intervals around each time-series. We also combined the country-level results to generate a set of continent-level indicator time-series, representing overall coverage levels among populations at risk in the 40 endemic countries. Third, the bivariate histograms allowed analysis of over-allocation: certain cells of the histogram represented households owning more ITNs than were required to achieve access on a one-between-two basis, and the proportion of the total net crop falling in this category was examined through time for every country.

We took a different approach for the final indicator, % population who slept under an ITN the previous night. ITN use is less directly linked to national net crop and is primarily determined by the availability of nets within households (Eisele et al., 2009). A total of 83 of the 102 national surveys contained data allowing the relationship to be explored between ITN use and each of the three access indicators with, perhaps unsurprisingly, % population with access to an ITN within their household displaying the largest correlation (adjusted R2= 0.96). We fitted this relationship across the 83 surveys using a simple Bayesian regression model (see Supplementary Information) and used it to predict time-series of the ITN use indicator for every country. The ratio of population use to access revealed the 'usage gap'—the fraction of the population with access to ITNs not using them—and between-country variation in this ratio was also explored.

Estimating ITN requirements to achieve universal access

Our two-stage modelling framework represented the pathway from ITN delivery into countries through to resulting levels of net access and use in households. It also accounted for two potential factors that act to reduce access levels, and allowed these to be quantified through time for each country. Using this architecture, it was possible to simulate delivery of any hypothetical volume of ITNs to a given country over a given future time period, to predict the levels of access and use that would result, and to examine the impact of different amounts of over-allocation and net loss. The current needs assessment exercise that countries are undertaking (RBM-HWG, 2014; RBM, 2014) is designed to identify the number of LLINs required to achieve coverage targets by 2017. We used our model to estimate the levels of access likely to be achieved if these forecast LLIN commodity needs were met across the 2014–2017 period under a 'business as usual' scenario, that is, with current levels of over-allocation and net loss, and compared these predicted levels with the objective of universal access among target populations. We then generalized this experiment to predict the likely level of coverage (for all four indicators) achievable by 2017 under a broad spectrum of LLIN delivery levels, equivalent to a total for sub-Saharan Africa (two of the 40 endemic countries in our study did not participate in the RBM-HWG needs assessment exercise [Djibouti and Equatorial Guinea], and so our scenario analysis is based on the set of 38 remaining countries; to maintain comparability through time, we combined needs assessment data for mainland Tanzania and Zanzibar, and for Sudan and South Sudan) of between zero and 2.5 billion nets across the 4-year period. Further, we ran these simulations under four scenarios: (i) 'business-as-usual' (where current levels of over-allocation and net loss were maintained); (ii) with no over-allocation (new LLINs are distributed preferentially to those households with zero LLINs, then to those with less than one-between-two); (iii) with reduced LLIN net loss by households (using a modelled 3-year median retention time); and (iv) with both no over-allocation and a 3-year median retention time.

eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.

Thank you for submitting your work entitled "Coverage and system efficiencies of insecticide-treated nets in Africa from 2000 to 2017" for peer review at eLife. Your submission has been favorably evaluated by Prabhat Jha (Senior editor), Catherine Kyobutungi (Reviewing editor) and two reviewers, one of whom has agreed to reveal her identity: Eline Korenromp.

The reviewers have discussed the reviews with one another and the Reviewing editor has drafted this decision to help you prepare a revised submission.

Summary:

The authors, using data from 102 national surveys, triangulated against delivery data and distribution reports, developed a Bayesian model to generate year-by-year estimates of four ITN coverage indicators: i)% households with at least one ITN; ii)% of children under five who slept under an ITN the previous night; iii)% household with at least one ITN for every two people; and iv)% of the population with access to an ITN within their household.

The authors also explored the impact of two potential 'inefficiencies': uneven net distribution among households and rapid rates of net loss from households. The findings show worsening of over-allocation (a measure of inefficiency) over time as overall net provision has increased, and more rapid ITN loss from households than previously thought. This has implications on the universal coverage (only 77%) that can be expected from the current estimate of 920 million additional ITNs that have been computed as the need through 2017. By improving efficiency, however, the 920 million ITNs could yield population access as high as 95%.

The study presents an important situation analysis, of progress in distribution and resulting protective coverage with ITNs, the primary malaria prevention strategy in most of the high-malaria-burden countries in Africa. The dynamic model developed for this purpose is elegant, building on and refining earlier analysis, and adds value by producing more credible and considerably different estimates of the extent that ITNs are mis-allocated to some households, and the high rates of loss of ITNs from households – with the resulting inefficiencies varying among countries. The up-to-date estimates of ITN use will be useful to other researchers. Building on similar previous work, the authors have developed a model which allows them to break down coverage into estimates of multiple indicators that take account of the variation in household size, and this is an important advance on previous work. The paper also has projections of the coverage achieved if distribution can be done more efficiently or if people retain their nets for longer – this is also novel.

Essential revisions:

The areas of concern include the following:

1) The absence of concrete suggestions on how national malaria control programs could aim to reduce the identified inefficiencies, specifically over-allocation of ITNs. The authors should give examples of the operational strategies for better targeting and/or pre-screening of households' current ITN situation, and indicate which of these has proven to work, in what settings.

2) Different choices could have been made in the modelling, and this makes some of the conclusions more uncertain than what is currently presented. The authors should discuss these points and consider which outputs are most sensitive to the modelling choices. The estimates of the various indicators up to the present might not be much affected, but the projections forwards would be.

3) The net loss function (the rate at which people discard nets) is estimated by combining the numbers of nets entering a country, the number reported as distributed by control programmes, and the numbers owned by households. Flaxman et al. found that nets were retained were much longer than would be expected from matching up the distribution data and coverage data from cross-sectional surveys and as a result they introduced bias parameters to account for the discrepancy between data sources. It is not clear how this was handled in the modelling. The authors should describe this aspect of the methodology and the implications on the findings.

Minor points:

The estimated sharp fluctuations over time in ITN coverage indicators in some countries: are these real (if periodic campaigns dominate distributions?), or reflecting missing or incomplete data, e.g. in NMCP-reported distributions? In particular, comment on the odd time patterns in DRC and Ghana and/or a few other countries with irregular patterns.

[Editors' note: further revisions were requested prior to acceptance, as described below.]

Thank you for resubmitting your work entitled "Coverage and system efficiencies of insecticide-treated nets in Africa from 2000 to 2017" for further consideration at eLife. Your revised article has been favorably evaluated by Prabhat Jha (Senior editor), a Reviewing editor, and two reviewers. The manuscript has been improved but there are some remaining issues that need to be addressed before acceptance, as outlined below:

The reviewers still have concerns about the conclusions on net retention. The model in this paper only estimates ITN retention indirectly while the six studies that Flaxman depended on were directly measuring it. The bias parameters of Flaxman et al. were introduced because of a discrepancy between the different sources of data, if a loss function of longer duration is used. This discrepancy could be due to systematic differences between the data sources, and so the reviewers feel that the conclusion of the present paper that nets are retained for a shorter time than was previously thought is less certain than the authors suggest. This also has implications for the point about sharp fluctuations in the predicted ITN coverage over time, which presumably would be smoother with a loss function of longer duration.

We recommend that the authors conduct a sensitivity analysis that assumes longer retention of nets, showing how any projections change. In any case this uncertainty should be acknowledged and the implications for the projections discussed.

In addition, net loss and retention consider the physical presence of the ITNs in households, but not their effectiveness in repelling mosquitoes. According to manufacturer and WHOPES evidence and guidance on ITN's effective durability, is it justified to assume a median 36-month retention (in the optimized scenario), or does this imply a proportion of ITNs modelled as effective for longer than is realistically assumed? The authors should expound on this point in the discussion – for LLINs specifically

For the country-level estimates: Are the 4 international indicators (ownership, access, usage) calculated and shown for each country's overall national population (including sub-populations at no risk, just as the DHS, MICS and MIS surveys) or for the sub-population at risk of malaria? What assumptions, if any, are made about (mis-)allocation of ITNs to populations not at risk of malaria? Please clarify this in the appropriate section of the manuscript

Minor suggested edits:

If the world limit allows, please define 'Population access' in the Abstract.

Paragraph three, subsection “Net Loss”, 'minimised over-allocation': From the annex (and subsection “Estimating ITN Requirements to Achieve Universal Access”) it seems that this is ZERO over-allocation; please qualify or rephrase accordingly.

Appendix 1—figure 2:

Appendix figure 2.

Compartment model dynamics for LLINs.

The compartment model predicts the total number of nets in a country (thick purple line). For a given country at given years there will be LLINs delivered to the country from manufacturers (green arrows), and LLINs distributed at some point in that or subsequent years within the country (blue arrows). Country-wide survey reports give a cross-sectional measure of the total number of nets in households (red line). Evidence synthesis of three data sources can then be inferred by estimating a rate of loss (via a loss function). LLINs, long-lasting insecticidal nets.

DOI: http://dx.doi.org/10.7554/eLife.09672.013

a) Parts of the diagram are confusing or not clear – please correct as needed:b) An arrow (or, equality sign) is missing from HH-level ITN ownership to the box below (National total ITNs in HH?). c) An arrow missing from National net crop to, Observed bivariate histogramsd) National net crop equals National total ITNs in HH 5 boxes below, but the equality is not indicated.

Appendix 1: The scaling across countries to get to continent-totals is not very clear.

Essential revisions:

The areas of concern include the following:

1) The absence of concrete suggestions on how national malaria control programs could aim to reduce the identified inefficiencies, specifically over-allocation of ITNs. The authors should give examples of the operational strategies for better targeting and/or pre-screening of households' current ITN situation, and indicate which of these has proven to work, in what settings.

In this work we sought simply to identify and measure the magnitude of over-allocation and under-retention that exists and the changes in these factors over time. We felt this was a vital initial step and, of course, agree that it now paves the way for subsequent work to explore strategies for better targeting. We would propose, though, that this substantial body of new work would form one (or likely many) new studies in their own right, and would not be appropriately placed within the current submission which already covers a lot of new ground.

Solving the over allocation problem from an operational standpoint is something currently being discussed in policy fora. In the early years of integrated campaigns nets were allocated to a specific target group (e.g. children under five), but since the shift of policy to universal coverage this has become more difficult. Currently the distribution of nets requires a registration process within a county in order to identify beneficiaries thereby leaving large scope for under- and over-allocation. Work has been done at the Global Fund to strengthen the registration process but they have yet to identify concrete solutions that work at scale. Mozambique, Burundi and Madagascar have implemented systems based on village hierarchy that have been partially effective at small scales, but these do not lend to implementations at the scale of our analysis (country or continent level). Increasingly it is thought that solutions to the problem of over allocation need to be addressed in the planning stages rather than post hoc.

Given these complications, it is clear that defining strategies to address over-allocation in the field will remain a challenge – one to which modelling may be able to make a contribution alongside those with expertise of the reality of the logistics on the ground. For now, however, we maintain that by identifying that the phenomenon of over-allocation exists, is widespread, and contributing substantially to lower coverage yields, our study makes an important first step in motivating future work to improve distribution systems in-country.

2) Different choices could have been made in the modelling, and this makes some of the conclusions more uncertain than what is currently presented. The authors should discuss these points and consider which outputs are most sensitive to the modelling choices. The estimates of the various indicators up to the present might not be much affected, but the projections forwards would be.

We thank the reviewers for this comment. We felt that, because of the policy nature of this paper, most modelling assumptions were better described in full in the Appendix. For every component of the model we provide a conceptual and formal distribution of the model that outlines all the assumptions made and justifications for why we made them. As the reviewers expressed concern for the projection assumptions, we have added additional text to the assumptions section for future projections in the Appendix (Future predictions).

3) The net loss function (the rate at which people discard nets) is estimated by combining the numbers of nets entering a country, the number reported as distributed by control programmes, and the numbers owned by households. Flaxman et al. found that nets were retained were much longer than would be expected from matching up the distribution data and coverage data from cross-sectional surveys and as a result they introduced bias parameters to account for the discrepancy between data sources. It is not clear how this was handled in the modelling. The authors should describe this aspect of the methodology and the implications on the findings.

The Flaxman et al. paper was the inspiration for this work. However, there were several fundamental problems with their work. Their loss function was constant over a given year with very minor reductions for subsequent years until all nets are discarded at once after three years (see brown line Figure 7). This function is almost piece-wise linear and does not resonate with available evidence on net retention from field settings (e.g. see Killian et al.’s Net Calc). This limitation of their loss function was compensated for by introducing a ‘bias’ parameter that helped triangulate the entire modelling chain. As a result of this statistical compensation we do not believe their loss function represented reality particularly well – it is inconceivable that everyone in a given country discards all their nets at once at the three year mark. Additionally the Flaxman et al. net loss was fitted from 6 studies outside the main compartment model and fixed, whereas we learn a data driven, time varying loss function directly from the survey, manufacturer and NMCP data. We have discussed these limitations in the Methods section.

Minor points:

The estimated sharp fluctuations over time in ITN coverage indicators in some countries: are these real (if periodic campaigns dominate distributions?), or reflecting missing or incomplete data, e.g. in NMCP-reported distributions? In particular, comment on the odd time patterns in DRC and Ghana and/or a few other countries with irregular patterns.

We believe these fluctuations to be real. The priors on distributions are set as uniform and therefore in the absence of concrete data driving these sharp fluctuations the model will prefer smoother distributions. Sharp fluctuations only arise when triangulation between the NMCP, manufacturer and survey data demand it. In the absence of data the model will choose a more parsimonious and even distribution pattern. It should also be noted that these sharp distributions are accompanied by full Bayesian credible intervals capturing our uncertainty about the distribution pattern (if that uncertainty is there).

[Editors' note: further revisions were requested prior to acceptance, as described below.]

The reviewers still have concerns about the conclusions on net retention. The model in this paper only estimates ITN retention indirectly while the six studies that Flaxman depended on were directly measuring it. The bias parameters of Flaxman et al. were introduced because of a discrepancy between the different sources of data, if a loss function of longer duration is used. This discrepancy could be due to systematic differences between the data sources, and so the reviewers feel that the conclusion of the present paper that nets are retained for a shorter time than was previously thought is less certain than the authors suggest.

The key point of contention here revolves around how the NMCP distribution data triangulates with the loss function. We disagree that the measurement of the loss function, as modelled in Flaxman et al., is suitable and disagree that the bias parameter allows a loss function of longer duration. Below we explain the shortcomings of the Flaxman et al. approach and the true purpose of the bias parameter.

There are two modelling choices that can be made here:

1) The approach of Flaxman et al. – use a loss function based on 6 in field studies, and adjust the NMCP distribution data for a hypothesised systematic bias.

2) Our approach – learn a highly flexible loss function directly from the data and only adjust the NMCP distribution data for random noise.

To convince you that our approach is inherently a less subjective modelling choice consider the two following issues with the Flaxman et al. approach:

a) Net loss function:

Flaxman et al. paramaterised their loss function based on 6 studies with differing time points under no standardised conditions between studies. In these highly controlled studies key factors of net loss predominantly driven by living conditions, household behaviour and attitudes (Killian et al., 2015) are unaccounted for by the very nature of the study. Additionally some of the studies used in Flaxman et al. are retrospective studies where recall bias was a significant and unaccounted for confounder (Killian et al., 2015). We also must stress again just how implausible the loss function resulting from these studies is – the function is essentially constant (reductions with an average of 5% per year) for 3 years and then a complete stepwise reduction to zero at the 3-year mark. Under this loss function, using the upper 95% confidence interval, at most only 30% of nets can be lost after 3 years; immediately after which all nets are discarded. We believe that the implausibility of this loss function highlights the inconsistencies in paramaterising a function based on a small sample of unstandardized studies.

In contrast to this approach, we learn the loss function directly from the data, imposing an agnostic uninformative prior on the underlying half-life and duration. In our approach nets are lost smoothly over time (with no sudden discards) and the model has the choice of a long or short half-life depending the country and the year. From our fitted function (Figure 7) we find large variability between countries that would not be able to be represented using the Flaxman et al. loss function. Additionally, we actually say the duration of nets is longer than what Flaxman et al. say, with nets lasting up to 5 years on average (Figure 7 red curve). The key disagreement between our fitted loss function and that of Flaxman et al. is the year-to-year rate of reduction, and that all nets are lost exactly on the three year mark.

To further assure the reviewers of the implausibility of Flaxman et al. loss function, consider looking at the raw household survey data and extracting information on net age directly from each survey. When averaged across all surveys used in our study the average proportion of the total number of nets that were:

Less than one year old was 57%,

Between one and two years old was 20%

Between two and three years old was 5%

Greater than three years old was 18%

Obviously there are a lot more complexities to these simple summaries, including when the survey was sampled, the distribution pattern, recall bias etc. But one thing is clear, the year-to- year loss rate is very non-linear and probably much greater than 5% (which Flaxman et al. claim), and a large number of nets are older than 3 years (which the Flaxman et al. does not allow). This once again calls into question the plausibility of the Flaxman et al. loss function, but is consistent with our loss function.

b) The bias parameter:

The bias parameter in Flaxman et al. is used to systematically adjust the NMCP distribution data to match the household survey data. This is essential in their approach as given their constrained loss function (described above) the survey data would not match the distribution data without some form of adjustment. There are several conceptual and implementation problems with this bias parameter. Firstly this bias parameter is a very poor linear fit (see Figure 5, Supplementary information Flaxman et al.), and does not account for the sampling time of the survey or the recall bias inherent in the survey questions. The poor linear fit suggests substantial country specific variation. Second the prior does not model country specific variation making the assumption that all NMCP distribution data is biased in the same way.

Conceptually the problem with the bias parameter is a reliance on the assumption: the NMCP data is systematically biased (constant) across all countries and years. Flaxman et al. provide no justification for this assumption, and we believe it to be very unlikely that all NMCP data, reported from different countries, in different years are systematically biased in the same way. This is the equivalent to saying that for every country the number of nets reported to be distributed should be scaled by the same constant.

To further prove to the reviewers that using a bias parameter is not a sound modelling choice to explain the data, consider a like for like re-run of the Flaxman et al. bias parameter analysis. We take the weighted survey data totals for the survey questions:

Is your net less than one year old?

Is your net between 1 to 2 years old?

Is your net between 2 to 3 years old?

And regress these (on a logarithmic scale) against the corresponding NMCP distribution results in that year (with linear interpolation to match the average year of the survey). If the Flaxman et al. hypothesis of the NMCP data being systematically biased holds the intercepts of the three linear models (in Author response image 1 where the intercepts are in the brackets, the regression bias line is in green and the one-to-one line is in red) should be the same. However, these intercepts are orders of magnitude different (note the logarithmic scale) suggesting steep losses of nets and no systematic bias. A far more parsimonious explanation is that the differences in the number of LLINs from the Survey and NMCP are explained by the loss of nets occurring through time. This is supported by the increasing bias parameter, as nets get older. Indeed this analysis suggests that the sole purpose of Flaxman et al. bias parameter is to compensate for the deficiencies in their poorly characterised loss function.

Author response image 1.

DOI: http://dx.doi.org/10.7554/eLife.09672.019

In contrast we do not assume any systematic bias on the NMCP data, we only assume that the NMCP data contains random noise and let the variation be described by a flexible loss function.

Therefore in summary we hope we have made clear that opting for modelling approach (2) over (1) is a considerably more parsimonious choice making far fewer assumptions and letting the data speak. We therefore have high confidence in the fits of our modelled loss functions. We stress that we have been completely transparent with the modelling assumptions in the Appendix, and have made very few subjective decisions and focused on a data driven approach.

This also has implications for the point about sharp fluctuations in the predicted ITN coverage over time, which presumably would be smoother with a loss function of longer duration.

We believe the reviewers are mistaken that the cause of sharp fluctuations in ITN coverage is due to the loss function. The predominant and overwhelming cause of these fluctuations is the delivery and distribution data, not the loss function. Sharp fluctuations arise due to real, large distributions of nets captured in both the distribution and survey data. See a typical example of this below for Togo.

In Author response image 2, green dots are the manufacturer deliveries, purple triangles are the NMCP distributions, red triangles are the house-hold surveys, the red line is the number of LLINS, the yellow line is the number of cITNs and the black line is the residual stock. The fluctuations are caused by real large deliveries and distributions causing a saw tooth pattern in the number of nets. The reason the number of nets does not increase exponentially with the deliveries is due to loss, but this loss is smooth over time. Using the Flaxman et al. loss function would cause far less smoothness due to restriction of all nets being discarded at the three-year mark. To highlight this consider what would happen in the above plot if all nets were discarded after 3 years. As highlighted in the previous rebuttal the distribution of NMCP ITNs is uniform intra- years, as nothing in the data suggests a non-uniform pattern, thereby implicitly choosing a smooth distribution pattern unless the data would suggest otherwise.

Author response image 2.

DOI: http://dx.doi.org/10.7554/eLife.09672.020

We recommend that the authors conduct a sensitivity analysis that assumes longer retention of nets, showing how any projections change. In any case this uncertainty should be acknowledged and the implications for the projections discussed.

Given the above discussion about the suitability of our loss function we feel the reviewers will agree that a sensitivity analysis is unnecessary given our framework. If we used a loss function such as that in Flaxman et al., this sensitivity analysis would be warranted to test the validity of using such a constrained loss function. However, given the data driven, assumption free nature, of our analysis this is unnecessary as the sensitivity analysis is implicitly contained within the Bayesian framework. That is, during every fit, we explore a full range of candidate functions from differing retention times to sharp and smooth transitions and average over these to find the best function that fits the data. Therefore, during every fit, we perform a statistically robust sensitivity analysis. To state this differently our model allows for a loss function similar to that of Flaxman et al., but this loss function is not supported by the data at all.

That said, one of the important results of this paper is exploring the effects of a longer retention net half-life (3 year half-life). These results are presented and discussed in the main paper (Figure 8 red line) and in Appendix 1.

In addition, net loss and retention consider the physical presence of the ITNs in households, but not their effectiveness in repelling mosquitoes. According to manufacturer and WHOPES evidence and guidance on ITN's effective durability, is it justified to assume a median 36-month retention (in the optimized scenario), or does this imply a proportion of ITNs modelled as effective for longer than is realistically assumed? The authors should expound on this point in the discussion – for LLINs specifically

The scope of this paper was restricted to ownership and usage indicators and not on the effectiveness of nets once they are being used. Of course the insecticidal and barrier effectiveness of a nets diminish in a complex multifaceted way through time but due to these complexities, we did not expand our analysis to effectiveness and focus on the standard RBM definitions of ownership and use. We have expanded on this point in the Discussion.

For the country-level estimates: Are the 4 international indicators (ownership, access, usage) calculated and shown for each country's overall national population (including sub-populations at no risk, just as the DHS, MICS and MIS surveys) or for the sub-population at risk of malaria? What assumptions, if any, are made about (mis-)allocation of ITNs to populations not at risk of malaria? Please clarify this in the appropriate section of the manuscript

We use population at risk as the denominator in all the analysis in this paper and in doing so we avoid the problem of biased estimates by including populations not at risk. This however does make the assumption that there is no allocation of nets outside populations at risk. We have added a line in the main manuscript explicitly stating this assumption and thank the reviewers for pointing out this omission. However it is also vital to recognise that this adjustment by populations of risk only applies to a handful of countries in sub-Saharan Africa (e.g. Ethiopia, Kenya etc.) as all others are universally at risk.

Minor suggested edits:

If the world limit allows, please define 'Population access' in the Abstract.

These terms are fully described in the Introduction, subsection “Modelling coverage”.

Paragraph three, subsection “Net Loss”, 'minimised over-allocation': From the annex (and subsection “Estimating ITN Requirements to Achieve Universal Access”) it seems that this is ZERO over-allocation; please qualify or rephrase accordingly.

We thank the reviewers for this suggestion and have clarified this statement in the Methods and Appendix.

a) Parts of the diagram are confusing or not clear – please correct as needed:b) An arrow (or, equality sign) is missing from HH-level ITN ownership to the box below (National total ITNs in HH?). c) An arrow missing from National net crop to, Observed bivariate histogramsd) National net crop equals National total ITNs in HH 5 boxes below, but the equality is not indicated.

Appendix 1: The scaling across countries to get to continent-totals is not very clear.

Figure 2 describes the Bayesian compartment model only, we choose not to tie together the Poisson model for the indicators as felt this would make the figure too complex.