Taking the health aid debate to the subnational level: the impact and allocation of foreign health aid in Malawi.

OBJECTIVE: Cross-national studies provide inconclusive results as to the effectiveness of foreign health aid. We highlight a novel application of using subnational data to evaluate aid impacts, using Malawi as a case study.
DESIGN: We employ two rounds of nationally representative household surveys (2004/2005 and 2010/2011) and geo-referenced foreign aid data. We examine the determinants of Malawi's traditional authorities receiving aid according to health, environmental risk, socioeconomic and political factors. We use two approaches to estimate the impact of aid on reducing malaria prevalence and increasing healthcare quality: difference-in-difference models, which include traditional authority and month-of-interview fixed effects and control for individual and household level time-varying factors, and entropy balancing, where models balance on health-related and socioeconomic baseline characteristics. General health aid and four specific health aid sectors are examined.
RESULTS: Traditional authorities with greater proportions of individuals living in urban areas, more health facilities and greater proportions of those in major ethnic groups were more likely to receive aid. Difference-in-difference models show health infrastructure and parasitic disease control aid reduced malaria prevalence by 1.20 (95% CI -0.36 to 2.76) and 2.20 (95% CI 0.43 to 3.96) percentage points, respectively, and increased the likelihood of individuals reporting healthcare as more than adequate by 12.1 (95% CI 1.51 to 22.68) and 14.0 (95% CI 0.11 to 28.11) percentage points. Entropy balancing shows similar results.
CONCLUSIONS: Aid was targeted to areas with greater existing health infrastructure rather than areas most in need, but still effectively reduced malaria prevalence and enhanced self-reported healthcare quality.

Several studies have examined the association between health aid and health outcomes using cross-national data, where recent literature has come to conflicting results: some find significant associations between aid and key health outcomes, while others find no relation.

Evidence on aid impacts in Malawi remains limited, but suggests possible aid inefficiencies and attenuated impacts. Donor reports note inefficiencies in the health sector and that some funds have gone unaccounted for. Reports do note that, despite inefficiencies, health aid has had beneficial impacts, although conclusions were not based on empirically driven impact evaluations. One paper empirically examined how donor-financed expanded coverage of malaria interventions was associated with malaria infection rates at four hospitals in Malawi, finding no declines in malaria admission rates from 2000 to 2010.

There is a large literature examining determinants of aid allocation cross-nationally, although the literature on subnational aid allocation is limited. No study has examined the subnational allocation of health aid, specifically.

Using subnational data, this study finds that, in Malawi, health aid contributed to reducing malaria prevalence and improving self-reported healthcare quality. In particular, aid projects allocated towards improving basic health infrastructure and parasitic and infectious disease control had the most notable impacts.

Health aid was preferentially allocated to urban areas and areas with more existing health facilities; however, aid was not preferentially allocated to areas with the worst health conditions.

Subnational data can assist policymakers and practitioners in evaluating donor-funded health interventions and guiding allocation decisions.

In Malawi, refined aid targeting at the subnational scale may help to further bolster health outcomes.

Introduction

In the past two decades, health aid quintupled from US$7 billion in 1990 to US$36 billion in 2015 (2015 dollars).1 At the same time, health outcomes across developing countries have improved; estimates suggest that malaria death rates declined by 57% from 2000 to 2015 across sub-Saharan Africa.2 The growing commitment to address health challenges abroad alongside registered progress has been matched by an argument about the role development assistance has played in bolstering health outcomes in recipient countries. Advocates argue that health aid has reduced preventable diseases, particularly in light of cheap and readily available medical technologies.3 Even some aid critics argue that well-implemented health programmes can have beneficial impacts; for example, Deaton (2013) argues that aid undermines government accountability to citizens, but notes that health aid has caused measurable improvements in health outcomes.4 Aid critics, though, emphasise the frequency of poor implementation, pointing to projects failing to account for local conditions, poor donor coordination leading to duplicate efforts and corruption causing aid funds to disappear,5 6 suggesting that other factors—such as domestic health expenditures, economic growth and quality of governance—have driven gains in health outcomes.7–9

Aid effectiveness debates are largely informed by cross-national studies, where studies range from showing positive to no aid impacts. A number of studies fail to find health aid significantly improving key health indicators and suggest health aid is an ineffective policy tool.10 11 Other scholars find significant relations between health aid and health outcomes, particularly on reducing infant mortality and increasing life expectancy.12 13 Within nations, evidence highlights the effectiveness of specific interventions, such as insecticide-treated mosquito nets and indoor residual spraying;14 however, there is limited understanding of the impact of donor dollars allocated to these efforts.10

Conflicting evidence within aid effectiveness research suggests that alternate approaches are needed to analyse aid impacts. This paper employs a novel, subnational data set that allows for analysing aid in reference to the geographic location where it was allocated. By delineating areas that did and did not receive aid, these data readily allow for quasi-experimental approaches to estimating aid impacts. Moreover, a subnational approach allows for mitigating omitted variable bias inherent in estimating aid impacts with cross-national data.

With the emergence of geo-referenced data, scholars have begun to examine subnational aid dynamics. Existing findings suggest that aid is not consistently allocated to the poorest areas and that political motivations may drive aid allocation, especially in countries with weak political institutions.15 16 Additionally, a subnational approach has highlighted the effectiveness of the President's Emergency Plan for AIDS Relief (PEPFAR) in Tanzania and Rwanda.17 However, subnational data have not been used to examine aid impacts beyond targeted programmes, nor have scholars examined health aid allocation specifically. Despite the lack of empirical evidence, scholars have hypothesised about the subnational allocation of health aid, arguing that more accurate data about the distribution of disease may have led to improved targeting of aid, which in turn may have enhanced aid effectiveness.12

Using Malawi as a case study, we ask two questions that contribute to the discussion on the effectiveness of international development assistance. First, how is health aid allocated according to malaria prevalence, health infrastructure, ecological conditions that increase risk of disease, socioeconomic status and political affiliation? Second, what is the impact of aid on reducing malaria prevalence and improving health infrastructure?

Malawi provides a useful case study to investigate health aid impacts due to poor health conditions, significant donor attention and possible evidence of aid inefficiencies. In the 2000s, Malawi saw worsening socioeconomic and health conditions; for example, malaria incidence increased from 295 cases per 1000 people in 2005 to 458 in 2009.18 Deteriorating conditions led to increased attention from donors;19 estimates from AidData indicate that up to US$50 to US$200 million in health aid have been allocated annually in the past decade (see online supplementary appendix A). Broadly, Malawi remains highly donor dependent, with foreign aid making up about 40% of the national budget.20 Donor reports claim that development efforts have improved health outcomes, but note some aid funds have been unaccounted for and poorly documented, and that inefficiencies in the health sector, such as inefficient resource flows to health centres, have attenuated aid impacts.21 In addition, government officials have been accused of siphoning off funds into their own pockets, causing donors to question aid effectiveness and freeze aid at times.20 Empirical evidence examining the impact of donor dollars remains limited. One empirical study examined donor-financed expanded coverage of malaria interventions in four hospitals in Malawi, only to find malaria admission rates increasing or remaining unchanged between 2000 and 2010.22

Malawi is also uniquely suited for a subnational evaluation of foreign aid. Geographically referenced surveys were administered near the beginning of a large surge in foreign aid in the mid-2000s and near the peak of these aid flows in the early 2010s, providing a strong context for evaluations of aid (see online supplementary appendix A for trends of health aid disbursements). Malawi also contains ∼250 tertiary administrative divisions (traditional authorities), enabling a highly targeted subnational analysis. Moreover, the government of Malawi had very limited capacity during the 2000s,23 resulting in little chance that aid crowded out any government efforts.

Methods

Data sources

This analysis is based on five data sources: (1) AidData's Malawi Aid Management Platform (AMP) data set, which includes data on the locations of aid projects recorded by the Government of Malawi's AMP and includes roughly 80% of all aid reported to the Ministry of Finance since 2000;24 (2) Malawi's 2010/2011 Integrated Household Survey (IHS), which surveyed 12 271 households representing 56 218 individuals;25 (3) Malawi's 2004/05 IHS, which surveyed 11 280 households representing 51 292 individuals;25 (4) the locations of health facilities acquired from the Malawi Ministry of Health, which specified 783 health facilities constructed up to 199826 and (5) Malawi's 2008 census data.27 Online supplementary appendix B indicates the variables that were used from each data source.

Data in Malawi's AMP are hierarchically coded, where purpose codes represent the overarching purpose of aid projects and activity codes are subcategories of purpose codes indicating specific project activities. We restrict our analysis to projects that could feasibly have local impacts on disease burdens and health infrastructure, including any project with a purpose or activity code from the following aid sectors: medical services, basic healthcare, basic health infrastructure and infectious and parasitic disease control. Subsequent allocation and impact analyses examine the aid sectors separately and collectively.

Medical services aid includes projects directed towards specialised medical services and equipment, such as a US$7 million European Union blood transfusion service programme. Basic healthcare aid includes projects directed towards primary healthcare programmes and towards supplying basic medications, such as a US$3 million German Development Bank (KFW) project to bolster health services in the Chitipa district, which included rehabilitating six health centers.28 Basic health infrastructure aid includes projects directed towards enhancing or providing basic medical equipment to hospitals, clinics and dispensaries, such as a US$1.5 million Icelandic International Development Agency (ICEIDA) project supporting the Kamuzu Central Hospital in Lilongwe. Infectious and parasitic disease control aid includes prevention and control activities directed towards infectious diseases such as malaria, such as a US$33 million World Bank education support project that included US$3 million targeted towards enhancing school health programmes, including de-worming and treatment of malaria and fever.29 Online supplementary appendix C provides further descriptions of aid sector categories, and online supplementary appendix D lists all aid projects included in allocation and impact analyses.

Joining aid data with survey data

Each aid project carries a precision code that indicates the spatial certainty of where the project was allocated. Spatial precision ranges from a precise location to the entire country. We include all projects with a spatial precision at a specific location and at the district-level, excluding all projects with spatial precision at the region and country-level as these are too coarse for the analysis (of all project locations, 19% were excluded). Of the projects omitted from this study, a number were government-wide reform programmes, such as grants to the Ministry of Health and general support to health reform. Projects allocated to a specific location are coded as being allocated to a traditional authority, which is Malawi's third administrative division. In household surveys, we code individuals as receiving aid if they lived in a traditional authority where aid disbursements were above zero.

Despite the granularity of the aid data, it is difficult to distinguish which individuals in the survey data were beneficiaries of aid. For example, KFW's efforts to rehabilitate six health centres in the Chitipa district likely primarily benefited individuals that the health centres serviced, not everyone in the district. Rather than delineating individual beneficiaries of aid, our data should be viewed as delineating individuals who resided in areas that received aid. On average, individuals living in areas that received aid should exhibit improved health outcomes if aid is effective.

Aid allocation

Aid allocation is examined using traditional authorities as the unit of analysis. To incorporate the variation in where aid was allocated across time, we disaggregate aid into separate years. Years for aid projects are based on the agreement date of the project which extend from 2004 to 2011. We rely on the agreement date rather than the completion date of the project as aid allocation models are intended to examine where donors choose to allocate aid given current conditions in a country. The resulting data set includes 29 discrete projects from 9 different donors allocated across 317 project locations, totaling in US$170.3 million disbursed (online supplementary appendix D describes projects). We use logistic models to understand the determinants of whether a traditional authority received aid, and ordinary least squares models to understand the drivers of per capita aid among traditional authorities that received aid (population data to calculate per capita aid come from Malawi's 2008 census). In the aid data set, aid dollar amounts are provided at the project level; in imputing aid dollars to project locations, we assume aid was allocated equally across project locations. When projects cover multiple traditional authorities (eg, district-level projects), we assume aid was allocated equally across traditional authorities.

We examine how four categories of covariates are associated with receiving aid: malaria prevalence, health infrastructure, disease ecological risk factors, socioeconomic status and political affiliation (online supplementary appendix B provides further variable descriptions). Malaria prevalence is the percentage of individuals that reported falling ill with malaria/fever but no other symptoms in the two weeks prior to being surveyed; excluding illnesses with other symptoms is performed to distinguish malaria from other causes of fever. Health infrastructure includes perceptions of healthcare quality, percentage of households that use mosquito nets and the number of health facilities per 1000 people in a traditional authority. Disease ecological risk factors include temperature and precipitation, as higher temperatures and precipitation are associated with an increased risk of vector-borne diseases.30 Ecological risk factors may be used in aid allocation decisions because disease burden data are often scarce in sub-Saharan Africa; weak surveillance systems capture only about 10% of malaria cases.31 Wealth perception and whether an individual lived in a rural or urban area serve as socioeconomic variables. Scholars have hypothesised that health aid may be preferentially allocated to urban areas; however, this question has not been empirically examined.32 Covariates from household surveys are measured before any aid interventions were introduced (ie, at baseline levels).

Political variables include proportions of the population in traditional authorities that belong to certain ethnic groups and a variable indicating whether a traditional authority is in the president's birth district. Bingu wa Mutharika served as the president throughout the time period in this analysis—from 2004 to 2012.20 We include the four largest ethnic groups in Malawi: the Chewa (32.6% of the population), Lomwe (17.6%), Yao (13.5%) and the Ngoni (11.5%), of which the Lomwe is the ethnic group of the president.33 Political variables are examined as aid could be preferentially allocated to areas in support of or with close ties to the president or other government leaders.34 Political motivations can play a role in aid allocation as donors often cooperate with government agencies in order to allocate aid, and donors often delegate responsibility of aid management because recipient governments have greater information on local conditions.35

Aid impacts

We employ a repeated cross-section difference-in-difference approach to examine the causal impact of aid, using individual-level survey data from 2004/2005 and 2010/2011. Despite the two surveys not tracking the same individuals over time, a repeated cross-section approach is appropriate in this context as both surveys are nationally representative. We examine the impact of health aid on malaria prevalence and perceptions of healthcare quality. Malaria prevalence, specifically, is examined due to the particularly high prevalence of malaria in Malawi; out of a population of 15 million, up to 6 million cases of malaria are reported annually, and 34% of outpatient visits result from malaria.18 Healthcare quality, measured as individuals rating healthcare as more than adequate, is examined to understand whether aid caused individuals to view their healthcare as having improved.

Aid projects are included if their projected completion date fell between household survey years (2005–2009). Relying on completion dates of aid projects ensures all aid dollars are disbursed before the end-line survey (2010/2011), allowing time for projects to impact health outcomes. The resulting data set includes 13 discrete projects from 9 donors allocated across 103 project locations, totaling in US$113.8 million disbursed (online supplementary appendix D describes projects; online supplementary appendix E shows the spatial distribution of aid disbursements). In aid impact models, we use aid as a binary variable, indicating whether a traditional authority received aid or not. Doing so ignores the amount of aid that was allocated, where more donor dollars allocated may lead aid to have larger impacts; however, leaving aid as a binary variable avoids assumptions involved with imputing aid dollars from projects to project locations to individuals. Future work is needed to incorporate uncertainties involved in imputing aid down to smaller spatial scales in analyses.

The difference-in-difference model used for estimation is described in equation 1:where y is the binary outcome variable—in one set of models y indicates whether an individual fell sick with malaria or fever (and reported no other symptoms) in the two weeks prior to being surveyed, and in another set of models y indicates whether an individual rated their healthcare quality as more than adequate. Aid is a binary variable indicating whether an individual lived in a traditional authority that received aid, Year indicates the time period (either 2004/2005 or 2010/2011), Aid×Year is the treatment effect, xk…xK are a vector of socioeconomic and health controls, δta are traditional authority fixed effects and γm are month fixed effects, where month refers to the month in which the individual was interviewed.

The controls include time-varying socioeconomic and health factors that may influence malaria burden and healthcare quality where data were available for both time periods. The covariates help to control for time-varying factors that impact health conditions independent of aid, including compositional changes from surveying different people across the two cross-sections. Variables include mosquito net ownership, the wealth index (individuals rating how poor/rich they are on a 6 unit scale), whether an individual lived in an urban area, age, education level of the household head, healthcare quality (for models explaining malaria prevalence), malaria prevalence (for models explaining healthcare quality) and a set of asset variables which gauge socioeconomic conditions, including the toilet type in households, wall material of households, floor material of households and drinking water source (see online supplementary appendix B for further covariate descriptions). In sector-specific models, we include a dummy variable indicating whether an individual lived in a traditional authority that received other types of health aid. Traditional authority fixed effects control for time-invariant factors across traditional authorities. Month fixed effects control for seasonal variation in disease prevalence.

We examine how aid impacts vary across Malawi by interacting the impact of aid with relevant covariates. The model used for estimation adds an Aid×Year×Covariate interaction term to equation 1, where Covariate is either an indicator variable indicating whether the individual lived in an urban area, the wealth index, or a variable indicating the number of health facilities per 1000 people in a traditional authority.

We perform a series of robustness checks to check the stability of difference-in-difference results. First, to check the sensitivity of results to model specifications we report models without health-related covariates (malaria prevalence, healthcare quality and number of mosquito nets)—which could be endogenous with health aid—and models without any socioeconomic and health covariates (see online supplementary appendix F). Second, as a further check on model specification we report models using placebo outcomes as dependent variables (ie, variables one would not expect to be influenced by health aid) to ensure aid is not correlated with these variables (see online supplementary appendix G). Third, we report models excluding urban areas to check whether results are driven by urban dynamics (see online supplementary appendix H). Fourth, we report difference-in-difference models with survey weights (see online supplementary appendix I). In some cases, the inclusion of survey weights when estimating causal effects can impact the precision of estimates, which has led to debate about the appropriateness of including survey weights.36 However, to ascertain potential bias from this source we report models that include survey weights as a point of comparison to those that do not include them.

Difference-in-differences models rely on assuming that areas that did and did not receive aid would have experienced similar trends in the outcome variable absent of aid. We cannot check this assumption without pre-treatment data. Consequently, we use an alternative quasi-experimental strategy—entropy balancing—as a further robustness check on the results (see online supplementary appendix J). Entropy balancing does not assume parallel trends between treatment and control areas absent of treatment; however, entropy balancing requires specifying all potentially relevant covariates, while difference-in-differences allows for controlling for unobserved time-invariant factors among traditional authorities.

To better understand health aid impacts, we estimate the number of malaria cases in 2010 that aid prevented and the number of days people could carry out productive activities as a result of not being sick. For difference-in-difference models, estimates of cases prevented are calculated by multiplying the average treatment effect against the estimated number of people in treated areas. Average days individuals could not carry out productive activities are then multiplied against these values. Calculations are further discussed in online supplementary appendix K.

Results

Poor health conditions do not drive aid allocation among most health aid sectors (see tables 1 and 2). High rates of malaria and low measurements of healthcare quality were not associated with receiving most sectors of aid. Having malaria was associated with a 24 percentage point decrease in the likelihood of receiving basic healthcare aid (p<0.10), but a 56 percentage point increase in the likelihood of receiving basic infrastructure aid (p<0.01). Among areas that did receive aid, having malaria was associated with a 3.1% decrease in per capita disbursements. Additionally, greater environmental disease risk did not increase the likelihood of receiving aid.

Table 1

Allocation of health aid according to traditional authorities receiving aid

	Dependent variable: traditional authority received aid
	All health aid	Medical services aid	Basic health care aid	Basic health infrastructure aid	Parasitic disease control aid
Malaria	0.460	0.234	−0.243*	0.566***	−0.022
	(0.250)	(0.169)	(0.216)	(0.173)	(0.184)
Healthcare quality	0.074*	0.018	0.046	0.002	0.023
	(0.038)	(0.024)	(0.032)	(0.033)	(0.026)
Mosquito net	0.079	0.033	0.063	0.054	0.025
	(0.057)	(0.033)	(0.049)	(0.040)	(0.038)
Wealth index	−0.090***	−0.006	−0.059**	−0.013	−0.004
	(0.029)	(0.017)	(0.025)	(0.021)	(0.019)
Urban	0.196***	0.065*	0.049**	0.129***	0.019
	(0.037)	(0.032)	(0.033)	(0.029)	(0.028)
Number of health facilities (PC)	0.613***	0.235	0.621***	0.083	0.340**
	(0.249)	(0.155)	(0.221)	(0.178)	(0.149)
Temperature	−0.001	0.007	0.002	0.006	0.017
	(0.015)	(0.008)	(0.013)	(0.011)	(0.011)
Precipitation	−0.056***	−0.012	−0.036**	−0.028**	−0.021**
	(0.016)	(0.008)	(0.014)	(0.012)	(0.009)
Per cent Lomwe	0.393***	0.096*	0.195**	0.228***	0.101
	(0.077)	(0.048)	(0.067)	(0.056)	(0.047)
Per cent Chewa	0.161***	0.054**	0.101**	0.065**	0.029
	(0.044)	(0.023)	(0.038)	(0.031)	(0.026)
Per cent Yao	0.372***	0.059	0.305**	0.086**	0.109***
	(0.069)	(0.034)	(0.060)	(0.040)	(0.038)
Per cent Ngoni	0.065	−0.038	0.113***	−0.157***	−0.139***
	(0.059)	(0.028)	(0.049)	(0.057)	(0.042)
President birth district	−0.024	−0.058**	−0.003	−0.026	−0.194***
	(0.050)	(0.024)	(0.046)	(0.029)	(0.008)
Year fixed effects	Yes	Yes	Yes	Yes	Yes
Observations	1376	1376	1376	1376	1376

Note: Average marginal effects reported with robust SEs in parentheses. Covariates from household surveys are measured at baseline levels. PC, per capita; specifically, per 1000 individuals.

*p<0.1; **p<0.05; ***p<0.01.

Table 2

Aid allocation according to per capita health aid disbursements

	Dependent variable: log (per capita health aid disbursement)
	All health aid	Medical services aid	Basic health care aid	Basic health Infrastructure aid	Parasitic disease control aid
Malaria	−3.143**	−2.727	−5.741***	−3.743***	−2.167
	(1.538)	(2.871)	(1.461)	(1.446)	(2.175)
Healthcare quality	−0.114	−0.213	−0.485***	−0.076	−0.378
	(0.192)	(0.295)	(0.183)	(0.222)	(0.272)
Mosquito net	0.508	0.486	0.594*	0.135	0.713
	(0.334)	(0.528)	(0.343)	(0.418)	(0.509)
Wealth index	−0.085	−0.096	−0.098	0.015	0.083
	(0.179)	(0.237)	(0.182)	(0.185)	(0.225)
Urban	0.283	−0.056	0.007	0.151	0.520
	(0.215)	(0.319)	(0.195)	(0.233)	(0.318)
Number of health facilities (PC)	6.380***	5.885***	5.286***	4.852***	5.489**
	(1.457)	(1.275)	(0.977)	(1.036)	(2.147)
Temperature	−0.089	−0.087	−0.009	−0.013	−0.256*
	(0.096)	(0.140)	(0.083)	(0.105)	(0.148)
Precipitation	0.452***	0.400***	0.522***	0.377***	0.391***
	(0.085)	(0.112)	(0.081)	(0.081)	(0.092)
Per cent Lomwe	−1.908***	−1.828***	−1.620***	−1.427***	−2.791***
	(0.372)	(0.561)	(0.346)	(0.356)	(0.456)
Per cent Chewa	−0.347	−0.609	−0.510*	−0.417	−0.485
	(0.288)	(0.420)	(0.266)	(0.296)	(0.357)
Per cent Yao	0.007	0.796*	−0.045	0.303	−0.183
	(0.303)	(0.483)	(0.266)	(0.356)	(0.389)
Per cent Ngoni	0.391	0.116	0.682**	0.294	0.388
	(0.312)	(0.433)	(0.286)	(0.335)	(0.356)
President birth district	0.522**	0.845*	0.647**	−0.516
	(0.264)	(0.477)	(0.275)	(0.380)
Year fixed effects	Yes	Yes	Yes	Yes	Yes
Observations	556	221	435	290	249
Adjusted R2	0.472	0.359	0.511	0.359	0.283

Note: Coefficients reported with robust SEs in parentheses. No estimates for presidential birth district obtained for parasitic control aid because no parasitic control aid was allocated to the president's birth district. Covariates from household surveys are measured at baseline levels. Traditional authorities that did not receive aid are excluded from models.

*p<0.1; **p<0.05; ***p<0.01.

Despite aid not preferentially allocated to areas with worse health conditions, poorer traditional authorities were more likely to receive aid, particularly basic healthcare aid. A 1 unit decrease in the wealth index (where the index is based on a 6 unit scale) was associated with a 5.9 percentage point increase in the likelihood of a traditional authority receiving aid. Consequently, the poorest areas (ie, those with a wealth index of 1) had a 35.4 percentage point greater likelihood of receiving aid compared to the richest areas (ie, those with a wealth index of 6). However, among areas that received aid, aid was not preferentially allocated to poorer areas.

Traditional authorities that had greater existing infrastructure—in particular, urban areas and areas with more health facilities per capita—had a higher likelihood of receiving aid. Living in an urban area and an additional health facility per 1000 people were associated with a 19 and 61 percentage point increase in the likelihood of receiving aid, respectively (p<0.01). Moreover, among areas that did receive aid, areas with more health facilities per capita tended to receive more aid disbursements.

Traditional authorities with greater proportions of major ethnic groups tended to have a higher likelihood of receiving aid. Across all sectors, being in the Lomwe (the ethnic group of the president during the time, Bingu wa Mutharika), Yao or Chewa ethnic groups was associated with a 39, 37, and 16 percentage point increase in the likelihood of receiving aid, respectively (p<0.01). However, traditional authorities with higher proportions of those in the Lomwe group received less aid per capita among areas that received aid. Traditional authorities in the president's birth district were less likely to receive aid, particularly for medical services and parasitic disease control aid; however, among areas that received aid, areas in the president's birth district received greater medical services and basic healthcare aid.

Figure 1 shows changes in malaria prevalence and healthcare quality across traditional authorities between 2004 and 2010. On average, areas that received aid experienced reductions in malaria prevalence, while areas that did not receive aid experienced increases in malaria prevalence. Most traditional authorities saw increases in perceptions of healthcare quality; however, areas that received aid experienced slightly larger increases.

Figure 1

Trends in outcomes across time. Note: Thin lines denote individual traditional authorities, and thick lines denote averages.

Difference-in-difference models show that basic health infrastructure and parasitic control aid were associated with reducing malaria prevalence (table 3) and bolstering perceptions of healthcare quality (table 4). Average marginal effects show that basic infrastructure aid was marginally associated with reducing malaria prevalence by 1.2 percentage points (p<0.10) but increased the likelihood of an individual reporting above average healthcare by 12.1 percentage points (p<0.05). Parasitic and disease control aid was associated with reducing malaria prevalence by 2.2 percentage points (p<0.05) and was marginally associated with increasing the likelihood of an individual reporting above average healthcare quality by 14 percentage points (p<0.10).

Table 3

Impact of aid on malaria prevalence

	Dependent variable: individual reporting falling ill with malaria/fever
	All health aid	Medical services aid	Basic health care aid	Basic health infrastructure aid	Parasitic disease control aid
No interaction term
Health Aid×Year	–0.008	–0.002	0.001	−0.012*	−0.022**
	(0.007)	(0.009)	(0.009)	(0.008)	(0.009)
Observations	68 719	68 719	68 719	68 719	68 719
Moran's I (p value)	0.492	0.448	0.446	0.433	0.512
Socioeconomic and health controls	Yes	Yes	Yes	Yes	Yes
Other health aid control	NA	Yes	Yes	Yes	Yes
TA fixed effects	Yes	Yes	Yes	Yes	Yes
Month fixed effects	Yes	Yes	Yes	Yes	Yes
Health aid impact interacted with wealth
Health Aid×Year	−0.008	−0.003	−0.001	−0.014	−0.014
	(0.008)	(0.012)	(0.010)	(0.009)	(0.012)
Health Aid×Year×Wealth index	0.000318	0.001	0.001	0.002	−0.003
	(0.003)	(0.005)	(0.003)	(0.004)	(0.006)
Observations	68 719	68 719	68 719	68 719	68 719
Moran's I (p value)	0.561	0.547	0.528	0.498	0.514
Socioeconomic and health controls	Yes	Yes	Yes	Yes	Yes
Other health aid control	NA	Yes	Yes	Yes	Yes
TA fixed effects	Yes	Yes	Yes	Yes	Yes
Month fixed effects	Yes	Yes	Yes	Yes	Yes
Health aid impact interacted with number of health facilities
Health Aid×Year	−0.005	−0.007	0.003	−0.005	−0.003
	(0.010)	(0.016)	(0.012)	(0.012)	(0.023)
Health Aid×Year×Health facilities	−0.052	−0.130	−0.015	−0.157	−0.503
	(0.099)	(0.195)	(0.125)	(0.210)	(0.393)
Observations	68 719	68 719	68 719	68 719	68 719
Moran's I (p value)	0.54	0.475	0.505	0.466	0.449
Socioeconomic and health controls	Yes	Yes	Yes	Yes	Yes
Other health aid control	NA	Yes	Yes	Yes	Yes
TA fixed effects	Yes	Yes	Yes	Yes	Yes
Month fixed effects	Yes	Yes	Yes	Yes	Yes
Health aid impact interacted with urban
Health Aid×Year	−0.008	0.005	−0.001	−0.009	−0.024**
	(0.007)	(0.010)	(0.009)	(0.009)	(0.010)
Health Aid×Year×Urban	0.009	−0.023*	0.021	−0.009	0.025
	(0.011)	(0.014)	(0.015)	(0.013)	(0.027)
Observations	68 719	68 719	68 719	68 719	68 719
Moran's I (p value)	0.607	0.373	0.564	0.418	0.548
Socioeconomic and health controls	Yes	Yes	Yes	Yes	Yes
Other health aid control	NA	Yes	Yes	Yes	Yes
TA fixed effects	Yes	Yes	Yes	Yes	Yes
Month fixed effects	Yes	Yes	Yes	Yes	Yes

Note: Average marginal effects reported with SEs in parentheses. SEs clustered on traditional authorities. NA refers to not applicable. TA refers to traditional authorities. The health aid variable reflects the aid sector listed in the column heading.

*p<0.1; **p<0.05; ***p<0.01.

Table 4

Impact of aid on healthcare quality

	Dependent variable: individual reporting above average healthcare quality
	All health aid	Medical services aid	Basic health care aid	Basic health infrastructure aid	Parasitic disease control aid
No interaction term
Health Aid×Year	0.009	0.034	−0.024	0.121**	0.140*
	(0.027)	(0.057)	(0.021)	(0.054)	(0.072)
Observations	68 719	68 719	68 719	68 719	68 719
Moran's I (p value)	0.609	0.634	0.848	0.570	0.834
Socioeconomic and health controls	Yes	Yes	Yes	Yes	Yes
Other health aid control	NA	Yes	Yes	Yes	Yes
TA fixed effects	Yes	Yes	Yes	Yes	Yes
Month fixed effects	Yes	Yes	Yes	Yes	Yes
Health aid impact interacted with wealth
Health Aid×Year	0.037	0.000027	−0.030	0.222***	0.321***
	(0.034)	(0.031)	(0.023)	(0.070)	(0.091)
Health Aid×Year×Wealth index	−0.011*	0.011	0.003	−0.019**	−0.031***
	(0.006)	(0.010)	(0.006)	(0.009)	(0.009)
Observations	68 719	68 719	68 719	68 719	68 719
Moran's I (p value)	0.692	0.554	0.852	0.669	0.912
Socioeconomic and health controls	Yes	Yes	Yes	Yes	Yes
Other health aid control	NA	Yes	Yes	Yes	Yes
TA fixed effects	Yes	Yes	Yes	Yes	Yes
Month fixed effects	Yes	Yes	Yes	Yes	Yes
Health aid impact interacted with number of health facilities
Health Aid×Year	0.011	0.204	−0.054*	0.246**	0.136*
	(0.037)	(0.179)	(0.030)	(0.124)	(0.078)
Health Aid×Year×Health Facilities	−0.044	−0.778	0.798	−0.748	−0.002
	(0.412)	(0.722)	(0.519)	(0.643)	(0.303)
Observations	68 719	68 719	68 719	68 719	68 719
Moran's I (p value)	0.559	0.66	0.701	0.546	0.762
Socioeconomic and health controls	Yes	Yes	Yes	Yes	Yes
Other health aid control	NA	Yes	Yes	Yes	Yes
TA fixed effects	Yes	Yes	Yes	Yes	Yes
Month fixed effects	Yes	Yes	Yes	Yes	Yes
Health aid impact interacted with urban
Health Aid×Year	0.007	0.126*	−0.027	0.191***	0.201*
	(0.027)	(0.065)	(0.021)	(0.067)	(0.106)
Health Aid×Year×Urban	0.018	−0.052***	0.034	−0.042***	−0.034
	(0.029)	(0.013)	(0.050)	(0.016)	(0.023)
Observations	68 719	68 719	68 719	68 719	68 719
Moran's I (p value)	0.590	0.544	0.861	0.778	0.957
Socioeconomic and health controls	Yes	Yes	Yes	Yes	Yes
Other health aid control	NA	Yes	Yes	Yes	Yes
TA fixed effects	Yes	Yes	Yes	Yes	Yes
Month fixed effects	Yes	Yes	Yes	Yes	Yes

Note: Average marginal effects reported with SEs in parentheses. SEs clustered on traditional authorities. NA refers to not applicable. TA refers to traditional authorities. The health aid variable reflects the aid sector listed in the column heading.

*p<0.1; **p<0.05; ***p<0.01.

Interaction terms on the impact of health aid are insignificant in models explaining malaria prevalence. However, the impact of health infrastructure and parasitic control aid on increasing perceptions of healthcare quality diminishes in more wealthy areas, and the impact of health infrastructure aid is smaller in urban areas. In models that include an interaction of health aid impacts with urban areas, medical services aid appears marginally significant in bolstering perceptions of healthcare quality (p<0.10); however—like with health infrastructure aid—the effect is smaller in urban areas. Across all models, the number of health facilities (per capita) interacted with health aid impacts is insignificant.

Entropy balancing shows all sectors of aid strongly associated with reducing malaria prevalence, with coefficients ranging from aid causing a 9% to 21% reduction in malaria prevalence (p<0.05; see online supplementary appendix J). In addition, entropy balancing shows health infrastructure and parasitic control aid associated with increasing individuals’ perceptions of healthcare quality by 5% and 10%, respectively (p<0.05), while medical services and basic healthcare aid are not associated with perceptions of healthcare quality.

Results are generally robust to sensitivity analyses. One concern is that models excluding all covariates and models including survey weights show basic healthcare aid associated with decreasing the likelihood of individuals’ reporting above average healthcare quality. However, models including survey weights show that as the number of health facilities per 1000 individuals increases by one the impact of basic healthcare aid on improving healthcare perceptions increases by 92 percentage points. Overall, results are unclear on the effect of basic healthcare aid, although models suggest that basic healthcare aid appears primarily effective at bolstering perceptions of healthcare quality in areas with existing health facilities.

Owing to using spatial data, spatial autocorrelation could be present in models. We cluster SEs on traditional authorities, which allows dependence among individuals within traditional authorities; however, spatial autocorrelation could still exist between traditional authorities. To test whether spatial autocorrelation exists between traditional authorities, we add residuals within each traditional authority and calculate Moran's I—a metric of spatial autocorrelation—across these summed residuals. Spatial autocorrelation appears insignificant across all models (p>0.10, see tables 3 and 4).

Discussion

This research provides evidence that foreign health aid effectively reduced malaria and enhanced perceptions of healthcare quality in Malawi. Aid effectiveness results vary according to the specific sector of health aid examined; the most robust results point to the effectiveness of parasitic control and basic infrastructure aid. Aid effectiveness results are economically substantial, with aid preventing a conservatively estimated 85 000 cases of malaria in 2010 alone, enabling people able to engage in up to a collective 535 000 additional days of productive activities due to not being sick (see online supplementary appendix K).

Strong determinants of whether a traditional authority received aid were existing levels of infrastructure, exhibited by greater levels of urbanisation and a higher number of health facilities per capita. However, despite aid allocated to areas with greater infrastructure, donors showed a preference towards allocating aid to poorer areas for only one sector of aid: basic healthcare aid. These results suggest that existing infrastructure was an important factor for donors; however, among areas with similar infrastructure, donors tended to allocate basic healthcare aid to poorer areas. However, other aid sectors were not targeted to poorer areas and, among areas that received aid, donors did not preferentially allocate aid to poorer areas or areas with high levels of malaria prevalence.

Even if aid was not targeted to the most destitute areas, pervasive poverty across Malawi (∼62% of Malawi's population lives below US$1.25/day)37 suggests even relatively better-off areas would benefit from aid. To this point, aid appeared more effective in less wealthy areas; however, results still showed aid having significant effects in wealthier areas. Consequently, while results indicate that aid was effective across areas with different socioeconomic levels, donors better targeting aid towards areas where it is most needed would likely enhance aid effectiveness.

Results indicate possible evidence of political factors driving aid allocation. Areas with greater proportions of major ethnic groups, including the ethnic group of the president, tended to have a higher likelihood of receiving aid compared to other areas. Further, areas in the president's birth district tended to receive more aid per capita among areas that received aid. These trends are consistent with Malawi's political situation. For example, President Bingu wa Mutharika's second term (2009–2012) was plagued by corruption as he gave leaders in his ethnic group power throughout all branches and different levels of government.38 While this research does not provide conclusive evidence of such corruption, it does suggest that further inquiry may be warranted.

In total, US$113.8 million was allocated in aid for projects used in aid impact analyses. Directly comparing aid flows to cases of malaria averted yields US$1300 per case averted; however, true costs per case averted will be lower as not all aid dollars were directly targeted towards malaria projects. For example, a number of projects were not exclusively focused on reducing malaria (eg, projects directed towards strengthening local health systems or clinics) and health initiatives were sometimes secondary efforts in aid projects; of the World Bank's US$33.8 million Education Sector Support project, US$3 million was directly targeted towards health efforts. Owing to difficulties in ascribing costs specifically targeted for combatting malaria, caution should be taken in making judgements about cost-effectiveness of aid based solely on these results.

While our research highlights aid effectiveness, it is important to note limitations of the data. First, the AidData data set does not include data from donors without an in-country office, which represents ∼10% of total aid.39 Second, non-governmental organisations (NGOs) are not required to report data to the AMP; however, NGO data are captured if donors that report to the AMP fund the NGO. Consequently, the true amount of total aid used in this analysis is likely a conservative underestimate. To this point, the AidData data set underestimates health aid flows compared to data from the Institute for Health Metrics and Evaluation; however, aid disbursements are roughly comparable to the Creditor Reporting System database (see online supplementary appendix L for comparisons of AidData to other data sources). Third, we do not incorporate the timing of projects in the aid impact analyses (ie, we collapse projects completed anytime from 2005 to 2009 into one variable instead of treating projects completed in different years separately). Treating aid as timeless between 2005 and 2009 is carried out due to only having two years of survey data; however, future research would be useful in exploring the timing of aid and changes in outcome variables. Fourth, while the evidence presented in this paper does not suggest that there is a substantial population of individuals travelling outside of their traditional authority of residence to receive healthcare (ie, no spatial autocorrelation in difference-in-difference models), such patterns were not the focus of this analysis and bias from these movements could exist in our results. This provides an important path for future research. Fifth, we do not explicitly control for government and private investments in health that are not funded by foreign donors, which could bias impact estimates. However, the Malawi government's contribution to health expenditure is small compared to donors. In 2005/2006, foreign donors contributed US$15 per capita to Malawi's health expenditure, while the government contributed US$5 per capita.40 Moreover, ∼90% of medicine costs are covered by donors.41 Consequently, any biases from exclusively government-funded interventions should be minimal.

Beyond data concerns, aid impact analyses across sensitivity analyses were less consistent for basic healthcare aid compared to other aid sectors. In particular, difference-in-difference models excluding socioeconomic and health covariates and models using survey weights show basic healthcare aid having a negative impact on healthcare perceptions, although results show basic healthcare aid having beneficial impacts in areas with greater numbers of health facilities per capita, and no negative effect is found in difference-in-difference models without survey weights and in entropy balancing models. Taken together, the results are not clear on the impact of basic healthcare aid; future work investigating aid projects in this sector would be useful. Despite issues in models for basic healthcare aid, beneficial effects of health infrastructure and parasitic control aid are more robust across sensitivity analyses.