Effectiveness of provider incentives for anaemia reduction in rural China: a cluster randomised trial.

OBJECTIVES: To test the impact of provider performance pay for anaemia reduction in rural China.
DESIGN: A cluster randomised trial of information, subsidies, and incentives for school principals to reduce anaemia among their students. Enumerators and study participants were not informed of study arm assignment.
SETTING: 72 randomly selected rural primary schools across northwest China. PARTICIPANTS: 3553 fourth and fifth grade students aged 9-11 years. All fourth and fifth grade students in sample schools participated in the study.
INTERVENTIONS: Sample schools were randomly assigned to a control group, with no intervention, or one of three treatment arms: (a) an information arm, in which principals received information about anaemia; (b) a subsidy arm, in which principals received information and unconditional subsidies; and (c) an incentive arm, in which principals received information, subsidies, and financial incentives for reducing anaemia among students. Twenty seven schools were assigned to the control arm (1816 students at baseline, 1623 at end point), 15 were assigned to the information arm (659 students at baseline, 596 at end point), 15 to the subsidy arm (726 students at baseline, 667 at end point), and 15 to the incentive arm (743 students at baseline, 667 at end point). MAIN OUTCOME MEASURES: Student haemoglobin concentrations.
RESULTS: Mean student haemoglobin concentration rose by 1.5 g/L (95% CI -1.1 to 4.1) in information schools, 0.8 g/L (-1.8 to 3.3) in subsidy schools, and 2.4 g/L (0 to 4.9) in incentive schools compared with the control group. This increase in haemoglobin corresponded to a reduction in prevalence of anaemia (Hb <115 g/L) of 24% in incentive schools. Interactions with pre-existing incentives for principals to achieve good academic performance led to substantially larger gains in the information and incentive arms: when combined with incentives for good academic performance, associated effects on student haemoglobin concentration were 9.8 g/L (4.1 to 15.5) larger in information schools and 8.6 g/L (2.1 to 15.1) larger in incentive schools.
CONCLUSIONS: Financial incentives for health improvement were modestly effective. Understanding interactions with other motives and pre-existing incentives is critical. TRIAL REGISTRATION NUMBER: ISRCTN76158086.

RCT Entities:   Population Interventions Outcomes

Introduction

Inexpensive, efficacious technologies and services exist for improving human health in developing countries, but implementation and coverage are often low. Clear examples include point-of-use water disinfectants, insecticide treated bed nets, oral rehydration therapy, fortified food staples, condoms, improved cooking stoves, and basic primary healthcare services. Why have efforts to disseminate these technologies and services not produced greater population health gains? Given tremendous efforts by donors and international organisations in recent years, the answer cannot simply be that they are unavailable or unaffordable.

Misalignment between supplier incentives and the ultimate objective of improving health may be an important part of this puzzle. Building on the logic of performance pay in human resource management, a straightforward solution may be rewarding providers directly for producing socially desirable outcomes. Beyond well known applications in wealthy countries (such as the British National Health Service’s quality and outcomes framework), a growing number of aid organisations are also experimenting with pay-for-performance incentives under the umbrella of “results based financing,” rewarding measurable increases in the use of traditionally underused health inputs such as the WHO’s Expanded Programme on Immunisation bundle of childhood vaccinations.1 2 3 4 5 6 7 8

The full promise of pay-for-performance incentives extends far beyond current applications to increase the use of specified technologies and services, however.1 9 In particular, rewarding the ultimate objective—health improvement—without specifying how it should be achieved can strengthen incentives for creativity and innovation in service delivery. Given broad decision making authority, local providers are better able to use their superior knowledge of what is likely to work (and not work) in local settings. To the best of our knowledge, performance pay for health improvement in a developing country has not been tried or evaluated.

This paper presents first evidence on the effectiveness of rewarding providers for better health outcomes. Despite China’s rapid economic development, the prevalence of anaemia among children in rural China ranges from 20% to 60%, implying more than 10 million affected children.10 11 12 In addition to causing debilitating fatigue and retarding growth, childhood anaemia may also impair cognitive development and inhibit human capital accumulation—lowering socioeconomic status throughout the life course.13 14 15 16 The high prevalence of childhood anaemia in China and many other developing countries is remarkable given that it can (in principle) be confronted through simple, low cost nutritional interventions.17 18 We therefore conducted a cluster randomised field experiment in rural Chinese primary schools (a natural contact point with children19) to study the impact of information, subsidies, and financial incentives for school principals to reduce anaemia among their students.

Methods

Setting and participants

Through a canvass survey, we first created a sampling universe of all primary schools in 10 nationally designated poor counties spread across two provinces with high anaemia rates—Ningxia and Qinghai. We then identified all schools having six grades (that is, “complete” primary schools, or wanxiao) and boarding facilities. These criteria were used because China’s government is currently consolidating existing rural schools into new ones with these characteristics. A total of 85 schools met these criteria, and we randomly selected 72 for inclusion in our study. Finally, we randomly selected half of fourth and fifth grade students in study schools (sampling 3944 students in total). Fourth and fifth grade students were chosen because they are old enough for test scores to be relevant but also young enough not to have reached puberty (at which point nutritional requirements differ more markedly from childhood and vary by sex).

Of the 3944 students enrolled in the study, we were able to obtain haemoglobin concentration measurements for 3553 (90.1%) in the follow-up survey. Attrition was almost entirely due to students not being present at the time of the endpoint survey; the refusal rate was less than 1%. This refusal rate is consistent with previous studies conducted by the authors in China.20 The probability of attrition was not statistically different across the study arms.

The study received ethical approval from the Stanford University Institutional Review Board (IRB) on 21 July 2009 (protocol number 17071). All necessary permissions were obtained from the Chinese government as well. All participating children gave their assent to participate in the project, and their legal guardians provided consent. All study participants were aware of the (minimal) risks involved and understood that their participation was purely voluntary.

Study outcomes and data collection

Primary outcomes

Our study includes two primary outcomes. The first is change in altitude adjusted haemoglobin concentration. Using estimates from our previous studies,20 we calculated that we would require 30 schools (15 per trial arm) and 55 students per school for each inter-arm comparison to detect a standardised effect size of 0.4 for haemoglobin concentration with 80% power at the 5% significance level. We assumed an intra-cluster correlation of 0.25, a pre-intervention and post-intervention correlation of 0.5, and 10% loss to follow-up. The actual unadjusted intra-cluster correlation in our sample was lower than expected (0.14 for student level changes in haemoglobin concentration, our primary outcome; 0.23 at baseline survey; and 0.17 at follow-up in the control group). Our second primary outcome variable is anaemia status after the intervention. Anaemia is defined as haemoglobin concentration below 115 g/L at the time of the final survey.

Primary school surveys

We conducted baseline and follow-up surveys of principals and students in all study schools. These surveys collected detailed information about (a) nutritional characteristics of school meals (meat, soybean, fruit, and vegetable composition); (b) use of anaemia related nutritional supplements (such as iron or multivitamin or mineral supplements); and (c) school characteristics (class size, infrastructure, school budgets, and school expenditure records). Because the effectiveness of any incentive scheme depends critically on local context,21 22 we also collected administrative information about explicit financial incentives that local education bureaus offer for good test scores. During our follow-up school surveys, we gathered information from principals about strategies they pursued to reduce anaemia among students as well.

Performance in response to any set of incentives must be verifiable, so, with the assistance of nursing teams from Xi’an Jiaotong Medical School, we also measured student haemoglobin concentration directly using finger prick blood samples (with HemoCue 201+). We randomly retested 10% of all sampled students as well; if the second measure differed from the original one by more than 3.0 g/L among three or more students, we retested all sample students in that school. Short surveys were also administered to students to collect basic information about their ages, socioeconomic characteristics, and eating patterns. Specifically, students were asked about their consumption of a variety of different foods over the past week.23

Household surveys

We conducted baseline and follow-up household surveys for each child in our school based sample. Specifically, we interviewed each child’s household head (usually a parent), collecting information about household socioeconomic characteristics, individual health behaviours, and nutritional characteristics of household meals (consumption of meat, beans, fruits, and vegetables).

Figure 1 depicts the flow of participants through each stage of the study, and table 1 shows student characteristics by trial arm at the time of our baseline survey.

Fig 1 Flow of participants through study

Table 1

 Baseline characteristics of 3553 primary school students in rural China included in trial of information, subsidies, and incentives for reducing anaemia. Values are numbers (percentages (95% CI)) of students in trial arm unless specified otherwise

Characteristic	Trial arm*
	Control (n=1623 students, 27 schools)	Information (n=596 students, 15 schools)	Subsidy (n=667 students, 15 schools)	Incentive (n=667 students, 15 schools)
Mean (95% CI) haemoglobin concentration (altitude adjusted g/L)	126 (123 to 128)	125 (120 to 129)	124 (119 to 129)	124 (119 to 129)
Anaemic (haemoglobin <115 g/L)	338 (20.8 (14.4 to 27.3))	130(21.8 (9.9 to 33.7))	160(24.0 (12.4 to 35.5))	161(24.1 (10.3 to 38.0))
Female	778(47.9 (46.2 to 49.7)	281(47.2 (45.0 to 49.3)	317(47.5 (43.2 to 51.9)	319(47.8 (43.0 to 52.7)
Mean (95% CI) age (months)	123.3 (120.6 to 125.9)	124.7 (121.8 to 127.6)	123.4 (118.4 to 128.3)	120.5 (116.4 to 124.7)
Boarding student	554(34.1 (25.5 to 42.8))	248(41.6 (29.3 to 53.9))	236(35.4 (24.1 to 46.7))	244(36.6 (30.5 to 42.7))
Migrant mother	386(23.8 (15.3 to 32.3))	156(26.2 (16.7 to 35.6))	158(23.7 (8.3 to 39.0))	165(24.7 (14.5 to 35.0))
Mother has primary school education or less	1403(86.4 (80.8 to 92.1))	545(91.4 (86.8 to 96.1))	572(85.8 (77.9 to 93.6))	569(85.3(78.8 to 91.8))

*Control arm (no intervention); information arm (school principals received information about anaemia); subsidy arm (principals received information and unconditional subsidies for reducing anaemia); incentive arm (principals received information, subsidies, and financial incentives for reducing anaemia).

Randomisation and interventions

After the baseline survey, our research team randomly assigned study schools to a control arm or one of three experimental arms (described below). To account for our three treatment arms while maximising power given project resources, we allocated 27 schools to the control group (following Dunnet,24 25 who shows that the required number of control units is approximately N√t, where N is the number of treatment arm units and t is the number of treatments), 15 schools to the “information” arm, 15 to the “subsidy” arm, and 15 to the “incentive” arm. The allocation of schools to treatment arms was done by one of the authors in private using STATA 10. The randomisation procedure was repeated until achieving balance in haemoglobin concentration across the four arms (with 95% confidence). Participants and enumerators were not explicitly informed of study arm allocation, but we cannot rule out that any of them became aware of it.

Schools in all four groups were located an average of 30 km apart, and because there was only one study school per township, there was no communication among study schools through local school districts. It is therefore unlikely that principals or other study participants would have had an opportunity to communicate about the study across study clusters.

The four trial arms were:

Control—Schools in the control group did not receive any intervention. Principals were told only that they would be participating in a study on how better nutrition might affect iron deficiency.

Information arm—We provided three types of information to principals: (a) the share of enrolled students who were anaemic, (b) descriptions of effective methods for reducing iron deficiency anaemia (including vitamin supplementation, adding more meat at lunch, and other dietary changes, as well as the potential importance of educating parents about anaemia), and (c) details about anaemia’s relation with school attendance, educational performance, and cognitive development (as reported in peer reviewed academic studies).14 15 16 19 Principals did not receive financial assistance of any kind.

Subsidy arm (information + subsidy)—Principals were given the same information as in the information arm, but, because purchasing inputs to reduce anaemia might be difficult with current school operating budgets (which are small and allow for little discretion), schools in this arm also received earmarked operating budget subsidies. The subsidy schools were given ¥1.5 (about $0.22, £0.14, €0.18) per student per day, an amount sufficient to buy two to three ounces (55-85 g) of red meat. These subsidies were in principle only to be used for nutrition related expenses (we studied reallocation of school funds as well). Principals did not receive financial incentives of any kind.

Incentive arm (information + subsidy + incentives)—To test the effectiveness of direct rewards for health improvement, school principals in this group received performance payments for anaemia reductions among their student bodies (in addition to the information and subsidies described above). Under the governance structure of Chinese primary schools, school principals make executive decisions about school operations.26 Incremental incentive payments were made as salary supplements to principals on a per student basis (¥150 per student who changed from anaemic to non-anaemic over the course of the intervention). This amount was chosen to equal roughly two months of salary (¥3000) for successfully reducing the total number of students with anaemia by 50%—a feasible reduction according to our early pilot experience.

Statistical analysis

We estimated student level changes in haemoglobin concentration in the intervention arms relative to the control arm using multilevel mixed effect linear models with hierarchical error structures allowing for random effects at the county and school levels. Per our trial protocol, baseline adjusted models controlled for student haemoglobin concentrations before our interventions. Our multivariate adjusted models also controlled for student sex, student age, whether the student boards at school, mother’s migration status, mother’s education level, and whether the principal was eligible for a bonus based on student exam scores. For probability of anaemia at study end, we report average marginal effects from baseline adjusted and multivariate adjusted mixed effect logistic regressions with the same error structure.

Additionally, we investigated how the impact of treatment arms varied by pre-existing principal incentives for good test scores. Roughly 20% of schools in our sample had explicit test score incentives through the cadre evaluation system (ganbu kaohe zhidu).27 These were balanced across experiment arms and often gave principals up to one month of additional salary for mean test scores ranking in their district or county’s top decile. To examine differences in intervention effects between schools with and without pre-existing principal incentives for good test scores, we included interactions between treatment arms and a dummy variable indicating local government use of test score incentives in the multivariate adjusted models described above. We calculated the interaction effect for each intervention by taking the difference between the estimated marginal effect of each intervention among principals with test score incentives and the marginal effect estimated for principals without test score incentives. All analyses were performed using STATA 10.

In the linked data supplement, we also analyse behavioural responses to each intervention arm, testing for differences (and differential changes) in (a) principals’ strategies to reduce anaemia, (b) principals’ efforts to educate parents about good nutrition and anaemia, (c) the composition of students’ diets (at home and at school), and (d) types of school expenditures.

Our trial was registered in January 2011 after the completion of project fieldwork in May 2010. In conducting the trial, we originally intended to write for social scientists, among whom trial registration is a relatively new phenomenon.28 Through the project, we more clearly recognised the role of trial registration (including documenting studies whose findings are not ultimately published and preventing deviation from originally planned analyses). Our paper and the results we report are consistent with our original focus. We conducted one set of analyses that was not included in the original protocol: our analyses of heterogeneous effects by test score incentives. However, these analyses provide insight into the contextual factors and cultural context that mediate the impact of performance incentives. Throughout the rest of the paper we distinguish between planned and unplanned analyses.

Results

Primary outcomes: changes in haemoglobin concentrations and anaemia prevalence

Figure 2 shows the distribution of student haemoglobin concentration (g/L) both at baseline and follow-up in each study arm (in separate panels). Each distribution, including the control group, shifted right between the baseline and follow-up surveys. The increase in haemoglobin concentration in the control group over the six month study period presumably reflects a well documented seasonal effect.29 Using non-parametric Kolmogorov-Smirnov tests that account for our cluster randomised design to compare the distribution of changes (baseline to follow-up) across trial arms, we are able to distinguish each intervention arm from the control group (information arm v control, P=0.012; subsidy arm v control, P<0.001; incentive arm v control, P=0.003), but there are no statistically meaningful differences among intervention arms.

Fig 2 Distribution of primary school students’ haemoglobin concentrations at baseline and follow-up analysis by trial arm. (Control arm = no intervention; information arm = school principals received information about anaemia; subsidy arm = principals received information and unconditional subsidies for reducing anaemia; incentive arm = principals received information, subsidies, and financial incentives for reducing anaemia)

Table 2 presents baseline adjusted and multivariable adjusted estimates for both mean change in haemoglobin concentration and for percentage point change in anaemia status. Among the three intervention arms, only the incentive group was associated with a statistically significant increase in haemoglobin concentration compared with the control group (baseline adjusted estimate 2.1 g/L (95% confidence interval –0.4 to 4.7); multivariate adjusted estimate 2.4 g/L (0 to 4.9)). The accompanying reductions in anaemia at study end associated with the incentive intervention were 4 percentage points (95% confidence interval –0.08 to 0) for baseline adjusted analysis and 5 percentage points (–0.09 to –0.01) for multivariate adjusted analysis. These declines imply reductions of 18% (36% to 0%) and 23% (41% to 5%), respectively, relative to the baseline prevalence of 22%. We find no statistically significant changes in haemoglobin status between the intervention arms (rows 3 to 6 in the body of table 2). The school level intra-class correlations for change in haemoglobin concentration in the baseline adjusted and multivariate adjusted models were 0.086 and 0.082, respectively.

Table 2

 Effects of different combinations of information, subsidies, and incentives for reducing anaemia on haemoglobin (Hb) concentration and anaemia prevalence among primary school students in rural China

Trial arms compared*	Baseline adjusted						Multivariable adjusted
	Change in Hb concentration (g/L)†			Anaemic (Hb <115 g/L)‡			Change in Hb concentration (g/L)§			Anaemic (Hb <115 g/L)¶
	Difference (95% CI)	P value		Marginal effects (95% CI)	P value		Difference (95% CI)	P value		Marginal effects (95% CI)	P value
Information v control	1.5 (–1.2 to 4.1)	0.275		–0.02 (–0.06 to 0.03)	0.430		1.5 (–1.1 to 4.1)	0.245		–0.02 (–0.06 to 0.02)	0.364
Subsidy v control	1.0 (–1.6 to 3.6)	0.471		–0.03 (–0.07 to 0.01)	0.188		0.8 (–1.8 to 3.3)	0.548		–0.03 (–0.07 to 0.02)	0.220
Incentive v control	2.1 (–0.4 to 4.7)	0.099		–0.04 (–0.08 to –0.00)	0.042		2.4 (–0.0 to 4.9)	0.054		–0.05 (–0.09 to –0.01)	0.019
Subsidy v information	–0.5 (–3.6 to 2.5)	0.74		–0.01 (–0.06 to 0.04)	0.67		–0.8 (–3.7 to 2.2)	0.62		–0.01 (–0.05 to 0.04)	0.80
Incentive v information	0.7 (–2.3 to 3.6)	0.66		–0.02 (–0.07 to 0.02)	0.33		0.9 (–2.0 to 3.8)	0.53		–0.03 (–0.07 to 0.02)	0.25
Incentive v subsidy	1.2 (–1.8 to 4.1)	0.44		–0.01 (0.67 to 0.59)	0.59		1.7 (–1.2 to 4.6)	0.26		–0.02(–0.06 to 0.02)	0.37
Adjusted school level intra-class correlations	0.08			0.09			0.08			0.09

*Control arm (no intervention); information arm (school principals received information about anaemia); subsidy arm (principals received information and unconditional subsidies for reducing anaemia); incentive arm (principals received information, subsidies, and financial incentives for reducing anaemia).

†Estimated with multilevel mixed effect linear regression adjusting for baseline Hb status and clustering at the school and county level.

‡Estimated with multilevel mixed effect logistic regression adjusting for baseline Hb status and clustering at the school and county level.

§Estimated with multilevel mixed effect linear regression adjusting for baseline Hb status, whether the principal was eligible for a bonus based on student exam scores, student sex, student age, whether the student boards at school, mother’s migration status, and mother’s education level, and clustering at the school and county level.

¶Estimated with multilevel mixed effect logistic regression adjusting for baseline Hb status, whether the principal was eligible for a bonus based on student exam scores, student sex, student age, whether the student boards at school, mother’s migration status, and mother’s education level and clustering at the school and county level.

Subgroup analysis: test score incentives

Although this was not originally planned, we next analysed how the impact of each intervention varied with pre-existing incentives for principals to achieve good academic performance among students. Table 3 shows marginal effects separately for schools with and without principal incentives for good academic results, and the last two columns report differences associated with these incentives. In the presence of the incentives, the information and incentive interventions were substantially more effective, raising mean haemoglobin concentration by 9.8 g/L (4.1 to 15.5) and 8.6 g/L (2.1 to 15.1) more than schools without them. Corresponding reductions in the probability of anaemia at study end were 11 percentage points (–0.19 to –0.03) in the information arm and 6 percentage points (–0.12 to 0.01) in the incentive arm (declines of 50% and 27% relative to the baseline prevalence of 22%). The effect of the subsidy arm did not vary significantly with academic performance incentives.

Table 3

 Interaction effects of allocation to trial interventions (information, subsidies, and incentives for reducing anaemia) with pre-existing incentives for school principals to achieve good academic performance among students on haemoglobin (Hb) concentration and anaemia prevalence

Trial arms compared*	Without incentive for good academic results (n=2871 students, 58 schools)†						With incentive for good academic results (n=682 students, 14 schools)‡						Interaction effect§
	Change in Hb concentration (g/L)¶			Anaemic at follow-up (Hb <115 g/L)**			Change in Hb concentration (g/L)			Anaemic at follow-up (Hb <115 g/L)			Change in Hb concentration (g/L)			Anaemic at follow-up (Hb <115 g/L)
	Marginal effect (95% CI)	P value		Marginal effect (95% CI)	P value		Marginal effect (95% CI)	P value		Marginal effect (95% CI)	P value		Difference (95% CI)	P value		Difference (95% CI)	P value
Information v control	–0.6 (–3.2 to 2.0)	0.66		0.02 (–0.03 to 0.07)	0.46		9.2 (4.0 to 14.4)	<0.01		–0.09 (–0.14 to –0.03)	<0.01		9.8 (4.1 to 15.5)	<0.01		–0.11 (–0.19 to –0.03)	0.01
Subsidy v control	–0.2 (–2.8 to 2.5)	0.91		–0.01 (–0.06 to 0.04)	0.65		2.6 (–2.5 to 7.7)	0.32		–0.04 (–0.10 to 0.02)	0.24		2.7 (–3.1 to 8.6)	0.35		–0.03 (–0.10 to 0.05)	0.51
Incentive v control	1.1 (–1.4 to 3.6)	0.38		–0.03 (–0.07 to 0.01)	0.12		9.7 (3.7 to 15.6)	<0.01		–0.09 (–0.15 to –0.03)	<0.01		8.6 (2.1 to 15.1)	0.01		–0.06 (–0.12 to 0.01)	0.07
Subsidy v information	0.4 (–2.6 to 3.4)	0.78		–0.03 (–0.09 to 0.03)	0.30		–6.6 (–12.6 to –0.6)	0.03		0.05 (–0.01 to 0.11)	0.09		–7.0 (–13.7 to –0.4)	0.04		0.08 (–0.00 to 0.17)	0.06
Incentive v information	1.7 (–1.2 to 4.5)	0.25		–0.05 (–0.11 to 0.00)	0.07		0.5 (–6.3 to 7.3)	0.89		–0.00 (–0.03 to 0.03)	0.94		–1.2 (–8.6 to 6.1)	0.75		0.05 (–0.01 to 0.11)	0.11
Incentive v subsidy	1.3 (–1.6 to 4.1)	0.39		–0.02 (–0.07 to 0.03)	0.42		7.1 (0.3 to 13.9)	0.04		–0.05 (–0.12 to 0.01)	0.09		5.8 (–1.5 to 13.2)	0.12		–0.03 (0.04 to 0.04)	0.40

*Control arm (no intervention); information arm (school principals received information about anaemia); subsidy arm (principals received information and unconditional subsidies for reducing anaemia); incentive arm (principals received information, subsidies, and financial incentives for reducing anaemia).

†Marginal treatment effects evaluated for schools where principal did not have bonus incentives based on student academic performance.

‡Marginal treatment effects evaluated for schools where principal had bonus incentives based on student academic performance.

§Interaction effect is the difference in marginal treatments effects between schools in which principals had bonus incentives for academic performance and those in which they did not.

¶Results for change in Hb concentration between baseline and follow-up are estimated with multilevel mixed effect linear regression adjusting for baseline Hb status, whether the principal is eligible for a bonus based on academic performance, student sex, student age, whether the student boards at school, mother’s migration status, and mother’s education level and clustering at the school and county level.

**Results for anaemia status at follow-up are estimated with multilevel mixed effect logistic regression adjusting for baseline Hb status, whether the principal is eligible for a bonus based on academic performance, student sex, student age, whether the student boards at school, mother’s migration status, and mother’s education level and clustering at the school and county level.

Supplemental analysis: behavioural responses to interventions

The appendix and its figure in the data supplement present analyses of how the school principals and parents responded to the study’s interventions—both to reduce anaemia and in potentially unintended ways. We summarise its main findings here.

Principals’ strategies to reduce anaemia

The appendix presents three stylised facts about principals’ major strategies for reducing anaemia.

(1) Compared with those in the subsidy group, school principals in the incentive group were more likely to use their subsidies to pursue iron supplementation strategies that increased only students’ multimicronutrient intake and not their energy intake (for example, with vitamins or minerals and iron fortified wheat) and were less likely to pursue broad feeding strategies that increased both multimicronutrient intake and energy intake (for example, with meat and other food added to school lunches). This response is consistent with these principals’ narrow incentive to raise iron levels.

(2) Consistent with research linking broader nutritional gains to school performance,30 principals with incentives to achieve good academic performance focused relatively more on feeding.

(3) Compared with those in the control group, principals in all the intervention groups provided nutritional information to parents to reduce anaemia. However, principals in the information arm with academic performance incentives seemed to have done so most intensively. This may be because these principals understood the link between anaemia and school performance (an explicit part of the information they received) and also had incentives to improve academic performance but lacked the resources to intervene through schools.

Unintended behavioural responses by parents and principals

We also investigated the possibility of unintended (perverse) behavioural responses to the study’s interventions. First, we examined if there were offsetting reductions in dietary quality at home in response to improvements in school meals.31 32 Second, we estimated if school spending on functions other than nutrition (on administration or teaching, for example) was diverted to reduce anaemia. We found little evidence of either.

Discussion

This paper reports first evidence on the effectiveness of direct provider incentives for better health outcomes. On balance, we found that these incentives were moderately effective in reducing anaemia rates among school children in rural China. Importantly, we also found that analyses failing to recognise their interactions with other incentives could miss important population health gains associated with them. Primary school principals with incentives for good academic performance seem to have made more effective use of subsidies to reduce anaemia—and performance pay for anaemia reduction may have nearly doubled their impact. Moreover, although the provision of information about anaemia to school principals was ineffective on average, when combined with principal incentives for good academic performance, information was associated with significant health gains on a par with the full bundle of information, subsidies, and anaemia reduction incentives. School principals in the information arm with incentives for good academic performance had strong incentives to reduce anaemia, but, because they lacked resources to do so through schools, they pursued innovative strategies to improve students’ diets at home that ultimately proved effective. Considerably more research is needed, but our findings broadly suggest that policymakers may be able to exploit synergies between financial incentives and other motives or pre-existing incentives (often embedded in local institutions and cultural context) to produce substantial population health gains.

Several limitations are important to note. First, as in other studies,33 34 there is presumably error in our haemoglobin concentration measurement despite our explicit protocol to minimise it. However, this error should be uncorrelated with assignment to experimental arms. Second, we selected study schools according to criteria set by China’s government in its policy for consolidating rural schools. However, our results may not generalise to other Chinese schools or to other cultural and institutional settings—particularly where performance incentives are less common. Third, our specific randomisation procedure was re-randomisation until achieving balance in haemoglobin concentration across the four study arms (with 95% confidence). This approach is valid, but other procedures offer additional benefits.35 Fourth, our study was not powered ex ante to detect meaningful differences between intervention arms. Finally, because incentives for academic performance were not randomly assigned, interactions with trial arms should be interpreted cautiously.

Further research is needed to understand several important dimensions of performance pay for health improvement. These include (a) understanding how to structure performance pay in multitasking environments—in particular, for providers with broader clinical responsibility36; (b) identifying optimal combinations of fixed (unconditional) and variable (performance based) compensation (research on executive compensation suggests that smaller unconditional components may be appropriate—but this varies with the degree to which agents can influence the rewarded outcome)37 38 39; (c) examining the relative merits of performance pay at different staff levels within organisations; and (d) documenting the comparative effectiveness and cost effectiveness of financial and non-financial incentives (given that financial incentives can be costly).

Overall, our study suggests that performance incentives—and direct rewards for health improvement in particular—may hold promise for promoting a broad class of seemingly under-used health technologies more effectively. However, it also offers a cautionary tale about introducing incentives naively without understanding the local institutions within which they are applied. Failing to do so may hamper their effectiveness, while harmonising them with other motives and incentives can amplify their impact.

Inexpensive, highly effective technologies and services exist for improving human health in developing countries, but implementation and coverage are often low

Rewarding the ultimate objective—health improvement—without specifying how it should be achieved can strengthen incentives for creativity and innovation in service delivery

However, performance pay based directly on health outcomes in a developing country has not been tried or evaluated

Direct provider incentives for better health outcomes were moderately effective in reducing anaemia among primary school children in rural China

Importantly, analyses failing to recognise interactions with pre-existing motives and incentives may miss important sub-population health gains