Impact of the extension of a performance-based financing scheme to nutrition services in Burundi on malnutrition prevention and management among children below five: A cluster-randomized control trial.

Malnutrition is a huge problem in Burundi. In order to improve the health system response, the Ministry of Health piloted the introduction of malnutrition prevention and care indicators within its performance-based financing (PBF) scheme. Paying for units of services and for qualitative indicators is expected to enhance provision and quality of these nutrition services. The objective of this study is to assess the impacts of this intervention, on both child acute malnutrition recovery rates at health centre level and prevalence of chronic and acute malnutrition among children at community level. This study follows a cluster-randomized controlled evaluation design: 90 health centres (HC) were randomly selected for the study, 45 of them were randomly assigned to the intervention and received payment related to their performance in malnutrition activities, while the other 45 constituted the control group and got a simple budget allocation. Data were collected from baseline and follow-up surveys of the 90 health centres and 6,480 households with children aged 6 to 23 months. From the respectively 1,067 and 1,402 moderate and severe acute malnutrition transcribed files and registers, findings suggest that the intervention had a positive impact on moderate acute malnutrition recovery rates (OR: 5.59, p = 0.039 -at the endline, 78% in the control group and 97% in the intervention group) but not on uncomplicated severe acute malnutrition recovery rate (OR: 1.16, p = 0.751 -at the endline, 93% in the control group and 92% in the intervention group). The intervention also had a significant increasing impact on the number of children treated for acute malnutrition. Analyses from the anthropometric data collected among 12,679 children aged 6-23 months suggest improvements at health centre level did not translate into better results at community level: prevalence of both acute and chronic malnutrition remained high, precisely at the endline, acute and chronic malnutrition prevalence were resp. 8.80% and 49.90% in the control group and 8.70% and 52.0% in the intervention group, the differences being non-significant. PBF can contribute to a better management of malnutrition at HC level; yet, to address the huge problem of child malnutrition in Burundi, additional strategies are urgently required.

RCT Entities:   Population Interventions Outcomes

Introduction

The latest State of Food Security and Nutrition in the World report signals a rise in world hunger and food insecurity and a reversal of trends after a prolonged decline [1]. Causes of this increase are multiple, with climate variability and extremes identified as among the key drivers. Urgent and considerable action is needed if we want to achieve the SDG goals on food security and nutrition (by 2030, end all forms of malnutrition). Malnutrition rates in many low-income countries (LICs) remain terrible. Stunting is declining too slowly while wasting still impacts the lives of far too many young children. Poor nutrition in the first 1,000 days of a child’s life can also lead to stunted growth, which is associated with impaired cognitive ability and later on reduced school and work performance [2].

The state of nutrition in Burundi is particularly alarming. Burundi is a small landlocked country in Eastern Central Africa, with a very high and increasing population density (more than 420 people per sq. km of land area in 2017 [3]). This country is in a fragile situation, being among the poorest in the world in terms of income per capita, and regularly affected by political crises. In 2016/2017, 55.9% of children under five years old suffered from chronic malnutrition (which places the country in the world top 3) and 5.1% from acute malnutrition [4]; these figures were respectively 57.7% and 5.8% in 2010, showing almost no improvement [5].

Malnutrition is multifactorial: poverty and food security are central elements in low-income countries, while water, sanitation and hygiene are found to be important factors of chronic malnutrition among children [6]. There is also a consensus that the health system has a role to play [7], especially at the health centre level, where basic perinatal care, paediatric prevention and care services are provided: generic health services can prevent and manage child health issues that could lead otherwise to malnutrition, while specific to malnutrition interventions aim to prevent, identify and manage malnutrition. In Burundi, a national program is dedicated to the fight against malnutrition [8, 9]. It has adopted and rolled out different strategies promoted by international agencies, including the systematic screening and outpatient treatment for moderate and non-complicated severe acute malnutrition (MAM and SAM) children at health centre (HC) level. Community-based management of acute malnutrition (CMAM), although highly relevant [7], was not sufficiently invested in Burundi at the time of the study. CMAM includes 1) community outreach as the basis for early identification and referral of malnourished children, follow up through home visits, sensitisation and mobilisation of the community; 2) management of MAM; 3) outpatient treatment for children with uncomplicated SAM; and, 4) inpatient treatment for children with complicated SAM. In Burundi, active screening was to be performed by CHWs, and systematic screening at health centres through pre-school consultations and curative consultations of children under five; outpatient treatment services for MAM and uncomplicated SAM cases are integrated in the primary health care services, in about one third of health centres. Complicated SAM cases are managed in the hospital, before being transferred back to outpatient treatment and follow-up at the health centre. As for sensitisation at community level, this was mainly performed by NGOs, in only a few provinces.

Over the last decade, Burundi has taken several bold measures to boost the performance of its health system. One of them has been a nationwide performance-based financing (PBF) policy. Since 2010, a part of health facilities revenue is linked to their performance (measured as a combination of volume and quality of health services). There is evidence that this strategy had positive effects on child and maternal health services in Burundi; studies found that PBF was associated with an increase in the health workforce as well as in health service utilization (incl. vaccination, assisted deliveries, antenatal care) and quality of care [10-13].

In 2012, the Ministry of Health and the World Bank decided to pilot the extension of the PBF system to nutrition services, as a measure to reinforce the recently developed malnutrition prevention and management program [9].

As a strategy, PBF has expanded rapidly in low- and middle-income countries in the last decade. This has been accompanied by a large research program to assess the effectiveness of the strategy across settings and health conditions. Only three studies examined the impact of PBF on any nutrition outcome, with mixed results. Following the Cochrane review’s general recommendations for solid evaluations of PBF schemes [14, 15], it was decided that the extension of the PBF scheme to nutrition (the first ever to our knowledge) would be subject to an impact evaluation.

This paper reports findings from this evaluation, with a focus on the two main research questions: (1) did the extension of PBF to nutrition (hereafter, referred as PBF-N) contributed to improve nutritional health services for children below five years old at HC level and; (2) did it reduce the prevalence of malnutrition among children below two years old at community level?

Methods

Study design

This interventional study followed a cluster randomised controlled trial (cRCT) design. The intervention design and implementation were under the responsibility of the Ministry of Health. Data collection was carried out by two national survey institutes (the Institut National de Santé Publique (INSP) for the facility surveys and the Institut de statistiques et d’Etudes Economiques du Burundi (ISTEEBU) for the household surveys). The general coordination of the research was under the responsibility of the Institute of Tropical Medicine of Antwerp, with logistical assistance from Blue Square Bujumbura. The intervention covered all provinces of Burundi, but the urban province of Bujumbura, the capital city. Among the 590 health centres in the country, 193 health centres provided nutrition services in 2014, and among them, 90 were randomly selected for the study. In the intervention group (concerning the 45 randomly allocated HCs), hospitals, HCs and the related community health workers (CHWs) were subject to PBF-N, for two years. In the control group, the 45 HCs received, monthly, a simple budget allocation while hospitals were also subject to PBF-N.

The study protocol was reviewed and approved by ethics committees of the Institute of Tropical Medicine of Antwerp, the University of Antwerp, and the National Ethics Committee of Burundi (see S3–S6 Files). This study is registered with ClinicalTrials.gov, number NCT027211160 [16], and the research protocol has been published elsewhere [17]. Informed written consents were obtained for all participants of the study, that is: health workers, household heads, and mothers (or child caregivers) for themselves and their children.

The CONSORT flow diagram (Fig 1 below) displays the progress of enrolment and intervention allocation of health centres, as well as of follow-up and data analysis along the study. The CONSORT checklist can be found in S1 File.

Fig 1

CONSORT flow diagram.

HC means health centre, S.D. means standard deviation; *Information about the existence of both MAM and SAM services was initially obtained from the Ministry of Health; once the randomization was performed, while gathering more information from each HC, we realised than nine of them did not provide either the MAM (one) or the SAM (eight) services. Providing both services was an inclusion criterion, so these nine HC were removed from the list. Source: authors.

Participants

HCs were eligible to participate in the study if, at the time of recruitment, i.e. Summer 2014, they provided both moderate and severe acute malnutrition management services, they were already subject to PBF, they were in a rural province, and they were separated from another participating health centre by at least 5 km. Surveyed households, health workers and patients were recruited on the field, at the time of the survey (see the procedures below).

Randomisation and masking

In order to study rather similar groups of HCs, the 90 participating HCs were paired on various parameters before being allocated to either the intervention or the control groups. Based on data collected during the baseline survey, we first assigned HCs to four different groups, according to the continuation (or not) of MAM services despite the treatment supply discontinuation, and to their volume of activity (above or below the median). Then in each group, we paired HCs according to their performance in acute malnutrition services, using the acute malnutrition recovery rates. For each pair, the random allocation to the intervention or the control group took place in a public session with HC managers one month before the start of the intervention.

The sample size was computed on the smallest difference in the main outcome that could be considered of public health significance, i.e. a reduction of ≅ 25% in acute malnutrition prevalence (2.5% points in absolute terms) in the communities around the intervention HCs as compared to control HCs. Assuming that the intervention would result in decreasing the prevalence of acute malnutrition in children aged 6–23 months from 10% to 7.5% (DHS survey 2010), and assuming that 65 children aged 6–23 months would be surveyed in the catchment area of each health centre, 90 health centres needed to be randomized to either the intervention or control group, for an α-error of 5% and a β-error of 20%, and taking a conservative intraclass correlation (ICC) of 0.25. This was calculated with the formula of [18]. The number of children per HC was increased to 72 to allow for missing or incomplete data, amounting to a total of 6,480 children aged 6–23 months over the 90 selected HCs, at each survey round (baseline in 2014 and endline in 2016). Now, according to the baseline data, the prevalence of acute malnutrition in children aged 6–23 months is 6%, instead of 10%, with an ICC of 0.007. Considering this, the explanatory power for a reduction of 25% (from 6% to 4.5%) was reduced from 80% to 60%, while the explanatory power for a reduction from 6% to 4% remained at 80%.

Intervention

Interventions intended for each group

In the intervention group, the PBF-N intervention consisted in paying for improved performance at three levels—hospital, HC and community health workers (CHWs)–for malnutrition screening, growth promotion as well as acute malnutrition management interventions, all focusing on children below five years old (cf. Table 1 and Fig 2).

Table 1

Measures used for the PBF-N payments.

Level	Payments are given for:
Hospital	(1) success in managing and rehabilitating severe acute malnutrition cases with medical complications and(2) length of stay (number of days) at the hospital due to malnutrition.
Health Centre	(1) screening children for malnutrition and caring (or referring if services not available) for rehabilitating acute malnutrition cases, and(2) growth monitoring and promotion activities.
Community	(1) screening and referring acute malnutrition cases to health centres,(2) organising classes promoting good food and nutrition behaviours, and(3) organising cooking classes.

Source: Ministry of Health technical note [20].

Fig 2

PBF-N system.

Source: authors. Note: ‘comp.’ refers to the ‘compensation intervention’.

A quality of care checklist, with 19 indicators specific to nutrition, was added to the existing one, and was considered for the calculation of payment to facilities (this checklist can be found in the supporting information, S2 File). The whole reliability of the payment system was ensured through a verification system, itself subject to a regular systemic audit [19]. HC and hospital managers as well as controllers received a training regarding the PBF-N, during a workshop that took place in Bujumbura in December 2014, one month before the start of the intervention (January 2015). The intervention at community level was delayed and CHWs received a training in September 2015, just before the intervention officially started in October 2015. At all levels, the intervention lasted until December 2016, i.e. two years at HC and hospital levels and fifteen months at CHW level.

PBF-N subsidies were transferred to the bank accounts of hospitals, HCs and CHW associations. Revenues from the PBF-N were fungible with other revenues raised by the hospitals and HC (the national PBF scheme, user fees, etc.); their use is under the responsibility of the facility staff who based their allocation decisions on a management tool prioritising purchase of key inputs (e.g. medicines); surplus (if any) is distributed to staff as bonuses, again using a specific management tool. As for CHW associations, one part of the funds could be invested into revenue-generating activities and the rest was shared among CHWs.

In the control group, the hospitals were also subject to the PBF-N, i.e. they received subsidies according to their performance with complicated SAM management (Table 1). HCs were subject to a ‘compensation intervention’: they received a financial amount equivalent to the PBF subsidies received in the intervention group (weighted average, considering volume of activities and staff). The purpose of this payment was to assure that any future significant difference between the intervention and control groups could be attributed to the payment-for-result rule and not to the extra money injected in the HCs. The CHWs were not subject to any intervention, i.e. they received nothing more than usual.

The initial theory of change of the intervention is described in details in the research protocol [17]. To sum it up, the introduction of nutrition activities into the PBF program translates policy makers’ belief that PBF can trigger some positive changes in the performance of the health personnel, facilities or system which will eventually impact on households and children. To reap the extra income from the PBF-N, service providers (CHWs, HCs and hospitals) are expected to take proactive steps to improve the delivery of nutrition services. In addition, through the PBF-N contract, staff has a clearer view on what performance should be, as far as nutrition services are concerned. In addition, verifications, essential to enforce any PBF intervention, may result in more systematic supervisions and qualitative feedbacks, and guide the health workers decisions towards ‘performance’. A chain of expected results can be found in S1 Fig. In another paper, recently published, we report on more extensively our findings as for the theory of change [21]. Qualitative data suggest that the most influencing mechanisms related to PBF-N were incentives as well as information provided in the contract and during the initial training (stating clearly the service performance indicators).

Procedures

Cross-sectional household surveys

Cross-sectional household surveys were performed in all clusters at baseline, in December 2014 and January 2015, and at end-line, in March and April 2017 (i.e. about two years after the PBF-N intervention had started). Surveyed households were recruited at the time of the survey. Eligible households were those living in the area of the study health centres and having a member aged 6–23 months at the time of the survey. As described in the randomization subsection above, the number of children per cluster to be surveyed at community level was set to 72. Around each HC, twelve sub-hills were randomly selected among the sub-hills covering the HC catchment area, and in each sub-hill, six eligible households were selected as follows: a random direction from the centre of the sub-hill was selected and following it led to the first contact with a household, which in turn indicated the next households with children aged 6–23 months. Demographic details about households and children characteristics at baseline, in 2014, are displayed in S1 Table. Chronic malnutrition indicators computed for our sample of children were similar to those found in the nationally representative DHS 2010 survey for same age children, which make us confident that our sample can be considered representative of all households with a toddler living in Burundi at that time.

The nutritional status of the child was assessed through anthropometric measurements; health care behaviour and diet of the child were assessed through a questionnaire administered to the mother (or caregiver). The questions in these sections were a combination of those adapted from the WHO guidance on assessing infant and young child feeding practices [22] as well as those designed by the Child Health and Nutrition Unit of the ITM. Socio-economic characteristics and food security of the household were assessed through a questionnaire administered to the head of the household. The questions used for the assessment of household food security were based on the 2007 Household Food Insecurity Access Scale (HFIAS) Generic Questions, created by the Food and Nutrition Technical Assistance (FANTA) project [23].

Before each survey, fieldworkers received a one-week training. For each survey, 72 surveyors were deployed across clusters in teams of 12 surveyors (working in duos). Each team was managed by a supervisor. Interviews were performed using Android smartphones with Open Data Kit® software. Data were transferred every evening to the ONA internet data management platform and checked regularly by two data managers for consistency and completeness. The electronic data entry has the advantage of reducing risks of errors in recording the answers (thanks to automatic validity checks) and eliminating the need for double data entry from the paper to software transcription and to decrease considerably the time for transcription. Lot quality assessment surveys were also performed by the field coordinator to assess accuracy of anthropometric data in the records.

Health centre surveys

The 90 participating HCs were surveyed at baseline, in September-October 2014 and at endline in March-April 2017. Various survey tools were administered at the HC level, in 2014 and 2017. Surveyed health workers and patients were recruited at the time of the surveys, as described below.

To get information on malnutrition recovery rates, in each study HC, a total of 24 individual clinical files were to be transcribed, totalling 1,080 MAM and 1,080 SAM clinical files for each survey wave. This sample size was computed on the smallest difference in the main outcome that can be considered of public health significance in intervention health centres, and assuming that the intervention would result in increasing the recovery rate of both MAM and SAM in children from 80% to 90%, for an α-error of 5%, a power of 80% and an ICC of 0.15 [18]. These 24 clinical files were randomly selected among all the closed files of all children enrolled acute malnutrition services (12 files for the MAM care program and 12 files for the SAM care program) during a six-month period preceding the survey where both services were procured with malnutrition treatments (March-September 2013 for baseline and April-October 2016 for endline). In addition, organizational aspects of the HCs as well as of the nutrition services were collected through interviews of managers.

To assess the quality of services, we combined two techniques: patient-provider observation carried out on six paediatric consultations (performed by a maximum of two HWs) and exit interviews at the end of each of these observed consultations, in order to get information on the satisfaction level as well as to record anthropometrics of the children (540 expected observations per survey). For the latter, two health workers were randomly selected among those performing paediatric curative consultations the day of the survey; then six children aged 6–59 months, coming for a consultation the day of the survey, were randomly selected, among the eligible patients in the waiting room, for being observed in consultation and surveyed afterwards. Finally, to assess knowledge of the observed HWs, we used vignettes to measure the practical knowledge on different tasks to perform: a pattern of consultation was proposed and the health worker could ask all the questions (related to history and physical exams) necessary to arrive at a diagnosis and propose a treatment. Three vignettes were administered to every HW observed in consultation.

The 90 HC selected for the study cover all areas of Burundi and all provinces but Bujumbura city have at least one HC in the sample. Although this sample is not representative of the whole health sector in Burundi, since neither hospitals nor community health workers were surveyed, we are confident that this sample is representative of all health centres of the rural part of Burundi. Health centre characteristics and nutrition services performance indicators were rather similar to other nationally representative data.

PBF Routine heath centre data

In addition, routine data from the PBF-N program was obtained from the PBF technical unit of the Ministry of Health for the intervention HCs over 2015 and 2016. It provided regular information about the 45 HCs, as for their quantitative and qualitative ‘nutrition performance’ and the related payments.

Outcomes

The primary outcomes to test were the prevalence of chronic and acute malnutrition measured at the community level, as well as the MAM and SAM recovery rates among children below five years old at the HC level. At the community level, for statistical power issues, the focus was made on the age group with the highest prevalence, where we could expect the largest impact, that is, children aged 6–23 months.

Chronic malnutrition identification is based on length-for-age and height-for-age z-scores (LAZ and HAZ) respectively for children below and above two years old. Length-for-age and height-for-age deficits indicate past or chronic inadequacies of nutrition and/or chronic or frequent illness [24]. A child below two years old is stunted when their LAZ is lower than minus two standard deviations (SD) and severely stunted when it’s lower than minus three SDs. Acute malnutrition identification, for children below two, is based on weight-for-length z-scores (WLZ), mid-upper arm circumference (MUAC) and the presence of oedema. Severe acute malnutrition is diagnosed either when the WLZ is below minus three SDs or when the MUAC is below 115mm or when oedema is present. MAM is diagnosed when there is no oedema and either the WLZ is between -3SD and -2SD or MUAC is between 115mm and 125mm.

MAM and SAM recovery rates among children below five years old and managed at the HC are based on the transcribed clinical files collected from both MAM and SAM management services. HWs report various elements on these files, including the reason for the end of treatment, which may be recovery, transfer, desertion, non-responding, death, or any other reason.

As for secondary outcomes, at community level we also analysed WLZ, LAZ and MUAC, and at HC level, treatment duration among the MAM and SAM cases followed up and recovered at the HC.

As for intermediary outcomes, we focused on prevention and screening of malnutrition. At community level, we checked the extent to which acute malnutrition cases diagnosed among all surveyed children were indeed followed-up at the HC. At HC level, we looked at the quality of diagnosis during paediatric curative consultations, by (1) administrating fictive cases with three vignettes with three different cases of acute malnutrition (MAM, non-complicated and complicated SAM), and (2) by observing real consultations (combined with anthropometric measurements taken during an exit interview): was acute malnutrition diagnosed during consultation, and, if so was it a false positive (i.e. positively diagnosed by the health worker among those without malnutrition detected from anthropometric measurements of surveyors), or otherwise, was it a false negative (i.e. negatively diagnosed by the health workers among those suffering from acute malnutrition according to anthropometric measurements taken by surveyors)? Using clinical files and register transcripts, we analysed the number of MAM and SAM cases followed-up at the HC per semester. There were serious issues with MAM treatment supply (which was based on corn-soy blend mixed with sugar and oil) during the project and many HCs did not offer a MAM management service anymore; therefore, we also checked whether the PBF-N intervention had an impact on offering this service at all. As for growth promotion, we checked to what extent the growth curve was mentioned during consultations, and whether growth monitoring sessions were reported as regularly organised by the nutrition service managers in the HCs.

Statistical analysis

Difference-in-difference estimates were performed in order to assess the impact of PBF-N on primary and secondary outcomes as well as on intermediary outcomes. The following model was used: Where i indicates the individual (when appropriate), h the health centre, and t the survey wave (before or after the intervention); Y refers to the dependent variable (whether primary or secondary or intermediary outcome); f(.) is the indicator function. The variable PBFnut is an interaction term multiplying group with period: it equals 1 only in the intervention HCs and after the intervention, and 0 otherwise; the beta coefficient provides an estimate of the impact of the PBF-N on the dependent variable. We controlled through dummy variables for time effects (t) that were common across all HCs, and group specific effects (intervention or control) that were common over time.

For binary variables, logit estimates were performed, and for the impact estimates, we reported odds ratios (OR) as well as marginal impacts when appropriate (i.e. the absolute change in a rate expressed in percentage points due to the intervention). For continuous variables, the Ordinary Least Square (OLS) estimator was used, except for the number of MAM and SAM cases managed at HC during the last semester where the Poisson estimator was used. For all estimates we used the sandwich estimator to take into account the clustering at HC level.

In displayed results, no control variable was added; however, as for robustness checks, equations using individual and HC (fixed or time varying) characteristics were included to gain precision (X’), see model below: At HC level, control variables include dummies indicating (1) whether staff of the health centre was in line with national norms, (2) whether anthropometric measurements equipment was complete, (3) whether supervision was recently performed at the HC, (4) whether MAM treatment was available, (5) whether the MAM service was closed. For outcomes retrieved from the observed consultations or vignettes, HW related variables were used, such as their age, gender, salary level, whether salary was received, whether acute malnutrition management was in their terms of reference, whether they were supervised in the last six months. For outcomes retrieved from household data, we added a wealth quintile variable, age, gender, and education of household head, age and education of mother, age of the child, a food security index, a dietary index of the child related to the frequency and food groups, and the distance to HC.

Conditional effects were tested, using an interactive variable combining the PBFnut variable alternatively with control variables such as the time distance to the HC (less than 30 minutes versus more), whether health workers in the HC had been recently trained on nutrition, the quality of malnutrition screening in the HC (as found out from vignettes and from clinical files), wealth quintile of children, and the existence in the HC of a health worker dedicated to health promotion and in charge of supervising CHWs.

Findings

In total, respectively 6,199 and 6,480 children aged 6–23 months were surveyed during the baseline and the endline household surveys. With the health facility baseline and endline surveys, we got respectively 971 and 674 MAM clinical files transcribed, and 963 and 1,043 SAM clinical files transcribed (out of 1,080 expected each time). The low rate of transcription for MAM clinical files within the endline survey is attributed to the control group, where only 29% of the expected files were found and transcribed, compared to 96% in the intervention group. This different compliance for the medical record can be interpreted as a first result of the PBF-N; it indeed stems from a different response in front of the discontinuation in the MAM treatment supply. Because of the PBF-N reward, HCs in the intervention group had still an incentive both to carry out MAM and to complete the clinical files (as data from the medical files was used for the payment calculation); conversely, HCs in the control group had no incentive neither to look for strategies to circumvent the shortages nor to complete the patient files; in several control HCs, MAM services were even stopped for a while. We also collected information at the baseline and endline on respectively 515 and 529 observed paediatric curative consultations followed by exit interviews; as well as on the HC general and nutrition specific situation in the 90 HCs.

Baseline data found no significant difference between the two groups (intervention and control) as for the variables of primary interest for this research, whether at the HC or household levels. Among the 386 variables tested overall, we observed a significant difference between the intervention and control groups (p<0.05) for 26 (6.7%) of them (see details in the baseline report, S7 File). The findings showed that prevention and management of malnutrition were very problematic, technical capacities limited [25] and global malnutrition prevalence very high [26].

The PBF-N intervention started in January 2015, at the hospital and HC levels, and was delayed to October 2015 at the community level. The intervention suffered from implementation problems such as delays in payments to providers. A more important issue concerned the MAM treatment supply, which was discontinued at the time of the implementation; a local solution was initiated one year after the intervention started but suffered from communication and quality of product issues. Finally, political followed by economic instability affected the project. Nevertheless, the intervention occurred and HCs in the intervention group were paid accordingly to their performance in nutrition activities. More details on the implementation and on the context can be found in the endline report and in other papers [21, 25, 27].

Findings at the health centre level

As explained above, we did not gather the 1,080 clinical files initially planned. Furthermore, in some patient files, the outcome as for the end of the treatment (recovered or not) was missing. At baseline, the MAM and SAM outcomes were reported in respectively 628 and 665 clinical files. This affected the explanatory power of our tests. At baseline, the ICC for MAM recovery outcome was assessed to be up to 0.41 (based on observations from 26 HC for which we had the outcome information for 12 clinical files), which reduced the explanatory power to 60% for a recovery rate of increase from 80% to 95%. The ICC for SAM was estimated at 0.06, and the explanatory power remained at 80% with an α-error of 5% assuming that the intervention would result in increasing the recovery rate of SAM from 85% to 95%.

Management of cases has improved thanks to the intervention. MAM recovery rate increased from 84% in 2014 to 97% in 2017 in the intervention group, whereas it stayed the same in the control group (respectively 76% and 78%). The logit estimates display a significant positive impact of the PBF-N of 5.59 in terms of odds ratio (p = 0.039) and +14.7 percentage points in terms of marginal impact on the recovery rate (p = 0.007). This may be underestimated since only a few clinical files have been collected in the control group. We indeed think that the bias is towards underestimates, as the HCs in the control group that kept their MAM management services running were probably the most committed to the nutrition domain and/or those assisted by aid actors (most of MAM clinical files in the control group come from two provinces assisted by the World Food Program). The treatment duration for the recovered patients also significantly improved according to the OLS estimates (-33 days, p = 0.040). Uncomplicated SAM recovery rate increased from 84% to 92% in the intervention group and from 87% to 93% in the control group, and impact was not found significant (OR: 1.16, p = 0.751). However, uncomplicated SAM treatment duration among the recovered patients also significantly improved thanks to the intervention (-20 days, p = 0.025; cf. Table 2).

Table 2

Impact of PBF-N on MAM and uncomplicated SAM services’ performance.

		Control group			Intervention group			PBF-N impact
								Logit estimates
		%	n	N	%	n	N	Odds ratio	p-value	(95% CI)	Nb of obs
MAM recovery rate	before	76%	234	308	84%	268	320	5,59	0,039	(+1,09;+28,70)	1 067
	after	78%	87	112	97%	317	327
Uncomplicated SAM recovery rate	before	87%	306	352	84%	262	313	1,16	0,751	(0,46; 2,92)	1 402
	after	93%	359	387	92%	322	350
								OLS estimates
		Mean	SD	N	Mean	SD	N	Coef.	p-value	(95% CI)	Nb of obs
Treatment duration among the MAM cases who recovered	before	70.79	47.18	225	78.08	53.78	252	-33.59	0.040	(-65.5; -1.7)	842
	after	70.26	39.82	77	43.96	40.55	288
Treatment duration among the uncomplicated SAM cases who recovered	before	56.97	41.22	304	61.19	34.16	252	-20.42	0.025	(-38.3; -2.5)	1 155
	after	59.06	41.08	317	42.87	32.35	282

These estimates correspond to model (1); a model using control variables were also tested, leading to a higher explanatory power and similar coefficients and p-values regarding the PBF-N impact. Meanings: % is proportion, n is number of occurrences, SD is standard deviation, N is number of observations, and CI is confidence interval.

Similarly, PBF-N had a significant impact on the number of children treated for acute malnutrition. According to our Poisson estimates, the intervention had a multiplying effect of 17.9 on the number of MAM cases per semester per health centre (p<0.001), and of 3.06 on the number of SAM cases per semester per health centre (p<0.001). Yet the intervention did not have any influence on malnutrition screening during paediatric curative consultations. In 2017, only 18 cases were diagnosed as suffering from malnutrition in the regular consultation, while according to the anthropometric data collected during the exit interview, 70 children were malnourished (out of 528 cases). We assessed the impact of the intervention on false positive cases of acute malnutrition (AM) among the non-AM cases according to the anthropometrics taken at exit interview, and on false negative cases among the AM cases: no significant impact was found on false positive and false negative rates. The 2017 fictive vignette cases confirmed the appalling situation with capacities of health workers in AM diagnosis already observed in 2014 [25]: only three HWs in 2014 and two in 2017 succeeded in diagnosing the three acute malnutrition cases correctly (out of 145 in 2014 and 168 in 2017). Unsurprisingly we found no significant impact of the intervention on diagnosis success rate, whatever the vignette (cf. Table 3). Findings do not show either any influence of the PBF-N intervention on activities on growth monitoring, whatever the source of information: the growth curve is mentioned in 4% of consultations in 2017 (even less than in 2014), it is drawn in less than 3% of health booklets of children surveyed in the household survey (three times less than in 2014; cf. Table 4).

Table 3

Impact of PBF-N on acute malnutrition diagnosis and screening.

		Control group			Intervention group			PBF-N Impact
								Logit estimates
		%	n	N	%	n	N	Odds ratio	p-value	(95% CI)	Nb of obs
Within consultations observed:
Acute malnutrition was diagnosed	before	3.46%	9	260	1.97%	5	254	1.77	0,527	(0,30; 10,42)	1 042
	after	3.42%	9	9263	3.40%	9	265
False positive of acute malnutrition diagnosis	before	1.00%	2	205	1.33%	3	226	2.98	0,512	(0,11; 78,34)	876
	after	0.45%	1	223	1.80%	4	260
False negative of acute malnutrition diagnosis	before	86.54%	45	52	92.59%	25	27	0.98	0,983	(0,11; 8,50)	162
	after	80.00%	32	40	88.37%	38	43
Within fictive cases:
Vignette 1: diagnosis of complicated SAM was found	before	7.0%	5	71	6.8%	5	74	1,34	0,718	('0,27; 6,67)	313
	after	12.9%	11	85	16.0%	13	81
Vignette 2: diagnosis of uncomplicated SAM was found	before	15.5%	11	71	12.2%	9	74	1,85	0,384	(0,46; 7,40)	313
	after	20.0%	17	85	25.9%	21	81
Vignette 3: diagnosis of MAM was found	before	28.2%	20	71	24.3%	18	74	0,83	0,722	(0,30; 2,29)	313
	after	42.4%	36	85	33.3%	27	81
Using clinical files transcripts:								Poisson estimates
		Mean (median)	SD	N	Mean (median)	SD	N	IRR	p-value	(95% CI)	Nb of obs
Nb of MAM cases per semester	before	62.0 (29)	109.4	45	33.4 (20)	39.5	45	17,9	<0.001	(7,11; 45,07)	180
	after	12.7 (0)	27.1	45	122.2 (66)	182.7	45
Nb of uncomplicated SAM cases per semester	before	40.1 (33)	36.4	45	24.9 (17)	21.4	45	3,06	<0.001	(1,92; 4,87)	180
	after	41.5 (28)	46.4	45	78.9 (62)	68.8	45

These estimates correspond to model (1); a model using control variables were also tested, leading to a higher explanatory power and similar coefficients and p-values regarding the PBF-N impact. Meanings: % is proportion, n is number of occurrences, SD is standard deviation, N is number of observations, IRR is incidence rate ratio and CI is confidence interval.

Table 4

Impact of PBF-N on growth monitoring activities.

		Control group			Intervention group			PBF-N impact
								Logit estimates
		%	n	N	%	n	N	Odds ratio	p-value	(95% CI)	Nb of obs
Growth curve mentioned in consultations observed	before	6.2%	16	260	8.3%	21	254	0.80	0.776	(0,17; 3,80)	1 042
	after	3.8%	10	263	4.2%	11	265
Growth monitoring sessions are reported as regularly organized	before	80.0%	36	45	84.4%	38	45	2,47	0,200	(0,62; 9,82)	180
	after	51.1%	23	45	77.8%	35	45

These estimates correspond to model (1); a model using control variables were also tested, leading to a higher explanatory power and similar coefficients and p-values regarding the PBF-N impact. Meanings: % is proportion, n is number of occurrences, N is number of observations and CI is confidence interval.

Findings at the community level

The improvements of malnutrition management at the HC level did not translate into better results at the community level. Prevalence of chronic malnutrition among children aged 6–23 months remained high, above 50%. Prevalence of acute malnutrition increased from 6% in December 2014 (6.3% in control group, 5.8% in intervention group) to 8.8% in March 2017 (8.8% in control group, 8.7% in intervention group); we attribute this increase to seasonality and, possibly, to the general deterioration of the country’s economy. PBF-N had any significant impact on neither chronic nor acute prevalence rates (cf. Table 5).

Table 5

Impact of PBF-N at the community level.

		Control group			Intervention group			PBF-N impact
								Logit estimates
		%	n	N	%	n	N	Odds ratios	p-value	(95% CI)	Nb of obs
Acute malnutrition prevalence	before	6.22%	193	3100	5.78%	179	3099	1,08	0,628	(0,79; 1,47)	12 679
	after	8.73%	283	3240	8.70%	282	3240
Chronic malnutrition prevalence	before	52.97%	1642	3100	53.21%	1649	3099	1,01	0,893	(0,88; 1,16)	12 679
	after	49.88%	1616	3240	51.91%	1682	3240
								OLS estimates
		Mean	SD	N	Mean	SD	N	Coef	p-value	(95% CI)	Nb of obs
Weight for length z-score	before	-0.34	1.09	3100	-0.34	1.08	3098	+0.02	0.725	(-0.08;+0.12)	12 677
	after	-0.47	1.12	3233	-0.46	1.14	3246
Length for age z-score	before	-2.10	1.22	3100	-2.11	1.24	3099	-0.01	0.783	(-0.11;+0.08)	12 673
	after	-2.06	1.30	3230	-2.08	1.23	3240
Mid-Upper Arm Circumference (MUAC)	before	139.75	12.42	3100	140.10	12.32	3099	-0.07	0.893	(-1.1;+1.0)	12 678
	after	137.96	12.36	3239	138.24	12.66	3240
								Logit estimates
		%	n	N	%	n	N	Odds ratios	p-value	(95% CI)	Nb of obs
Among AM cases, follow-up at the HC for AM	before	8.81%	17	193	8.38%	15	179	1,53	0,341	(0,64; 3,67)	937
	after	13.78%	39	283	18.79%	53	282
Among the SAM cases, follow-up at the HC for AM	before	12.90%	4	31	19.51%	8	41	1,08	0,935	(0,18; 6,43)	191
	after	15.00%	9	60	23.73%	14	59

These estimates correspond to model (1); a model using control variables were also tested, leading to a higher explanatory power and similar coefficients and p-values regarding the PBF-N impact. Meanings: % is proportion, n is number of occurrences, SD is standard deviation, N is number of observations, and CI is confidence interval.

We also checked with the household survey data whether more malnourished children were under HC management. It appeared that, among the children identified by our surveyors with acute malnutrition, only a few of cases were managed in HCs: less than 10% in both groups in 2014, and 13.8% and 18.8% respectively in the control and intervention groups in 2017. We found a non-significant impact of the PBF-N intervention of 1.53 OR (p = 0.341) on the management rate (cf. Table 5). Using a set of control variables improved the explanatory power of the models but the PBF-N impact coefficient remained non significantly different from zero (cf. Table 6).

Table 6

Conditional impact of PBF-N on follow-up at the health centre for acute malnutrition.

Logit estimates		Follow-up at the HC for AM (yes/no)
		Among both MAM and SAM cases			Among SAM cases only
PBF-N impact	Odds ratio	1,24	0,97	0,7	1,26	1,06	0,56
	p-value	0,685	0,957	0,525	0,843	0,964	0,655
PBF-N impact x Short distance	Odds ratio		2,47			1,9
	p-value		0,058			0,544
PBF-N impact x supervision of	Odds ratio			3,9			34,31
CHWs at HC level	p-value			0,001			0,003
Control variables		Yes	Yes	Yes	Yes	Yes	Yes
Nb of observations		712	712	712	141	141	141

These logit estimates correspond to model (1), using in addition control variables and interactive variables.

Discussion

Summary of findings

This study is to our knowledge, the first cRCT assessing the effect of an extension of a PBF scheme to nutrition services. Our findings indicate that the PBF-N scheme piloted over 2015–2016 in rural Burundi improved malnutrition management at HC level, despite serious resource and capacity constraints. Thanks to the PBF-N incentives, there was an improvement in the outcome of the treatment: MAM recovery rates increased by 14.7 percentage points (p = 0.007), while MAM and uncomplicated SAM duration of treatment was reduced by respectively 34 and 20 days (resp. p = 0.040 and p = 0.025), suggesting more efficient practices. There was also an increase in the number of malnourished children managed by the HCs (see in the evaluation report, S8 File). Yet, the volume increase at facility level was not massive enough to generate large benefits at population level: we did not find any significant impact on acute and chronic malnutrition prevalence among children aged 6–23 months. This is in line with other studies on PBF in Burundi, which also reported improvements in service utilisation [10, 12, 28] and quality of care [11, 29], but not always health benefits at population level [10, 29, 30].

Further data analyses found out that there were several missed opportunities, a major one—the insufficient effort to detect children during the general paediatric consultation—directly stems from a lack of knowledge and know-how at the level of health workers [31]. This weak capacity, already present at the baseline, has not been modified by the intervention. In addition, growth monitoring remains a marginal activity while it is in fact essential to growth promotion, malnutrition screening, and, in fine, to the reduction of child malnutrition [32]. Further analyses displayed in the final report detect that the PBF-N intervention was not enough to tackle several other structural constraints (e.g. sets of anthropometrics’ equipment -with scales, measuring boards, WHZ tables, etc.- were not fully available and functional in most of the HCs), at least not within the two-year period of the study. The discontinuation of the MAM treatment supply as well as the political and financial instability also hindered the PBF-N to operate [21, 24, and S8 File].

At community level, prevalence of chronic malnutrition among children aged 6–23 months remains high, above 50%, and there was no impact of the intervention on neither chronic nor acute malnutrition prevalence. This is not surprising, as malnutrition is multifactorial, and prevention, identification and management of malnutrition at health system level can compensate but cannot fix food security nor water, sanitation and hygiene failures. Burundi is highly dependent on agriculture [33], 91% of our surveyed households suffered from moderate to severe food insecurity, and about three quarters of children aged 6–23 months did not have a diet that met the required diversity and frequency. Improved water source and sanitation are accessible by respectively 82% and 39% of rural inhabitants [4] and according to our findings, 65% of them had soap. PBF-N did not address these issues, mainly because this goes beyond its scope: this is not a mechanism that can address socio-economic determinants such as wealth, education of mothers and number of children [26].

Still, among the children identified with acute malnutrition by our surveyors, too few were managed by the HCs. According to us, the main priority should be to increase the number of detected children: at community level, among children aged 6–23 months, there were still, at the endline, more than 80% of AM cases not followed-up in HCs in the intervention group. The improvement in screening should happen both at community and facility levels. At facility level, during paediatric consultations, health workers’ practices towards malnutrition detection were found to be weak, and it appeared that PBF-N had no impact on this: the first constraint probably is health workers’ technical capacities. Nevertheless, our study indicates a strategy which seems particularly powerful: PBF-N had a significant impact on the volume of detected children in HCs where there was a health worker dedicated to health promotion and in charge of supervising CHWs. This indicates that community health could be a central strategy for the screening, detection and referral of acute malnutrition cases. This finding also suggests that the CHW strategy did get a boost from the PBF-N intervention, both at community and HC level, i.e. both through the provision of revenues, conditional on performance, to CHWs, and the PBF-N intervention at the health centre level which encouraged some HCs to work with and integrate CHWs to their malnutrition screening activity. This kind of supervision of CHWs should be scaled up.

This study was based on a pilot project, intended to inform policy decisions related to health financing and nutrition. To our mind, the main policy lessons do not directly relate to PBF-N: they bear on fundamental weaknesses of the health system. There is a need for much more attention to malnutrition management in Burundi. Many children could benefit from MAM or SAM services. The priority is to detect those children and deliver them quality services. Better detection will come from improved collaboration with and supervision of CHWs, growth monitoring and capacity to perform integrated management of childhood illnesses, with having personnel specifically in charge of health promotion. Staff technical capacities are a bottleneck, and there is an urgent need for specific training and closer supervision. Issues with supplement shortages are another major constraint and a source of staff demotivation (what is the point to screen malnourished children if you have nothing to give them?).

Our findings have also relevance beyond Burundi. There is a case for expanding PBF schemes to nutrition services in some other low-income countries, but there must be a clear understanding that PBF will not suddenly solve all systemic weaknesses: in many poor-resource settings, complementary health system strengthening efforts will probably be needed. There is also a need for more research to confirm our findings and their validity in other contexts.

Limitations

This impact evaluation followed a cRCT design–this ensures robust findings. However, results coming from the MAM management clinical files should be taken with some caution because of the attrition in the control HCs. Because of the lack of food treatment, most control HCs have closed the MAM service; the only ones that kept the service running were those helped by other aid organisations or for which nutrition was a priority (for instance, because of some intrinsic motivation at the level of the staff). This selection bias among control HCs suggests that the effect of PBF-N on treatment outcome is under-estimated. At the community level, the short duration of the intervention (two years) may not have allowed to show an impact on chronic malnutrition, although we believe other factors than the health system itself, like food security, water, sanitation and hygiene elements, are much more structural determinants.

Conclusion

Child malnutrition is a major health issue in Burundi. It also compromises the present and future economic development of the country. Major action is required. Our findings indicate that PBF can be part of the strategy. Yet, its effectiveness will gain from a general improvement of malnutrition screening and identification, both at health centre and community levels. Built-in in a health system approach, with improved coordination with CHWs, capacities of HWs and nutrition services, its effects should be more important than what we documented in this pilot study. Still, PBF won’t be enough. Malnutrition in Burundi calls for much broader and multisectoral effort from a large national and international coalition.

Baseline characteristics of households and children.

Notes: N refers to the number of observations; n refers to the number of positive (yes to a categorical variable); SD refers to the standard deviation. Source: Authors.

(DOCX)

Click here for additional data file.

Chain of results with PBF-N.

Source: Authors.

(TIF)

Click here for additional data file.

CONSORT checklist.

Source: Authors.

(PDF)

Click here for additional data file.

Quality indicators specific to PBF-N.

Source: MSPLS. Note technique relative à l’intégration de la nutrition dans la stratégie nationale de financement basé sur la performance. 2013.

(PDF)

Click here for additional data file.

Study protocol, household level, as approved by the University of Antwerp Ethics Committee, in French (original language).

Source: Authors.

(PDF)

Click here for additional data file.

Study protocol summary in English, household level, as approved by the University of Antwerp Ethics Committee.

Source: Authors.

(PDF)

Click here for additional data file.

Study protocol, health centre level, as approved by the University of Antwerp Ethics Committee, in French (original language).

Source: Authors.

(PDF)

Click here for additional data file.

Study protocol summary in English, health centre level, as approved by the University of Antwerp Ethics Committee.

Source: Authors.

(PDF)

Click here for additional data file.

Baseline report, in French: « FBP Nutrition au Burundi.

Rapport des enquêtes de référence au niveau des centres de santé et des ménages ». Source: Authors.

(PDF)

Click here for additional data file.

Impact evaluation report, in French: « FBP Nutrition au Burundi.

Evaluation d’impact au niveau des centres de santé et des ménages ». Source: Authors.

(PDF)

Click here for additional data file.

Synthesis of datasets used for analysis, with summary statistics (mean, median, sum, variance, standard deviation, number of observations) by survey and treatment group.

Source: Authors.

(XLSX)

Click here for additional data file.

Transfer Alert

This paper was transferred from another journal. As a result, its full editorial history (including decision letters, peer reviews and author responses) may not be present.

7 Apr 2020

PONE-D-19-30259

Impact of the extension of a performance-based financing scheme to nutrition services in Burundi on malnutrition prevention and management among children below five: a cluster-randomized control trial

PLOS ONE

Dear Dr. Korachais,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by May 22 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Seth Adu-Afarwuah

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. In your Methods section, please provide additional information about the participant recruitment method and the demographic details of your participants. Please ensure you have provided sufficient details to replicate the analyses such as: a) the recruitment date range (month and year), b) a description of any inclusion/exclusion criteria that were applied to participant recruitment, c) a table of relevant demographic details, d) a statement as to whether your sample can be considered representative of a larger population, e) a description of how participants were recruited, and f) descriptions of where participants were recruited and where the research took place.

3. Please correct your reference to "p=0.000" to "p<0.001" or as similarly appropriate, as p values cannot equal zero.

4. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified (1) whether consent was informed and (2) what type you obtained (for instance, written or verbal). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

5. Thank you for stating the following in the Competing Interests section:

"CK, SN and MN declare no conflict of interest.

BM has contributed to the emergence of PBF as a global health policy, through technical assistance, research and knowledge management. He is the lead facilitator of the PBF Community of Practice. He holds minority shares in Blue Square, a firm developing health system software solutions. "

Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

6. Please remove your figures from within your manuscript file, leaving only the individual TIFF/EPS image files, uploaded separately.  These will be automatically included in the reviewers’ PDF.

7. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Yes

Reviewer #3: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This paper reported an important national study that aimed to address two research questions: (1) did the extension of performance-based financing to nutrition (PBF-N) scheme contributed to improve nutritional health services for children below five years old at HC level and; (2) did it reduce the prevalence of malnutrition among children below two years old at community level? Of the 193 eligible health centres providing nutrition services in Burundi, 100 HCs were randomised and 90 were selected to the study. Cross-sectional household surveys were conducted separately at baseline and at end-line about two years after the PBF-N intervention has started. In addition, health centre surveys were performed at both visits and routine data were obtained from the PBF-N program. The primary outcomes were the prevalence of chronic and acute malnutrition among children aged 6-23 months measured at the community level, and the MAM/SAM recovery rates among children below five years old at the HC level.

The study has followed a cluster-randomised controlled trial design, with half of the 90 selected HCs randomly allocated to intervention and the other half constituted the control group. Data were collected at both cluster and individual levels using two cross-sectional surveys. The methods were well documented; however, I have major concerns on the overall trial design, statistical analysis and results presented in this paper.

The CONSORT flow chart suggested that 193 HCs were assessed for eligibility, 100 were randomised, and 90 were selected and enrolled to the study. The authors did not explain why only half of the HCs were randomised, and the discrepancy between 100 and 90 HCs that were randomised. In principle, only eligible clusters should be randomised and included in final analysis.

The sample size of the household surveys was based on a reduction of 25% in acute malnutrition prevalence between HCs in the two groups. The number of children recruited in each HC was set to 72, from 12 sub-hills per cluster and 6 eligible households per sub-hill. This adds up to a total of 6,480 children aged 6-23 months surveyed at the community level. Was the same target used for baseline survey, which as reported included a total of 6,199 children?

For a cluster randomised trial, sample size calculation should be based on both the number of clusters and average cluster size with pre-specified statistical power (>80%) and significance level (normally 5%). Another important parameter to consider is the inflated design effect related to the intra-cluster correlation coefficient (ICC), which is the proportion of the total variance of the outcome that can be explained by the variation between clusters. The flow chart should also include information on both clusters and individuals, as recommended by the CONSORT 2010 statement with extension to cluster randomised trials.

For outcome analysis, the statistical model has included both time points while only the end-line survey data were potentially impacted by the intervention. The beta coefficients estimated the difference-in-difference, which would be hard to quantify the actual effect sizes between intervention and control groups on the defined outcomes. A standard approach is to analyse the end-line survey data only using the mixed models with a random cluster effect, with the assumption that the clusters were similar at baseline by randomisation and matching.

As required by the CONSORT guidelines, baseline characteristics at cluster and individual participant levels should be presented in a table for each group. Continuous variables can be summarised by the mean and standard deviation, and numbers and proportions should be reported for categorical variables. The supplementary table 1 needs to be modified following the guidelines. The outcome tables are hard to follow, and the numbers are inconsistent. For example, how were the total numbers of observation 1,254 and 1,549 obtained in Table 2? The column titles Mean and N were used for all outcomes regardless of their distributions. With an interaction term between group and period in the main model, which term was used to estimate the effects of intervention on different outcomes should be clearly stated so that the estimated coefficients, odds ratios, and incidence rate ratios can be interpreted appropriately. With more than one primary outcome and a large number of variables tested overall (386), any significant differences observed between intervention and control groups should be concluded with caution without controlling for overall Type 1 error rate.

Reviewer #2: 1. Abstract. The word “data” is plural so it should be “data were”.

2. p. 9. The last paragraph purports to explain the theory of change of the intervention. It does not do so. Having this explanation is essential especially for understanding why the authors expected health system improvements to have reduced chronic or acute malnutrition prevalence, an expectation that on the face of it does not seem warranted because health systems cannot remove most of the causes of the malnutrition (as the authors note in the Discussion). What the authors have provided here is useless for explaining the theory of change. The authors have basically said to the reader to go find out for yourself by going to the supplementary material, which is irritating and not acceptable. S2-S5 appear to have two files duplicated twice. The protocol is in French; PLoS One is an English language publication. Please revise to explain clearly in the main text of the manuscript the theory of change of the intervention for both recovery from and prevention of malnutrition.

3. p. 14, top. The authors write “We talk about shortness when LAZ is lower than minus two standard deviations (SD) and stunting when it’s lower than minus three SDs.” WHO (and just about everyone else) defines stunting as less than -2 SD, not -3 SD. Please revise and use the standard definitions to avoid confusion.

4. p.15. Where it says “The time varying variable PBFnutht is an an interaction term…” delete “time varying”. This term is not time varying in the common use of this descriptor.

5. p. 22. P-values, by definition, cannot be zero. All p-values given as 0.000 should be given as <0.001.

6. The manuscript requires careful technical editing and preparation. Overall, the writing is understandable, but many technical problems are throughout the manuscript. Furthermore, the manuscript was not carefully prepared, for example, line numbers in only part of the manuscript.

Reviewer #3: • General comments

This is a quality paper on an important nutrition issue using state-of-the art methodology (RCT) with particularly careful pairing of control and intervention HCs. It has often been assumed that health personnel and workers did not have enough incentives to devote more effort to nutrition services and therefore, it was hypothesized that performance-based monetary incentives would boost the nutrition performance and impact. There are some issues, however minor, that need to be addressed.

• Specific comments

1. Abstract

The objectives should be given. The results section is not very clear : 97% - 78% = 19%, not 14.7pp (what is “pp”?). Unless this is not the way the results should read.

2. Introduction :

a. Hunger and food insecurity are among the root causes of malnutrition. Malnutrition encompasses several forms including undernutrition. In addition to poverty and food insecurity, there are factors of child undernutrition which are in the health realm, beyond nutrition care and services, and management of undernutrition, that is, poor access to health care as well as WASH. This could be clearer in the paper.

b. Community-base management of undernutrition (and prevention) is now advocated, although it is clear that health personnel are involved, at community, as well as HC and hospital level. The expected nutrition activities and services at these three levels should be described, perhaps in general and in the Burundi context.

3. Methods

a. How many health centers in total for 200 providing nutrition services?

b. The rationale for the different age range in the health center and the community assessment should be given.

c. Fig. 1: The unit is the health center, please specify

d. Institut national de santé publiQUE

e. Who gets the subsidy and how is it shared in the case of hospitals, HCs and CHWs?

f. What is the MAM treatment supply? It should be explicit as it is often referred to.

g. Shortness and stunting: are these widely accepted cut-offs and terms? Above the age of 24 months, it should be stated that standing height is measured, not length.

h. Please clarify that the duration of the intervention was two years.

i. Is it possible to provide the list of the 19 indicators of quality of nutrition services, perhaps as supplementary data?

j. According to Table 2, CHWs are to screen and refer acute malnutrition cases to HCs. However, is this not accomplished through growth monitoring and promotion activities, which are HC’s activities?

k. The protocol is not needed even in appendix; besides, it is in French.

4. Results

a. On p. 18, there is a distinction of the intervention at hospital, HC, and community level. This is why the specific activities expected at these three levels have to be described in the methods section.

b. Fig. 3 is not very useful.

c. What about the % of defaulters in MAM management, in intervention vs control HCs?

d. It has to be clear that the improved SAM management performance only refers to uncomplicated SAM treatment outside the hospital since control hospitals also get PBF for the management of SAM cases with complications.

e. In Table 2, what is the meaning of **?

f. What about infant and child feeding practices in the community survey? It is referred to it in the discussion (p. 32, line 29) but the results are not shown.

5. Discussion and conclusion

a. Training of health personnel (although recent training apparently had no impact) and supervision of CHWs would appear to be major conditions of performance that the PBF strategy did not address. This should be discussed in more depth.

b. In the recommendations, the need for health centre staff training and CHW supervision should be emphasized, along with having personnel specifically for health promotion.

c. Is the fact the control hospitals and HCs also received subsidies not a limitation of the study, although this was intended to clearly identify the impact of the monetary incentives’?

d. The short duration of the intervention (2 years) may not have allowed to show an impact on stunting; this can be regarded as a limitation of the study.

e. P. 31, line 19: reference needed

f. P. 31, line 21: What equipment?

g. P. 32, line 27: also failures in WASH, not only food security

h. P. 32, line 40: “CHW strategy got a boost from PBF”: Please explain

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

18 Jun 2020

Dear editor and reviewers,

Thank you for your feedbacks, we appreciated the detailed and committed review. We considered and took care of each comment, and feel that this made the paper clearer. You can appreciate our responses in the "responses to reviewers" document (18 pages), and modifications of the text in the revised manuscript.

Sincerely,

the authors

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

28 Jul 2020

PONE-D-19-30259R1

Impact of the extension of a performance-based financing scheme to nutrition services in Burundi on malnutrition prevention and management among children below five: a cluster-randomized control trial

PLOS ONE

Dear Dr. Korachais,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Sep 11 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Seth Adu-Afarwuah

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: (No Response)

Reviewer #3: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: (No Response)

Reviewer #3: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: (No Response)

Reviewer #3: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: (No Response)

Reviewer #3: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #3: Only two minor points:

1. In the abstract, HC should be defined. Perhaps the conclusion should be clearer that performance-based financing had not substantive effect on the prevention of malnutrition and therefore alternative or additional strategies are required.

2. If the protocol is already published [17], one wonders why it is provided in supplementary files (S3-S6).

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

28 Jul 2020

The two reviewers who responded reported that all comments had been addressed. We are happy that we have responded to all your questions and requests and that we got to a common understanding on this work.

There were two additional minor comments sent by reviewer #3. Here below are the two minor comments and our answers to them:

1 / His first comment was “In the abstract, HC should be defined. Perhaps the conclusion should be clearer that performance-based financing had not substantive effect on the prevention of malnutrition and therefore alternative or additional strategies are required.” We would like to thank him, that was indeed a problem. We replaced “HC” by “health centres (HC)” at its first appearance in the abstract. We also added a conclusive sentence to the abstract: “PBF can contribute to a better management of malnutrition at HC level; yet, to address the huge problem of child malnutrition in Burundi, additional strategies are urgently required.”

2/ His second comment was: “If the protocol is already published [17], one wonders why it is provided in supplementary files (S3-S6).” We added these supplementary files because this corresponds to the protocol that was approved by the ethics committee, and this is a requirement from PLOS ONE to provide such a copy. These files are not accessible from the published version of the protocol.

31 Aug 2020

Impact of the extension of a performance-based financing scheme to nutrition services in Burundi on malnutrition prevention and management among children below five: a cluster-randomized control trial

PONE-D-19-30259R2

Dear Dr. Korachais,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Seth Adu-Afarwuah

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: (No Response)

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: (No Response)

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: (No Response)

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: No

8 Sep 2020

PONE-D-19-30259R2

Impact of the extension of a performance-based financing scheme to nutrition services in Burundi on malnutrition prevention and management among children below five: a cluster-randomized control trial

Dear Dr. Korachais:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Seth Adu-Afarwuah

Academic Editor

PLOS ONE