Effect of results-based financing on facility-based maternal mortality at birth: an interrupted time-series analysis with independent controls in Malawi.

INTRODUCTION: The aim of this study was to assess the impact of a results-based financing (RBF) programme on the reduction of facility-based maternal mortality at birth. Malawi is a low-income country with high maternal mortality. The Results-Based Financing For Maternal and Newborn Health (RBF4MNH) Initiative was introduced at obstetric care facilities in four districts to improve quality and utilisation of maternal and newborn health services. The RBF4MNH Initiative was launched in April 2013 as a combined supply-side and demand-side RBF. Programme expansion occurred in October 2014.
METHODS: Controlled interrupted time series was used to estimate the effect of the RBF4MNH on reducing facility-based maternal mortality at birth. The study sample consisted of all obstetric care facilities in 4 intervention and 19 control districts, which constituted all non-urban mainland districts in Malawi. Data for obstetric care facilities were extracted from the Malawi Health Management Information System. Facility-based maternal mortality at birth was calculated as the number of maternal deaths per all deliveries at a facility in a given time period.
RESULTS: The RBF4MNH effectively reduced facility-based maternal mortality by 4.8 (-10.3 to 0.7, p<0.1) maternal deaths/100 000 facility-based deliveries/month after reaching full operational capacity in October 2014. Immediate effects (changes in level rather than slope) attributable to the RBF4MNH were not statistically significant.
CONCLUSION: This is the first study evaluating the effect of a combined supply-side and demand-side RBF on maternal mortality outcomes and demonstrates the positive role financial incentives can play in improving health outcomes. This study further shows that timeframes spanning several years might be necessary to fully evaluate the impact of health-financing programmes on health outcomes. Further research is needed to assess the extent to which the observed reduction in facility-based mortality at birth contributes to all-cause maternal mortality in the country.

Many low-income countries have adopted performance payments in the form of results-based financing (RBF) programmes to improve quality and utilisation of maternal health services.

Current evidence on RBF impact is largely focused on immediate or intermediate health service outcomes (ie, service utilisation, service quality), but remains so far rather inconclusive.

Our findings suggest that RBF programs with a strong focus on quality of service delivery and in combination with demand-side interventions can play a role in reducing maternal mortality, in settings with high utilisation of facility-based childbirth services but inadequate service quality.

This study provides further insight into how health-financing interventions implemented in low-income settings require several years before reaching full operational capacity.

Selection and evaluation of performance incentives might require a stronger focus on their actual contribution to population health outcomes.

Introduction

Although maternal deaths have decreased globally, sub-Saharan Africa (SSA) remains the region with the highest maternal mortality ratio (MMR), with 546 deaths per 100 000 live births in 2015.1 The majority of maternal deaths are attributable to direct obstetric causes, such as haemorrhage, eclampsia, puerperal sepsis or obstructed labour.2 In most African settings, lack of access to care (due to financial and distance barriers) and poor health service delivery are key factors hampering countries’ ability to adequately address the underlying clinical causes of maternal mortality.3

In recent years, results-based financing (RBF) has caught traction as a health system strengthening approach in improving both utilisation and quality of health services in low-income countries (LICs).4 RBF refers to a set of financial arrangements linking payments to defined healthcare outputs (eg, performance payments for service providers) or health-seeking behaviours (eg, conditional cash transfers (CCT) or vouchers for service users).5 Many LIC health systems therefore adopted RBF to gain further improvements in the utilisation and quality of primary care services, especially those related to maternal and newborn health (MNH). Current evidence on the effect of RBF is rather inconclusive given the differences in implementation contexts, and furthermore focused on immediate or intermediate health service outcomes, such as service utilisation, health worker motivation, patient satisfaction and clinical quality.6 7 While few authors have looked at the impact of RBF on ultimate MNH outcomes, such as mortality, this work has almost exclusively addressed demand-side RBF programmes (ie, use of RBF to improve service utilisation).8–10 It follows that, to date, there is no clear evidence available on the causality between supply-side RBF programmes (ie, use of RBF to improve service provision) and maternal mortality reduction in SSA.

Our study contributes towards filling this knowledge gap by presenting results from a quasi-experimental impact evaluation of an RBF intervention on facility-based maternal mortality in Malawi. Our work is based on the assumption that the current evidence on RBF falls short to gauge the ultimate role RBF programmes can play in improving MNH outcomes in LIC. As a result, we postulate that comprehensive assessments of RBF ought to include analyses of its impact on relevant mortality indicators. Moreover, using exclusively routine data for our analysis, we demonstrate the feasibility of secondary data for RBF impact evaluations.

Methods

Study setting

Malawi is an LIC in SSA with an estimated MMR of 439 deaths per 100 000 live births in 2015.11 Obstetric care services are provided through the country’s essential health package offered free of charge at public and contracted not-for-profit health facilities.12 In 2015, 91% of births occurred in health facilities, with 90% of births attended by a skilled provider.11 In 2014, unmet need for emergency obstetric care (EmOC) among women with obstetric complications was estimated at 75%, given the majority of health facilities failed to fully meet the applicable EmOC standards.13 Shortages in human resources and stock-outs of essential drugs and supplies further challenge the health system’s ability to reliably provide EmOC.

Intervention design

In April 2013, the Ministry of Health launched the Results-Based Financing For Maternal and Newborn Health (RBF4MNH) Initiative in four districts (Balaka, Dedza, Mchinji, Ntcheu) to improve quality and utilisation of facility-based childbirth care services.14 15 The RBF4MNH includes two components: (1) performance contracts with facilities and district health management teams (DHMTs) linked to defined childbirth care quality targets; and (2) CCT to pregnant women linked to giving birth and spending a 48-hour postpartum observation period at their respective catchment facility.14 15 District selection was non-random and a result of a political decision specifically supporting districts with weaker maternal health outcomes and EmOC structures.

The RBF4MNH was rolled out at the facility level. Initially, 18 non-randomly selected EmOC facilities (4 hospitals, 14 health centres) across the 4 districts received RBF (ie, intervention phase 1). In October 2014, 15 additional EmOC facilities (3 hospitals, 12 health centres) were added within the same districts (ie, intervention phase 2). Facilities received performance payments in addition to their usual budget allocation. As part of RBF, most facilities also benefited from upfront investments in minor infrastructure repair or essential equipment procurement (eg, renovation of labour rooms, purchase of disinfectants, replacement of blood pressure machines). Previous research related to the RBF4MNH demonstrated positive effects of the programme on clinical performance and supply chain management,16 an overall positive but statistically non-significant impact on effective coverage of pregnant women with obstetric care services,17 a significant improvement in the timelines of care-seeking for women with pregnancy-related complications,18 and no evidence for the erosion of overall intrinsic health worker motivation.19

Study design and outcome variable

Our study adopted a quasi-experimental approach based on an interrupted time series (ITS) design with independent controls.20 We used monthly data on the number of direct infacility maternal deaths (ie, occurred during intrapartum or early postpartum period) and deliveries (ie, excluding abortions and miscarriages) reported by obstetric care facilities into the District Health Information System version 2 (DHIS-2)-based national health management information system and computed facility-based maternal mortality at time of birth as the outcome variable. Beyond the lack of reliable population-based maternal mortality data in our study setting, we preferred this facility-based outcome because it better reflects the RBF4MNH theory of change, which targeted specifically effective childbirth care coverage at time of birth. Our outcome variable, facility-specific maternal mortality at birth per month, was calculated as the following:

To estimate the impact of the RBF4MNH on facility-based maternal deaths at birth, we compared monthly mortality ratios over consecutive time points between all obstetric care facilities in the 4 intervention districts (Balaka, Dedza, Mchinji, Ntcheu) and all obstetric care facilities in 19 out of the country’s remaining 25 districts as controls (district of ‘Nkhata Bay and Likoma’ treated as two separate districts). We excluded six control districts due to lack of a priori comparability: the four urban districts of Blantyre, Lilongwe, Mzuzu and Zomba, and the island districts of Likoma and Mwanza. Based on the 2015/2016 Democratic Health Survey, averages across districts for both use of facility-based delivery services and accessibility of skilled birth attendants are comparable between intervention and control districts (93.5% vs 93.7% and 90.5% vs 90.3%, respectively).11

We decided to compare estimates aggregated at the district rather than at the facility level for two reasons: first, to account for the substantially higher number of deaths reported by hospitals compared with health centres, as risk profiles inevitably differ across levels of care; second, to account for the fact that RBF4MNH performance contracts in the four intervention districts also targeted each DHMT, linking incentives to quality of service delivery in the districts at large. Given this particular intervention feature, we postulated the existence of a district effect due to spillover to non-RBF facilities.

Data extraction and cleaning

For each facility, monthly data points were extracted from the Health Management Information System (HMIS) for a total period of 57 consecutive months starting July 2012 (ie, 9 months before RBF4MNH launch) and ending March 2017 (ie, 48 months after RBF4MNH launch). During data preparation, we omitted all data points that were of irretrievably poor quality (eg, number of maternal deaths reported higher than number of deliveries), three individual data points judged as outliers (ie, extremely high numbers of maternal deaths observed in two control districts during the preintervention period) and single facilities with reported numbers of deliveries missing for more than 40% of time points. The proportion of omitted facilities was higher in the control (39%) compared with the intervention districts (18%). We further conducted sensitivity analyses comparing how different data cleaning decisions might have affected the resulting estimates, and found that results only very minimally differed (data not shown) and thus not affected the overall findings of the study as reported here.

Data analysis

We used multiple-group segmented linear regression to analyse the ITS21 comparing maternal mortality at birth between intervention and control districts, and between the preintervention (July 2012 until March 2013), early postintervention (April 2013 until September 2014) and late postintervention (October 2014 until March 2017) periods, according to the following model:

where y represents the maternal mortality outcome variable measured at each monthly time point t, T a continuous variable representing the months since observation start, x dummy variables representing each study period (x =0 for t in the preintervention period; x =1 for t in the early and late postintervention periods; x =0 for t in the preintervention and early postintervention periods; x =1 for t in the late postintervention period), and z a dummy variable representing the treatment group (0=control, 1=RBF). In this model, and indicate the estimated differences in level (intercept) and slope (trend), respectively, of maternal mortality between treated and controls prior to the intervention; and represent the estimated difference-in-differences in level and slope, respectively, attributable to the intervention during the early intervention period; and and represent the estimated difference-in-differences in level and slope, respectively, attributable to the intervention during the late intervention period.

We defined two interruption points in our analysis to reflect the beginning of the two RBF4MNH intervention phases in April 2014 and October 2014. Our model estimates the respective coefficients by Ordinary Least Squares (OLS) regression using Newey-West SEs to handle autocorrelation and potential heteroskedasticity. The Cumby-Huizinga test for autocorrelation22 demonstrated the presence of serial autocorrelation up to a lag of 1; hence, we adjusted the model accordingly. In two separate sensitivity analyses (see online supplementary appendix), we adjusted the model to account for seasonality (hypothesised to affect labour patterns due to climate variability), and we estimated a more parsimonious model based on district matching. Stata V.14.2 was used for all analyses.

Patient and public involvement statement

This study did not involve any patients.

Results

Sample characteristics are shown in table 1. The 23 districts (4 intervention and 19 controls) contained a total of 456 health facilities offering obstetric care services for which HMIS data were available for more than 40% of observation points. Over the 57-month study period, a total of 23 964 complete observation points were included in the analysis. The average number of deliveries per month differed significantly between groups (p<0.01), but reported mortality rates were statistically not significantly different between groups and thus comparable during the preintervention period.

Table 1

Sample distribution and sample characteristics

Characteristics	Intervention	Control	Total
Total number of districts	4	19	23
Total number of health facilities	63	245	308
Total number of complete observations across all included facilities (entire study period)*	4948	18 980	23 964
Mean (SD) and median of monthly number of facility-based deliveries (entire study period)	65.4 (104.3)†, 37	61.4 (91.5)†, 34	62.2 (94.3), 35
Mean (SD) of monthly facility-based maternal deaths per 100 000 facility-based deliveries
Preintervention period	158.0 (54.0)	120.7 (22.5)	139.3 (44.5)
Postintervention period 1	158.4 (53.3)†	103.7 (21.8)†	131.0 (81.1)
Postintervention period 1	123.5 (66.0)†	83.5 (24.9)†	103.5 (53.4)

*Complete information on both indicators (ie, number of monthly facility-based deliveries and monthly facility-based direct maternal deaths) feeding into outcome indicator for the entire study period.

†Difference in means statistically significant at 0.05 level (based on two-group t-test).

For the entire time series, the distribution of monthly observations by intervention and control districts is shown in the scatterplot in figure 1. Regression lines depict the predicted values of maternal mortality for each period. Dashed vertical lines indicate the two interruption points. Intervention and control districts experienced similar declines in mortality levels and slopes between preintervention and first postintervention periods. For both groups, mortality trends (slopes) in the first postintervention period were only slightly lower than the estimated mortality levels at the end of the postintervention period. Going from the end of the first to the beginning of the second postintervention period, neither intervention nor control districts experienced a statistically significant drop in estimated mortality rates. However, the intervention districts experienced a marginally significantly greater decline in maternal mortality over the course of the later compared with the earlier intervention period.

Figure 1

Time trends of facility-based maternal mortality by district and months. Dots represent maternal mortality ratios averaged across facilities within each study arm (ie, intervention vs control); lines represent predicted maternal mortality ratio trends for each period based on linear regression.

Table 2 displays the results from the controlled ITS model. Maternal mortality at observation start (ie, July 2012) was estimated at 111.4 deaths/100 000 facility-based deliveries for the control and 134.8 deaths/100 000 facility-based deliveries for the intervention districts. During the 9-month preintervention period, maternal mortality increased by 2.3 deaths/100 000 facility-based deliveries per month in the control and by 5.8 deaths/100 000 facility-based deliveries per month in the intervention districts. The differences between levels and trends were statistically not different, indicating that control and intervention districts were sufficiently comparable prior to the RBF4MNH intervention. During the first intervention period, we observed a slope reduction attributable to the RBF4MNH of 0.1 fewer maternal deaths/100 000 facility-based deliveries per month with an immediate reduction in maternal mortality levels (comparing the end of the preintervention with the beginning of the first postintervention period) attributable to the RBF4MNH of 28.9 deaths/100 000 facility-based deliveries. These effects, however, are not statistically significant. During the second postintervention period, we observed a marginally significant negative trend effect of 4.8 fewer deaths/100 000 facility-based deliveries per month attributable to the RBF4MNH, coupled with a statistically non-significant immediate reduction in maternal mortality attributable to the RBF4MNH of 26.9 deaths for every 100 000 facility-based deliveries. The results of the sensitivity analyses confirm the patterns observed in the primary analysis.

Table 2

Effect of the RBF4MNH on facility-based maternal mortality

	Estimated maternal deaths per 100 000 facility-based deliveries (95% CI)
Preintervention period
Control level (July 2012)	111.4	(100.8 to 122.0)*
Difference in levels, intervention vs control (July 2012)	23.4	(−37.6 to 84.5)
Control monthly trend	2.3	(−2.0 to 6.6)
Difference of intervention vs control in trend change	3.5	(−12.2 to 19.1)
Effects related to phase 1 (postintervention period 1)
Control level change	−29.8	(−63.9 to 4.2)*
Difference of intervention vs control in level change	−28.9	(−119.4 to 61.7)
Control monthly trend change	−2.2	(−6.7 to 2.4)
Difference of intervention vs control in trend change	−0.1	(−17.8 to 17.7)
Effects related to phase 2 (postintervention period 2)
Control level change	−10.8	(−34.7 to 13.1)
Difference of intervention vs control in level change	−26.9	(−91.0 to 37.1)
Control monthly trend change	−0.9	(−2.4 to 0.6)
Difference of intervention vs control in trend change	−4.8	(−10.3 to 0.7)*

Estimates based on interrupted time-series analysis.

*P<0.1

RBF4MNH, Results-Based Financing For Maternal and Newborn Health Initiative.

Discussion

Statement of principal findings

Our study makes a unique contribution to the existing literature being the first to assess the impact of a combined supply-side and demand-side RBF intervention on facility-based maternal mortality at birth. The significant reduction by 4.8 deaths/100 000 deliveries (CI −10.3 to 0.7, p<0.1) per month attributable to the RBF4MNH intervention is remarkable considering that the intervention had been operative for only 4 years at the time of evaluation.

Strengths and weaknesses of the study

With under-reporting of maternal deaths being likely in both intervention and control facilities, our mortality estimates are probably rather conservative. Throughout the study period, monthly average ratios of facility-based maternal mortality were higher and more fluctuating in the intervention compared with the control districts (figure 1). In fact, the numbers of birth-related deaths varied greatly for any given facility when measured monthly. This fluctuation was more pronounced in the four intervention districts given their smaller sample size compared with the control arm. The higher mortality in the four intervention districts might be explained by the non-random selection of the RBF4MNH districts,14 and the RBF4MNH incentives to improve HMIS reporting, perhaps reducing previous under-reporting of birth-related maternal deaths in the intervention facilities, might explain the higher mortality observed in the four intervention districts.15

The similarity in trends and levels in both study groups during the preintervention and first postintervention periods likely demonstrates a pre-existing general decline in maternal mortality that continued far into the initial programme phase (April 2013–September 2014). This could be an indication of both the programme’s limited capacity to produce any measurable effects in its early phase and the existence of a nationwide secular trend. In fact, early intervention was characterised by several adjustments to the initial design, eventually improving the programme’s operational capacity prior to expansion.15 23 Coexistence of many independent MNH programmes across Malawi during the pre-2015 period could explain the presence of a secular trend.24 Also, given the relatively high mortality rates observed in the intervention districts, we cannot determine to what extent the observed effect size would have been different in scenarios with higher or lower baseline levels prior to intervention start.

Strengths and weaknesses in relation to other studies

The size of the impact on mortality identified in this study was surprising given our prior analyses of the programme’s effect on intermediate outcomes based on a controlled pre–post test design using primary data. While we found significant improvements in equipment maintenance and selected drugs and consumables (ranging between 9% and 52% point increases for selected items), RBF4MNH effects were less conclusive or less extensive in terms of birth attendants’ adherence to obstetric treatment protocols (for instance, non-significant increases between 8% and 21% points in activities related to infection prevention, accompanied by decreases between 18% and 46% points for activities related to postpartum haemorrhage prevention)16 and effective childbirth care coverage (increase by 7.1% points with a p=0.07 in effective coverage).17

Three factors may explain this discrepancy. First, our prior analyses used endline data collected only 2 years after the RBF4MNH launch compared with 48 months in this ITS analysis. This analysis confirms that RBF4MNH gains were not yet realised in the 2 years after programme launch and were mainly accrued later once the intervention had reached its full operational capacity. Second, the RBF4MNH might have produced changes in service quality early on that our previous studies failed to capture, and those early changes led to remarkable reduction in maternal mortality later on. Third, with 62% of maternal deaths in Malawi occurring during the early postpartum period,25 the combined demand-side and supply-side effect of the RBF4MNH encouraging both women and providers to remain at facilities for the 48-hour postpartum observation period15 likely removed pre-existing delays in postpartum care-seeking (not focus of our prior work).26

Meaning of the study

The observed reduction in maternal mortality is highly relevant from a policy point of view. Although remarkable reductions,27 Malawi continues to experience high rates of maternal mortality.11 About 71% of maternal deaths in Malawi occur around the time of birth and 63% among women who delivered in a facility.25 A recent survey indicated that 62% of maternal deaths occurred at health facilities and an additional 21% among mothers who just returned home after delivering in a facility.26 Hence, reducing facility-based maternal deaths at birth by acting to improve quality of service delivery and extending women’s in-hospital stays is likely to bear an important impact on the country’s overall maternal mortality, considering Malawi’s situation with over 90% of women giving birth at a facility11 in the context of poor obstetric care quality.28 29 However, we cannot fully appraise the mortality reduction produced by the RBF4MNH in relation to other maternal care interventions due to the current lack of comparable studies.

Unanswered questions and future research

Our study inevitably suffers from a number of limitations. First, reliance on HMIS data implied that overall reductions in population-based maternal mortality could not be estimated. Given the crucial role quality obstetric care plays in shaping maternal health outcomes beyond the early postpartum period, it is plausible to assume that the RBF4MNH is likely to have produced broader impacts on overall maternal mortality. Further research relying on other data sources is needed to test this hypothesis.

Second, due to extremely poor quality of HMIS data with extreme proportions of missing values on newborn outcomes, we were unable to assess the impact of the RBF4MNH on neonatal mortality. While the DHIS-2 platform likely contributed to improved HMIS data quality in Malawi (especially for Millennium Development Goal-relevant indicators, such as facility-based deliveries and related direct maternal deaths),30 31 and although the assuring findings of our sensitivity analysis regarding our data cleaning approach, the fact that we still had to exclude single facilities due to poor data quality might have biased our findings. This is unfortunate given that improving delivery and early neonatal care is likely to bear a more visible impact on neonatal than maternal mortality.32

Third, while the quasi-experimental application of the ITS allowed us to establish causality between the RBF4MNH and maternal mortality, additional non-observed confounders (eg, maternal health programmes with local or regional effects) might have biased our estimates. To our knowledge, however, the only other large programme likely to have produced changes in health system structures capable of inducing changes in maternal mortality is the Support for Service Delivery Integration, a United States Agency for International Development-funded programme implemented in parallel to the RBF4MNH,33 which we think is unlikely to have shaped results since it was implemented in one of four RBF districts but in 14 controls. If so, our analysis is likely to have produced lower bound estimates of the true effect of the RBF4MNH Initiative.

Fourth, in spite of the observed completeness of HMIS data on maternal deaths, we need to acknowledge the possibility that providers may under-report deaths. Again, however, such under-reporting does not invalidate our analysis, since we have no reason to imagine that under-reporting differs systematically between intervention and control facilities and/or districts.

Last, we need to acknowledge the limited generalisability of our findings to other RBF settings. Unlike most other RBF programmes where payments linked to quantity aspects of service delivery dominate,34 the RBF4MNH kept a stronger focus on payments linked to quality of care processes, such as drug and supply procurement, equipment maintenance, routine death audits and selected aspects of clinical case management. We therefore need to caution the reader when extrapolating our results to other RBF settings.