Assessing Combined Longitudinal Mentorship and Skills Training on Select Maternal and Neonatal Outcomes in Rural and Urban Health Facilities in Malawi.

INTRODUCTION: Despite successful efforts to improve clinical access and skilled birth attendance in Malawi, it still faces high rates of maternal and neonatal mortality. In 2017, the UCSF-GAIN partnership began a nurse-midwifery clinical education and longitudinal mentorship program. While it has received positive reviews, it is unclear whether routinely collected indicators can assess such a program's impact.
METHOD: A longitudinal review of the Malawian DHIS2 database explored variables associated with maternal and newborn care and outcomes before and after the intervention. Data were analyzed using generalized estimating equations (GEE) to account for facility-level correlations over time.
RESULTS: Quality issues with DHIS2 data were identified. Significant changes potentially associated with the GAIN intervention were noted. DISCUSSION: The GAIN approach appears to be associated with positive trends in maternal and neonatal care. National summary databases are problematic, however, for evaluating targeted interventions and the provision of care to specific outcomes.

Background

There are not enough nurses in the world, particularly in low-resource settings, even as nurses are responsible for delivering care to eight out of every 10 patients globally (Uwizeye et al., 2018). Most pregnant women in the developing world receive most, if not all, of their care from a nurse or nurse-midwife. Amid their immense contributions to population health, nurses face many challenges including high caseloads, limited resources, and consequently unnecessarily high rates of poor outcomes in their patients (Bradley et al., 2015; Chimwaza et al., 2014). These can contribute to unnecessary morbidity and mortality, including for maternal and newborn patients. Despite global campaigns and continued calls for improvement, significant disparities between countries still exist in outcomes for mothers and babies, with those in sub-Saharan Africa (SSA) the most affected.

Despite being one of the poorest countries in SSA, Malawi has made significant strides in its efforts to improve the health outcomes of its population. The country spends a greater proportion of its public expenditure on health care than most others in the region, yet as of 2019 this was roughly US$30 per capita, making cost-effective interventions all the more critical (World Health Organization, 2022). In maternal health, the Malawian Ministry of Health (MoH) has achieved great success in increasing the rate of skilled birth attendance to 89% and facility-based deliveries to 91%, as well as removing user fees for maternity care across the country (Chansa & Pattnaik, 2018; Malawi Ministry of Health, 2012; Malawi National Statistical Office, 2017). However, challenges remain as both the neonatal death rate (35 per 1,000 births) and the maternal mortality ratio death rate (439 per 100,0000 births) are more than double the global averages (Malawi National Statistical Office, 2017). Once facility births become the norm, further improvements in outcomes require improvements in the quality of care at the facility level (Souza et al., 2014). For Malawi, this requires improving access to skilled nurses and midwives, as currently the country has only 4.4 nurses for every 10,000 people, far below the World Health Organization (WHO) recommendation of 50 per 10,000 (World Health Organization, 2020).

However, increasing staffing is only the first step toward significant reductions in negative outcomes. Therefore, interventions must improve the skills of clinicians, the quality of care they provide—both technical and material—and the experience of care received by patients (Tunçalp et al., 2015; White Ribbon Alliance, 2019). To do this, numerous studies and conceptual models highlight the importance of mentorship, especially when provided longitudinally, in tandem with clinical skills training in improving patient outcomes in SSA (Barnhart et al., 2020; Manzi et al., 2017; Prasad et al., 2019). For maternal care, this appears especially important, as trainings and coaching alone may have an immediate but limited impact on reducing maternal and neonatal negative outcomes, suggesting that the addition of longitudinal mentorship may be beneficial (Semrau et al., 2017; Street et al., 2021).

Global Action in Nursing

The University of California San Francisco (UCSF) Global Action in Nursing (GAIN) project is a nurse-designed education and mentorship model currently in use in Malawi, Liberia, and Sierra Leone. The model is designed to address maternal and neonatal morbidity and mortality by improving the quality of nursing and midwifery care through an integrated training/mentorship intervention. The intervention with 143 nurses and midwives at participating facilities in Malawi is the focus of this article and described in detail below. Throughout the process, the GAIN team engages with the District Health Management Team to share findings and ensure the success of knowledge dissemination.

While advanced clinical training and mentorship have been shown to be effective at increasing provider competencies, agency, and patient outcomes, it is less clear as to whether the impact of such programs are able to be measured via aggregate MoH reports. Therefore, this study sought to quantify the impact—if any—of the GAIN clinical training and long-term mentorship intervention on DHIS2 aggregates of routinely collected patient outcomes.

Method

Setting

This study is a longitudinal analysis of maternal and newborn health outcomes at partnering sites in the Neno and Blantyre districts of Malawi during the implementation of the GAIN clinical skills and mentorship intervention. For purposes of this study, sites are referred to as “health facilities” to encompass both the health centers and primary-level health facility designations. Neno District is a rural district in the southwestern area of Malawi with a total population of 138,000, most of whom are sustenance farmers. The district has two hospitals, one district hospital and one community hospital. Eleven surrounding health centers complete access to health care in the district. The hospitals are owned and operated by the Malawi MoH while the health centers are operated by both MoH and the Christian Hospitals Association of Malawi (CHAM) with support from a nongovernmental organization (NGO) partner, Partners In Health (PIH). On average these facilities saw 5,164 births per year, with ranges between the facilities from one birth at a peripheral health center to 194 per month at Neno District Hospital.

Conversely, the Blantyre district is urban and includes 28 peripheral health facilities. One of Malawi’s four central hospitals is located in Blantyre; while it is not technically part of the district, it serves as a referral hospital where complex cases are referred for higher level care. In both districts, primary maternity care is delivered by Registered Nurse Midwives (RNMs) and Nurse-Midwife Technicians (NMTs); in the rural district, midwives practice with the support of Community Midwife Assistants (CMAs). A busier and more urban district than Neno, the facilities averaged 10,413 births per year with the facilities ranging from 94 to 303 births per month.

Training and Mentorship Intervention

The GAIN model consists of a combined intervention that supplements an initial in-depth midwifery-focused short course on key areas of intervention followed by a year-long longitudinal mentorship. A sample of the training curriculum is provided in Table 1. The course was offered to RNMs, NMTs, and CMAs at partnering sites in the partnering districts. The trainings were open to all eligible maternity ward staff, and 15 to 20 were selected from across the participating facilities per course by a committee led by the District Nursing Officer from the MoH. The full-day courses lasted between 5 and 10 days in length and included content led by Malawian and international clinical experts. In both districts, the intensive training included midwifery care, neonatal care, and leadership which has been continually refined based on MoH priorities and participant feedback. For instance, in Neno District, an additional day of quality improvement training was included. After completion of the training, participating facilities received at least 1 year of on-site longitudinal mentorship by an experienced clinical nurse-midwife. Mentors were chosen among a pool of international applicants with more than 3 years of midwifery practice and a minimum of a bachelor’s degree in midwifery. The GAIN nurse-midwife mentors traveled to each health facility every week, to support the integration of the course material into local site-specific contexts, focusing on three primary areas: (a) the care of mothers and neonates during labor, delivery, and postpartum; (b) assessment of and assistance sourcing facility-level essential supplies and medications; and (c) overall quality improvement project support (currently only in Neno District). All training content and care provided were in accordance with Malawi national reproductive health service guidelines and protocols.

Table 1.

Sample GAIN Training Short-Course Intensive Training Curriculum for Nurses and Midwives at Participating Facilities.

Day 1: Introduction and leadership	Day 2: Leadership and normal newborn	Day 3: QI	Day 4: Nursery topics	Day 5: Nursery topics and closing	Day 6: Midwifery topics
Registration	Announcements and Recap	Announcements and Recap	Announcements and Recap	Announcements and Recap	Announcements and Pretest
Welcome and Introductions	Effective Communication and Documentation	Fundamentals of QI	Birth Asphyxia and Convulsions	Respiratory Disorders in Neonate	Introduction to WHO Safe Childbirth Checklist
Pretest and Consent forms	Principles for Triage and Task Allocation	How to Design and Implement a QI Project	Neonatal Resuscitation Updates	Fluids, Electrolytes and Nutrition	Laboring Mother and Using the Safe Childbirth Checklist
Course Overview	Advocacy and Resource Mobilization	Generating Change Ideas in QI	Neonatal Resuscitation Practice	Fluid and Feeding Calculations	Identification and Management of Fetal Distress
Country Health Profile	Care and Assessment of the Normal Newborn	QI Practice	Sepsis and Jaundice	Posttest	Preterm Births
Principles of Adult Learning	Care of Preterm Neonate	Closing and Daily Evaluations	Preterm/Low Birth Weight Practice	Closing remarks and certificates	Updates on Preeclampsia and PPH Management
Respectful Maternity Care (RMC)	Skills Practice: Newborn exam		Closing and Daily Evaluations	Closing and Daily Evaluations	Posttest
Team Based Care	Closing and Daily Evaluations				Closing and Daily Evaluations
Closing and Daily Evaluations

Note. GAIN = Global Action in Nursing; QI = quality improvement; WHO = World Health Organization; PPH = postpartum hemorrhage.

Study Population

The GAIN intervention was launched in Neno district in 2017. Training and mentorship started in two hospitals and four health centers and, over the course of the project, expanded to all eight health centers where births are attended. Four cohorts of training were offered between September 2017 and November 2019; mentorship began in January 2018. In Blantyre district, the first cohort of training began in January 2019 and mentorship began the following month in five of the district’s health centers. For evaluation purposes, the study included all DHIS2 data available for patients seen at GAIN participating health facilities during the study period.

Data Collection

The clinical outcomes reported in this study are based on the standardized Malawian Maternity Clinic-Facility Monthly Report. The MoH partographs are used for patient-level data management and key indicators are logged in the standardized clinical Maternity Registers. Examples of the partographs and Maternity Registers are seen in Appendix A. On a monthly basis, staff (often the nurse-in-charge) aggregates Maternity Register data into a monthly report. A copy of the monthly report is kept at each facility and a copy is sent to the District Health Office to be recorded in the national DHIS2 database, which the team was granted access to for this study. To establish a minimum of 1-year preintervention baseline, we used DHIS2 monthly clinical outcomes data from 2017 for Neno district and 2018 for Blantyre district. Postintervention data were based on the year/month combinations in DHIS2 for the 2018 rollout in the Neno district and 2019 for Blantyre district based on when facilities were included in the intervention. The data were accessed from DHIS2 in February 2020. As this study used aggregate data from participating facilities, it was impossible to directly engage the patients whose data were included in the design or implementation of the study.

To gather missing DHIS2 data, the GAIN Malawian nurse mentors worked with the relevant sites to access their retained hard-copy form, which was then used to update the master dataset. In addition, prior to analysis, each variable was assessed for significant outliers (more than two standard deviations from the mean), which may have resulted from data-entry errors. When identified, the study team again compared the DHIS2 data with clinical hard copies and corrected discrepancies. In six cases, the team was unable to identify facility hard copies, and so these facility-month pairings were removed from analysis for those facility/months.

Descriptive Variables

The variables found in the Malawian Maternity Register were included in the analysis for this study. A full list can be seen in Tables 2 to 4. During the study period, an updated version of the Malawi Maternity Register forms was released and rolled out at participating facilities at various points during 2018 and 2019. The new version allowed providers to report more than one maternal or neonatal complications, whereas prior instructions allowed the selection of only one (primary) complication. The updated forms also added new options for obstetric complications for fetal distress, premature labor, and retained placenta, as well as new emergency interventions. Due to the timing of these new forms relative to the GAIN intervention, these could not be compared pre/post and so were analyzed separately to better understand their trends in participating facilities. In addition, listed cesarean rates in the DHIS2 in Blantyre district do not reflect the population rate as GAIN-affiliated facilities are primary-level facilities (one has an operating theater which only services elective cases); patients requiring surgical delivery are referred to local hospitals.

Table 2.

Proportion of Births Associated With Select Maternal and Neonatal Variables as Listed in the Malawian Maternity Registry by Intervention Status With Associated P-Values for Neno (Rural) Facilities.

	Total	Intervention status		p-value
		Preintervention	Postintervention
Variable of Interest	M % (SD)	M % (SD)	M % (SD)
Number of deliveries at all facilities	43.5 (±43.7)	43.8 (±40.1)	43.3 (±46.3)	.21
Referral percent	16.4 (±20.7)	15.0 (±16.9)	17.5 (±23.1)	.87
Delivery route
Spontaneous vertex (vaginal)	94.0 (±9.2)	93.8 (±9.8)	94.2 (±8.9)	.81
Vacuum extraction	0.6 (±1.6)	0.6 (±1.4)	0.6 (±1.8)	.45
Breech	1.4 (±2.9)	1.4 (±2.5)	1.5 (±3.2)	.51
Cesarean section	3.7 (±7.9)	3.7 (±8.0)	3.7 (±7.9)	.92
Obstetric Complication
Ante-partum hemorrhage (APH)	1.0 (±2.7)	1.0 (±2.3)	1.1 (±3.1)	.45
Fetal distress[a]	0.4 (±1.8)	—	0.7 (±2.4)	—
Any other direct obstetric complication	7.1 (±10.7)	6.4 (±8.5)	7.7 (±12.1)	.98
Postpartum Hemorrhage (PPH)	2.2 (±3.7)	2.6 (±4.3)	1.9 (±3.2)	.078
Preeclampsia or eclampsia	0.7 (±2.5)	0.7 (±1.8)	0.8 (±2.9)	.49
Premature labor[a]	1.1 (±6.4)	—	1.9 (±8.4)	—
Obstructed or prolonged labor	7.1 (±8.9)	7.4 (±7.1)	6.9 (±10.1)	.096
Retained placenta[a]	0.1 (±0.8)	—	0.2 (±1.1)	—
Ruptured uterus	0.0 (±0.4)	0.0 (±0.2)	0.0 (±0.5)	.43
Postpartum sepsis	0.3 (±1.6)	0.3 (±1.1)	0.4(±1.9)	.61
Emergency obstetric care
Antibiotics	3.6 (±7.0)	4.4 (±8.2)	3.0 (±5.8)	.62
Anti-convulsants	1.3 (±9.3)	2.4 (±13.7)	0.6 (±2.9)	.18
Blood transfusion	0.2 (±0.5)	0.2 (±0.6)	0.1 (±0.5)	.14
Oxytocin	88.4 (±29.3)	81.6 (±35.4)	93.5 (±22.5)	.004
Manual removal of placenta	0.3 (±1.3)	0.3 (±1.1)	0.3 (±1.4)	.65
Neonatal complications
Total neonatal complications	16.3 (±13.7)	17.1 (±12.6)	15.8 (±14.5)	.085
Asphyxia	4.7 (±5.8)	5.1 (±5.6)	4.4 (±5.9)	.093
Other newborn complications	0.8 (±1.7)	0.5 (±1.1)	1.0 (±2.1)	.089
Prematurity	3.6 (±5.4)	3.8 (±4.7)	3.6 (±5.9)	.14
Low birthweight, less than 2,500 g	6.5 (±11.9)	6.7 (±11.9)	6.3 (±12.0)	.27
Sepsis	1.0 (±3.2)	1.4 (±4.0)	0.7 (±2.4)	.038
Perinatal death	0.6 (±1.3)	0.7 (±1.4)	0.5 (±1.1)	.26
Referral to care	16.4 (±20.7)	15.0 (16.9)	17.5 (23.1)	.87

These variables were only rolled out for inclusion in the DHIS2 at the end of 2018 for full use in 2019 and therefore do not have “preintervention” levels to record.

Table 4.

Generalized Estimating Equation (GEE) Models Showing Changes in the Prevalence of Select Maternal and Neonatal Variables With PE Reflecting the Change in Percentage of the Variables’ Occurrence, SE, and 95% CI Lower and Upper Bounds for Both Fixed and Random Effects Using an AR1 Correlation Structure.

Variable	Neno district								Blantyre district
	GAIN intervention (post compared with pre)				Increase by year				GAIN intervention (post compared with pre)
	PE	SE	95% CI		PE	SE	95% CI		PE	SE	95% CI
			Lower	Upper			Lower	Upper			Lower	Upper
Delivery method
Spontaneous vertex (vaginal)	−1.18	1.31	−3.75	1.39	1.13	2.02	−2.83	5.09	−0.33	0.81	−1.92	1.26
Vacuum extraction	0.30	0.37	−0.43	1.03	−0.19	0.31	−0.80	0.42	0.12	0.04	0.04	0.20
Breech	−0.20	0.29	−0.77	0.37	0.16	0.17	−0.17	0.49	−0.07	0.21	−0.48	0.34
Cesarean section	1.19	1.12	−1.01	3.39	−0.27	2.15	−4.48	3.94	0.87	0.78	−0.66	2.40
Obstetric complication
Antepartum hemorrhage (APH)	0.46	0.52	−0.56	1.48	−0.33	0.51	−1.33	0.67	0.17	0.16	−0.14	0.48
Any other direct obstetric complication	2.21	1.85	−1.42	5.84	−1.01	1.30	−3.56	1.54	−1.95	2.65	−7.14	3.24
Postpartum hemorrhage (PPH)	−0.29	0.62	−1.51	0.93	−0.35	0.39	−1.11	0.41	1.06	2.00	−2.86	4.98
Preeclampsia or eclampsia	−0.06	0.59	−1.22	1.10	0.13	0.32	−0.50	0.76	−1.67	0.73	−3.10	−0.24
Obstructed or prolonged labor	−3.43	1.12	−5.63	−1.23	0.61	1.18	−1.70	2.92	−1.81	0.88	−3.53	−0.09
Ruptured uterus	−0.07	0.04	−0.15	0.01	−0.07	0.04	−0.15	0.01	0.00	0.00	0.00	0.00
Postpartum sepsis	−0.14	0.20	−0.53	0.25	0.19	0.16	−0.12	0.50	0.15	0.11	−0.07	0.37
Emergency obstetric care
Antibiotics (Abx)	−1.51	1.89	−5.21	2.19	−0.71	2.41	−5.43	4.01	0.03	0.20	−0.36	0.42
Anti-convulsive (AC)	1.69	1.09	−0.45	3.83	−1.82	0.99	−3.76	0.12	0.16	0.14	−0.11	0.43
Blood transfusion (Tr)	−0.06	0.04	−0.14	0.02	−0.07	0.06	−0.19	0.05	0.02	0.01	0.00	0.05
Oxytocin (Ox)	−0.04	0.25	−0.53	0.45	0.05	0.15	−0.24	0.34	0.52	0.55	−0.56	1.60
Manual placenta removal (MRP)	−0.56	3.45	−7.32	6.20	8.91	4.36	0.36	17.46	−46.24	9.67	−65.19	−27.29
Neonatal complications
Total neonatal complications	0.03	1.28	−2.47	2.54	−1.12	0.93	−2.94	0.70	0.26	0.26	−0.25	0.77
Asphyxia	0.31	0.26	−0.20	0.82	0.15	0.15	−0.14	0.44	0.32	0.09	0.14	0.50
Other newborn complications	0.13	0.85	−1.54	1.80	−0.24	0.49	−1.20	0.72	−0.35	0.24	−0.82	0.12
Prematurity	−1.11	0.55	−2.19	−0.03	−0.33	0.37	−1.06	0.40	0.02	0.07	−0.12	0.16
Weight less 2,500 g	−0.13	2.00	−4.05	3.79	0.12	1.58	−2.98	3.22	−0.15	0.43	−0.99	0.69
Neonatal death	−0.39	1.91	−4.13	3.35	0.42	1.18	−1.89	2.73	−1.10	0.88	−2.82	0.62
Referral to advanced care	−0.58	0.47	−1.50	0.34	0.48	0.59	−0.68	1.64	−0.13	0.05	−0.23	−0.03

Note. PE = point estimates; CI = confidence interval; AR1 = auto-regressive correlation structure; GAIN = Global Action in Nursing.

Summary variables were generated to assess the total number of birthing mothers and neonates born at each facility to create the denominator from which to assess changes in the percentages of outcomes over time. The monthly maternity reports include aggregate numbers of births by delivery route and referrals but lack a single variable for total number of mothers treated. As these numbers naturally differ, the team ultimately used the variable “total births at this facility.” This was seen to most accurately account for all complications and subsequent care resulting from a home or in-transit birth which was treated and recorded at a facility, those at the facility, and those beginning at a facility and leading to a referral for advanced care. The denominator for all neonatal variables was based on a summation of all neonatal outcomes (live birth, stillbirth, and neonatal death). An additional composite variable was created to assess perinatal death, combining the numbers for neonatal deaths and fresh stillbirths (intrapartum deaths).

This study does not include an analysis of maternal or neonatal death. Most deaths originating in Blantyre District facilities are not recorded in the DHIS2 because complex perinatal cases are referred to the Central Hospital and any subsequent mortality, including deaths in transit, is registered in that facility’s statistics.

Data Analysis

All analyses were conducted using R v.3.6.0 (The R Foundation for Statistical Computing, 2019). Data were initially explored using univariate descriptive statistics, both overall and by each district for select maternal and neonatal variables, including the type of delivery, maternal complications, neonatal complications, clinical interventions given, and referral rates. Due to the differences in the underlying number of births each year and at the facilities, the variables were explored both in terms of raw numbers and their percentage of all births.

Subsequent bivariate analyses explored the impact of the GAIN intervention on maternal and neonatal outcomes stratified by the district. This decision was made a priori and sought to account for the potential impact of factors associated with the urban/rural divide (e.g., socioeconomic status, access, distance to facilities), the staggered implementation by district, and the fact that while the intervention occurred in all facilities in Neno district, it only was in a few in the larger Blantyre district.

Next, due to the longitudinal nature of the study, the presence of population-level as opposed to individual-level data, and likely presence of facility-level correlations in trainings, treatment practices, and subsequent outcome, we explored the impact of the GAIN intervention using General Estimating Equations (GEE). Analyses were run using the “GeePack” Package V1.3-1 in R. Each GEE model used the individual facility as the grouping unit of observations and an auto-regressive correlation structure (AR1) due to the hypothesis that issues impacting outcomes would be more likely correlated at closer timepoints than further ones. This fit with an a priori assumption that changes in practices and behavior occur over time. Each GEE model focused on the receipt of ongoing GAIN mentorship and training (yes/no) as the primary exposure of interest. To account for the impact of Malawian contextual factors such as changes to government protocols over time, we included year as a potential confounder in the GEE models for Neno district where GAIN has been working for 3 years. This was not done for the Blantyre models as the intervention has only occurred over the past 2 years and correlated with the yearly 2018/2019 separation. Each GEE model reports the point estimate change in the proportion of the outcomes’ occurrence, the standard error for this difference, and 95% confidence interval (95% CI). Statistically significant variables are highlighted in bold. The occurrences of fetal distress, premature labor, and retained placenta were not included in these analyses due to their absence in the preintervention period.

Ethical Considerations

Study approval was obtained from the Malawi National Health Science Research Committee (protocol #17/09/1906 and #2210) and the University of California San Francisco Committee for Human Research (protocol #17-23849 and #18-26842).

Patient and Public Involvement

In Malawi, GAIN works in concert with the NGOs GAIA Global Health (GAIA), Partners In Health (PIH), and members of the District Health Management Team (DHMT) were involved in the design, recruitment, and evaluation process of the study. Dissemination of results occurs on an ongoing basis directly through quarterly reviews and reports with the nurse-midwives and nursing leadership and to stakeholders through regular knowledge dissemination meetings including quarterly reports to district leadership.

Results

This study included data on 36,049 births from the start of 2017 to the end of 2019, through 474 monthly facility reports at 15 facilities in Blantyre and Neno. This is broken down by 16,838 pre-GAIN births (46.7%) from 213 (44.9%) facility reports and 19,211 births (53.3%) postintervention from 261 (55.1%) monthly reports. Over the course of the study period, GAIN participating health facilities saw an average of 76 births per month, although this varied significantly from a single birth at a rural referral health center to 303 at the Neno District Hospital (median 35).

Figures 1 and 2 show the percentage of births with a reported maternal or neonatal complication, respectively, by month, year, and district. Normal vaginal deliveries were the most common mode of delivery, representing 95.0% of all births at participating facilities during the study period, followed by cesarean sections (2.9%), breech births (1.3%), and vacuum deliveries (0.5%).

Figure 1.

Percentage of Maternal Complications by Year and District

Figure 2.

Percentage of Neonatal Complications by Year and District

Tables 2 and 3 show the proportion of births associated with the maternal and neonatal outcomes for Neno and Blantyre districts, respectively. In Neno, the proportion of births reporting neonatal sepsis almost halved (p=.038) following the intervention despite the ability for it to be recorded as one of the multiple other complications in the postintervention period. The proportion of births where the use of oxytocin was documented differed significantly in both districts but inversely with an 11.9% increase from 81.6% to 93.5% (p=.004) in Neno and a 29.5% decrease in Blantyre from 99.5% to 70.0% (p = .009). The data for the new maternal complications of fetal distress, premature labor, and retained placenta were not present in preintervention periods so their proportions are only listed for the postintervention period in Tables 1 and 2.

Table 3.

Proportion of Births Associated With Select Maternal and Neonatal Variables as Listed in the Malawian Maternity Registry by Intervention Status With Associated P-Values for Blantyre (Urban) Facilities.

	Total	Intervention status		p value
		Preintervention	Postintervention
Variable of Interest	M % (SD)	M % (SD)	M % (SD)
Number of deliveries at all facilities	174.2 (±47.2)	169.1 (±45.7)	179.5 (±48.4)	.33
Referral percent	31.0 (±13.0)	31.4 (±11.0)	30.6 (±14.9)	.57
Delivery route
Spontaneous vertex (Vaginal)	98.1 (±6.0)	98.3 (±2.3)	97.9 (±8.2)	.11
Vacuum extraction	0.4 (±1.0)	0.4 (±1.2)	0.3 (±0.8)	.56
Breech	0.8 (±0.9)	0.7 (±0.9)	0.8 (±0.9)	.33
Cesarean section	0.6 (±1.5)	0.3 (±0.9)	0.8 (±1.9)	.69
Obstetric complication
Ante-partum hemorrhage (APH)	0.9 (±1.2)	0.8 (±0.8)	1.0 (±1.4)	.97
Fetal distress[a]	0.8 (±2.6)	—	1.6 (±3.6)	—
Any other direct obstetric complication	12.8 (±7.7)	13.1 (±8.2)	12.4 (±7.2)	.76
Postpartum hemorrhage (PPH)	2.7 (±6.9)	2.2 (±4.9)	3.3 (±8.6)	.45
Preeclampsia or eclampsia	3.5 (±3.1)	4.3 (±3.7)	2.7 (±2.1)	.036
Premature labor[a]	1.0 (±2.6)	—	2.1 (±3.4)	—
Obstructed or prolonged labor	6.1 (±3.2)	6.9 (±3.6)	5.2 (±2.5)	.013
Retained placenta[a]	0.2 (±1.1)	—	0.5 (±1.6)	—
Ruptured uterus	0.0 (±0.0)	0.0 (±0.0)	0.0 (±0.0)	—
Postpartum sepsis	0.2 (±0.5)	0.1 (±0.4)	0.3 (±0.5)	.028
Emergency obstetric care
Antibiotics	0.7 (±1.2)	0.6 (±1.2)	0.7 (±1.1)	.15
Anti-convulsants	0.3 (±0.6)	0.3 (±0.5)	0.4 (±0.6)	.15
Blood transfusion	0.0 (±0.1)	0.0 (±0.0)	0.0 (±0.1)	.15
Oxytocin	85.0 (±34.6)	99.5 (±4.5)	70.0 (±44.5)	.009
Manual removal of placenta	0.4 (±3.0)	0.1 (±0.3)	0.6 (±4.3)	.85
Neonatal complications
Total neonatal complications	9.3 (±3.8)	9.1 (±4.2)	9.4 (±3.4)	.22
Asphyxia	2.5 (±1.6)	2.4 (±1.7)	2.6 (±1.6)	.55
Other newborn complications	1.2 (±1.2)	1.0 (±1.0)	1.4 (±1.3)	.2
Prematurity	2.1 (±1.5)	2.3 (±1.8)	1.9 (±1.2)	.49
Weight less 2,500 g	3.3 (±2.7)	3.2 (±3.3)	3.3 (±1.9)	.12
Sepsis	0.2 (±0.6)	0.2 (±0.7)	0.2 (±0.4)	.61
Perinatal death	1.1 (±1.8)	1.2 (±2.3)	0.9 (±0.9)	.55
Referral to advanced care	31.0 (±13.0)	31.4 (±11.0)	30.6 (±14.9)	.57

These variables were only rolled out for inclusion in the DHIS2 at the end of 2018 for full use in 2019 and therefore do not have “preintervention” levels to record and therefore only rates from 2019 are recorded.

Table 4 reports the results from the GEE models exploring the longitudinal impact of the GAIN intervention on the maternal and neonatal outcomes in Neno and Blantyre District. Only a slight increase in vacuum deliveries was seen in Blantyre. The cesarean section rate in Neno District was unchanged; the cesarean section at Blantyre facilities was also unchanged; however, this does not account for patients who underwent cesarean after referral to higher level facilities. In Blantyre district, a large decrease of 46.24% (95% CI: [27.29%, 65.19%]) was seen in the documented use of oxytocin, while in Neno district, it was seen to increase each year by 8.91% (95% CI: [0.36%, 17.46%]). With regard to neonatal complications, the intervention was associated with a 1.11% reduction in neonatal sepsis in Neno District (95% CI: [0.03%, 2.19%]) and a slight increase of 0.32% in “other complications” in Blantyre District (95% CI: [0.14%, 0.50%]). Finally, a minor reduction in patients being referred to advanced care was seen in the Blantyre district of 0.13% (95% CI: [0.03%, 0.23%]) from a 2018 average of 31.4%

Discussion

This study sought to assess whether the implementation of the GAIN program was associated with changes in maternal and neonatal outcomes as reported in DHIS2 data. Program participant evaluations and ongoing meetings with the Malawian MoH have documented the positive impact and confidence felt by nurse-midwife participants while caring for women during childbirth. While this article sought to quantify these changes—if any—utilizing advanced statistical techniques that can account for longitudinal changes and facility-level correlations in outcomes, the ability to do so was limited by the underlying nature of facility-level DHIS2 data. These findings are consistent with others exploring the use of DHIS2 and similar databases (Battle et al., 2019; Joseph Wu et al., 2018). However, the team sought to surmount the common issues of data-entry error by a robust tracking process that highlights the critical role of buy-in and participation at every level of the research process.

GAIN nurse mentors (authors I.M., O.J., R.M.) were able to detect several data errors within DHIS2 based on their knowledge of the facilities, which would not have been caught by external analytical staff. For instance, concerning increases in the incidence of ruptured uterus in several facilities in the DHIS2 data were flagged by clinical staff as potentially wrong, and then confirmed upon review of the facility records. Similarly, our analysis found statistically significant changes in documented use of oxytocin in emergency obstetric care, which was recorded in 87.5% of all births in the Neno and Blantyre districts over the course of the study. However, within the districts, Neno—after accounting for the GAIN intervention—saw an average increase of 8.9% each year while Blantyre District saw a decrease of 29.5%. Explorations of the underlying data showed that in 2019, multiple facilities reported single-digit to no instances of oxytocin use, which were confirmed in hard-copy registers with a review of program supply checklists showing no major stockouts of oxytocin. Insight from local staff (authors OJ, RM) suggests this likely resulted from discrepancies in the interpretation of the reporting requirements by the staff at the facilities. Because oxytocin is routinely given as part of active management of the third stage of labor, some staff recorded oxytocin use for all deliveries, while others recorded only when oxytocin was given to treat a postpartum hemorrhage as part of “emergency obstetric care”—the heading for variable selection in the form. Therefore, what appeared to be a significant research finding likely reflected a different individual compiling the monthly report and interpreting the variables in a slightly different way. This process of reconciling a national database with the experiences of frontline healthcare workers highlighted the importance of including frontline healthcare workers in data management and strengthening efforts. Including nurses and midwives in discussions around data strengthening is critical as they are primarily the ones who complete the initial forms and are most familiar with trends in their facilities. Similarly, this shows the importance of being able to access patient-level longitudinal data directly to reduce steps—and potential errors—in the data-entry chain as well as better assess the direct impact of programs designed to improve the quality of care.

The analysis also suggests that aggregate national databases, while valuable in many areas, are limited in their ability to detect targeted interventions such as those seeking to link changes in care to an outcome or similarly focused programs. For example, the register combines outcomes of hypertensive disorders of pregnancy (preeclampsia/eclampsia) and labor dystocia (obstructed/prolonged labor) into composite variables; perhaps a more apt indicator would be the percentage of preeclamptic women who become eclamptic or prolonged labors that lead to obstruction. As improvements in the quality of nursing and midwifery care become a primary driver of maternal and neonatal survival, it is essential that indicators sensitive to quality of care are included in data systems. This was partially reflected by the addition of new emergency interventions, such as antenatal corticosteroids and non-pneumatic anti-shock garment, in the 2018 register revision, as well as the addition of additional complications such as preterm labor and retained placenta. However, the aggregate nature of DHIS2 makes it made it impossible to link types of obstetric care to given complications to assess whether appropriate care was being given to individual patients. As DHIS2 data are used by policymakers to identify areas of concern, allocate funds, and prioritize interventions at a national level, these data remain useful and open new areas of inquiry and discussion (Dehnavieh et al., 2019).

Neonatal sepsis remains a significant health concern in SSA, and so it is interesting to note a significant decrease associated with the intervention in Neno district, where the GAIN project has been for a longer period, where an increase may have been more likely due to increased vigilance including regular temperature checks (Kortz et al., 2018; Li et al., 2020; Ranjeva et al., 2018). This change was not reflected in Blantyre, however, where a small but statistically significant increase in “other complications” was noted.

Finally, it is interesting to note a small but significant decrease in the percentage of patients referred to advanced care in Blantyre, a more urban district with easier access to high-quality referral facilities. The decrease of 0.13% accounted for roughly 14 women in 2019 or 130/100,000 women seen at the participating facilities. It is unclear whether this shift is a positive or negative one: some local stakeholders suggested this may be due to nurse midwives’ increased confidence in handling complications at primary-level facilities, although others suggested it could be due to transportation issues and/or patient preferences to avoid the busy central hospital. Again, aggregate data make it challenging to link referral rates to corresponding diagnoses and outcomes, and so it is impossible to report whether this change has any impact on the “right” patients being referred to the hospital.

Strengths and Limitations

As noted above, the reliance on DHIS2 facility-level rather than individual-level data limited our ability to fully assess the impact of the GAIN intervention, which other researchers have echoed as resulting from the data-entry chain (Battle et al., 2019; Kapito et al., 2019). However, we significantly reduced the potential for errors by hand-reviewing facility hard copies of the reports to ensure the data were as representative as possible. This has also led to an area for potential expansion in the GAIN project to work with partnering facilities on data strengthening to better identify, allocate for, and treat areas of concern as they appear. In addition, the exploration of numerous potential outcomes leaves open the possibility for spurious Type I errors resulting from multiple testing. While this cannot be excluded as a possibility, the variable associations found to be significant fit within a priori expectations and tended to track with prior research. Finally, although the statistically significant findings were often associated with relatively small clinical percentage shifts in outcomes, it may not be possible to see massive shifts across all indicators as not all measured complications are amenable to changes in nursing quality of care. These small percentage changes are also reflected in more than 12,000 births at the participating facilities, making shifts of even a percent able to impact the lives of 120 mothers and children.

Conclusion

Clinical skills training coupled with long-term bedside mentorship may help improve care in low-resourced settings. However, tracking these changes, identifying areas for improvement, and instituting effective change are limited when forced to rely on aggregated facility-level data, which can be subject to data-entry errors. The GAIN project is already working to identify ways to access patient-level data to better evaluate the effectiveness of training and mentorship on appropriate care when complications are identified, as well as mapping outcomes to the presence of critical supplies.