Impact of the implementation of the WHO Safe Childbirth Checklist on essential birth practices and adverse events in two Brazilian hospitals: a before and after study.

OBJECTIVE: The WHO Safe Childbirth Checklist (SCC) is a promising initiative for safety in childbirth care, but the evidence about its impact on clinical outcomes is limited. This study analysed the impact of SCC on essential birth practices (EBPs), obstetric complications and adverse events (AEs) in hospitals of different profiles.
DESIGN: Quasi-experimental, time-series study and pre/post intervention.
SETTING: Two hospitals in North-East Brazil, one at a tertiary level (H1) and another at a secondary level (H2). PARTICIPANTS: 1440 women and their newborns, excluding those with congenital malformations.
INTERVENTIONS: The implementation of the SCC involved its cross-cultural adaptation, raising awareness with videos and posters, learning sessions about the SCC and auditing and feedback on adherence indicators. PRIMARY AND SECONDARY OUTCOME MEASURES: Simple and composite indicators related to seven EBPs, 3 complications and 10 AEs were monitored for 1 year, every 2 weeks, totalling 1440 observed deliveries.
RESULTS: The checklist was adopted in 83.3% (n=300) of deliveries in H1 and in 33.6% (n=121) in H2. The hospital with the highest adoption rate for SCC (H1) showed greater adherence to EBPs (improvement of 50.9%;p<0.001) and greater reduction in clinical outcome indicators compared with its baseline: percentage of deliveries with severe complications (reduction of 30.8%;p=0.005); Adverse Outcome Index (reduction of 25.6%;p=0.049); Weighted Adverse Outcome Score (reduction of 39.5%;p<0.001); Severity Index (reduction of 18.4%;p<0.001). In H2, whose adherence to the SCC was lower, there was an improvement of 24.7% compared with before SCC implementation in the composite indicator of EBPs (p=0.002) and a reduction of 49.2% in severe complications (p=0.027), but there was no significant reduction in AEs.
CONCLUSIONS: A multifaceted SCC-based intervention can be effective in improving adherence to EBPs and clinical outcomes in childbirth. The context and adherence to the SCC seem to modulate its impact, working better in a hospital of higher complexity. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first study on the impact on essential birth practices and safety of the Safe Childbirth Checklist (SCC) in Brazil, which is a country with high maternal and neonatal mortality rates.

The study assesses the impact of SCC on adherence to essential birth practices and incidence of adverse events and severe complications in childbirth in hospitals with different levels of care complexity.

The participation of only two hospitals with different capacities limited the comparison with other studies involving the SCC.

Due to its quasi-experimental design, this study produces only moderate evidence on the challenges to the implementation and impact of SCC.

Introduction

Despite the increase in the number of women who deliver in health institutions,1–3 patient safety incidents, including adverse events (AEs), are common and require improvement.4 5 The most serious AEs are maternal and newborn death. Others, such as uterine rupture and trauma at birth, tend to be more frequent and are associated with important failures in the quality of care that can be prevented with evidence-based practices. Low adherence to essential birth practices (EBPs), which are those with proven effectiveness, efficiency and safety, increases the risk of unnecessary interventions and harm, resulting in more costs and a negative experience for the families involved.6

To address these safety concerns during facility-based childbirth, the WHO has developed the Safe Childbirth Checklist (SCC), a tool that synthesises the evidence-based practices that must be offered before, during and after delivery.7 The checklist contains 29 items with succinct reminders to prevent, detect and treat the main causes of maternal death (haemorrhage, hypertensive diseases and infection) and foetal death due to inadequate delivery assistance and neonatal deaths (asphyxia, infection and prematurity). Based on the ‘SCC Collaboration’ initiative, the WHO recommended its adaptation and use worldwide as well as additional studies that assess the barriers and facilitators of the effective use of the SCC and its effects on the quality and safety of childbirth care.7 8

Available evidence has demonstrated that the SCC is effective in increasing EBPs,9–16 but fewer studies have analysed its impact on clinical outcomes and the findings are mixed.11 16–18 In a single hospital in Namibia, a quality improvement project based on SCC has been successful in increasing EBPs and reducing perinatal mortality, mainly by decreasing stillbirths.11 A reduction in neonatal mortality was also found in two other studies conducted in India17 and in Kenya and Uganda,18 which used a package of interventions that included the SCC. However, a large randomised study of 60 pairs of institutions in India showed that a coaching-based SCC programme also increased adherence to EBPs, but had no effect on any of the measured clinical outcomes.16 These inconsistent results signal the importance of the implementation context of implementing checklists and the required supporting environment to make them successful. They also reflect the need for studies of other implementation approaches and AEs not previously studied. This study aimed to analyse the impact of the SCC on adherence to EBPs and on the incidence of AEs and severe complications in hospitals of different levels of complexity in Brazil.

Methodology

Study design and context

The study design was a quasi-experimental pre/post intervention time series. It was developed in the context of the Safe Childbirth Project (approval protocol number 1 562 300/2015), an initiative for the implementation of WHO SCC in hospitals in Brazil and Mexico, which was part of the ‘WHO SCC Collaboration’. The validation of indicators used in the project,19 20 the descriptive baseline results4 and the process of adapting the SCC for Brazil21 have been previously published. The present study tests the hypothesis that the adapted SCC can improve the quality of care during childbirth, both in processes and clinical outcomes.

The study was carried out in a state in the northeast region of Brazil, which stands out for having a maternal mortality ratio of 64.3 maternal deaths per 100 000 live births and early neonatal mortality of 8.6 per thousand live births. In Brazil, these rates are 57.9 and 9.5, respectively.22 Two public hospitals linked to a Federal University participated: H1, a referral centre for high-risk births (tertiary level), which is located in the capital of a state in the northeast of Brazil and performs an average of 11 births/day; H2, low-risk hospital (secondary level), located in the interior of the state and performs an average of six births/day.

The clinical staff for obstetric care in the participating facilities included 60 gynaecologists and obstetricians in H1 and 30 in H2 as well as 45 specialised midwifery nurses in H1 and 59 in H2. Regarding newborn care, there were 51 paediatricians and neonatologists in H1 and 30 in H2. The number of nurses specialising in neonatology was 23 in H1 and 2 in H2. The number of beds for maternal and neonatal care included 88 beds for gynaecology and obstetrics and 40 beds for neonatology in H1 and 41 beds and 22 beds, respectively, in H2.

The number of births during 2015 and 2016 in H1 was 4147, of which 1603 (38.6%) were vaginally delivered and 2544 (61.3%) were by caesarean section. In H2, the total number of vaginal deliveries in the same period was 1239 (60.2%) and 819 (39.8%) deliveries were by caesarean section.

Intervention

The intervention for implementing the SCC was developed through extensive discussions with professionals from both hospitals for the cross-cultural adaptation of the WHO SCC to the Brazilian context, using nominal group techniques (three meetings), consensus conference (two conferences), pilot study and interview with professionals. The two hospitals in the study implemented the SCC adapted for Brazil and incorporated it into the medical records of all patients admitted for childbirth.21 The approved version included the 29 items from the original checklist and 20 new items. Justifications for Caesarean section and episiotomy, delayed clamping of the umbilical cord and care for the newborn (such as administration of vitamin K, vaccines and diagnostic tests) were some of the items added to the SCC (see online supplemental file 1 in Portuguese).21

The implementation of the SCC was carried out by the Patient Safety Units of the participating hospitals with the support of the study researchers. The intervention included training professionals to adapt and implement the checklist, learning sessions to use the checklist and definition of those responsible for completing the checklist and monitoring the implementation. In addition, simulations of using the checklist were carried out, along with the production of posters and explanatory folders, featuring videos on television media, auditing and feedback on adherence indicators.

Population

The study included all deliveries performed at the two participating hospitals between July 2015 and August 2016, excluding cases of newborns with congenital malformations to avoid overestimation of AEs.

The sample consisted of 30 medical charts every 2 weeks for 1 year, 6 months before and 6 months after the SCC implementation. It is known that random samples with successive measurements of 30 cases are considered feasible and useful for quality monitoring and decision-making in health services.23 The sample size per hospital was 720, representing a total of 1440 evaluated deliveries. The cases were selected by systematic random sampling.

Variables

The measures used to assess the level of the SCC implementation by hospital were the percentage of deliveries adopting SCC and the percentage of items and pause points filled out on the checklist.

EBP indicators were evaluated in simple and compound form. Four indicators of EBPs were evaluated for the woman (use of partogram, adherence to the antihypertensive protocol, adherence to the magnesium sulphate protocol and administration of oxytocin in the first minute after delivery) and three of EBPs for the newborn (timely clamping of the umbilical cord, skin-to-skin contact after birth and breastfeeding in the first hour). The simple indicators were aggregated into three composite measures: average percentage of compliance with four EBPs for the woman; average percentage of compliance with three EBPs for the newborn and average percentage of EBPs compliance in general (seven EBPs).

The analysis of the adherence to the antihypertensive and magnesium sulphate protocols was applied to all women in the sample and was considered as compliant when these drugs were used appropriately for the clinical indication as well as not used in the absence of indication. The classification of appropriate use of antihypertensive medications and magnesium sulphate was performed according to the clinical criteria established by WHO and the International Society for the Study of Hypertension in Pregnancy.24–26

Outcome indicators included measures of severe maternal morbidity and AEs. The composite indicator of the delivery rate with severe complications was calculated for the main causes of maternal mortality in Brazil and in the world: severe acute hypertension; eclampsia and obstetric haemorrhage.27–30

For the AEs, we used the indicators proposed by Mann et al:5 Adverse Outcome Index (AOI), which measures the incidence of deliveries with one or more maternal and neonatal AE; Weighted Adverse Outcome Score (WAOS), which measures the severity of AEs in relation to the total number of deliveries and Severity Index (SI), which is the score of the sum of severity scores of births with AEs divided by the total number of births with AEs.5 The WAOS and SI severity scores were decided on through a consensus process carried out by the American College of Obstetricians and Gynaecologists Committee on Quality Improvement and Patient Safety, which attributed a weighted score to each measure that represented the severity of the AE. It was predetermined that the sum of the scores of all other outcomes could not be greater than the score for a maternal death (750 points). The individual scores for the 10 AEs were: 750—maternal death; 400—intrapartum or neonatal death >2500 g; 100—uterine rupture; 65—maternal admission to the Intensive Care Unit (ICU); 60—birth trauma; 40—return to operating/delivery room; 35—admission to Neonatal Intensive Care Unit >2500 g & for >24; 25—Apgar <7 at 5 min; 20—blood transfusion; 5—3rd or 4th degree perineal laceration.5 The AOI, WAOS and SI are measures recommended by the National Health Surveillance Agency of Brazil.31 In addition to these measures, the AEs that make up the AOI were evaluated as two other composite indicators: percentage of deliveries with maternal AE and percentage of deliveries with neonatal AE.

Data collection

After training and a pilot study to validate the indicators in both hospitals,19 20 the data were collected with a prospective review of medical records. The reliability of the instrument during the pilot study in Brazil20 showed Kappa indices with substantial agreement (>0.76) for most indicators and, when not, adjustments were made for greater clarity. The pilot study was carried out in the first and second week of July 2015. The pilot study cases were not part of this study.

Data analysis

A descriptive analysis of maternal age, type of delivery and length of stay was performed. Percentage estimates of filling in the SCC (by items, by pause point and in general), adherence to EBPs, the incidence of severe complications and AEs and severity of AEs were calculated. The graphical representation of the improvement of the composite EBPs and AE severity indicators (WAOS) was performed with a statistical control graph.

All indicators were stratified by hospital. EBP indicators and clinical outcomes were compared in an aggregated and longitudinal way before and after the intervention with SCC. The improvement estimate after the intervention was calculated by means of absolute improvement (difference between the levels of compliance before and after the checklist) and relative improvement (ratio between the absolute improvement and the possible improvement space). The statistical improvement test performed was the unilateral Z test of the difference between the proportions (for the composite indicators of EBPs, complications and AEs) and the difference between the means (for the WAOS and SI indicators). For all these analyses, the level of statistical significance of 5% was considered.

Patient and public involvement

Patients were not directly involved in this study as data collection was based only on medical records and researchers ensured the confidentiality of data for the institutions and patients involved.

Results

Characterisation of women, mode of delivery and length of stay

The quality of care at 1440 births and their clinical outcomes in the mothers and newborns involved were evaluated. Women seen at the tertiary hospital (H1) had a longer hospital stay (average of 3.4 days and SD 3.2) and a higher frequency of Caesarean sections (67.5%; p<0.001) than women seen at the secondary hospital (H2) (hospital stay of 2.5 days and 41.0% of Caesarean sections). The average age of women did not vary between hospitals, being 26.1 years in H1 and 25.4 in H2.

Compliance with filling the checklist

As shown in table 1, the rate of adoption of the adapted SCC (percentage of deliveries in which SCC was used) was significantly higher in H1 (83.3%; 300 deliveries) than in H2 (33.6%; 121 deliveries), with no difference in the level of completion between the items from the SCC adapted for Brazil and the items from the original SCC (83.3% in H1 and 31.7% in H2). Among the births that adopted the checklist, the percentage of completion of all items was 38.1% in H1 and 22.9% in H2 (p<0.001). The level of completion of the checklist was significantly higher in H1 than in H2 for three of the four pause points of the SCC. In both hospitals, the moment of admission was the pause point with the highest completion (55.3% in H1 and 57.9% in H2), and the discharge pause point was the one with the lowest completion (17.6% in H1 and 5.9% in H2).

Table 1

Frequency of use and filling (%) of the SCC by pause points and items, stratified by hospital. Brazil, 2015 and 2016

Pause points	SCC items	H1 (N=360)% (n)	H2 (N=360)% (n)	P value*
	Deliveries adopting of adapted SCC	83.3 (300)	33.6 (121)	<0.001†
On admission	Items from the original SCC
	Does the pregnant woman need to be referred?	11.7 (35)	35.5 (43)	<0.001†
	Was the partogram initiated?	53.3 (160)	65.3 (79)	0.025†
	Does the pregnant woman need to take an antibiotic?	55.3 (166)	37.2 (45)	0.001†
	Does the pregnant woman need to take magnesium sulphate?	53.0 (159)	35.5 (43)	0.001†
	Does the pregnant woman need to take an antiretroviral?	48.7 (146)	34.7 (42)	0.009†
	Were there availability of hand*washing material and gloves for each vaginal examination?	69.0 (207)	83.5 (101)	0.002†
	Was the presence of a companion during the delivery encouraged?	68.0 (204)	82.6 (100)	0.002†
	Will the pregnant woman or companion ask for help during labour if necessary?	65.3 (196)	81.0 (98)	0.002†
	Items added‡
	Did the woman bring her prenatal card?	69.7 (209)	86.8 (105)	<0.001†
	Does the pregnant woman need to take an antihypertensive?	59.0 (117)	37.2 (45)	<0.001†
	Total pause point 1	55.3	57.9	0.250
Just before pushing or before Caesarean	Items from the original SCC
	Does the pregnant woman need to take an antibiotic?	21.0 (63)	23.1 (28)	0.629
	Does the parturient need to take magnesium sulphate?	19.0 (57)	16.5 (20)	0.553
	Was there essential material near the bed and preparation for the delivery confirmed? (for the pregnant woman)	24.3 (73)	31.4 (38)	0.136
	Was there essential material near the bed and preparation for the delivery confirmed? (For the newborn)	71.0 (213)	33.1 (40)	<0.001†
	Was the assistant identified and ready to help during the delivery if necessary?	77.3 (232)	29.8 (36)	<0.001†
	Items added ‡
	Does the parturient show signs of needing a Caesarean?	24.0 (72)	22.3 (27)	0.712
	Does the parturient show signs of needing an episiotomy?	8.3 (25)	17.4 (21)	0.007†
	Does the parturient need to take an antihypertensive?	21.7 (65)	18.2 (22)	0.424
	Does the current professional have recent updated neonatal resuscitation qualifications (maximum 2 years)?	77.0 (231)	30.6 (37)	<0.001†
	Total pause point 2	38.2	24.7	<0.001†
Soon after birth–within 1 hour	Items from the original SCC
	Is the mother bleeding more than expected?	17.0 (51)	12.4 (15)	0.240
	Does the mother need to start on antibiotics?	17.7 (53)	13.2 (16)	0.265
	Does the mother need to start on magnesium sulphate?	17.7 (53)	11.6 (14)	0.122
	Does the newborn need to be referred?	57.7 (173)	13.2 (16)	<0.001†
	Does the newborn need antibiotics?	56.0 (168)	13.2 (16)	<0.001†
	Does the newborn need special care/monitoring?	61.0 (183)	14.0 (17)	<0.001†
	Does the newborn need to start on antiretroviral therapy?	55.3 (166)	13.2 (16)	<0.001†
	Was there skin*to*skin contact (if the mother and the newborn are well)?	60.3 (181)	21.5 (26)	<0.001†
	Was breastfeeding initiated in the first hour (if the mother and the newborn are well)?	61.7 (185)	17.4 (21)	<0.001†
	Will the mother/companion ask for help if there are any signs of danger?	58.3 (175)	20.7 (25)	<0.001†
	Items added‡
	Does the parturient need to take an antihypertensive?	18.0 (54)	11.6 (14)	0.105
	Was the cord clamped between 1 and 3 min?	65.0 (195)	20.7 (25)	<0.001†
	Was vitamin K administrated?	73.7 (221)	21.5 (26)	<0.001†
	Did the NB have an identification bracelet on?	72.3 (217)	22.3 (27)	<0.001†
	Total pause point 3	49.4	16.2	<0.001†
Before discharge	Items from the original SCC
	Is the mother’s bleeding controlled?	4.3 (13)	10.7 (13)	0.013†
	Does the mother need to take an antibiotic?	3.7 (11)	1.7 (2)	0.280
	Does the newborn need to take an antibiotic?	19.0 (57)	1.7 (2)	<0.001†
	Does the baby breastfeed correctly?	23.7 (71)	10.7 (13)	0.003†
	If the mother is seropositive, did the mother and the newborn receive enough antiretrovirals (ARVs) for a period of 6 weeks?	10.0 (30)	0.8 (1)	0.001†
	Were family planning options discussed with the mother?	4.0 (12)	11.6 (14)	0.003†
	Was the mother instructed on the follow*up of the baby after discharge and the warning signs to ask for help?	20.7 (62)	10.7 (13)	0.016†
	Items added ‡
	Did the NB show any signs of jaundice?	18.7 (56)	0.8 (1)	<0.001†
	Did the newborn perform blood group and RH factor tests?	24.3 (73)	5.0 (6)	<0.001†
	Did the newborn receive BCG vaccine?	24.0 (72)	12.4 (15)	0.008†
	Did the newborn receive hepatitis B vaccine?	24.3 (73)	11.6 (14)	0.003†
	Was the neonatal heel prick test performed?	22.3 (67)	2.5 (3)	<0.001†
	Was the newborn hearing screening performed?	19.0 (57)	2.5 (3)	<0.001†
	Was the red reflex examination performed?	21.7 (65)	8.3 (10)	<0.001†
	Was the tongue screening test performed?	19.7 (59)	1.7 (2)	<0.001†
	Was the screening for critical congenital heart defects performed?	22.3 (67)	2.5 (3)	<0.001†
	Total pause point 4	17.6	5.9	<0.001†
	Completion of all SCC items	38.1	22.9	<0.001†

H1, tertiary hospital; H2, secondary hospital.

The frequency of the items refers only to the items completed by the professional in the SCC, it does not equal the frequency of compliance with the practice.

*The denominator used to calculate the percentage of filling of the SCC was 300 in H1 and 121 in H2, which are equivalent to the total number of childbirths that adopted the checklist in each hospital.

†Variable with p<0.05.

‡Items added in the SCC adapted and validated for the Brazilian version.21

SCC, Safe Childbirth Checklist.

Variation in EBPs

Before the implementation of the SCC, both hospitals had low adherence to EBPs for the newborn (less than 18%) and greater adherence to EBPs for the woman (62.0% in H1 and 89.9% in H2), with few variations in longitudinal analysis with control charts (table 2).

Table 2

Point (%) and interval (±95% CI) estimates of EBP indicators, before and after the SCC implementation, 2015 and 2016

Indicator	Tertiary hospital (H1)				Secondary hospital (H2)
	BeforeN=360n (%; 95% CI)	AfterN=360n (%; 95% CI)	Absolute improvement (%relative improvement)*	p value†	BeforeN=360n (%; 95% CI)	AfterN=360n (%; 95% CI)	Absolute improvement (% relative improvement)*	P value†
Average percentage of compliance with four EBPs for the woman	62.0±2.1	79.2±1.8	17.2 (45.2)	<0.001‡	89.9±1.5	91.0±1.4	1.1 (11.0)	0.305
Use of partogram	15 (4.2±2.1)	169 (46.9±5.2)	42.7 (44.6)	<0.001‡	316 (87.8±3.4)	314 (87.2±3.5)	−0.6 (NA)	–
Adherence to the antihypertensive protocol§	279 (77.5±4.3)	332 (92.2±2.8)	14.7 (65.3)	<0.001‡	356 (98.9±1.1)	351 (97.5±1.6)	−1.4 (NA)	–
Adherence to the magnesium sulphate protocol§	269 (74.7±4.5)	295 (81.9±4.0)	7.2 (28.5)	0.019‡	336 (93.3±2.6)	349 (96.9±1.8)	3.6 (53.7)	0.013‡
Administration of oxytocin in the first minute after delivery	330 (91.7±2.8)	344 (95.6±2.1)	3.9 (47.0)	0.016‡	287 (79.7±4.2)	297 (82.5±3.9)	2.8 (13.8)	0.341
Average percentage of compliance with three EBPs for the newborn	17.9±2.6	1,9 (62.6±4.3)	44.7 (54.5)	<0.001‡	16.9±2.5	0,6 (18.8±3.0)	1.9 (2.2)	0.258
Delayed clamping of the umbilical cord	76 (21.1±4.2)	259 (71.9±4.6)	50.8 (64.4)	<0.001‡	10 (2.8±1.7)	26 (7.2±2.7)	4.4 (4.5)	0.003‡
Skin†to†skin contact after birth	26 (7.2±2.7)	198 (55.0±5.1)	47.8 (51.5)	<0.001‡	78 (21.7±4.3)	79 (21.9±4.3)	0.2 (0.3)	0.476
Breastfeeding in the first hour	91 (25.3±4.5)	219 (60.8±5.0)	35.5 (47.5)	<0.001‡	95 (26.4±4.6)	98 (27.2±4.6)	0.8 (1.1)	0.405
Average percentage of EBP compliance in general (seven EBPs)	43.1±1.7	72.1±2.3	29.0 (50.9)	<0.001‡	58.7±1.4	68.9±1.5	10.2 (24.7)	0.002‡

Empty cells represent measures that did not improve at the end of the study.

The frequency of the items refers only to the items completed by the professional in the SCC, it does not equal the frequency of compliance with the practice.

*Absolute improvement=p2–p1, where p2 is the percentage of compliance after the checklist and p1 the percentage before the checklist; Relative improvement = (p2–p1) / (100-p1) * 100, quotient between the absolute improvement and the possible improvement space existing before the checklist.

†In the composite indicators, the average of the previous percentages is presented; in the others, the absolute values (n) are presented.

‡Variable with p<0.05. N: denominator; n: numerator.

§Adherence to antihypertensive and MgSO4 protocols were calculated considering the total number of women per sample. We classified as ’compliance’ when these drugs were used in the presence of clinical indication and when the drug was not used in the absence of clinical indication.

EBP, essential birth practice; SCC, Safe Childbirth Checklist.

With the intervention, an increase (p<0.001) in 17.2 (relative improvement of 45.2%) in EBPs for the woman (62.0% before and 79.2% after SCC) and 44.7 (relative improvement of 54.5%) in EBPs for the newborn (17.9% before and 62.6% after) was found in H1, representing an overall improvement of 50.9% in total of the EBPs. All EBP indicators in H1 showed a significant increase after the intervention. The EBPs that showed the most significant improvements (p<0.001) were compliance with the antihypertensive protocol (increasing from 77.5% to 92.2% after SCC) and timely clamping of the umbilical cord (increasing from 21.1% to 71.9% after SCC).

In H2, there was little variation in EBP indicators after the intervention, however, EBPs for the woman showed higher levels than in H1. The improvements were significant (p<0.05) for the indicators of adherence to the magnesium sulphate protocol (increasing from 93.3% to 96.9% after SCC) and timely clamping of the umbilical cord (increasing from 2.8% to 7.2% after SCC), resulting in a final increase in 24.7% in the EBPs compliance in general.

Impact of SCC on severe complications and AEs

The impact of implementing the SCC on health outcomes was more significant in the hospital of greater complexity where there was higher adoption of the SCC (H1), with a reduction (p<0.05) being detected both in the incidence of AEs (AOI decreased from 17.2% to 12.8% after SCC) and their severity (WAOS decreased 39.5% and SI reduced 18.4%). There was also a drop of 30.8% in the rates of deliveries with severe complications. In H2, the only significant improvement was in the rate of deliveries with severe complications, which dropped from 6.1% to 3.1% after SCC (relative improvement of 49.2%; p=0.05) (table 3).

Table 3

Point (%) and interval (±95% CI) estimates of outcome indicators and adverse events in childbirth care, before and after of the SCC implementation, 2015 and 2016

Indicator	Tertiary hospital (H1)				Secondary hospital (H2)
	BeforeN=360n (%; 95% CI)	AfterN=360n (%; 95% CI)	Absolute improvement (%relative improvement)*	p value†	BeforeN=360n (%; 95% CI)	AfterN=360n (%; 95% CI)	Absolute improvement (%relative improvement)*	P value†
Delivery rates with severe complications‡	91 (25.3±4.5)	63 (17.5±3.9)	−7.8 (30.8)†	0.005§	22 (6.1±2.5)	11 (3.1±1.8)	−3.0 (†49.2)	0.027§
AOI	62 (17.2±3.9)	46 (12.8±3.5)	−4.4 (25.6)†	0.049§	8 (2.2±1.5)	3 (0.8±0.9)	−1.4 (63.3)†	0.061
WAOS¶	17.3±7.1	10.5±4.3	−6.8 (39.5)†	<0.001§	1.4±2.2	1.1±2.2	−0.3 (20.4)†	0.189
SI	100.7±14.4	82.2±9.1	−18.6 (18.4)†	<0.001§	64.4±14.0	136.7±23.6	72.3 (NA)	–
Percentage of deliveries with adverse events	39 (10.8±3.2)	31 (8.6±2.9)	−2.2 (20.4)†	0.159	6 (1.7±0.4)	2 (0.6±0.8)	−1.1 (64.7)†	0.084
Percentage of deliveries with neonatal adverse events	27 (7.5±2.7)	19 (5.3±2.3)	−2.2 (29.3)†	0.113	2 (0.6±0.8)	1 (0.3±0.6)	−0.3 (50.0)†	0.274

Empty cells represent measures that did not improve at the end of the study.

*Absolute improvement=p2–p1, where p2 is the percentage of compliance after the checklist and p1 the percentage before the checklist; relative improvement = (p2–p1)/(p1) * 100, quotient between the absolute improvement and the possible improvement space existing before the checklist. A negative value of absolute and relative improvement indicates a reduction in complications and/or adverse events in the post-intervention period.

†In the composite indicators WAOS and SI, the average of the previous percentages is presented; in the others, the absolute values (n) are presented.

‡Severe complications considered: severe acute hypertension; eclampsia; and obstetric haemorrhage.

§Variable with p<0.05. N: denominator; n: numerator.

¶Total adverse events that make up WAOS=maternal death, intrapartum or neonatal death >2500 g, uterine rupture, maternal admission to the Intensive Care Unit (ICU), birth trauma, return to operating/delivery room, admission to Neonatal ICU>2500 g and for >24, Apgar<7 at 5 min, blood transfusion, third or fourth degree perineal laceration.5

AOI, Adverse Outcome Index; SCC, Safe Childbirth Checklist; SI, Severity Index; WAOS, Weighted Adverse Outcome Score.

Comparing the institutions, the final incidence of AEs in H1 (AOI of 12.8%), which decreased by 25.6% (p=0.049), was higher than in H2 (AOI of 0.8%), however, its SI after intervention significantly reduced and was lower than in H2. In the low complexity hospital (H2), AEs were less frequent, but more severe.

The control charts of the main outcome measures are shown in figure 1. The analysis of the H1 time series shows that there was a non-random and sustained improvement attributed to the SCC in EBP measures in general and to the WAOS; while in H2, the process remained stable, with no special cause of change towards an improvement in the quality of care.

Figure 1

Control charts of the longitudinal variation of the average percentage of compliance with essential birth practices (EBPs) in general and the average of the Weighted Adverse Outcome Score (WAOS) in the deliveries evaluated in each study institution, 2015 and 2016. ┼Total monitored EBPs=use of partogram, adherence to the antihypertensive protocol, adherence to the magnesium sulphate protocol, administration of oxytocin in the first minute after delivery, delayed clamping of the umbilical cord, skin-to-skin contact after birth and breastfeeding in the first hour. £Total adverse events that make up WAOS=maternal death, intrapartum or neonatal death >2500 g, uterine rupture, maternal admission to the Intensive Care Unit (ICU), birth trauma, return to operating/delivery room, admission to Neonatal ICU>2500 g and for >24, Apgar<7 at 5 min, blood transfusion, third or fourth degree perineal laceration.5 - The control charts were configured to identify violations of the following rules: points above or below the upper and lower control limits, 7 or more points above or below the centre line, 6 or more consecutive points on an up or down trend line, 2 of 3 consecutive points in zone A (between SD 2 and 3), 4 out of 5 consecutive points in zone B (between SD 1 and 2), and 14 consecutive points on an alternating line. SCC, Safe Childbirth Checklist.

Thus, it is observed in graph A1 that the compliance with EBPs before the checklist was below the average in all 12 initial measurements of H1. After the intervention, there is a sustained and above average improvement in all final measurements of the study. Regarding the WAOS measure of H1 (graph A2), it is observed that before the SCC, its value was higher than the average in 8 of the 12 points and, after the intervention, it remained below the central line in 10 of the 12 end points, having a series of 8 consecutive points below the average, representing a significant reduction in the severity of the AEs in H1.

Discussion

General study contributions

This study assesses adherence to the SCC and its impact on the quality of childbirth care in two hospitals with distinct implementation contexts in Brazil. We compare the extent of SCC implementation and use, levels of adherence to EBPs and clinical outcomes including AEs and childbirth complications.

The main results showed that the tertiary-level hospital that had the higher adoption and completion (H1) rates of the SCC also had the best performance in terms of EBPs and AEs. In this hospital, the intervention was significantly associated with a sustained improvement in adherence to EBPs (50.9% increase) and a 30.8% drop in the rate of deliveries with severe complications, further reducing severity measures of AEs (39.5% improvement in WAOS and 18.4% in SI). The secondary-level hospital with lower use of the SCC showed improvements in EBPs but did not show improvements in the incidence and severity of AEs, probably because the statistical power of the study was not enough to detect an improvement in this hospital, where the rates of complications and AEs are lower.

Even with the similar SCC implementation strategy in the two hospitals, completion of the SCC and adherence to EBPs was higher in H1, demonstrating that the enabling environment for the intervention may have been different between the two hospitals. The previous experience of using the SCC in H1, where a regular monitoring and feedback team on SCC indicators was established, may signal a more favourable environment for the Safe Childbirth Project intervention.32 This uneven improvement between the hospitals demonstrates the importance of contextual characteristics and of a systematic and continuous monitoring of adherence to the SCC.

The checklist and adherence to EBPs

We found an increase in adherence to the practices of using the partogram, management of hypertensive disorders and immediate care for the woman and newborn, which are consistent with findings from other studies linked to participants in the ‘WHO SCC Collaboration’.9–16 We believe that this is because the checklist functions as a brief reminder for the main evidence-based practices, encouraging communication and coordination between teams and, consequently, compliance with EBPs.7 9 11

Hypertensive disorders are a major cause of maternal morbidity and mortality in Brazil and worldwide.27–30 In H1, the increase in the adherence to the antihypertensive protocol was 65.3% and the adherence to the magnesium sulphate protocol was 28.5%, suggesting that the adoption of the checklist improved standardisation of care processes. In H2, there was also a significant improvement in adherence to the magnesium sulphate protocol, with levels close to the total in these two indicators. The greater and similar completion in the two hospitals of the items of the ‘On admission’ pause point, without variation between them, reinforces the SCC’s role in prompting adherence to EBPs for the management of hypertensive disorders in hospitals of different complexities.

In the third stage of labour, the administration of oxytocin in the first minute is the main intervention for the prevention of postpartum haemorrhage (PPH).6 33 34 In H1, the 47.0% increase in administration of oxytocin in the first minute after birth has contributed to reducing the incidence of PPH. Thus, the present study shows that the introduction of the checklist helped to increase adherence to EBPs and reduce the incidence of the main causes of maternal death: severe hypertensive disorders and haemorrhage.27–30

Immediate neonatal care practices that have increased with the intervention are strongly recommended in the current WHO guidelines because they produce better health and nutrition outcomes for the newborn.6 Several studies show that the delayed clamping of the umbilical cord (EBP present only in the SCC adapted for Brazil and in the current obstetric guidelines) prevents childhood anaemia35–37 and skin-to-skin contact improves the bond between woman and newborn and encourages breast feeding.6 The significant increase (54.5%) in EBPs for the newborn only in H1 may be explained by the greater adherence to filling in the items in the pause point ‘Soon after birth’. Similarly, a pre and postintervention study conducted in Ethiopia on SCC implementation found a 26.2% improvement in EBPs at this pause point.38 Thus, this positive impact of the checklist on neonatal care denotes the importance of this tool for reducing early neonatal mortality,11 17 18 which showed the slowest improvement during the era of the Millennium Development Goals.39

Impact of the checklist on the complications and AEs of childbirth care

Scientific evidence on the effects of SCC on adverse outcomes is challenging due to the relatively rare occurrence of maternal and neonatal mortality. Therefore, we examined the reduction in the rate of births with severe complications; and the reduction in the incidence and severity of AEs in the hospital with the highest adherence to SCC.

The BetterBirth trial, the largest randomised controlled trial on the SCC, showed puzzling results with an increase in adherence to EBPs15 16 but no reduction on maternal and neonatal mortality.16 In subsequent investigations, a reduction in the rate of stillbirths and in early neonatal mortality11 17 was observed after implementing the SCC along with a reduction in these rates among low-birthweight and preterm babies.18 Even using SCC-based interventions, the different implementation context of these studies may explain the divergent results, especially due to the drop in the levels of adherence and checklist use after coaching ceased in The BetterBirth trial.16

Since severe morbidity and AEs are more frequent than maternal and neonatal deaths and still constitute the direct causes of these deaths, the 30.8% reduction in the rate of deliveries with severe complications and the 25.6% reduction in the AOI in the hospital that made greater use of the SCC (H1) found after the intervention signals the importance of the SCC as a patient safety tool. Reducing AEs is one of the main objectives of the checklist, and more studies are needed to evaluate the impact on a variety of AEs that occur during childbirth.

It is also important to clarify that, in general, complications and AEs were greater in the tertiary hospital, a result already expected because H1 is the reference hospital for high-risk pregnancies in that region, its patients naturally exhibit greater likelihood of complications. Most studies do not distinguish the frequency of AEs by the level of complexity of care, making it difficult to compare our results with the findings in the literature.

Analysis of the severity of the AEs with the WAOS and SI indexes showed that, although the secondary hospital had lower AOI and WAOS, its SI was higher than the tertiary hospital, suggesting that, although less frequent, the outcomes were more severe at the secondary hospital. As the general adherence to the SCC was low in this institution, a more effective implementation approach may be needed in H2, so that the benefits found in the hospital of high complexity can also be reproduced in low-risk hospitals.

Thus, even though our intervention did not have optimal adherence levels, we believe that the reduction identified in H1 in the incidence and severity of AEs and in severe childbirth complications was related to the use of the checklist and the increase in EBPs. Other contextual factors in H1, such as a culture of quality improvement, continuous monitoring and feedback on indicators, and the involvement of the clinical leadership and the patient safety unit in the intervention may have contributed to this result. This reinforces the usefulness of using SCC as a strategy to improve the quality and safety of care during childbirth and demonstrates that the improvement in quality is strongly dependent on the context of health services.40

This result reinforces the idea that where the SCC is best implemented, the processes and results improve. This was found in the BetterBirth study, where it was identified that each additional SCC practice performed in care was strongly associated with a reduction in the chances of perinatal mortality and early neonatal mortality.41 Thus, effective implementation of the checklist is needed, including strategies for Improvement Science and Quality Management, team training and monitoring and evaluation with continuous feedback.7 10

Study limitations

This study may contain limitations related to registration bias, since the collection of data in medical records depends on the quality and regularity of the information recorded. This bias may have happened because it involves routine events in which data are simply not recorded or because data collection is related to the responsibility of professionals.

Another limitation may be related to the nature of the quasi-experimental design, where the absence of a control group may have confounded the analysis and variations in the indicators. As it was not possible to carry out a randomised controlled trial, this study produces moderate evidence on the impact of SCC.

The participation of only two hospitals with different care levels and the use of a single data source limited the comparison with other studies involving the SCC, which, in their majority, integrate multiple institutions and different data sources.

Conclusions

We found that SCC improves EBPs in a secondary-level and tertiary-level hospitals in Brazil, which is consistent with previous studies. We also demonstrated a reduction in severe complications and the incidence and severity of AEs in childbirth after SCC implementation. Despite the difficulties in filling out the checklist, the improvements found in the hospital with the highest adherence highlight the usefulness of this tool for the prevention and management of the main complications of childbirth, especially in a tertiary-level setting.

The persistent high maternal and neonatal morbidity and mortality rates require complex interventions to improve the quality of care. The SCC is one tool that can improve some aspects of safety and quality in childbirth but may require additional initiatives to achieve impact on mortality.

Finally, it is necessary to carry out new studies that evaluate the benefits of using the SCC in other processes and results of childbirth care, as well as studies that evaluate the influence of the context on the effectiveness of this tool.