Effect of a cluster randomised team training intervention on adverse perinatal and maternal outcomes: a stepped wedge study.

OBJECTIVE: To study the effect of an intervention based on Crew Resource Management team training, including a tool for structured communication, on adverse perinatal and maternal outcomes.
DESIGN: Stepped wedge.
SETTING: The Netherlands. POPULATION OR SAMPLE: Registry data of 8123 women referred from primary care to a hospital during childbirth, at ≥ 32.0 weeks of singleton gestation and with no congenital abnormalities, in the period 2012-15.
METHODS: Obstetric teams of five hospitals and their surrounding primary-care midwifery practices participated in the intervention. In total, 49 team training sessions were organised for 465 care professionals (75.5% participated). Adverse perinatal and maternal outcomes before, during and after the intervention were analysed using multivariate logistic regression analyses. MAIN OUTCOME MEASURES: Adverse Outcome Index (AOI-5), a composite measure involving; intrapartum or neonatal death, admission to neonatal intensive care unit, Apgar < 7 at 5 minutes, postpartum haemorrhage and/or perineal tear.
RESULTS: In total, an AOI-5 score was reported in 11.3% of the study population. No significant difference was found in the incidence of the AOI-5 score after the intervention compared with before the intervention (OR 1.07: 95% CI 0.92-1.24).
CONCLUSIONS: We found no effect of the intervention on adverse perinatal and maternal outcomes for women who were referred during childbirth. Team training is appreciated in practice, but evidence on the long-term impact is still limited. Upcoming studies should build on previous research and consider more sensitive outcome measures. TWEETABLE ABSTRACT: A cluster randomised team training intervention showed no effect on adverse perinatal and maternal outcomes for women referred during childbirth.

Introduction

Poor collaboration and communication are well‐known contributing factors to adverse health outcomes.1, 2 For example, the Joint Commission on Accreditation of Healthcare Organisations identified miscommunication as a contributing factor in two of every three sentinel events in 2014.3 Breakdown in communication is at risk, especially during handovers.4 A safe transfer of responsibility of patient care is even more challenging between care professionals working at different locations and between care professionals at different levels of care. In an obstetric team, obstetricians, midwives and nurses work together in a complex care setting and teams change within and between shifts. In addition, in settings with community midwives, referral situations are quite common. These referral situations have been identified as an important risk factor for deficits in team performance.5

Efforts to improve collaboration and communication in obstetric care often include team training interventions based on Crew Resource Management (CRM). CRM team training focuses on team performance and team coordination to promote safety and enhance efficiency.6 Instead of technical skills, in CRM team training cognitive and interpersonal skills are addressed such as teamwork, communication, decision making and leadership.7 A number of studies and reviews have been performed exploring the effectiveness of CRM team training in healthcare in general and obstetric care in particular. Findings indicate that classroom‐based CRM team training has had positive effects on participants’ reactions, attitudes towards teamwork and safety and team behaviours.6, 7, 8, 9, 10 However, there is a lack of evidence regarding the impact of CRM team training on patient outcomes.

Team training seems most effective when implemented as a bundled intervention that includes tools to support effective collaboration and communication.11 CRM team training programmes are being increasingly used to support implementation of checklists for structured communication, such as SBARR (situation, background, assessment, recommendation, read‐back). SBARR provides a common and predictable structure for an accurate information exchange in a brief and concise way.12 For example, this structure can be used during intrapartum referrals to enhance communication and mutual understanding between obstetric care professionals of a woman's situation. In addition, implementation strategies should be taken into account as well as ways to sustain effects in daily clinical practice.

The LOCoMOTive study (Local Obstetric Collaboration Onsite Multicentre Teamtraining effectiveness study) was designed to examine the effectiveness of an intervention in an obstetric care setting, based on CRM team training, including the SBARR tool for structured communication during intrapartum referrals. It is a longitudinal multicentre study with an explicit focus on interprofessional collaboration and communication. We aimed to study the effect of this intervention on adverse perinatal and maternal outcomes.

Methods

Design

We performed a stepped wedge study in the northwestern region of the Netherlands. A stepped wedge design is a one‐way crossover design in which different clusters switch from control to intervention conditions at regular intervals.13 To uphold a high quality of the intervention, this design was considered to be appropriate as the intervention periods in different clusters did not coincide. In the current study, five Local Obstetric Collaborations (LOCs) represent the different clusters. An LOC refers to a hospital and the surrounding primary‐care midwifery practices that are their preferred referring practices. The intervention was rolled out sequentially in all five LOCs at 3‐monthly intervals. The five LOCs were randomly assigned to a specific intervention period using an online randomisation tool. Figure 1 shows a scheme of the study periods following a stepped wedge design. Patients were not involved in the development of this research.

Figure 1

Scheme of the three study periods per Local Obstetric Collaboration (LOC) between 2012 and 2015, visualising the period before the intervention, intervention period (I) and period after the intervention.

Setting

In the Netherlands, obstetric care is provided by different groups of professionals working in three levels of care; primary, secondary and tertiary. Independent primary‐care midwives take care of women in the locality with low risks of pathology. Women are referred to secondary care, a general or teaching hospital, if risks of adverse fetal or maternal outcomes are high, or if complications arise during pregnancy or childbirth. In hospitals, obstetricians take over responsibility and care, in collaboration with nurses, clinical midwives and specialist registrars. More specialised care for complex and acute cases is provided in academic hospitals, representing the third level of care. Besides clinical relevance, there are elective reasons for referring women from primary care to a hospital, such as analgesia during childbirth. A consequence of this system is a high referral rate between care settings for women. In 2016, more than 85% of all pregnant women in the Netherlands started prenatal care provided by primary‐care midwives. Of all women, 35.2% (n = 59 210) were referred to a hospital during pregnancy and an additional 21.5% (n = 36 250) were referred during childbirth.14

Intervention

The intervention focused on intrapartum referrals between primary care and secondary or tertiary care and is based on classroom‐based CRM team training including the SBARR tool for structured communication. In every LOC, two complementary 3‐hour team training sessions were organised for the entire multidisciplinary team of obstetric professionals in a period of 3 months. The teams in every LOC consisted of primary‐care midwives, obstetricians, specialist registrars, nurses and clinical midwives. In order to include the entire team per LOC, multiple sessions were scheduled for each part of the training. In total, 49 team training sessions were organised for 465 care professionals in the five LOCs. The training served a dual purpose: (i) to develop the interpersonal knowledge and skills needed for optimum collaboration within the chain of obstetric care; and (ii) to improve interprofessional communication during intrapartum referrals by using SBARR for a structured communication procedure. The team training sessions were highly interactive and included plenary sessions, group discussions and role‐playing reflecting referral situations. As shown in Figure 1, the ‘Intervention period (I)’ represents the period in which the team training sessions were organised. The details of the intervention have been described elsewhere, including an overview of the programme of the team training sessions.15

Study population

We used national registry data from women who gave birth in one out of the five LOCs, at ≥ 32.0 weeks of singleton gestation and with no congenital abnormalities, with date of birth between 1 January 2012 and 31 December 2015. The hospitals assigned to the LOCs represent different types of hospitals. We included two general hospitals, two teaching hospitals and one academic hospital. Because the focus of the intervention was on collaboration and communication during intrapartum referrals, we selected only the women who were referred during childbirth from primary care to secondary or tertiary care.

Birth data were obtained from the Netherlands Perinatal (Perined) registry (Perined approval 15.51). The Perined registry is a national registry containing information on pregnancies, childbirth and (re)admissions until 28 days after birth. The database is formed by validated linkage of three profession‐based registries on a yearly basis. These registries involve the midwifery registry (LVR1), the obstetricians registry (LVR2) and the paediatricians and neonatologists registry (LNR).16 All data in the Perined registry are voluntarily recorded by care professionals during prenatal care, childbirth and the neonatal period.

Primary outcome measure: adverse outcome index

Adverse outcomes were assessed using the Adverse Outcome Index (AOI). The AOI was developed by Mann et al. to measure the frequency of births with one or more of ten predefined adverse events.17 In the Netherlands, a study was conducted to examine the validity and reliability of the AOI based on the Perined registry. Based on the results, a shorter version of the AOI has been recommended as suitable for the Dutch context and registry, involving five instead of ten adverse events.18 Currently, the AOI‐5 is a no core outcome set and is operationalised as a national obstetric quality indicator in the Netherlands. The AOI‐5 score is a dichotomous variable (0: no adverse events, 1: one or more adverse events). Box 1 shows the components of the AOI‐5 as well as inclusion and exclusion criteria. These criteria and components were used as outcome measures in the current study.

Secondary outcome measure

The secondary outcome measure involved an evaluation of the classroom‐based CRM team training sessions. After the second team training all participants were asked to complete a ten‐item questionnaire. The self‐reported questionnaire assessed the trainers, perceived strong and weak elements of the training programme, added value for everyday practice and the applicability of the content, especially regarding the use of SBARR during referrals. These results were used to illustrate the participants’ reaction to the intervention.

Statistical analysis

Descriptive statistics were used to study population characteristics before, during and after the intervention. These characteristics involved maternal age (continuous), gestational age (continuous), parity (categorised as primiparous or multiparous), ethnic background (categorised as Caucasian or non‐Caucasian) and socio‐economic status. Data for socio‐economic status (educational level, (un)employment and income) are based on the four‐digit postal code and were obtained from the Netherlands Institute for Social Research for the most recent year 2010 and categorised as low p75. In addition, descriptive statistics were used to study the participants’ reaction to the team training sessions as well as the variation of the AOI‐5 score per quarter in the study period 2012–15.

The incidence of the AOI‐5 before, during and after the intervention was determined and differences were tested with a chi‐square test, for the three study periods (before, during and after) as well as two study periods (before and after). To investigate the effect of the intervention on the AOI‐5 score, we performed multivariate logistic regression analyses comparing the study periods ‘after intervention’ with the ‘before intervention’. After an initial crude analysis, the first adjusted model added the variable ‘location’ to correct for inclusion of five LOCs (with LOC = 1 as reference). The second adjusted model included the variable location and the five variables related to the population characteristics; maternal age, gestational age, parity, ethnic background and socio‐economic status.

We tested for interaction (effect modification) between the study periods and the five LOCs to evaluate whether the intervention effect was different for the different LOCs. Finally, we performed two additional analyses; first comparing the study periods ‘during intervention’ and ‘before intervention’ to investigate a short‐term effect of the team training sessions, second excluding the LOC referring to an academic hospital. A significance level of P ≤ 0.05 was used as an indication of statistical significance. All statistical analyses were performed with SAS for Windows XP (version 9.3, SAS Institute Inc. Cary, NC, USA).

Results

The obstetric teams in five LOCs involved 465 care professionals. These teams consisted of obstetricians, including specialist registrars (n = 87), nurses (n = 120), clinical midwives (n = 46) and primary‐care midwives (n = 212). In total, 334 care professionals participated in the first team training (71.8%) and 351 care professionals participated in the second team training (75.5%). The overall participation rate varied between the professions involved: primary‐care midwives (92.5%), obstetricians (87.4%), specialist registrars (84.8%), nurses (63.7%) and clinical midwives (92.5%). On average, the care professionals rated the quality of the team training sessions a score of 7.7 (SD 0.9) on a ten‐point Likert scale. In addition, care professionals expected collaboration, communication in acute situations and quality of care to improve, respectively scoring these issues 4.1 (SD 0.8), 4.3 (SD 0.8) and 4.1 (SD 0.7) on a five‐point Likert scale.

In total, 42 611 women gave birth, at ≥ 32.0 weeks of singleton gestation and with no congenital abnormalities, between January 2012 and December 2015. Of these women, 8123 (19.1%) were referred during childbirth and were included in this study. The percentage of women referred during childbirth increased over the study periods, from 18.1% before the intervention to 20.4% in the period after the intervention. Table 1 shows the characteristics of these women. On average, maternal age was 30.4 years and gestational age was 39.5 weeks. The percentage of primiparous women before, during and after the intervention ranged from 65.3% to 68.8%. In addition, the percentage of Caucasian women ranged from 65.9% to 68.1%. Socio‐economic status showed the only significant difference between the three study periods (P < 0.01). During the intervention period, more women with low socio‐economic status (38.6%) were referred to a hospital compared with before (30.1%) or after (34.5%).

Table 1

Population characteristics of women referred during childbirth from 2012 to 2015

	Total (n = 8123)	Before intervention (n = 4298)	During intervention (n = 489)	After intervention (n = 3336)
Maternal age (years), mean (SD)	30.4 (4.8)	30.5 (4.8)	30.3 (4.9)	30.3 (4.7)
Gestational age (weeks), mean (SD)	39.5 (1.4)	39.5 (1.4)	39.6 (1.4)	39.5 (1.4)
Parity, n (%)
Primiparous	5387 (66.3)	2871 (66.8)	336 (68.7)	2180 (65.3)
Multiparous	2736 (33.7)	1427 (33.2)	153 (31.3)	1156 (34.7)
Ethnic background, n (%)
Caucasian	5454 (67.1)	2926 (68.1)	328 (67.1)	2200 (65.9)
Non‐Caucasian	2669 (32.9)	1372 (31.9)	161 (32.9)	1136 (34.1)
Socio‐economic status, n (%)
Low	2635 (32.4)	1295 (30.1)	189 (38.6)	1151 (34.5)
Medium	2909 (35.8)	1577 (36.7)	150 (30.7)	1182 (35.4)
High	2579 (31.8)	1426 (33.2)	150 (30.7)	1003 (30.1)

In total, the incidence of an AOI‐5 score was 11.3% (n = 916) of the study population. This means that in 11.3% of births one or more adverse events had occurred for women who were referred during childbirth. A broad range of the AOI‐5 was found per quarter during the study period. The AOI‐5 score ranged between 9.5% and 14.5%. Table 2 shows the percentage of births with an AOI‐5 score and the adverse outcomes before, during and after the intervention. No evidence of a difference was found between the three study periods (P = 0.52) nor between the two periods ‘after intervention’ and ‘before intervention’ (P = 0.29). In addition, no evidence of a difference was found between the study periods for the five separate adverse outcomes that comprise the AOI‐5 score. Neither was there a significant trend of the AOI‐5 score within the ‘before period’ or ‘after period’. Postpartum haemorrhage was the most prevalent adverse outcome, ranging between 6.5% and 7.2%.

Table 2

Adverse Outcome Index (and components) of women referred during childbirth from 2012 to 2015

	Total (n = 8123)	Before intervention (n = 4298)	During intervention (n = 489)	After intervention (n = 3336)	P valueb	P valuec
AOI‐5 scorea	916 (11.3%)	472 (11.0%)	52 (10.6%)	392 (11.8%)	0.29	0.52
AOI components, n (%)
Intrapartum or neonatal death	3 (< 0.1)	0 (0)	0 (0)	3 (0.1)	–	–
Admission to NICU	47 (0.6)	25 (0.6)	2 (0.4)	20 (0.6)	0.92	0.87
Apgar < 7 (5 minutes)	122 (1.5)	65 (1.5)	8 (1.6)	49 (1.5)	0.88	0.96
Postpartum haemorrhage	551 (6.8)	278 (6.5)	32 (6.5)	241 (7.2)	0.19	0.42
Perinatal tear	262 (3.2)	137 (3.2)	12 (2.5)	113 (3.4)	0.63	0.54

AOI‐5 score: A dichotomous variable defined as the percentage of births with one or more of the adverse events (0: no adverse events, 1: one or more adverse events).

P value: Two‐group comparison (before and after the intervention).

P value: Three‐group comparison.

Table 3 shows the results of the logistic regression analyses. No evidence of a crude difference of AOI‐5 score was found between the period ‘after the intervention’ compared with the period ‘before the intervention’ (OR 1.08, 95% CI 0.94–1.24). Neither of the two adjusted analyses showed evidence of a difference between these two periods. The adjusted odds ratio was 1.07 (95% CI 0.92–1.24). There was no evidence of an interaction between the three study periods and the five LOCs (P = 0.12). Repeating the analyses for the comparison between the periods ‘during intervention’ and ‘before intervention’ showed similar results. In addition, the final analysis without the academic LOCs showed no evidence of a difference in AOI‐5 score between the study periods. These results of the additional analyses are shown in the Supplementary material (Table S1 and Table S2).

Table 3

Intervention effect based on logistic regression analyses

	After intervention compared with before intervention
	OR	95% CI
Crude	1.079	0.936–1.244
Adjusteda	1.048	0.906–1.213
Adjustedb	1.069	0.924–1.237

Odds ratio adjusted for location.

Odds ratio adjusted for location, maternal age, gestational age, parity, ethnicity, socio‐economic status.

Discussion

Main findings

This study showed no effect on the incidence of adverse perinatal and maternal outcomes of a classroom‐based CRM team training intervention including the SBARR tool for structured communication during intrapartum referrals. In total, an AOI‐5 score was reported in 11.3% of the study population. No significant difference was found in the incidence of the AOI‐5 score for women who were referred during childbirth before, during or after the intervention. In addition, no significant difference was found between the three study periods for the five separate adverse outcomes that comprise the AOI‐5 score.

Strengths and limitations

Evidence on the impact of CRM team training on adverse patient outcomes is limited. As individual adverse events are rare, we used the composite measure AOI.17 Using this index lessened the need for an excessively large study population and allowed for statistical analysis of data from the five LOCs. By including five LOCs there is a risk of chance imbalance. However, we did not find significant differences in the baseline period and found no evidence of different effects for the five LOCs. Other studies outside the Netherlands have used the AOI score as well as modified versions.19, 20 We were able to use a modified version of the AOI score as outcome measure, operationalised to the Dutch context.18 No other Dutch study has yet been published including this AOI‐5 score.

Although the AOI has been used in other studies, different applications and compositions make it difficult to compare and interpret the results. For example, some reported mean scores in studies that used the original ten‐item AOI score, developed by Mann et al., varied from 2.5% to 9.4%.21, 22, 23 In the current study, the mean scores also varied considerably during the study period. Moreover, the AOI‐5 score is dominated by the most prevalent component; postpartum haemorrhage. As a result, the broad range of this composite measure makes it difficult to measure an effect before, during and after the intervention over a 4‐year study period. Mann et al. developed a scoring system to correct for variation in severity.17 However, there is currently no validated scoring system for the Dutch AOI‐5. Therefore, we presented the individual components, to increase understanding of the composition of the AOI‐5 in our study population. The prevalence of five components in this study population is in line with national data.14

Interpretation

Previous studies have reported mixed results on the impact of classroom‐based CRM team training on clinical outcomes. In obstetrics, Pettker et al. reported a decrease of the AOI score in a single academic centre, whereas Nielsen et al. found no effect on this outcome after classroom‐based CRM team training in a multicentre setting.21, 22 Various effects on patient outcomes have also been reported in other care settings. For example, a decline in operative technical errors and a non‐significant reduction in hospital stay have been reported in a study including a single operating theatre.24 In contrast, a CRM study in the trauma room of an emergency department reported an increased length of hospital stay and unaltered mortality rates.25 In the current multicentre study, no significant effect has been found of a classroom‐based team training intervention on the AOI‐5 score for women who were referred during childbirth.

In recent years, several studies have focused on simulation‐based CRM team training rather than classroom‐based training. In 2014, a review on the impact of simulation‐based CRM training showed that only five studies examined changes in patient outcomes despite an abundance of literature on simulation‐based education. These studies found some improvements in patient outcomes, including complications, length of stay and mortality rates. Still, only one study reported a significant impact of simulation‐based CRM training on mortality rates, a cohort study in a single hospital with no control group.26 In addition, Fung et al. found only two out of 12 studies demonstrating sustained reductions in adverse outcomes after a single simulation‐based CRM team intervention.27 Overall, classroom‐based and simulation‐based CRM interventions have limited effects on patient outcomes at best, which are often reported in single‐centre studies.

Despite a lack of clear evidence, several elements to optimise team training can be derived from the literature. These elements include, onsite repetitive training, low technological base, a highly realistic approach and integration of an interventional or implementation programme.10, 11, 28 The current study included these elements and has built on previous research on the impact of CRM team training incorporating critical success factors and an implementation strategy including the SBARR tool for structured communication during intrapartum referrals. This has resulted in a high response rate, in total 75.5% of the care professionals in the study population participated in the team training sessions. Also, the intervention received high ratings comparable to other studies. In addition, in a dynamic context for research the intervention was rolled out as planned, according to the stepped wedge design. Overall, the study has met the need for a multicentre longitudinal approach to investigate the effectiveness of a complex social intervention.

Yet, it is possible that the lack of an effect of classroom‐based CRM in this study may relate to the study population, context as well as intervention format. Women who are referred during childbirth from primary care to a hospital are at low risk of pathology at the onset of childbirth. Women can be referred if a complication arises; however, a referral is also indicated for elective reasons such as analgesia during childbirth. In addition, we found no significant difference in effect between the five LOCs. Still, we must acknowledge that there are local differences in patient safety practices and implementation strategies aimed to improve collaboration and communication. Moreover, intrapartum referrals are complex situations. Besides collaboration and communication, the transfer of patient care between care settings requires many other skills, such as clinical assessment and decision making. The intervention addressed generic skills of teamwork whereas the outcome measure involved specific adverse perinatal and maternal outcomes.29 The alignment between intervention and outcome measure could be improved by incorporating these specific measures in the team training, for example using postpartum haemorrhage in case scenario's.

Conclusion

We found no effect of classroom‐based CRM team training including the SBARR tool for structured communication on adverse perinatal and maternal outcomes for women who were referred during childbirth. As the adoption of CRM team training in healthcare settings continues to grow, evidence on the long‐term impact of CRM team training is still limited. However, the discussion of whether there are specific combinations of CRM‐inspired interventions that could have positive outcomes is likely to continue. This multicentre longitudinal study has built on previous research on CRM effectiveness. Further research should take into account context and practical considerations of CRM team training as well as sensitive outcome measures aligned with the intervention.

Disclosure of interests

None declared. Completed disclosure of interest forms are available to view online as supporting information.

Contribution to authorship

AR conducted the LOCoMOTive study and drafted the manuscript. AR performed statistical analyses in collaboration with ACJR and JWRT. AR, ACJR, MCdB, JWRT, CW, CJMdG and PWT made contributions to interpretation of the data and revisions to the manuscript; they also read and approved the final manuscript.

Details of ethics approval

The LOCoMOTive study was approved by the Medical Ethical Committee of the VU University Medical Centre in the Netherlands on 11 September 2013. The study protocol is in accordance with Dutch privacy regulations. Participating hospitals received approval from their local boards and all obstetric departments and independent midwifery practices provided written consent to access and use data from the Perined registry. In addition, the committee for research and ethics of Perined approved the study protocol (Perined approval 15.51) on 22 September 2015. The study is registered within the Dutch Trial Registry, record NTR 4256.

Funding

This work was supported by ZonMw, the Dutch Organisation for Health Research and Development, Grant Number 209020001. ZonMw had no role in the design of this study or execution, analysis and interpretation of results.

Table S1. Intervention effect based on logistic regression analysis, comparing study periods ‘during intervention’ and ‘before intervention’

Click here for additional data file.

Table S2. Intervention effect based on logistic regression analyses, including four Local Obstetric Collaborations (LOCs) (excluding the LOC referring to the academic hospital).

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.

Adverse Outcome Index (AOI‐5)a

Components

Intrapartum or neonatal death (≥ 2500 g or ≥ 37.0 weeks of gestation) Admission to Neonatal Intensive Care Unit (NICU) (≥37.0 weeks of gestation) APGAR <7 at 5 minutes Postpartum haemorrhage (≥1000 ml)b Third‐ or fourth‐degree perineal tear

Inclusion criteria

Singleton pregnancies ≥ 32.0 weeks of gestation

Exclusion criteria

Congenital abnormalities

AOI‐5 score: A dichotomous variable defined as the percentage of births with one or more of the adverse events (0: no adverse events, 1: one or more adverse events).

The component blood transfusion as defined by Mann et al.17 is replaced by postpartum haemorrhage.