Relationship between non-technical skills and technical performance during cardiopulmonary resuscitation: does stress have an influence?

BACKGROUND: Non-technical skills, such as task management, leadership, situational awareness, communication and decision-making refer to cognitive, behavioural and social skills that contribute to safe and efficient team performance. The importance of these skills during cardiopulmonary resuscitation (CPR) is increasingly emphasised. Nonetheless, the relationship between non-technical skills and technical performance is poorly understood. We hypothesise that non-technical skills become increasingly important under stressful conditions when individuals are distracted from their tasks, and investigated the relationship between non-technical and technical skills under control conditions and when external stressors are present.
METHODS: In this simulator-based randomised cross-over study, 30 anaesthesiologists and anaesthesia residents from the VU University Medical Center, Amsterdam, the Netherlands, participated in two different CPR scenarios in random order. In one scenario, external stressors (radio noise and a distractive scripted family member) were added, while the other scenario without stressors served as control condition. Non-technical performance of the team leader and technical performance of the team were measured using the 'Anaesthetists' Non-technical Skill' score and a recently developed technical skills score. Analysis of variance and Pearson correlation coefficients were used for statistical analyses.
RESULTS: Non-technical performance declined when external stressors were present (adjusted mean difference 3.9 points, 95% CI 2.4 to 5.5 points). A significant correlation between non-technical and technical performance scores was observed when external stressors were present (r=0.67, 95% CI 0.40 to 0.83, p<0.001), while no evidence for such a relationship was observed under control conditions (r=0.15, 95% CI -0.22 to 0.49, p=0.42). This was equally true for all individual domains of the non-technical performance score (task management, team working, situation awareness, decision-making).
CONCLUSIONS: During CPR with external stressors, the team's technical performance is related to the non-technical skills of the team leader. This may have important implications for training of CPR teams. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

What is already known on this subject?

Previous data suggest that technical performance by a cardiopulmonary resuscitation (CPR) team is affected by the presence of external stressors. However, non-technical skills, including task management, situational awareness, communication and decision-making, are increasingly recognised as crucial elements for optimal team performance. It is unclear whether these non-technical skills are affected by stress, and if, in turn, this affects the team’s technical performance.

What this study adds?

In this randomised, cross-over simulation study, we found that non-technical skills were affected by external stressors. When stressors were present, better technical performance of the team was positively associated with better non-technical performance by the team leader. This may have important implications for training of CPR teams.

Introduction

Background

High quality cardiopulmonary resuscitation (CPR) is crucial for survival of patients in cardiac arrest.1 In this context, optimal technical performance of interventions is emphasised in resuscitation guidelines.2 3 However, coordinated performance of such interventions within CPR teams requires more than mastering the technical skills. Non-technical skills such as task management, leadership, situational awareness, communication and decision-making refer to cognitive, behavioural and social skills that contribute to safe and efficient team performance.4 Herein, the team leader plays a pivotal role in coordinating the team effort, and previous research has shown that absence of efficient leadership is associated with poor CPR performance.5

Relationship between non-technical and technical skills

Data on the actual strength of the relationship between the team leader’s non-technical skills and the technical performance of the CPR team are scarce.6 Moreover, it is unclear whether and how non-technical performance itself as well as the relationship between non-technical and technical performance are influenced by other factors that might interfere with CPR performance, such as the stress level. A more detailed understanding of the relationship between non-technical skills of the team leader and technical performance of the team during CPR may have important implications for training and clinical performance of resuscitation teams.

In a prior study, we demonstrated that additional external stressors during CPR adversely affect the technical performance of resuscitation teams regardless of the experience level of the team leader.7 We hypothesised that non-technical skills become increasingly important to maintain adequate CPR performance under stressful circumstances when individuals are distracted from their tasks.

Therefore, in this analysis, we determine whether non-technical performance is affected by the level of external stressors and whether specific domains of non-technical performance are differentially affected.

Methods

Study design

This study is part of a project which addressed effects of stress on technical and non-technical performance during CPR. Data describing the relationship between stress and the technical performance as well as the technical score itself have been described in detail previously.7

This study was performed at the clinical simulation centre of the VU University Medical Center in Amsterdam, the Netherlands (ADAM SimLab). The institutional review board was informed of the study; however, formal approval was not required according to Dutch law because no patients were involved.

The study was designed as a randomised cross-over investigation. Team leaders were assigned to lead two CPR scenarios. One scenario featured a patient with history of chest pain who presents with ventricular fibrillation (VF), while the other scenario was a patient who suddenly collapses with pulseless ventricular tachycardia (pVT). Both scenarios involved a shockable rhythm and required exactly the same treatment, allowing direct comparison of CPR performance. External stressors, consisting of a scripted family member who distracted the team leader at predefined crucial moments as well as a constant static radio noise at approximately 70 decibels, were randomly added to either the first or second scenario. Both, the sequence of scenarios (VF first vs pVT first) as well as the presence of stressors (scenario with external stressors first vs scenario without external stressors first), were randomised using a sealed envelope technique.

Setting and study subjects

Consultant anaesthetists or anaesthesia residents who assume the role of team leader in our hospital’s resuscitation team were approached to participate in this study. Ten first/second year residents, 10 fourth/fifth year residents and 10 consultant anaesthetists agreed to take part on voluntary basis with informed consent. Each participant was assigned the role of team leader in two simulated CPR scenarios. Their team consisted of three CPR-trained team members who are part of the research group (an anaesthesia resident, a nurse anaesthetist and a medical student) and who were instructed to act on command of the resuscitation team leader. Before the first CPR scenario, participants were familiarised with the full-scale patient simulator (SimMan, Laerdal Medical, Stavanger, Norway) and with the simulation environment. Participants (the team leaders) were informed that their performance would be evaluated in the context of a scientific investigation, but they were not aware of the specific aims of the study.

Measurements

Non-technical and technical performances were individually scored by two trained reviewers (LZ, MWK) using video recordings of the CPR scenarios. Non-technical performance was scored using the Anaesthetists’ Non-Technical Skills (ANTS) score.8 This framework is a validated scoring system for assessment of non-technical performance.8 9 While this system has initially been developed for anaesthetists, it is now also increasingly used in other medical subspecialties such as intensive care and prehospital emergency medicine.10 11

The ANTS system uses a total of 15 skill elements (figure 1). For each element, 1 to 4 points can be assigned, resulting in a total score ranging from 15 to 60 points. These skill elements are grouped into four domains of non-technical performance, which are Task Management (four skill elements, score range 4–16 points), Team Working (five skill elements, score range 5–20 points), Situation Awareness (three skill elements, score range 3–12 points) and Decision-Making (three skill elements, score range 3–12 points). When assessing the non-technical capabilities of a team leader, it is essential to assess his or her leadership performance. Since the flexible ANTS framework can be used to assess team leaders as well as team members, leadership is not a separate category but has been incorporated into the team working domain.12

Figure 1

Anaesthetists’ non-technical skills score.

Technical performance was assessed using a score developed by our research group for the purpose of this research project as described in detail previously because no validated instrument was available to assess technical performance during CPR.7 Briefly, the score was based on similar scoring protocols used in the aviation industry to rate pilot performance We identified all required technical tasks during CPR. Weights were assigned depending on relevance, and negative scores could be assigned for omission of crucial tasks such as defibrillation. The weighting process was refined in an internal and external audit process. The technical performance score represents the weighted sum of a number of technical elements that are considered crucial during advanced life support (table 1).7

Table 1

Items of the technical performance score

Item	Score (minimum to maximum)
Confirming and communicating of circulatory arrest, calling for help	0 to +4
Detection of VF/pVT	0 to +4
Timely first defibrillation	−4 to +4
Chest compressions, timely onset and quality	−4 to +6
Ventilation and airway management	0 to +3
Timely subsequent defibrillation	0 to +2
Timely resumption of chest compressions after defibrillation	0 to +4
Intravenous epinephrine timing and dose	−4 to +5
Diagnostic actions (interdisciplinary addressing of potentially reversible causes, blood sample, re-evaluation)	0 to +2
Total	−12 to +34

pVT, pulseless ventricular tachycardia; VF, ventricular fibrillation.

Statistical analysis

Results were analysed using the STATA 13.0 statistical package. ANTS scores were normally distributed as assessed by a Q-Q plot and the Shapiro-Wilk test.

Paired t-tests were used to compare differences in ANTS scores between the first and second scenario, and between scenarios with and without external stressors. In addition, to account for the cross-over design of the study, an analysis of variance (ANOVA) was performed in which the ANTS score was the outcome variable and in which the subjects, stress level and period were entered as factors in order to test and adjust for period effects.

We used Pearson product–moment correlation to examine whether there is a linear relationship between non-technical scores (ANTS) and technical performance scores, that is, whether high values of one score tended to be associated with high values of the other score and vice versa. Because the technical performance score was not normally distributed, the score was first transformed to a Gaussian distribution using a Box-Cox transformation with a transformation parameter (lambda) of 1.7, as described previously.7 To formally evaluate whether the association between technical and non-technical performance was affected by stress, a regression model was specified in which the null hypothesis of no interaction between stress and non-technical performance was tested. Herein, to account for non-independence of repeated observations per participant, a generalised estimating equation (GEE) for a normally distributed dependent variable (transformed technical performance score) was used. In a second analysis, the model was also adjusted for a period effect, which had been observed in the ANOVA (see Results section). For all hypothesis tests, a p value<0.05 was considered statistically significant.

Sample size calculations for the project had been performed with SAS 9.2 based on the technical performance score as described in detail previously.7 Briefly, calculations using different plausible estimates of mean differences, SD and correlations between repeated measurements suggested that a sample size of around 20–30 pairs would be necessary to detect a mean difference of 5 in the technical performance score between stress and no-stress conditions at a two-sided alpha level of 0.05 and with a power of 0.8. We therefore empirically chose a sample size of 30 pairs.

Results

All participants completed both CPR scenarios. Mean (±SD) age of participants was 34.8±6.1 years, and 17 of the 30 were male.

Effects of stress on non-technical performance

Non-technical performance scores were significantly lower when external stressors were present compared with control scenarios. In the group of participants who were not confronted with stressors during the first scenario, adding a stressor to the second scenario was associated with a significant decrease in ANTS scores (44.7±3.3 (mean±SD) vs 42.6±4.1, mean of the paired differences −2.0, 95% CI −3.5 to −0.6, paired t-test p value=0.011, figure 2). In the group of participants who were presented the scenario with external stressors first, ANTS scores significantly increased in the second scenario (40.7±4.6 vs 46.5±4.5, mean of the paired differences 5.8, 95% CI 3.0 to 8.6, paired t-test p value<0.001, figure 2). Overall, significantly lower ANTS scores were observed in scenarios where external stressors where added, with and without adjustment for period effects (41.7±4.4 (mean±SD) vs 45.6±4.0, mean difference 3.9, unadjusted 95% CI 2.3 to 5.6, adjusted 95% CI 2.4 to 5.5, unadjusted paired t-test p value <0.001, adjusted ANOVA p value<0.001). This effect of external stress was observed across all four domains of the ANTS score (table 2).

Figure 2

Non-technical performance scores (Anaesthetists’ Non-Technical Skills (ANTS) scores) during cardiopulmonary resuscitation scenarios for participants in whom external stressors were added to the second scenario (A) or to the first scenario (B). Individual ANTS scores of each participant are connected by a line. In (A), ANTS scores significantly decrease when external stress is added in the second scenario (p=0.011), while ANTS scores increase on (B) in the second scenario without external stressors (p<0.001).

Table 2

Relationship between external stressors and ANTS scores

	Without additional stressors[*] (mean±SD)	With additional stressors[*] (mean±SD)	Difference between means[†] (95% CI)	p Value†
Total ANTS score	45.6±4.0	41.7±4.4	3.9 (2.4 to 5.5)	<0.001
Domains
Task management	12.4±1.6	11.5±1.6	0.9 (0.3 to 1.5)	0.003
Team working	15.4±1.7	14.2±1.5	1.3 (0.7 to 1.8)	<0.001
Situation awareness	9.5±0.8	8.8±1.0	0.7 (0.2 to 1.1)	0.004
Decision-making	8.3±1.4	7.2±1.0	1.1 (0.6 to 1.5)	<0.001

*Unadjusted summary statistics

†Adjusted for period effects

ANTS, Anaesthetists’ Non-Technical Skills.

ANTS scores were slightly higher in the second scenario than in the first scenario (44.6±4.7 vs 42.7±4.4, and we did observe a significant period effect (adjusted mean difference 1.9, 95% CI 0.4 to 3.4, ANOVA p value=0.016), suggesting a learning effect from the first to the second scenario.

Relationship between non-technical and technical performance

Overall, a significant positive correlation was observed between non-technical (ANTS) and technical performance scores (r=0.53, 95% CI 0.32 to 0.69, p<0.001), meaning that better non-technical performance was associated with better technical performance. This overall correlation, however, is explained by a significant positive correlation in scenarios with external stressors (r=0.67, 95% CI 0.40 to 0.83, p<0.001), while no evidence for a relationship between technical and non-technical performance was observed when no external stressors were present (r=0.15, 95% CI −0.22 to 0.49, p=0.42, figure 3). Likewise, all scores of the domains of the ANTS framework showed a significant positive relationship with the technical performance when additional stressors were present, but not under control conditions (table 3).

Figure 3

Scatter plot showing the relationship between non-technical (Anaesthetists’ Non-Technical Skills (ANTS)) and transformed technical performance scores. Solid dots and the dashed line show the significant correlation in scenarios with external stressors (r=0.67, p<0.001). Open circles and the dotted line show the non-significant correlation in scenarios without external stressors (r=0.15, p=0.42).

Table 3

Correlations between the non-technical (ANTS) score and the transformed technical performance score.

	Overall		Without additional stressors		With additional stressors
	r	95% CI	r	95% CI	r	95% CI
Total ANTS score	0.53	0.32 to 0.69	0.15	−0.22 to 0.49	0.67	0.40 to 0.83
Domains
Task management	0.52	0.31 to 0.68	0.27	−0.10 to 0.58	0.63	0.35 to 0.81
Team working	0.49	0.27 to 0.66	0.18	−0.20 to 0.50	0.63	0.35 to 0.81
Situation awareness	0.45	0.22 to 0.63	0.14	−0.24 to 0.47	0.54	0.22 to 0.75
Decision-making	0.25	−0.01 to 0.47	−0.17	−0.50 to 0.21	0.43	0.09 to 0.69

Correlations are reported as Pearson correlation coefficients (r) for the total ANTS score as well as its domains, and under conditions with and without external stressors.

ANTS, Anaesthetists’ Non-Technical Skills.

The GEE analysis provides strong evidence for an interaction between non-technical performance and stress (unadjusted p=0.015, adjusted for period effects=0.001), confirming that the relationship between non-technical and technical performance varies depending on the presence or absence of external stressors (table 4).

Table 4

Generalised estimating equation (GEE) models to explore the effects of stress on the relationship between the non-technical (ANTS) score and the transformed technical score. Model 1 is not adjusted for period effects, while model 2 adjusts for period effects

	Regression coefficient	95% CI	p Value
Model 1
ANTS score (scenario without stressors)	2.7	−3.5 to 9.0	0.392
ANTS score (scenario with stressors)	12.2	7.7 to 16.8	<0.001
External stressor present (vs no stressor)	−448.8	−787.2 to −110.4	0.009
Interaction ANTS score*stress	9.5	1.8 to 17.1	0.015
Intercept (constant)	83.8	−198.6 to 366.1	0.561
Model 2
ANTS score (scenario without stressors)	0.4	−4.9 to 5.6	0.895
ANTS score (scenario with stressors)	10.2	5.9 to 14.5	<0.001
External stressor present (vs no stressor)	−474.2	−718.1 to −230.4	<0.001
Interaction ANTS score*stress	9.9	4.2 to 15.5	0.001
Period	48.3	19.4 to 77.1	0.001
Intercept (constant)	119.8	−110.7 to 350.3	0.308

Interpretation of regression parameters (based on the model with adjustment, analogue considerations apply to the other model): There is no evidence for a relationship between the ANTS score and the transformed technical score in scenarios without external stressors (mean increase of the transformed technical score of 0.4 (–4.9 to 5.6) points per 1-point increase in the ANTS score, p=0.895). In contrast, in scenarios with external stressors, a 1-point increase in the ANTS score is significantly associated with a 10.2 (5.9 to 14.5, p<0.001)-point increase in the transformed technical score (Note that the regression coefficient for the ANTS score in scenarios with stressors is the linear combination between the parameter for the ANTS score in scenarios without stressors and the coefficient for the interaction between the ANTS score and stress). The parameters can also be used to calculate expected scores of the transformed technical score at different ANTS scores, periods and stress levels: For example, at an ANTS score of 40 in the second scenario (second period), with external stressors, we would expect a transformed technical score of (40*10.2)–(1*474.2)+(2*48.3)+119.8=150.2 points. At an ANTS score of 45 in a first scenario without stressors, we would expect a transformed technical score of (45*0.4)–(0*474.2)+(1*48.3)+119.8=186.1 points.

ANTS, Anaesthetists’ Non-Technical Skills.

Discussion

We found that external stressors lowered the non-technical performance of resuscitation team leaders. Technical performance of the teams was only correlated with the non-technical performance of the leader scenarios when external stressors were present.

Stress can adversely affect cognitive functions such as attention, working memory and decision-making,13 and this is reflected by a deterioration of non-technical performance in the presence of external stressors in our study.

Deficiencies in non-technical performance have been considered major contributors to avoidable errors in healthcare, and it has been suggested that training in non-technical skills can reduce such errors and may improve patient outcome.14 As a consequence, the need for training in non-technical skills is increasingly acknowledged in various fields of healthcare, including CPR.15 16 However, the relationship between non-technical and technical performance is still poorly understood, and data on whether there is actually such a relationship at all are conflicting. Most previous studies relating to this topic have addressed surgical or anaesthesiological performances.17–19

Two prior studies have demonstrated variability in the relationship between technical and non-technical performance during CPR.6 20 In a study with anaesthesia residents responding to simulated intraoperative cardiac arrests, Riem et al 6 report a significant correlation between all domains of non-technical performance and technical performance, while Marsch et al 20 observed that deficiencies in the specific domains of leadership behaviour and absence of task distribution were associated with inadequate treatment of simulated cardiac arrests among intensive care personnel. The conflicting available data suggest that the relationship between non-technical and technical performance may not be the same for different domains of non-technical performance. Moreover, the relationship may depend on the specific situation or on circumstances that have not been addressed in previous studies, such as the stress level of participants.

In line with the results reported by Riem et al, we observed an overall significant relationship between non-technical performance of the team leader and technical performance of the resuscitation team, although this was only demonstrated when there were external stressors. During CPR, team members follow a well-defined algorithm,2 3 and non-technical skills may be of subordinate importance for technical performance of trained CPR providers during routine and straightforward CPR situations. As soon as routine is disturbed, for example, through interaction of bystanders, noise or other distractors, non-technical skills may become increasingly important to maintain good technical performance. Indeed, we found no association between non-technical and technical performance under control conditions, while a significant correlation was observed only when additional stressors were present.

These observations have important implications for training of resuscitation teams and their leaders. Hunziker et al 21 have previously demonstrated that training in leadership and communication skills (both belonging to the domain of team working) can improve CPR performance. Our data support this approach, because team working scores were clearly related to technical CPR performance under conditions with external stressors. However, our data also suggest that such training should not merely focus on general teamwork aspects but should address all domains of non-technical performance (task management, team working, situation awareness and decision-making) as this may have the potential to further improve CPR performance, especially under stressful conditions. Our data would suggest that additional selective non-technical skills training for team leaders—in addition to training resuscitation teams as a whole—could have an incremental benefit on team performance.

While the role of non-technical skills seems to be less crucial during routine CPR without external stressors, it is important to note that CPR is seldom routine in the clinical setting. Distractors are commonly observed, and the baseline stress level while treating a real patient in cardiac arrest may likely be higher than during simulated scenarios. Moreover, there are wide interindividual differences in response to stressors.13 Hence, it is possible that a given resuscitation is routine for one team member whereas the same situation is perceived as stressful by another team member. Hence, CPR training should include an emphasis on non-technical skills so that team members, and especially team leaders, are equipped with the necessary tools to ascertain high-quality CPR performance at all times regardless of the stress level.

Limitations

We used simulated CPR scenarios to assess the relationship between non-technical and technical performance. Performing a randomised study under clinical conditions would be unfeasible. Cardiac arrests are acute events that involve a heterogeneous patient population and take place in various settings. Standardisation of study conditions and controlling for confounders is barely possible. In contrast, the simulator environment allows standardising experimental conditions such that the scenarios and distractors were identical for all participants. Simulation is a well-established research instrument to study human factors and team interaction during CPR.5 The particular simulator that we used provides realistic features for training in resuscitation,22 and has repeatedly been used as research tool in various settings.7 23–25

For this study, we designed two different scenarios requiring the same treatment protocol. Using identical CPR scenarios twice—except for the fact that one scenario contains additional stressors—could lead to bias in the way that participants might recognise that they are confronted with the same clinical situation and that they would simply have to do the same things as in the first scenario. Although we created a different mindset in the second scenario, we observed a period effect which most likely reflects a learning effect from the first to the second scenario. Interestingly, there was no feedback or debriefing of the participants during or after the first scenario, suggesting that simulator-based training induces self-reflection and improves non-technical performance even if this has not been specifically addressed during the session. To avoid bias due to this learning effect, the analysis of the relationship between external stressors and non-technical performance has been adjusted for the observed period effect.

Conclusions

During CPR, non-technical performance of the team leader is related to technical performance of the entire team when external stressors are present. This relationship was observed for all domains of non-technical performance, suggesting that training of the leader and improving task management, team working and leadership, situation awareness as well as decision-making may substantially improve technical performance during CPR, especially under stressful conditions.