Measuring non-technical skills during prehospital advanced cardiac life support: A pilot study.

AIM: To analyse non-technical skills of mobile medical teams during out-of-hospital cardiac arrests (OHCA) using the validated Team Emergency Assessment Measure (TEAM) tool. To research the correlation between non-technical skills and patient outcome.
METHODS: Adult patients who experienced an out-of-hospital cardiac arrest between July 2016, and June 2018, and were treated by a mobile medical team from the University Hospital Leuven, were eligible for the study. Resuscitations were video recorded from the team leader's perspective. Video recordings were reviewed and scored by emergency physicians, using the TEAM evaluation form.
RESULTS: In total 114 OHCAs were analysed. The mean TEAM score was 34.4/44 (SD = 5.5). The mean item score was 3.1/4 (SD = 0.8). On average, 'effective team communication' had the lowest score (2.4), while 'acting with composure and control' and 'following of approved standards/guidelines' scored the highest (3.4). The average non-technical skills theme scores were 2.9 (SD = 0.9) for 'Leadership', 3.1 (SD = 0.8) for 'Teamwork' and 3.3 (SD = 0.7) for 'Task management'. 'Leadership' was rated significantly lower than 'Teamwork' (p = 0.004) and 'Task management' (p < 0.001). No significant correlation was found between TEAM and return of spontaneous circulation (p = 0.574) or one month survival (p = 0.225).
CONCLUSION: The mean overall TEAM score was categorized as good. Task management scored high, while leadership and team communication received lower scores. Future training programs should thus focus on improving leadership and communication. In this pilot study no correlation was found between non-technical skills and survival.

Introduction

Emergency mobile medical teams (MMTs) are faced with out-of-hospital cardiopulmonary resuscitation (CPR) on a daily basis. Despite tremendous efforts, return of spontaneous circulation (ROSC) is achieved in only 29.7% of all patients. Favorable neurological outcome at hospital discharge varies from 2.8% to 18.2%.

Optimal resuscitation efforts not only require technical skills and knowledge, but also the mastering of non-technical skills (NTS) such as leadership, teamwork, decision-making and situational awareness. Research in simulation training has demonstrated that good quality teamwork, especially communication and team coordination abilities, is particularly important for safe patient care during the management of life-threatening events by professional teams. Failures in teamwork have even led to the occurrence of preventable medical errors. Also, several studies have linked lack of leadership skills and poor teamwork to poor clinical outcomes in a large variety of medical settings.5., 6., 7.

Several assessment tools have been developed to measure teamwork performance of MMTs in crisis situations. Although these tools were made to measure performance during simulations in order to improve teamwork, they might offer the opportunity to equally assess NTS during clinical crisis events.8., 9.

Only a few tools have been developed specifically for the context of resuscitations. The Observational Skill-based Clinical Assessment tool for Resuscitation (OSCAR) focuses on individuals as part of a team and has been exclusively tested and used in in-hospital simulation-based training. The Team Emergency Assessment Measure (TEAM) and the Modified Non-Technical Skills Scale for Trauma (T-NOTECHS) were validated both in in-hospital simulated9., 10., 11., 14., 15. and in in-hospital clinical settings.9., 12., 13. While T-NOTECHS was validated for trauma team resuscitations and paediatric emergencies, TEAM specifically focuses on in-hospital cardiac arrests (IHCA) and emergency resuscitations in the clinical setting, primarily in the emergency department (ED).9., 12., 13. The TEAM instrument focuses on team performance at the team level.8., 9., 10. Cooper et al. and Cant et al. validated TEAM with clinical in-hospital resuscitation teams in a variety of resuscitations (neurological, trauma, respiratory, shock-all causes and cardiovascular).12., 13. In this pilot study TEAM is used for the first time in an out-of-hospital clinical resuscitation setting.

The present pilot study primarily aims to assess the NTS of MMTs during advanced cardiac life support (ACLS) specifically in an out-of-hospital setting. Secondarily, the influence of TEAM performance on ROSC and survival after one month is evaluated.

Methods

Design

A prospective observational pilot study was performed. The primary objective was to analyse the NTS of MMTs during resuscitations of out-of-hospital cardiac arrests (OHCA) using the TEAM tool. Secondarily, this study aimed to assess the correlation between TEAM performance and survival (ROSC and one month survival).

Setting

This study was conducted by the emergency medical services connected to the ED of the University Hospital Leuven, a tertiary referral center. Patients experiencing an OHCA are managed by MMTs, consisting of an emergency physician and an emergency nurse. The out-of-hospital team is reinforced by at least two paramedics, intermediate life support schooled. All emergency medicine attending physicians and residents (minimum 3 years of clinical experience) in this study have an ACLS certificate and regular experience in handling CA. The MMTs of the University Hospital Leuven annually perform approximately 2900 prehospital interventions. During the study period, the MMTs used the 2015 guidelines algorithms for CA resuscitation efforts. A defibrillator (Philips HeartStart MRx) was available for rhythm analysis and, if needed, defibrillation.

Instrument

The TEAM instrument is designed specifically to evaluate NTS in emergency medicine teams. TEAM includes 11 items rated on a five-point Likert scale ranging from 0 (never/hardly never) to 4 (always/nearly always) covering three themes: 'Leadership' (Q1-2), 'Teamwork' (Q3-9) and 'Task management' (Q10-11), which are added up to a total item score (TEAM score) ranging from 0 to 44. The 12th item (Q12) is an overall score of the team performance (Global score) on a scale of 1 (poor) to 10 (excellent). Content validity, construct validity, concurrent validity,10., 11., 12., 13., 14. internal consistency,13., 14., 15. inter-rater reliability10., 14., 15. and test–retest reliability were demonstrated in previous studies.8., 9., 10., 11., 12., 13., 14., 15. The TEAM items are listed in Table 2 (including the scores for this study).

Table 2

Mean item score TEAM.

Category	Item	Mean (SD)	Min	Max
Leadership	Q1. The team leader let the team know what was expected of them through direction and command	2.8 (1.0)	0	4
	Q2. The team leader maintained a global perspective	3.1 (0.8)	1	4
Teamwork	Q3. The team communicated effectively	2.4 (0.9)	0	4
	Q4. The team worked together to complete tasks in a timely manner	3.3 (0.7)	2	4
	Q5. The team acted with composure and control	3.4 (0.7)	1	4
	Q6. The team morale was positive	3.3 (0.7)	1	4
	Q7. The team adapted to changing situations	3.3 (0.7)	2	4
	Q8. The team monitored and reassessed the situation	3.3 (0.7)	1	4
	Q9. The team anticipated potential situations	3.1 (0.7)	1	4
Task management	Q10. The team prioritized tasks	3.2 (0.8)	0	4
	Q11. The team followed approved standards and guidelines	3.4 (0.7)	1	4
Global	Q12. Global score (/10)	7.0 (1.4)	2	9
Sum	Total TEAM score (/44)	34.4 (5.5)	11	42
Total	Total score (/54)	41.4 (6.8)	13	51

5-point Likert rating scale: 0: never/hardly never; 1: seldom; 2: about as often as not; 3: very often; 4: always. SD: standard deviation.

TEAM is validated in in-hospital simulation events,9., 10., 11., 14., 15. as well as in in-hospital clinical settings.12., 13. Cooper et al. proposed a performance benchmark for in-hospital resuscitation based on 106 in-hospital resuscitation episodes (of which 41% IHCA). TEAM scores below 34 were considered poor, between 34 and 39 good and above 40 team performance was considered excellent.

Study population

All consecutive adult patients (≥18 years) that experienced an OHCA and were treated by a MMT from July 1, 2016 to June 31, 2018, were eligible for inclusion in the study. All aetiologies of OHCA, medical or traumatic, were included. The medical team had to consist of at least three resuscitation members (including an emergency physician) and the resuscitation effort lasted for at least five minutes. Patients were excluded if: 1. the video recording of the CA was not complete; 2. the quality of the video recording did not allow for reliable data collection; 3. the patient was found with rigor mortis or other obvious signs of irreversible death; 4. The resuscitation was shorter than 5 minutes; or 5. The patient was a candidate for ECMO.

Data collection

A mobile digital real time video recording GoPro HERO 4 device (black edition) was used. The GoPro was body-mounted on the team leader with a chest harness (Chesty), and oriented to the team leader’s perspective. The team leader was responsible for recording the resuscitation process and the security of the videotape afterwards. The recording was started by the team leader upon arrival at the scene.

Video recordings of all included resuscitations were independently reviewed by two ACLS-certified emergency physicians (PDW and MV), who made a full description of the actions taken during ACLS using the detailed TEAM evaluation form. Inter-rater reliability was calculated using the Cohen’s kappa coefficient and the results of the expert reviewer (PDW) were used for analysis. When analysis of the recording could not provide a definite answer, the variable was left blank. Missing items were not included in the statistical analysis.

Data analysis

All data were imported into SPSS, version 26 (IBM, Armonk, NY) for statistical analysis. Data were described as percentage and frequency of occurrence for categorical variables and as central tendency measures for continuous data. The correlation between the TEAM score and Global score was investigated using a two-tailed student’s t-test. One way ANOVA was used to compare Total score and outcome. Multiple regression analyses were performed to investigate whether scores of individual items could predict the Global score and/or survival. A p-value < 0.05 was considered to be statistically significant.

Ethical approval

The study was approved by the UZ Leuven Institutional Ethical Review Board (identification number S58657). The videotapes could only be viewed by the team members taking part in a particular resuscitation and by the two investigators. All team members were informed of the recording before the start of the project and prior to the resuscitation.

Results

From July 1, 2016, to June 31, 2018, the MMTs from UZ Leuven performed 244 resuscitations on patients experiencing an OHCA. Of these, 85 resuscitations were not video recorded since there was no evidence for need of resuscitation prior to arrival on scene. From the 159 cases where the prehospital teams planned to film, 45 video recordings were excluded due to poor video quality, memory card problems, battery issues, unavailability of the GoPro, irreversible death, ECMO or CPR duration < 5 minutes (Fig. 1).

Fig. 1

Flow diagram video-recorded cardiac arrests.

Descriptive analysis of video recorded resuscitations

In total 114 video recorded OHCAs were analyzed. The median number of professional team members involved during resuscitation was 5 (range: 4 – 8). ROSC occurred in 40.4% of all patients, with a one month survival rate of 8.8% (Table 1).

Table 1

Resuscitation characteristics.

Variables	All OHCA (n = 114)	No ROSC (n = 68)	ROSC (n = 46)
Age (years, mean (IQR))	73.5 (63 – 81)	73.5 (63 – 81)	74.0 (63 – 81)
Gender (% male)	68.4	69.1	67.4
Witnessed (%)	59.6	42.4	90.9
Bystander CPR (%)	66.7	66.7	80.0
Initial shockable rhythm (%)	18.4	13.2	26.1
CPR duration (min, mean (IQR))	15.0 (10.5 – 22.1)	18.7 (5.6 – 51.6)	10.7 (5.2 – 48.6)

Non-technical skills ratings

Overall, the mean item score was 3.1 out of 4 (SD = 0.8). On average, ‘effective team communication’ (Q3) had the lowest score (2.4), while ‘acting with composure and control’ (Q5) and ‘following of approved standards/guidelines’ (Q11) scored the highest (3.4) (Table 2). Inter-rater reliability (Cohen’s kappa) was 0.973 (CI 95%: 0.963–0.982).

The average NTS theme scores were 2.9 for ‘Leadership’ (SD = 0.9), 3.1 for ‘Teamwork’ (SD = 0.8) and 3.3 for ‘Task management’ (SD = 0.7). ‘Leadership’ was rated significantly lower than ‘Teamwork’ (p = 0.004) and ‘Task management’ (p < 0.001).

The mean episode-based TEAM score (sum of all individual items) was 34.4 out of a maximum of 44 (SD = 5.5), equivalent to a NTS performance score of 78.2%. The 25, 50 and 75 scale score percentiles were respectively 30, 36 and 38 points. In total 23 resuscitations had a rating in the lower 25th percentile. The mean Global score (Q12) was 7.0 out of 10 (SD = 1.4). The average total score (TEAM score + Global score) was 41.4 out of 54 (SD = 6.8). A significant correlation was found between the sum of all individual items (TEAM score) and the Global score (p < 0.001).

Additional multiple regression analyses showed a significant correlation between the score for the individual items and the Global score (F(11, 102) = 44.24, R2 = 0.827, p < 0.001). Looking at the individual items, items Q1-Q4, Q7 and Q9 were found to be significant predictors for the Global score (all p < 0.05). Also a significant correlation between the category scores and the Global score was found (F(3, 110) = 144.58, R2 = 0.798, p < 0.001). All classes were found to be predictors for the Global score (p < 0.05).

TEAM rating and survival

ROSC was achieved for 46 patients (40.4%), with 10 patients surviving for one month (8.8%). No significant correlation was found between the total score and ROSC (p = 0.574) or one month survival (p = 0.225) (Table 3). Additional multiple regression analyses showed no significant correlation between the scores on the individual items and ROSC (R2 = 0.141, p = or survival after one month (R2 = 0.128, p = 0.202).

Table 3

Total score in function of ROSC and 1 month survival.

		Mean (SD)
ROSC	Yes	41.8 (6.7)
	No	41.1 (6.9)
1 month survival	Yes	43.9 (7.1)
	No	41.1 (6.8)

SD: standard deviation; CI: confidence interval.

Discussion

In this prospective observational pilot study, 114 resuscitations in an out-of-hospital setting were video recorded. The primary aim of the study was to analyse the NTS of MMTs during OHCAs using TEAM, a validated tool to measure teamwork performance during resuscitations. Secondarily, the influence of TEAM performance on survival was evaluated.

The average overall TEAM score in this study was 34.4. In the clinical in-hospital study of Cooper et al., based on 106 in-hospital resuscitation episodes (of which 41% IHCA), the mean overall TEAM score was 34.6. A clinical study by Cant et al. reported a higher mean overall score of 39.2 for 80 in-hospital resuscitation episodes (of which 20% IHCA). Following the performance benchmark of Cooper et al., team performance in this study can be considered as good despite the possible extra out-of-hospital challenges.

The MMTs seem to have a very good knowledge of the guidelines during CPR, reflected by the average maximum score of 3.4 for ‘following the clinical standards and guidelines’. Task Management, including ‘prioritization and execution of clinical standards and guidelines’, scored the highest of the NTS classes. The items ‘direction and command by the team leader’ and ‘effective communication within the team’ on average received the lowest score. The same trend was observed in other studies, with a lower score for leadership and a higher score for task management.12., 13.

In this pilot study, no significant correlation was found between NTS scores and survival. These findings contradict a previous study by Cant et al., that describes lower TEAM scores for non-survivors compared to survivors. Cant et al. however describes a lower number (80) of all origin in-hospital resuscitations.

In a prehospital setting, MMTs are confronted with many different confounders while performing ACLS. The prehospital team often needs to work with many uncertainties, like the medical history of the patient, the duration of the no flow time and/or the context of the event. Furthermore work space can be limited, bystanders can disturb the intervention and the environment can be unsafe. All these unknown or unpredictable factors can greatly influence a patient’s chance of survival. Overall, we can conclude that the sample size in this study was too small to account for these multiple confounders.

While video recording resuscitation events has the potential to serve as a valuable resource in assessing adherence to guidelines, insights into human factors and the effect on team performance will be critical. Video recording provides multi-dimensional data and allows us to explore hypotheses regarding the cause of resuscitation outcomes. Understudied aspects of resuscitation including NTS will now move from the shadows into plain sight.

It has been demonstrated that teamwork and care coordination during CA have an influence on CPR quality. Together with knowledge and technical skills, NTS can improve ACLS guideline compliance. Although all individual items contribute to NTS performance, based on this study, global perspective, effective communication, team expectations, adaptation, anticipation and teamwork deserve special attention during future trainings.

A recent systematic review demonstrated that team simulation training results in improved NTS and a reduction in the time required to complete a simulated CA. Also our results demonstrated more limited skills regarding leadership and communication. Therefore, an introduction of a NTS program into ACLS courses, with a special focus on leadership and communication, might improve NTS not only in simulations but also in real-life.

A NTS rating scheme can be used for OHCA events and can thus aid in improving team performance. TEAM is a user friendly tool to discuss the team approach. It is already recommended as a discussion facilitator in post resuscitation events enabling a constructive feedback platform for nurses, paramedics and emergency physicians to assess leadership, teamwork and communication. It can encourage post-event ‘hot debriefs’ between team members to identify points of improvement for future interventions. Furthermore, since feedback and repetitive practice are key factors for training NTS, every real-life resuscitation provides an opportunity to observe, discuss and improve NTS.

Limitations

This observational study has several limitations. As the study was limited to one ED and emergency medical service, it is not possible to generalize these results for all MMTs. Secondly, although video recording is increasingly integrated in CA care protocols, direct observation might alter the behaviour of team members and thus guideline adherence.21., 22. Also, since TEAM does not score individual team members, the global score can be influenced by weaker or stronger team members. Fourthly, as only 10 patients survive one month after OHCA, more OHCA events are needed to interpret the relation between the TEAM score and one month survival. Fifthly, we only included adult patients. The results could be very different in pediatric patients considering the different emotional situation and the exceptional occurrence.

Finally, as both reviewers work at the same ED and item definitions were discussed upfront, inter-rater reliability might be influenced. Raters from different simulation centers, simulation programs and different team positions (doctor/nurse/paramedic) would probably have a lower inter-rater reliability.12., 16.

Conclusions

The NTS of MMTs were analysed, using TEAM as a validated assessment tool. The mean overall TEAM score was categorized as good. Task management was scored high, while leadership and team communication received lower scores. Future training programs should thus focus on improving leadership and communication. Despite the growing evidence, in this pilot study no correlation was found between NTS and survival.

CRediT authorship contribution statement

Philippe Dewolf: Conceptualization, Data curation, Investigation, Methodology, Supervision, Visualization, Writing – original draft, Writing – review & editing. Maïté Vanneste: Conceptualization, Data curation, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing. Didier Desruelles: Formal analysis, Methodology. Lina Wauters: Data curation, Methodology, Project administration, Visualization, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.