Leaders' and followers' individual experiences during the early phase of simulation-based team training: an exploratory study.

BACKGROUND: A growing body of evidence shows that team training can develop essential team skills and contribute to better patient outcomes. Current simulation-based team training (SBTT) programmes most often include targets and feedback focused on the whole team and/or leader, ignoring the follower as a unique entity. By considering followers' individual experiences, and tailoring behavioural targets for training and feedback, SBTT could be improved. Our aim was to explore the individual experiences and behaviours of leaders and followers during the early phase of SBTT, and we hypothesised that leaders and followers would show different responses.
METHODS: Medical students (n=54) participated in half-day SBTT including three video-recorded scenarios. Self-efficacy was assessed pretraining and post-training. For each scenario (n=36), the individual teamwork behaviours, concentration, mental strain and the team's clinical performance were recorded. Data were analysed using a mixed model allowing for participants to be their own control in their roles as leader or follower.
RESULTS: Self-efficacy improved. In the role of leader, participants communicated to a greater extent and experienced higher mental strain and concentration than they did in the role of follower. DISCUSSION: The increased self-efficacy enables a positive learning outcome after only three scenarios. Individual experiences and behaviours differed between the role of leader and that of follower. By shedding further light on leaders' and followers' individual experiences and behaviours, targets for training and feedback could be specified in order to improve SBTT.

Introduction

Teamwork is of critical importance for patient safety.1 Effective team coordination is not a case of good luck. It is trainable through training interventions.2 3 High-quality training requires deep knowledge, needs analysis of the task to be performed, clarification of the training objectives, choice of training methods, and evaluation and assessment of the training delivered.4 Training in teamwork comprises growth in aspects of competence, such as knowledge, skills, attitudes and behaviours (KSAB).5 A prominent feature of medical simulation that leads to effective KSAB learning is repetitive practice and iterative scenarios during one training session, or repetitive training sessions over time.6 Hours of high-fidelity simulator practice have a positive, functional relationship with standardised learning outcomes as concluded by McGaghie et al.6 The minimal decisive dose of simulation-based team training (SBTT) that can add to learning, either on its own or for later reiteration, is unknown.5 7 Due to resource constraints, a number of institutions offer half-day training, although training using only one, two or three scenarios results in limited improvements in technical or non-technical skills.8–11

By contrast, at the very start of the learning process, when training has not yet passed the threshold needed to show improvements in KSAB competence, studies have shown that participants’ reactions to SBTT are extremely positive even after a very short training experience.12 13 The positive reaction in the early phases of SBTT may be a token of other individual experiences that trainers often witness: trainees’ motivation, concentration, commitment and efforts in further training.

One example of such an individual experience is self-efficacy, namely a belief in one's personal capability to perform given actions, an important prerequisite for learning, and a predictor for performance.14 Perceived self-efficacy motivates individuals to engage in pursuing their training goal. In general, the stronger the perceived self-efficacy, the higher the goal challenges people set for themselves, and the firmer is their commitment to them. Those with high self-efficacy in a specific task are more likely to make more of an effort, and persist longer than those with low self-efficacy.14 15 It is a critical feature of successful medical simulations,16 17 and has also been emphasised as an important individual factor in demonstration-based training.18 Overall, medical simulation training can improve individual as well as team self-efficacy.17 In this context, it should be noted that while self-efficacy is important for the success of a training process, there is no consistent correlation with observed measures of competence and self-efficacy.17

Another individual experience that has been positively associated with effective teaching and learning is concentration, the skill of selectively focusing on relevant information in the task(s) at hand and ignoring distracters, such as anxiety, mistakes and negative thoughts.19 20

A third individual experience that might have an effect on learning is perceived mental strain, vastly applied in clinical diagnostics and training.21 We expected that very high mental strain during training could jeopardise learning.

Current SBTT programmes most often include targets and feedback focused on the whole team and/or leader, ignoring the follower as a unique entity, as if the leader–follower relationship exists in a vacuum. This is a concern, as it does not fully appreciate and model the dynamism and complexities of team leadership as revealed in recent research on shared leadership.22

Leading and following are collaborative adjustment behaviours, an evolutionary strategy for solving social coordination problems.23 In particular, in high-performing teams, some attributes and characteristics associated with leaders are also prerequisites of followers; these may be referred to as, for example, ‘good leadership’ or ‘structuring leadership’, and ‘good followership’ or ‘content-oriented leadership’.24 25 When group members actively and intentionally shift the role of leader to one another as necessitated by the situation, a shared leadership occurs,22 implying that team members must master both roles in order to affect the team's attitudes, cognition, behaviour and effectiveness in a positive manner. Since the competencies for the leader and followers in a team are not identical, the learning processes and individual experiences of the two roles may also differ.

To the best of our knowledge, there are no data in the literature concerning leaders’ and followers’ individual experiences associated with effective learning in the early phase of SBTT. We therefore decided to undertake an exploratory study to expand the knowledge base.

Our aim was to explore individual experiences associated with learning and performance: we looked at levels of mental strain and concentration for leaders’ and followers’ separately during the early phase of SBTT, and their perceived self-efficacy pretraining and post-training. We also sampled data for training outcomes: individual behaviours and team endeavour in clinical performance. We hypothesized that it would be possible to detect changes in individual experiences early during SBTT, and that leaders and followers would show different responses.

Methods

Subjects

Medical students (4th or 6th year), attending one of three 2-week elective courses in emergency medicine, traumatology, or anaesthesia and intensive care, were offered the opportunity to participate in the study. All elective courses included standardised didactic lectures in top-to-toe physical examination and team coordination training. The half-day (3.5 h) SBTT was integrated in the curriculum and was mandatory for all courses, while participation in the study was voluntary. For anonymous video analysis and data sampling, participants were given a personal code to label their scrub shirt and questionnaire. Informed consent was obtained from all participants, and the institutional review board approved the study (Regionala etikprövningsnämnden i Stockholm, Dnr 358/02).

Simulation-based team training

Students in groups of three to five attended one of 12 half-day training sessions at the Center for Advanced Medical Simulation and Training, Karolinska University Hospital. Training was focused on patient management in emergency medicine or traumatology. Scenarios were carried out in a re-created emergency department using a patient simulator (Human Patient Simulator, METI Inc, Sarasota, Florida, USA). Nine standardised scenarios were designed to provide opportunities for training in the targets (table 1). The scenarios had the following diagnostic content: asthma, traumatic skull fracture due to alcohol intoxication, femur fracture and hypoglycaemia, traumatic femur fracture and pneumothorax, hypoglycaemia, heart failure and pulmonary oedema, ventricular fibrillation, diabetic acidosis, and traumatic spleen rupture with hypovolemia. Random samples from the standardised scenarios were used each half-day, offering the students different access to basic medical problems.

Table 1

Targets for training and video analysis

Targets for training	Method	Analysis	Note
Behaviour (1–5):
1. Take a team member role 2. Gather information and communicate 3. Contribute to a shared understanding of the situation 4. Make collaborative decisions 5. Coordinate and execute tasks	The A-TEAM programme, not yet validated, was used. Separate videos were used for calibration of rating. Active participants’ behaviours were analysed blind by two raters. They observed the study videos individually at random in 3 min sequences.	Each active participant was individually graded on a four-level scale (poor, in need of improvement, good, proficient). Team members were categorised as either leader or follower, by applying the behavioural elements for ‘Leader’ and ‘Follower’, respectively, from the category ‘Takes a team member role’ in the A-TEAM programme. For instance, a leader takes the initiative to provide structure and direct teamwork and task work, while a follower challenges constructively and assumes assigned responsibilities.	Annotation 1
Clinical performance (6–7):
6. Time to call for help, in seconds (s)	A standardised measure of call for help was calculated as a ratio by dividing 60 s with the team's measured time in seconds.	The time from the entrance of the first active participant into the scenario until the team called for help was measured.	Annotations 1 and 2
7a. Frequency of top-to-toe examinations (n×h−1)	The average frequency was calculated as the number of examinations divided by the length of the scenario. A standardised measure was calculated as the ratio between the team's measured frequency and the specialist team's reference frequency.	The number of top-to-toe physical examinations the team completed in a scenario was counted.	Annotations 1 and 2
7b. Frequency of team sum-ups (n×h−1)	The average frequency was calculated as the number of sum-ups divided by the length of the scenario. A standardised measure was calculated as the ratio between the team's measured frequency and the specialist team's reference frequency.	A sum-up includes the patient's present problem, clinical background, vital functions and further plan. The number of team sum-ups performed during a scenario was calculated.	Annotations 1 and 2
Medical management (8):
8. Stabilise the vital functions of the patient	Time in seconds.	The time from the entrance of the first active participant in a scenario until stabilisation of the vital functions of the patient was measured.	Not used (see Results)

Annotations:

(1) Two observers, one specialist in anaesthesia and intensive care (LM), and one senior research psychologist (LH), neither of whom were involved in the training process, analysed individual teamwork behaviours (Targets 1–5) using the A-TEAM programme, not yet validated.27 LM observed clinical performance and medical management.

(2) As the nine scenarios were of different complexity and, as such, not comparable, standardised measures for clinical performance were calculated for each separate scenario. A reference emergency team, including a senior consultant, a specialist nurse and a nurse assistant, carried out each of the nine scenarios, generating reference measures for Targets 6 and 7.

The training method used was highly structured, target-focused, team coordination training using the Crew Resource Management (CRM)-based A-TEAM programme.26 27 During training, active participants engaged in the scenario, while the observing participants reviewed the behaviours according to the targets. The facilitated debriefing session was highly structured, including an emotional ‘blow-out’, feedback on medical management and clinical performance, active participants’ experience and observers’ views of individual teamwork behaviours and feedback to active participants, and finally trainers’ concluding remarks and recommendations for the next scenario.26 28 The allocation of participants to active and observing roles was changed in consecutive scenarios in order to take advantage of different learning styles. Video recordings of each scenario were saved for the debriefing session, and were also used for later analysis of medical management, clinical performance and individual behaviour.

Targets for training

Targets for training (table 1) combined medical management (Target 8), the team's clinical performance (Targets 6 and 7) and individual teamwork behaviour (Targets 1 and 5). Targets for individual teamwork behaviour as a leader or follower were set at the ‘good’ level in the A-TEAM programme.27 All targets were explicitly explained in didactic lectures and demonstrated by the trainers prior to the start of the scenarios. Target 1 was to ‘take a team member role’, that is, assign roles within the team as situational leader or follower when there is no outside authority present. Thus, the training focused on the situational leader and his/her followers, not on formal leaders or chiefs. This issue was thoroughly approached during the didactic lecture (figure 1) in which team formation, the assignment of roles and deliberate changes in roles according to the clinical situation were discussed. The participants were explicitly asked to assign roles during the formation of the team, in the classroom, before entering the scenario or, ultimately, at the start of the scenario. Thus, the assignment of the role as leader or follower among the active participants was at the teams’ discretion and not predetermined by the trainers.

Figure 1

Flow sheet and collected data.

Due to the students’ lack of clinical competence, calling for help within 60 s was as one of the targets (Target 6) to achieve patient safety. To enable the active participants to work under stress, a consultant entered the scenario 5 min after a ‘call for help’ telephone call.

Video analysis

The video analysis is presented in table 1.

Individual experiences

Self-efficacy

Participants’ self-efficacy was self-assessed at the start and at the end of training using a questionnaire consisting of four items, where each proposal was rated on a 7-point Likert-type scale, where 1 = not true at all, and 7 = very true.28 The self-efficacy score was calculated as the mean value of all items.

Concentration

From a flow instrument, constructed and validated by Chen, we selected eight items (Cronbach's α >0.7) used for assessing aspects of concentration (focusing, time distortion, loss of self-consciousness, and telepresence).29 Concentration was self-assessed by all active participants immediately after each scenario. Scores were calculated as the mean of all items using a 9-point Likert-type scale, where 1=not at all, and 9=very much.

Mental strain

Mental strain was assessed by active participants immediately after each scenario using the Borg CR10 Scale and the prompt: ‘Please estimate your perception of mental strain during the session by putting a cross on the numbered scale’, where 0=none at all, and 10=the strongest you have ever experienced.30

Design of the study

The design of this exploratory study, including the flow of data sampling and lost data, is presented in figure 1.

Power analyses

Individual behaviour

If 40% of the subjects improve, and 10% deteriorate in the behaviour ‘gather information and communicate’ (Target 2) from scenarios I–III, there will be 80% power to detect this difference in proportions with a 0.05 two-sided significance level when the sample size is 46.

Individual experience

A power calculation revealed that 2×25 subjects would provide 80% power to detect a variance of 0.065 among the means of the two levels, self-efficacy start and end (the difference between the levels is 0.50), assuming that the between-groups (gender) error term is 1.47, the within-groups error term is 0.85, the measure of ‘sphericity’ of the covariance matrix, epsilon, is 1.00, and the correlation between the levels is 0.5 when the significance level is 0.05.

Statistical analysis

As this was an exploratory study, and subjects were their own controls, a linear mixed model in procedure Mixed in SAS (System 9.1, SAS Institute Inc, Cary, North Carolina, USA) was used to analyse the continuous variables, clinical performance and individual experiences.31 This model is preferred for use with questionnaire data because it can use all the data even if some values are missing, and all combinations of categories do not have to be fulfilled. It also takes into account covariance structures.

For the clinical performance (Targets 6 and 7), a mixed model with one within-groups factor consecutive scenario (I, II and III) was performed. In the mixed model for self-efficacy, two fixed factors and the interaction between the factors were included. The between-groups factor was ‘sex’ and the within-groups factor was ‘start/end’. For mental strain and concentration, a mixed model with two within-groups factors, consecutive scenario (I, II and III) and role (follower and leader), and one between-groups factor, sex, was performed. All possible interactions were also tested in this model.

A generalised estimating equations (GEE) model with the GENMOD procedure in SAS was performed to analyse the effect of the ordinal variables, clinical experience (level of medical studies) and order of training scenario (I–III), and categorical variables, team member's role (leader, follower) and sex (female, male), on the two raters’ scoring of the behavioural categories (Targets 1–5).32 With ordinal data, the GEE approach provides cumulative logits based on the cumulative probabilities. As all the students participated in more than one scenario with different memberships, a repeated-measures design was applied to these data.

A probability (p value) <0.05 was considered statistically significant. The SAS software was used for all statistical analysis.

Results

All 54 students, 26 men and 28 women (23–47 years of age), attending the elective courses in emergency medicine (n=30), traumatology (n=18), or anaesthesia and intensive care (n=6), agreed to participate in this study. Videos from 36 scenarios were analysed. All students participated in three scenarios, alternating as active participants and observers. The scenarios were numbered I, II and III chronologically, giving one set of three scenarios for each of the 12 half-day training sessions.

Medical management: Target 8

In no scenario were students able to stabilise vital functions before a consultant arrived. Consequently, Target 8 could not be used as the end point in this study of medical novices.

Clinical performance: Targets 6 and 7

The frequency of team sum-ups, the time to call for help, and the frequency of top-to-toe examinations are presented in box-plots, figures 2–4. Standardised variables (ratios) for ‘time to call for help’, ‘frequency of top-to-toe examinations’ and ‘frequency of team sum-ups’ were statistically analysed with respect to the classification variables, team (1–12), and consecutive scenario (I–III). The only significant change in response to training was an improvement in the frequency of sum-ups (p=0.04).

Figure 2

Time to call for help.

Figure 3

Frequency of top-to-toe examinations.

Figure 4

Frequency of team sum-ups.

Individual behaviour: Targets 1–5

Data for the five teamwork categories did not show any change between consecutive scenarios I, II and III (p=0.11–0.97). Team behaviours were not influenced by type of course (p 0.24–0.90) or sex (p 0.33–0.98). Both raters (p=0.00 and p=0.02, respectively) agreed that in the role of leader (n=36 observations), participants were rated higher on the item ‘gather information and communicate’ than in the role of follower (n=55 observations). The odds for the leader role having a higher score were over three times higher than for the follower role.

No differences in individual experiences between men and women were registered.

Cronbach's α for self-efficacy before training was 0.87. Perceived self-efficacy increased for the whole group from a mean (SD) of 5.15 (0.12) at the start of the training session to 5.42 (0.12) at the end. The mean difference was −0.267 (p=0.043 (95% Cl −0.526 to −0.008)).

Concentration and mental strain

Cronbach's α for concentration for scenario I was 0.78. There was no change in concentration or mental strain over the main effect, consecutive scenario (I–III). The interaction between consecutive scenario (I–III) and role (leader/follower) was not significant for either mental strain or concentration. In the role of leader, participants experienced a higher level of both concentration and mental strain as compared with the role of follower (table 2). Thus, the differences between the two roles can be generalised to all three scenarios.

Table 2

Data for mental strain and concentration for active participants

Membership	Role as leader		Role as follower		Difference in estimated mean	p	95% CI
	Estimated mean	SE	Estimated mean	SE
Mental strain (0–10)	5.88	0.38	4.31	0.27	1.57	<0.001	0.735 to 2.40
Concentration (1–9)	6.30	0.17	5.69	0.17	0.612	0.003	0.225 to 1.00

Table 2 shows the data for mental strain and concentration for active participants (n=46) after a scenario. The estimated means (SE) from the statistical model are presented here for the roles of leader and follower, respectively. The mean difference is calculated from the estimated mean values; the p value refers to the difference between the two roles.

Discussion

Summary

This study has explored individual experiences and behaviours in the roles of leader and follower, and the teams’ clinical performance during the early phase of SBTT.

After training in three scenarios, the participants’ individual self-efficacy improved in the absence of changes in observable individual teamwork behaviours. The teams’ clinical performance improved modestly even though the medical goal was not achieved.

In the role of leader, participants gathered information and communicated to a greater extent, and experienced higher levels of mental strain and concentration than they did in the role of follower during the early phase of SBTT. The combination of these differences in individual behaviour and experiences between the two roles demonstrates a higher degree of challenge in the training process of leader skills as compared with follower skills.33

The study's particular strengths

Our study population of medical novices provided an excellent group both for observing teamwork and the natural assignment as leader and follower, and for training in teamwork and the assignment of roles within a team. All participants acting as leader in one scenario had a follower role in another, that is, the participants were their own controls regarding differences in individual behaviours and experiences between the role of leader and that of follower. Cross-training as a leader or follower may also increase understanding of the difficulties specific to each role, and thus, enhance learning.34

Mixed models allow the analysis of data in field studies, that is, data occurring authentically. Applying a mixed model to the present data matrix is advantageous, as these models do not demand a data matrix that is balanced and complete, but make use of available information in the dataset, also taking covariation into consideration. Such models are not hampered by heterogeneity of data, and provide the opportunity for a sparse sampling approach. The statistical methods used in this study were appropriate for the dataset of continuous, ordinal and categorical data, respectively. The size required to indicate significant change in individual behaviour and self-efficacy was chosen to correspond to sound clinical experience in order not to overestimate the importance of minor differences.

In this study, we used a multimodal approach, recording the team's medical management and clinical performance, together with individual teamwork behaviours and experiences in both leader and follower roles. We found differences between individual behaviours and experiences in the two roles during a phase when training had not yet passed the threshold to show improvements in behaviour. These results support those of Kraiger et al,35–37 among others, who proposed that training should be evaluated considering cognitive, skill-based and affective learning outcomes to reach a more comprehensive picture of the training process.

Relation to other evidence

The lack of significant improvement in medical management and clinical performance in response to half-day SBTT in our study is in line with the findings of other studies.8 9 Teamwork behaviour was graded for the entire teams or individually for leaders, and limited changes were reported in early phases of SBTT.10 11 The same lack of change observed in individual behaviour in response to SBTT also applied to individual follower behaviour in this study. Despite this lack of visible success, participants consistently find SBTT extremely positive in published studies.12 13 In these studies, most probably the training curriculum has a high standard and trainers are well-educated, resulting in positive participant reactions.12 13

Improved self-efficacy regarding medical management, clinical performance and teamwork in response to simulation-based training has also been shown by others.38–40 For instance, a crisis resource management course with high-fidelity simulation improved self-efficacy in crisis management for emergency medicine residents,12 and final year medical students after simulation-based teaching reported an increase in self-efficacy.41 However, in the literature, we have thus far not found any data on mental strain and concentration in association with SBTT.

In this study, we used elements of ‘leader behaviours’ and ‘follower behaviours’, respectively, for the behaviour category ‘Take a member role’ in the A-TEAM programme in order to identify leaders and followers during teamwork in the scenarios.27 Although the literature on teamwork and leadership is vast, authors seldom declare precisely how leaders are identified, restricting opportunities to compare our findings with other authors. That participants in the role of leader communicated to a greater extent than in the role of follower was no surprise; the leader directs team and task work, asks for information, confirms and verbalises decisions, gives orders and takes the initiative in team sum-ups. All these actions, or a ‘structuring leadership’, are accompanied by intensive communication and, as such, differ between leader and follower.

The difference in behaviours between the leader and follower roles is in contrast with the lack of difference regarding the other four categories of teamwork behaviours; in the roles of leader and follower, participants scored at the same levels regarding ‘takes a team member role’, ‘contributes to a shared understanding of the situation’, ‘contributes to collaborative decisions’, and ‘coordinates and execute tasks’. This lack of difference in scoring between the roles of leader and follower is in line with recent studies on shared leadership.22 Among others, Kunzle and colleagues have shown that better performance and results in a team are associated with an even distribution of leadership behaviours between team members.25 42 The CRM-based A-TEAM programme used in the SBTT in this study emphasises that leaders should structure teamwork, and followers should focus on tasks, and also that members should be prepared to shift between the two roles depending on the situation, that is, all team members should embrace a ‘situational’ or ‘functional’ leadership. Participants were explicitly informed of the behavioural targets, and received diagnostic feedback for both their role as leader and as follower. As such, the A-TEAM programme appears to be successful in training a shared leadership, as suggested by Pearce et al.43

Limitations

In our training, medical management and clinical performance are discussed in balance with teamwork behaviours, whereas other types of SBTT emphasise teamwork and team processes over task work.7

Another limitation is the study population of medical students in that they were all young and clinically inexperienced. The results regarding individual experiences might not be generalisable to older, more experienced professionals who may be harder to motivate.

Interpretation

The observed increase in self-efficacy was a positive outcome of SBTT after three scenarios. Although behaviour and clinical performance showed only minor changes during the short training period, the increased self-efficacy suggests that the participants’ belief in their personal capability of participating in teamwork, and their motivation and commitment to engage in further training, increased.14 As such, the three scenarios enabled further learning in future sessions with repetitive practice and iterative scenarios.

Concentration was stable during training, helping participants to focus on targets for training. In this study, we were not able to tie concentration to learning outcomes, such as clinical skills and behaviour. Future studies could analyse the association between concentration, clinical skills and behaviour.

Mental strain was also stable, and moderate to strong, during the training period. If mental strain had been higher, it might have hindered learning and team performance.

In the role of leader, participants experienced higher levels of concentration and mental strain than in the role of follower. This difference in mental strain suggests that the acquisition of leader behaviours, and/or leading the team, was more demanding than the acquisition of follower behaviours and/or following in a team. A balanced combination of these higher demands and the higher levels of concentration might have been rewarding.33

Implications and future research

Even though individual experiences are not included in the behavioural targets for SBTT, trainers and researchers should recognise and acknowledge these reactions during training and teamwork.

If team effectiveness results from shared leadership, as suggested by recent research, then team members must be made aware of their individual accountability for leadership and followership as early as during training. It is possible that shared leadership will flourish when individual experiences and affective factors are recognised and acknowledged during training.

Further research is needed in order to shed more light on the participants’ individual experiences of teamwork training, not least in the two roles of leader and follower. Future studies might also explore whether fully qualified professionals show analogous responses in the roles of leader or follower in clinical practice, or in the roles of leader or follower during training.

Conclusions

Exploring the early phase of SBTT, we found improvements in self-efficacy, and differences in individual behaviour and experiences, between the roles of leader and follower. This study shows a new way to evaluate training during the early phase of SBTT. Using a multimodal approach, we identified differences in the training process between the roles of leader and follower. Training in leadership skills, such as the gathering of information and communication, was associated with a higher level of mental strain and concentration. This finding has implications for training and feedback; the desired targets for teamwork behaviours are different for the roles of leader and follower. Consequently, the targets for training and learning outcomes are different. The design of training curricula and scenario training should focus on both roles in order to foster the growth of all aspects of teamwork competence, and conceivably improve patient care.