Impact of Simulator-Based Crisis Resource Management Training on Collective Orientation in Anaesthesia: Pre-Post Survey Study With Interprofessional Anaesthesia Teams.

THEORY: Individuals have different qualities, levels of willingness, and degrees of engagement for working in teams. This behaviour is termed 'Collective Orientation' (CO). Collective orientation can be trained and has a positive influence on team processes. Here, we investigated the effect of a simulator-based, Crisis Resource Management team training upon the participants' CO. HYPOTHESES: We hypothesized (1) the scales of CO and Presence for lab-based microworld research (PLBMR) are applicable to the German anaesthesia teams, (2) the CO can be influenced by means of simulation training, (3) the training effect is dependent on sex and/or profession, and (4) the change of CO depends on the perceived presence of the participants in the scenario.
METHOD: In a pre-post study, 66 nurses and doctors from various anaesthetic departments took part in a 1-day training course to improve non-technical skills. The primary outcome was the mean difference between the CO measured (via questionnaires) immediately before (T1) and after (T2) training. The change was then tested for dependence upon other variables, such as sex, professional group, and immersion into the simulation scenarios.
RESULTS: Collective orientation improved significantly after training (mean difference: 0.2; P < .001; dz = 0.53). Considering the subscales, affiliation increased significantly (P < .001; dz = 0.59), whereas dominance remained unchanged. Furthermore, no correlation was found regarding sex, professional group, or immersion into the simulation scenarios.
CONCLUSIONS: Our study demonstrated that simulation-based training improves the participants' COs, primarily by increasing affiliation. Subjective scenario reality did not significantly influence this. Nonetheless, it remains unclear as to what factors categorically resulted in this benefit. The shared experience in the course by all team members might trigger the effects. However, further studies are needed to identify the modifiable factors that can improve teamwork attitudes.

Introduction

Analysis of critical incidents in industries that require staff to work as part of high responsibility teams (HRT)[1] resulted in the realization that adverse situations are often caused by ‘human errors’. The principles can also be applied to a health care environment, although there are some differences, eg, the working culture.[2] As HRT have to focus not only on technological issues under stressful situations but also teamwork processes – they also require proficiency in various non-technical skills (NTS).[3] The NHS (National Health Service) Education for Scotland developed the ‘Anaesthetists’ non-Technical Skills-Rating Scale’ with the dimensions of task management, decision-making, situational awareness, and team working to deal with a critical situation.[3] Fletcher et al defined teamwork as skills for working in a group context, in any role to ensure effective joint task completion and team member satisfaction. The European Society of Anaesthesiology (ESA) recommends safety trainings for anaesthesia teams to improve non-technical skills.[4]

An early finding regarding teamwork is that every individual brings different qualities, levels of willingness, and degrees of engagement for working in a team.[5] The individual behaviour of team members has been characterized by the term ‘Collective Orientation’ (CO). Diskrell et al[6] defines CO as the propensity to work in a collective manner in team settings. During the development of a measuring scale for CO (in the form of a questionnaire), 2 main factors became apparent. The first main factor was termed ‘Affiliation’ and consisted of 10 items. A high ‘Affiliation’ describes the ability to work both in a goal-oriented manner and with a higher regard for others in a team. Low ‘Affiliation’ is characterized by a preference to work on one’s own. The second factor was labelled ‘Dominance’, which was composed of 5 items. A high degree of ‘Dominance’ demonstrates a priority in having power and control. Low ‘Dominance’ reflects a cooperative working style. All items for both factors are listed in Table 1. The adapted and validated German version was developed by Hagemann.[7] As CO is an attitude which effectively supports teamwork processes,[6,8] it can be positively changed by training of health care providers, if the participants believe in it is important to their work.[9]

Table 1.

Items of collective orientation.[7]

Collective Orientation: Subscales and Items
Introduction: In the following you will see a series of statements. The concern is with your own option. Therefore, there are no ‘right’ or ‘wrong’ answers. Answer the question such that they best apply to you. Please respond to the statements in terms of your personal attitude. Items could be answered on a 5-point-Likert-type scale from 1 (stimme gar nicht zu / totally disagree) to 5 (stimme voll zu / totally agree).
German Version	English Version
Zugehörigkeit	Affiliation
Ich finde die Arbeit an Teamprojekten sehr zufriedenstellend.	I find working on team projects to be very satisfying.
Ich würde eher selbst handeln als auf den Input von anderen zu warten (R).	I would rather take action on my own than to wait around for others`
Ich bevorzuge es eine Aufgabe von Anfang bis Ende durchzuführen, ohne die Unterstützung von anderen (R).	I prefer to complete a task from beginning to end with no assistance from others.
Teams arbeiten normalerweise sehr effektiv.	Teams usually work very effectively.
Ich denke es ist normalerweise besser den Stier bei den Hörnern zu packen und etwas selber zu machen, als darauf zu warten Input von anderen zu bekommen (R).	I think it is usually better to take the bull by the horns and do something yourself, rather than wait to get input from others.
Bei den meisten Aufgaben würde ich eher allein arbeiten, als Teil einer Gruppe zu sein (R).	For most tasks, I would rather work alone than as a part of a group.
Ich kann normalerweise mehr leisten, wenn ich für mich alleine arbeite (R).	I can usually perform better when I work on my own.
Ich finde, dass es meist produktiver ist für mich alleine zu arbeiten als mit anderen (R).	I find that it is often more productive to work on my own
Ich arbeite gerne mit anderen zusammen. (only German version)	I find it easy to negotiate with others who hold a different viewpoint than I hold. (only English version)
Ich finde es nicht gut sich auf andere Teammitglieder verlassen zu müssen (only German version) (R)	I always ask for information from others before making any important decision. (only English version)
Dominanz	Dominance
Wenn ich anderen Teammitgliedern nicht zustimme, neige ich dazu meinem eigenen Bauchgefühl zu folgen (R).	When I disagree with other team members, I tend to go with my own gut feelings.
Wenn ich eine andere Meinung als ein anderes Teammitglied habe, versuche ich normalerweise bei meiner eigenen Meinung zu bleiben (R).	When I have a different opinion than another group member, I usually try to stick with my own opinion.
Es ist wichtig bei der eigenen Meinung zu bleiben, gerade wenn andere um dich herum versuchen dich zu einer Änderung zu bewegen (R).	It is important to stick on your own decision, even when others around are trying to get you to change.
Wenn andere widersprechen, ist es wichtig standzuhalten und nicht nachzugeben (R). (only German version)
Ich finde auch bei Teamarbeiten sollte man immer das tun, was man selbst für richtig hält. (R) (only German version)	When solving a problem, it is very important to make your own decision and stick by it. (only English version)
Wenn ich von etwas überzeugt bin bleibe ich bei meiner Meinung, egal was andere Teammitglieder dazu sagen (R).	When others disagree, it is important to hold one’s own ground and not give in.

Items marked with (R) are negatively worded and have to be reversed-scored.

Simulation-based training methods enable participants to learn in a virtual work environment, which helps foster their NTS. Despite technological advancements, a degree of artificiality in the simulation’s representation cannot be prevented. Factors that negatively affect the presence can also have a negative impact on the performance of teams in a virtual environment,[10] so it is interesting to consider training effects related to perceived reality. To measure presence, Frank and Kluge[11] devised a scale known as ‘Presence Scale for Lab-based Microworld Research’ (PLBMR) and applied it already in the medical context.[12]

The aim of this study was to assess the impact of simulation training for anaesthetic teams upon each participant’s CO in relation to the presence. The hypotheses were (1) the scales of CO and PLBMR are applicable to the German anaesthesia teams, (2) the CO can be influenced by means of simulation training, (3) the training effect is correlated sex and/or profession, and (4) the change of CO correlated with the perceived presence of the participants in the scenario.

Materials and Methods

Study design and population

Our study was a pre/poststudy using survey methodology with participants from various anaesthetic departments in our HAINS (Hannoeverscher Anaesthesie-Intensiv-Notfallsimulator)-Safety simulation programme. The first measurement (T1) was taken immediately prior to the training, and the second (T2) was completed immediately after the course. The items for assessing the presence were also completed after the course. The study was approved by the ethics committee of the Hannover Medical School (no. 2958-2015).

Setting and population

The participants of each course (8 hours) were represented near equally by nurses and doctors, with a maximum combined total of 12 people per date. All were required to be active in the field of anaesthesiology at the time of the study. Participation was voluntary and could be withdrawn at any time without giving reasons.

In terms of content, the topics were designed to increase competencies in teamwork, such as situational awareness, decision-making, and communication. As learning objectives for the competencies on NTS, the participants learned the CRM key principles of Rall et al[13] (Table 2).

Table 2.

Fifteen Crisis Resource Management (CRM) key principles published by Rall et al[13] considered as a template for the debriefing of the scenarios.

Know the environment

Anticipate and plan

Call for help early

Exercise leadership and followership

Distribute the workload

Mobilize all available resources

Communicate effectively

Use all available information

Prevent and manage fixation errors

Cross (double) check

Use cognitive aids

Re-evaluate repeatedly

Use good teamwork

Allocate attention wisely

Set priorities dynamically

Demographic data

Demographic data were gathered by the questionnaire at T1. The professional position (eg, assistant medical director, attending physician, resident physician, specialist nurse, nurse), the level of care (eg, outpatient care, standard-level care, advanced-level care, and maximum-level care [eg, university medical centre]), years of work experience, and sex was inquired.

‘CO’ questionnaire

Collective orientation was measured using the validated German version of a paper-based questionnaire.[7] Items could be answered on a 5-point Likert-type scale from 1 (trifft nicht zu/I totally disagree) to 5 (trifft vollständig zu/I totally agree). The internal consistency of the overall scale (α = .79) as well as the subscales ‘affiliation’ (α = .82) and ‘dominance’ (α = .70) proved to be satisfactory. The mean of all items was used as the marker for the CO.

‘Presence’ questionnaire

Frank and Kluge[11] devised the scale ‘Presence Scale for Lab-based Microworld Research’ (PLBMR) (Table 3). All 6 items of the questionnaire were rated with a 6-point Likert-type scale from 1 (‘totally disagree’) to 6 (‘totally agree’). The internal consistency of the scale proved to be α = .71. The mean of all items was used as the marker for the presence during the simulation scenarios.

Table 3.

Adapted items of the Presence Scale for Lab-based Microworld Research (PLBMR) for German and English trainees.[11]

German	Translated into English
Introduction: You will now read a series of statements that each describe their perception of the simulation world. Indicate to what extend the statement applies. The size of the number correlates with the approval. Three examples were given with big approval and big dislike. Items could be answered on a 6-point Likert-type scale from 1 (trifft nicht zu/I totally disagree) to 6 (trifft vollständig zu/I totally agree).
Ich habe mich als Teil der Simulationswelt gefühlt.	I felt like I was part of the Simworld.
Die Simulationswelt hat bei mir Emotionen (z.B. Ärger, Traurigkeit, Zufriedenheit) ausgelöst.	The simulation world triggered my emotions (eg, anger, sadness, satisfaction)
Die Arbeit mit der Simulationswelt war für mich zufriedenstellend.	Working in the simulation world was satisfying for me.
Während ich in der Simulation war, habe ich zwischenzeitlich vergessen, dass ich an einer Studie teilnehme.	While operating the simulation, I forgot for the time being that I was taking part in a study.
Die Arbeit in der Simulationswelt war für mich langweilig.	Working in the simulation world was boring for me.
Während ich in der Simulation war, bin ich gedanklich in die Simulationswelt abgetaucht.	While operating the simulation, my thoughts became immersed in the simulation world.

Protocol

The interventions were a single 1-day training course (8 hours), composed of both lecture material and multiple scenarios. Each course started with a lecture with the basics of Crisis Resource Management and non-technical skills, followed by an introduction to the simulation environment. Subsequently, the subjects participated in a total of 6 different medical situations. These critical situations included ‘difficult intubation’, ‘anaphylaxis’, and ‘severe bleeding’. Each scenario required approximately 15 minutes to complete. In case of technical problems with the material, a simulated nurse could help. When not actively participating in a scenario, they would observe the other scenarios via a 1-way mirrored pane. The SimMan3G (Laerdal Medical, Norway) was used as a patient simulator during the scenarios. A 20-minute debriefing took place after each scenario, during which time the participants discussed their performances with the facilitator and the learning group. All debriefings were conducted by a small team of anaesthesiologist, using the debriefing approach described by Rudolph et al.[14]

Statistical analysis

Demographic data and survey data were analysed in a descriptive manner. The normal distribution of the values was revised using the Kolmogorov-Smirnov test. For testing hypothesis (a), the reliability of the scales determined by Cronbach alpha was calculated. To test hypothesis (b), a t test for dependent samples was conducted. To test hypothesis (c), whether training effect correlates with professional group or sex, a univariate analysis of variance (ANOVA) with repeated measures was conducted. To test hypothesis (d), if a change in CO correlates with the perceived presence, a Pearson correlation was conducted. We assumed a P < .05 as being statistically significant. As effect size, Cohen d was calculated. All calculations were made using SPSS Statistics 24 (IBM Corporation, Foster City, CA, USA).

Results

Although all 66 participants filled out the questionnaires, not all information was completed – 3 participants provided no information regarding their profession, 10 gave no information regarding their work environment, 4 provided no information about their work experience, 6 had incomplete ‘Presence’ items, and 12 incomplete CO items (9 incomplete ‘Affiliation’ subscales, 3 incomplete ‘Dominance’ subscales). Incomplete questionnaires regarding subgroup ‘Affiliation’ or ‘Dominance’ were not included in the analysis of the respective item. A total of 60 sheets of the survey ‘Presence’ were complete and could be evaluated. In total, 53% of the participants were female, 47% were male. The professional position was 12 nurses, 14 specialist nurses, 5 assistant medical directors, 13 attending physicians, 19 resident physicians. The level of care ranged from outpatient care (1.5%), standard-level care (15.2%), advanced-level care (21.2%), and maximum-level care (31.0%). The work experience was given as an average of 11.65 years (1-36 years, SD: 9.9 years). Subgroups of professional groups and level of care were not statistically evaluated as the standard deviation was too big to process the data.

Reliability of the scales

To test hypothesis (a), Cronbach alpha was calculated. The result for the questionnaire CO was .74 (T1) and .84 (T2). Subdivided by subscale and time of measure, Cronbach alpha was .65 (T1) and .75 (T2), respectively, .65 (T1) for the subscale ‘affiliation’, and .81 (T2) for the subscale ‘dominance’. No item reduction led to a substantial increase in Cronbach alpha.

For the questionnaire PLMBR, Cronbach alpha was .47. The elimination of 1 item increases the value to 0.54. Therefore, we did not include this item in any further calculations.

Change in ‘CO’

To test hypothesis (b), the change in CO through a simulation training was evaluated via CO scores from the questionnaires before (T1) and after (T2) training (Figure 1). The normal distribution was given for all data. Collective orientation was M = 3.58 (SD = 0.43; n = 66) prior to training and M = 3.78 (SD = 0.47; n = 66) after the training. For the subscale ‘Affiliation’, the respective results were M = 3.63 (SD = 0.44; n = 66) and M = 3.88 (SD = 0.48; n = 66). For the subscale ‘Dominance’, we, respectively, found M = 3.51 (SD = 0.61; n = 66) and M = 3.62 (SD = 0.67; n = 66). In comparison with T1, the CO at T2 increased significantly, t(65) = −4.27, P < .001, dz = 0.53. Considering the subscales, the value for the dimension affiliation also increased significantly after training, t(65) = −4.76, P < .001, dz = 0.59. The change from T1 to T2 in the dimension dominance was not significant, t(65) = −1.68, P > .05, dz = 0.21.

Figure 1.

Changes in ‘collective orientation’ (CO) with its subscales ‘Affiliation’ and ‘Dominance’ prior to (T1) and after (T2) training. * indicates the significant difference for CO (P < .001; dz = 0.53) and ‘Affiliation’ (P < .001; dz = 0.59). ‘Dominance’ showed no significant change (P > .10).

Impact of profession and sex on ‘CO’

To test hypothesis (c), whether the potential training effect on CO is correlated with profession or sex, univariate ANOVAs with repeated measures were conducted. There were no significant results for the interaction between the change in CO and ‘professional group’ or ‘sex’. However, the regression coefficients show a difference for CO between the professional groups at measurement point T2 (β = 0.237; P < .05) as well as for affiliation (β = 0.247; P < .05). The increase in CO and affiliation was stronger for nurses than for physicians.

Impact of ‘presence’ on ‘CO’

To test hypothesis (d), whether the effect of CO is correlated to the perceived presence, the questionnaire PLMBR was carried out. Figure 2 shows that there is no systematic association between the change in the CO and the presence of the participants.

Figure 2.

Change in ‘Collective Orientation’ (CO) in relation to presence. There is no systematic association between the change in the CO and the presence of the participants (n = 54).

Further analyses showed that there was no systematic variation between duration of work experience and CO as well as the change in CO. In addition, no significant correlations were found between presence and duration of work experience.

Discussion

The aim of this study was to investigate the influence of a simulator-based crisis resource management training on the CO of the participants.

Our primary hypothesis of the study was to test the existing scales for measuring CO and presence in our targeted group. All values of Cronbach alpha are higher than .65 which are acceptable for low-stakes instruments designed for programmatic decision-making that do not have direct consequences for individuals.[15] The result of presence is .54, so the interpretation has to be carefully, even more as they are self-reported. Individual differences of the participants, such as real-life work experiences, may have influenced their presence outcomes. It has been described that participants may attempt to mentally ‘recreate’ actual life situations in a simulation[16] – they often experience simulations more realistically if they have undergone similar incidents previously. This may also be the reason of a wide variation of presence and low Cronbach alpha values.

The results supported hypothesis (b), namely that CO can be improved by a simulation-based team training. This improvement was mainly due to an increase in the dimension ‘Affiliation’. The participants experienced a critical situation together, where the importance of teamwork was not only emphasized by a theoretical introduction but also during the debriefings. The importance of team performance for the success of the patient care could be experienced by observing other high-quality teams as well as through participation in their own scenario. Own experience and the relevance of the topic are known determining factors for learning success.[17]

The effect size Cohen d shows a medium effect, which is satisfactory. The practiced skills, such as teamwork and communication, are very much shaped by corporate culture and are generally difficult to change.[18] The effect size shows that the simulation course can be a first step in change.

The dominance remained essentially unchanged during our study. As opposed to affiliation, dominance is a more difficult quality to assess regarding non-technical skills. On the one hand, a team leader in a critical situation must make rapid decisions and ensure that these decisions are efficiently executed. On the other hand, the team leader must remain open-minded and receptive to input from all team members so as to ensure that the treatment plan is the optimal strategy. The variable handling with these types of communication and task management will often be dependent upon the actual clinical situation. From the authors’ point of view, a single questionnaire, including those from our study, can hardly remotely predict or accommodate for this dynamic variability in dominance’s value during critical situations. The role of dominance and its relation to team leadership during critical situations is worthy of further study.

In addition, we saw no significant differences between the 2 professions and the effect of CRM training on CO. Both professional groups are likely to be difficult to fully classify due to the variabilities in their professional training, workplaces of employment, and subspecialization. Our result potentially reflects the special work environment for anaesthesiology in Germany which is characterized by a direct one-to-one proportion of nurses to physicians. In contrast to work at the ward level, the cooperation time in anaesthesiology with the respective ‘other’ professional group tends to be more intense than with their own profession.

Nonetheless, the increase in CO was higher (not significant) for nurses than for physicians. Perhaps the nurses became more aware of their own influence on the success of the treatment and their role in the team. Thomas et al[19] showed that physicians and nurses have discrepant attitudes about the teamwork they experience with each other. Further studies with a focus on these relationships would be interesting.

As for our fourth hypothesis, we found no correlation between presence and the effect of training upon CO. Few studies that have compared high- and low-fidelity simulation and their respective effects upon training have not been congruent: some studies have shown no difference at all,[20,21] whereas others have found an advantage in high-fidelity simulation[22] and some an advantage in low-fidelity simulation.[23] All of these studies had different study populations, learning objectives, and course structures. The aim of training and kind of participants have an influence.[24] Our explanation is that the interprofessional structure of the simulation course directly affects the CO, rather than the simulation scenarios themselves. In addition, debriefing is known to play a major role: Savoldelli et al[25] have shown that exposure to a simulation crisis without constructive debriefing offers little benefit to the trainees. In the debriefing, the participants have the opportunity to recognize and understand the point of view of the team members observing from outside of the stressful situation. Furthermore, while debriefing, participants learn the reflection process of any demonstrated behaviour.[26] These could be reasons for, or contribute to, improvements in the CO.

Limitations

There are some limitations associated with this study. First, the course content was created by trainers from a single centre and might not be directly comparable to other CRM trainings. Although the participants came from several anaesthesia departments, it is a monocentric study. Potentially, the subgroups were too small to see effects regarding sex and professional groups. Furthermore, the transfer into the clinical routine was not measured. It is possible that the measured effect only occurs in the simulated environment with the participating team members. Nor do we know how long any training effect might persist. The low values of Cronbach alpha in the measurement of presence suggest that there might be better instruments for assessment. Further research has to show whether there could be a better instrument for the participants of health care providers.

Conclusions

Simulation-based crisis resource management training can positively influence the CO of participants. The dimension ‘Affiliation’ is the main goal in a group of experienced health care providers. The factors that influence the training effect remain unclear; the shared experience and review of critical situations can be a reason. Presence does not appear to be an important factor for all participants. Further studies are required to assess for transfer of true benefits to the workplace after improvements in CO by these trainings.