In situ simulation improves perceived self-efficacy of OR nurses and anaesthesiologists during COVID-19 pandemic.

Introduction: Self-efficacy is defined as people's internal beliefs about their ability to have an impact on events that affect their lives. As part of the COVID-19 pandemic, we carried out in situ simulation for anaesthesiologists and operating room (OR) nurses. Simulation was focused on the recommendations on the use of specific personal protective equipment (PPE) as well as on airway management and intubation. We hypothesised that in situ procedural simulation should increase their perceived self-efficacy.
Methods: Between 16 March and 20 March 2020, 208 healthcare workers took part in in situ procedural simulation. A questionnaire was sent to participants on 21 April 2020. Six self-efficacy items related to PPE and airway manoeuvres were assessed before and after training on a Numeric Rating Scale from 0 to 10.
Results: Sixty-seven participants (32%) replied to the questionnaire. The before-after comparison of the six items revealed an increase in perceived self-efficacy for each of them. A before training difference was observed between nurses, board-certified anaesthetists and trainees in anaesthesia in perceived self-efficacy for putting on (6 (3-8) vs 4.5 (2.25-6) vs 2 (0-6), p=0.007) and remove PPE (8 (5-8) vs 4.5 (3.25-6) vs 4 (1-6), p=0.009). No difference in perceived self-efficacy after training was observed between nurses, board-certified anaesthetists and trainees in anaesthesia. Conclusions: In situ simulation improves the perceived self-efficacy of OR nurses and anaesthesiologists on specific skills related to the care of patients with COVID-19. © Author(s) (or their employer(s)) 2021. No commercial re-use. See rights and permissions. Published by BMJ.

The perception of being in control (self-efficacy), rather than the reality of being in or out of control, is a buffer of negative stress.

Working under extreme stress may cause healthcare professionals to deviate from clinical guidelines.

In situ simulation has been recognised as an educational strategy that might help change system-based risk factors and improve safety.

In situ procedural simulation improves the perceived self-efficacy of operating room nurses and anaesthesiologists on specific skills related to the care of patients with COVID-19.

Self-efficacy is positively related with the level of confidence of these healthcare professionals when taking care COVID-19 infected patients but only partially reduces stress.

Introduction

COVID-19 is predominantly caused by contact or droplet transmission related to the dispersion of relatively large respiratory particles contaminated by SARS-CoV-2 that are subject to gravitational forces and travel only approximately 1 m from the patient.1 Airborne transmission occur if patient respiratory activity or medical procedures generate respiratory aerosols.2–4 These aerosols contain particles that may travel much longer distances and remain airborne longer, but their infective potential is uncertain. Contact, droplet and airborne transmission are relevant during airway manoeuvres in infected patients, particularly during tracheal intubation.5

Therefore, it has been recommended to healthcare professionals to use a specific personal protective equipment (PPE) as well as to have recourse to specific airway management algorithms. Moreover, several authors have recommended specific training to facilitate the transfer of recommendations on the use of specific PPE as well as on airway management and intubation6 into practical skills and therefore to improve healthcare professional safety.7

Quickly, the need for such a training appeared in our institution for operating room (OR) healthcare professionals having to take care of SARS-CoV-2 positive patients in the context of surgical and obstetrical emergencies but also as part of the mobilisation of OR professionals to strengthen emergency and intensive care units.

We chose to train these specific guidelines with in situ simulation that has been recognised as an educational strategy that might help change system-based risk factors and improve safety, including during COVID-19 pandemic.8 9

We hypothesised that the use of two in situ simulation-based workshops would increase perceived self-efficacy of the team members regarding PPE and airway management specific procedures in addition to facilitating their transfer in order to improve safety when taking care of infected patients with SARS-CoV-2. Self-efficacy is defined as people’s internal beliefs about their ability to have an impact on events that affect their lives.10 Moreover, the perception of being in control, rather than the reality of being in or out of control, is a buffer of negative stress. This stress reduction should not be overlooked as a recent review11 of the mental health consequences of COVID-19 shows an increase of depression or depressive symptoms and anxiety, a poor quality of sleep or higher levels symptoms of obsessive-compulsive disorder in healthcare professionals. Furthermore, working under extreme stress may cause healthcare professionals to deviate from clinical guidelines.

The main objective of this work is to assess the effect of in situ simulation on self-efficacy of nurses and anaesthetists in relation with the use of PPE and airway procedures in the OR in the context of the COVID-19 pandemic.

The secondary aims are to assess the perceived value of this in situ simulation training in clinical practice and as compared with other learning methods and to assess self-confidence of OR nurses and anaesthetists when taking care of patients with COVID-19.

Material and methods

Study design

Retrospective before - after study by questionnaire. A waiver of written consent was granted.

Study population

Procedural simulation sessions were proposed to all OR nurses and members of the anaesthesia department, staff and trainees of the CHU de Liège. The sessions were set up from 16 March to 20 March 2020. For safety reasons, the members of the team presenting COVID-19 compatible symptoms were excluded.

Intervention: simulation sessions

The sessions took place in an unoccupied OR during working hours. The simulation sessions consisted of two 20 min workshops. Participants were assigned to groups of maximum four individuals in order to respect physical distancing. These sessions were conducted by two board-certified anaesthetists who are also validated simulation instructors.

The first workshop focused on the procedure to put on and remove PPE. Each participant completed the procedure in real working conditions, that is, usual environment and team help. Due to limited availability of disposable material, class 2 filtering facepiece (FFP2) masks were replaced for simulation session by coffee filters fitted with elastic bands placed over the surgical masks, and the disposable protective gown by reusable cloth gown washed after each use.

The second workshop focused on the specifics of OR intubation manoeuvres according to the guidelines of the Société Française d’Anesthésie-Réanimation,5 as well as on tracheal aspiration and extubating procedures. For this workshop, three participants from the group performed the manoeuvres with the instructor, dividing up the roles and positions in the OR workspace according to the guidelines. Namely the most experienced anaesthetist was at the head of the simulated patient, an assistant (nurse) on the left for the management of the intubation material, that is, video laryngoscope, tracheal tube and clamps, and an assistant (anaesthesiologist or nurse) on the right for the management of the narcosis, the anaesthesia machine (eg, gas flow pause) and the monitoring (eg, use of curarisation monitoring). To provide a realistic simulation of managing intubation, an intubation head was use. If present, the remaining participant observed the simulation session.

Data collection

An e-link to an anonymous questionnaire was sent to the 208 participants via their professional external email address on 21 April 2020. A weekly reminder was sent for the following 3 weeks. In addition to characteristics (gender, age, profession and years of expertise in the OR), the questionnaire assessed several items regarding perceived self-efficacy, usefulness of the simulation session or interest of the simulation tool. The items are listed below. A Numeric Rating Scale between 0 and 10 was used to assess the degree of agreement.

Perceived self-efficacy items were assessed before and after training. These assertions were

‘I feel competent to put on PPE to take care of a COVID-19 patient in the OR’.

‘I feel competent to check that an FFP2 mask is correctly placed’.

‘I feel competent to undress PPE without risk of contamination’.

‘I feel competent to perform an induction and intubation sequence while minimizing the risk of contamination’.

‘I feel competent to perform a tracheal suction while minimizing the risk of contamination’.

‘I feel competent to perform an extubation sequence while minimizing the risk of contamination’.

The perception of the usefulness of the simulation session was explored using to the following questions:

‘In general, I found the simulation session on putting on and removing PPE useful for my clinical practice when managing COVID-19 patients?’

‘In general, I found the simulation session on intubation and extubation useful for my clinical practice when managing COVID-19 patients?’

‘What I learned during this simulation session modified my clinical practice including for non-suspected patients’.

‘What I learned during the simulation session on putting on and removing PPE helped reduce my stress about taking care of COVID-19 patients’.

‘What I learned during the simulation session on intubation and extubation allowed me to reduce my stress about taking care of COVID-19 patients’.

The interest of the simulation tool as compared with other education tools was assessed using the relative compliance to the following statements:

‘Simulation was more useful compared to a written document’.

‘Simulation was more useful compared to a video’.

‘I enjoyed the simulation was performed in teams rather than by profession (nurses with nurses, doctors with doctors)’.

‘I would have preferred the simulation workshop to be carried out by profession’ and ‘I appreciated that the simulation was carried out in the OR rather than in a training room (eg, simulation center)’.

‘I would have preferred the simulation workshop being carried out in a training room (eg, simulation center)’.

Finally, the questionnaire explored the number of COVID-19 positive patients treated after the simulation session and the level of confidence in relation to the specific skills in this situation (putting on PPE, removing PPE, manipulation of the airways…) during the management of the first patient.

Statistical analysis

Results were expressed as proportions (percentage), mean±SD or median value (IQR) as specified. According to a Shapiro-Wilk normality test, parametric data between groups were compared by unpaired Student’s t-test and non-parametric data by Mann-Whitney rank-sum test. Categorical data were compared using χ2 test and Fisher’s exact test using a two-tailed probability. Paired data were compared using paired Student’s t-test. A multivariate logistic regression model (backward stepwise model) was used to determine independent risk factors for level of confidence in relation to the specific skills in this situation (putting on PPE, remove PPE, manoeuvres on airways,…) when taking care of the first patient, keeping in the equation the variables that were found relevant in the univariate analysis.

A p value less than 0.05 was considered to be significant. Statistical analysis was performed with JMP V.14.2 (14.0) (SAS Institute Inc).

Results

Participants characteristics

On 31 May 2020, 67 participants replied to the questionnaire, for a response rate of 32%. Among the respondents, there were 16 board-certified anaesthetists out of 26 participants (61%), 19 trainees in anaesthesia out of 37 participants (51%) and 32 nurses out of 145 participants (22%) (table 1).

Table 1

Respondent’s characteristics

	N=67
Male/female ratio	25 (37) / 42 (63)
Age category
<25 years old	3 (5)
25–40 years old	38 (57)
41–50 years old	11 (16)
>50 years old	15 (22)
Profession
Board-certified anaesthetists	16 (24)
Trainees in anaesthesia	19 (28)
Nurses	32 (48)
Operating room experience
<1 year	5 (7)
1–5 years	24 (36)
6–10 years	6 (9)
11–20 years	12 (18)
More than 20 years	20 (30)

Data are presented as numbers (%).

Perceived self-efficacy

The global perceived self-efficacy before and after training is shown in table 2. Their comparison that demonstrates a significant increase in perceived self-efficacy for each of the items is also shown in table 2.

Table 2

Before and after training global perceived self-efficacy and their comparison

Items	Before trainingn=67	After trainingn=67	Mean difference between before and after training	P value(two tailed)
I feel competent to put on personal protective equipment to take care of a patient with COVID-19 in the OR	5 (2–7)	8 (8–9)	3.51±0.36 (2.78 to 4.23)	<0.0001*
I feel competent to check that an FFP2 mask is correctly placed.	7 (4–8)	9 (8–10)	2.42±0.38 (1.67 to 3.17)	<0.0001*
I feel competent to remove personal protective equipment without risk of contamination	6 (3–8)	8 (8–9)	2.91±0.36 (2.20 to 3.62)	<0.0001*
I feel competent to perform an induction and intubation sequence while minimising the risk of contamination	5 (2–7)	8 (7–9)	3.09±0.33 (2.44 to 3.74)	<0.0001*
I feel competent to perform a tracheal suction while minimising the risk of contamination	4 (1–6)	8 (6–9)	3.08±0.34 (2.41 to 3.77)	<0.0001*
I feel competent to perform an extubation sequence while minimising the risk of contamination	4 (2–7)	7 (7–9)	3.18±0.31 (2.55 to 3.81)	<0.0001*

Data are presented as median (IQR); mean±SD (95% CI).

*Significant.

OR, operating room.

The perceived self-efficacy assessment by profession is shown in table 3. A significant difference in perceived self-efficacy before training was observed between nurses and anaesthetists (board certified and trainees) in putting on PPE (p=0.007) and removing PPE without risk of contamination (p=0.009). A difference was also observed between nurses and trainees in anaesthesia regarding extubation (p=0.05). No other difference was observed on the before training items between the nurses and the doctors.

Table 3

Self-efficacy before and after training by profession

Items	NursesN=32		Board-certifiedanaesthetistsN=16		Trainees in anaesthesiaN=19		P value
	Before	After	Before	After	Before	After	Before	After
Putting on personal protective equipment to take care of a COVID-19 patient in the OR	6 (3–8)	8 (7–9)	4.5 (2.25–6)	9 (8–9)	2 (0–6)	8 (8–9)	0.007*	0.50
Checking that an FFP2 mask is correctly placed	7 (3.5–8)	8.5 (8–9)	7 (4.5–8)	9 (8–10)	6 (2–8)	9 (7–10)	0.58	0.40
Removing personal protective equipment without risk of contamination	8 (5–8)	8 (8–9)	4.5 (3.25–6)	8.5 (8–9)	4 (1–6)	8 (8–9)	0.009*	0.44
Performing an induction and intubation sequence while minimising the risk of contamination	6 (2.25–8)	8 (7–9)	4.5 (2–6)	8 (8–9)	3 (2–5)	8 (6–9)	0.21	0.34
Performing a tracheal suction while minimising the risk of contamination	5 (1–7)	7 (6–8.75)	3.5 (2–5.75)	8 (7.25–9)	2 (0–5)	7 (5–8)	0.16	0.35
Performing an extubation sequence while minimising the risk of contamination	5.5 (2.5–7)†	8 (7–9)	3.5 (2–6)	8.5 (7–9)	2 (0–5)†	8 (6–9)	0.11	0.53

Data are presented as median (IQR),

*Significant.

†Significant difference between two groups, p=0.05.

OR, operating room.

No significant difference in perceived self-efficacy after training was observed between the nurses and board-certified or trainee anaesthetists for the different items.

A lesser increase in perceived self-efficacy was observed among nurses concerning putting on PPE with an average difference of 2.21±0.47 (95% CI 1.25 to 3.18) compared with 4.69±0.47 (95% CI 3.73 to 5.64) among doctors (p=0.0005). A lesser increase in perceived self-efficacy was also observed concerning removing PPE between nurses with a difference of 1.91±0.47 (95% CI 0.94 to 2.87) and doctors with a difference of 3.82±0.49 (95% CI 2.84 to 4.81) (p=0.006).

Perception of the usefulness of the simulation session

The participants agreed strongly (with 9 (8–10)) that the simulation session on putting on and removing PPE was useful for their clinical practice when managing patients with COVID-19 patients. Trainees in anaesthesia, however, find this part of the training less useful than board-certified anaesthetists with a score of 8 (8–10) versus 9.5 (9–10) (p=0.02). Participants also agreed at 9 (8–10) that the simulation session on airway maneuvers was useful for their clinical practice when managing patients with COVID-19.

Participants agreed with 8 (7–10) that what they learned during the simulation session changed their clinical practice, including for non-suspect patients.

Regarding their stress in relation to the care of patients with COVID-19, the participants moderately agreed that the putting on and removing PPE simulation session reduced their stress with a median of 7 (5–9). The feeling is similar for the intubation and extubation session with a median of 7 (6–9).

For participants who managed COVID-19 positive patients, the level of confidence in their specific skills was 7 (6–8) when the first patient was managed. A simple regression analysis showed a significant link between the level of confidence of the participants when taking care of the first patient and the perceived self-efficacy at the end of the training for each of the items analysed, namely putting on PPE (p=0.0005), checking the correct positioning of the FFP2 mask (p=0.003), removing PPE without risk of contamination (p<0.0001), performing an induction and intubation sequence (p<0.0001), performing tracheal aspiration (p<0.0001) and performing extubation (p<0.0001) by limiting the risk of contamination. In a multivariate analysis, only the perceived self-efficacy to perform an extubation while minimising the risks of contamination was found as a significant predictor of the level of confidence (p<0.0001).

Interest of the simulation tool as compared with other education tools

Participants strongly agree that the simulation had an advantage compared with a written document with a score of 10 (8–10). They also believe that the simulation was more useful than a video with a score of 8 (7–10).

Participants appreciated that the simulation was carried out in a team rather than by profession with a strong degree of agreement at 9 (8–10). In comparison, the score was 2 (0–5) for a workshop carried out by profession.

Participants also appreciated that the simulation was performed in the OR rather than in a simulation centre with a score of 9 (8–10). In comparison, the level of agreement with a simulation workshop carried out in a simulation centre was low with a score of 1 (0–4).

Training and patient care

Sixty-two respondents (92%) had to care for COVID-19 positive patients between the training and the survey. For half of the respondents, the number of patients treated was less than five patients. When the first patient was considered, they estimated their level of confidence in the procedures at 7 (6–8).

Discussion

The salient result of this study is that a session of in situ team procedural simulation improves perceived self-efficacy of OR healthcare professionals about OR-specific procedures for patients with COVID-19 namely the use of PPE and adapted airways manoeuvres. In addition, the participants found the learning made during these simulation sessions very useful for their clinical practice. Lastly, in situ simulation was favoured as compared with learning based on written document or video. Team and in situ sessions were preferred to sessions organised by profession or in a specific simulation centre.

The training increased the perception of self-efficacy of all participants, whatever their profession. Nevertheless, the increased self-efficacy was significantly lower for specific items related to putting on and removing PPE among nurses compared with doctors. This difference can be explained by the scarcity of basic hospital hygiene courses in the medical initial education, in particular on PPE, as compared with the nursing education. This is consistent with the higher level of self-efficacy on these items before training among nurses compared with doctors. The lower perceived self-efficacy in physicians after they received written recommendations before their training could also be explained by the following two factors. The first factor is the degree of uncertainty with regard to the situation and the management of patients at the beginning of the crisis, reinforced by the unsolved questions after reading the written information received.12 This could also explain the trend of the higher level of self-efficacy of nurses who did not receive the written documents before the training. A second factor is the nature of the tool: a written document or a video compared with simulation developing experiential learning. Indeed, experiential learning appears to be particularly effective in contexts in which complex information must be processed and contexts in which deeply ingrained behavioural attitudes are challenged,13 which was the case at the start of the crisis with regard to airway management in the OR.

In situ workshops practice might have favourably influenced practices for other patients also. Indeed, it has been shown that the participants in in situ simulations provide more ideas for changes.14 Regarding the difference in perception between board-certified anaesthetists and trainees on the usefulness of these simulations for putting on and remove PPE, the role of each intervener in the care of patients should be taken into account. Indeed, the management of the airways was attributed to the most experienced anaesthesiologist. Board-certified anaesthetists, generally older, might also have perceived the usefulness of the PPE procedures having in mind that the personal risk of serious illness was higher with age. Behavioural differences have already been observed, for example, among older healthcare workers who are more likely to be vaccinated against seasonal influenza.15

The moderate effect on the stress reduction associated with the training when managing patients should be seen in view of the other constraints imposed on the OR by the epidemic, such as limited availability of disposable equipment, the organisation of the team or also by the numerous pathophysiological uncertainties concerning the disease and the quick improvement evolution in disease knowledge.

Finally, it is interesting to note that the level of confidence when taking care of actual patients with COVID-19 is associated to the level of perceived self-efficacy at the end of the training. Likewise, the only factor predicting the level of confidence in patient care is the perceived self-efficacy for the extubation phase of COVID-19 positive patients. This is remarkable since no written recommendation has developed extubation of patients in the OR. These two points demonstrate the value of in situ procedural simulation training in the context of an emerging health crisis.

This study has several limitations. First, the overall response rate16 was mild: 32%. Although this is an adequate rate in relation to the methodology used, we cannot extrapolate that it is indeed a representative sample of the population. Indeed, if the medical population is well represented with more than 50% of responses, the response rate of nurses is quite low. This low rate is probably explained by the survey methodology chosen via the professional external email address. Indeed, the institution offers an internal email address and an external address. Nurses consult the external e-mail address less often than physicians. In addition, the link to the survey was blocked on some institutional computers by a firewall. On the opposite, doctors more often consult their professional address via personal devices that are prohibited for nurses within the institution during working hours. Second, the survey was an afterthought. The memory nature of the perceived self-efficacy can influence the response. However, as the training was decided and developed in less than 24 hours due to the urgency of the situation, we did not have time to conduct the survey in a prospective manner. Moreover, in this period of crisis, knowledge and protocols have evolved extremely rapidly. A discrepancy may have existed between the learning made during the training and the practices in progress during the survey. Finally, the responses spanned a little over a month and were carried out after the peak of the epidemic, which may have influenced or changed the participant’s perceived feeling.

Our results relate only to anaesthesiologists and nurses in the OR at the start of the COVID-19 pandemic in Europe, limiting their extrapolation to other target audiences in OR or for other anaesthetists working in, for example, in an intensive care unit, would have had the same impact on the self-efficacy. Likewise, we have not evaluated the impact of the context of an emerging pandemic on the effectiveness of the simulation training system.

Conclusions

The in situ procedural simulation has the potential to improve the perceived self-efficacy of OR nurses and anaesthesiologists on specific skills related to the care of patients with COVID-19.

The perceived self-efficacy is positively related with the level of confidence of these healthcare professionals when taking care of COVID-19 infected patients in the current course of this pandemic but only partially reduces stress.