Investigating effects of healthcare simulation on personal strengths and organizational impacts for healthcare workers during COVID-19 pandemic: A cross-sectional study.

INTRODUCTION: This cross-sectional study aimed at evaluating impacts of healthcare simulation training, either in-situ or lab-based, on personal strengths of healthcare workers (HCWs) and organizational outcomes during the COVID-19 pandemic.
METHODS: COVID-19 Taskforce was established to formulate standardized scenario-based simulation training materials in late-January 2020. Post-training questionnaires made up of 5-point Likert scales were distributed to all participants to evaluate their personal strengths, in terms of i) assertiveness, ii) mental preparedness, iii) self-efficacy, iv) internal locus of control, and v) internal locus of responsibility. Independent sample t-tests were used to analyze between-group difference in "In-situ" and "Lab-based" group; and one-sample t-tests were used to compare change in personal strengths with reference point of 3 (Neutral). Kirkpatrick's Model served as the analytical framework for overall training effects.
RESULTS: Between 05 February and 18 March 2020, 101 sessions of simulation training were conducted in "In-Situ" at either Accident & Emergency Department (20, 20%) or Intensive Care Unit (15, 14%) and "Lab-based" for Isolation (30, 30%) and General Wards (36, 36%). 1,415 hospital staff members, including 1,167 nurses (82%), 163 doctors (12%) and 85 patient care assistants (6%), were trained. All domains of personal strengths were scored 4.24 or above and statistically significantly increased when comparing with reference population (p < .001). However, no significant differences between in-situ and lab-based simulation were found (p > .05), for all domains of personal strengths.
CONCLUSION: Healthcare simulation training enhanced healthcare workers' personal strengths critical to operational and clinical outcomes during the COVID-19 pandemic.

Introduction

Collective Unconscious from SARS to COVID-19 in Hong Kong

A novel coronavirus (also known as “COVID-19” or “SARS-CoV-2”), initially emerged in Wuhan, Hubei Province in December 2019. At the end of the year, the Wuhan Municipal Health Commission reported that 27 unknown pneumonia cases had been exposed to the Huanan Seafood Wholesales Market. On 20 January 2020, Prof. Zhong Nanshan, head of the Health Commission, affirmed that two pneumonia cases in Gongdong were infected through human-to-human transmission. On 23 January 2020, the first imported case from the mainland China was admitted to Queen Elizabeth Hospital (QEH), a public hospital under the governance of Hospital Authority (HA)1 in Hong Kong Special Administrative Region (HKSAR). Flashbacks of Severe Acute Respiratory Syndrome (SARS) were still vivid.

HKSAR occupied over 1-fifth of SARS cases (1755/8096), with 0.77 increased risk ratio of Case Fatality Rate (CFR) among 29 countries in 2003.1, 2 Of 386 local Healthcare workers (HCWs) infected, 3 public and 2 private hospital medical officers, 1 nurse and 3 healthcare assistants have sacrificed their lives to safeguard public safety of Hong Kong during the epidemic.3, 4 In the concurrent battle with COVID-19, local HCWs’ safety and health conditions, physically and mentally, have been taken into consideration. Except more stringent infection control measures within hospital compounds, simulation training could be a hope for filling the knowledge gap and safeguarding psychological health and safety of HCWs.5, 6, 7, 8, 9

Healthcare Simulation Training

Healthcare simulation training replicates any clinical scenarios in a safe environment, bridging between immersive experience, acquired skills, and insights from the training and clinical practice. Not only had technical (e.g., cognitive and psychomotor skills in donning and doffing procedures of Personal Protective Equipment (PPE) in designated clean and dirty zone with buddy system) and non-technical skills (e.g., teamwork, leadership and communication among HCWs) enhanced,7, 8, 10 simulation training, either “Lab-based” in simulation training center or “In-situ” in genuine clinical environment with highest environmental fidelity, could strengthen personal qualities critical in healthcare industries.3, 6, 10, 11

Research & Knowledge Gap

Li and colleagues, through a recent British Medical Journal (BMJ) editorial, highlighted escalating needs of exploring simulation training effects on healthcare providers themselves rather than existing findings over-emphasizing scientific values of clinical, epidemiological, and laboratory characteristics, infection and transmission dynamics, and effect of pharmacological interventions. Through the lens of positive psychology, we explored specific personal strengths of HCWs associated with personal, organizational, and community safety, including:

Assertiveness --- Speak-up behavior

Mental preparedness --- Psychological capacity to response in high risk conditions towards resilience

Self-efficacy --- Capacity to execute task-oriented behavior with self-confidence

Internal locus of control --- Belief of clinical outcomes influenced by their action, not by chance

Internal locus of responsibility --- Self-motivated morality for patient care

This crisis served as a precious opportunity for our center to preliminarily investigate i) whether in-situ simulation would outweigh lab-based simulation in rating of personal strengths and ii) how simulation training brought prosperity to the society in Hong Kong. Recommendations regarding effective management approach of simulation training will be discussed as a wrap-up session.

Methods

Multi-disciplinary Simulation and Skills Centre (MDSSC), based in QEH, has been continuously improving quality and safety of hospital service through healthcare simulation. Fully accredited by the Society of Simulation in Healthcare (SSiH), our center is dedicated to facilitating innovations and exploring new frontiers in simulation-based learning and research.

Establishment of COVID-19 Taskforce

Back in the late-January 2020, a COVID-19 Taskforce was established under the leadership of high management and well-rounded support from administrative and clinical departments for the strategic direction and curriculum design on infection control training. The aim of this taskforce was to, through simulation training, unify and standardize hospital-wide practice and procedures to minimize contamination by and exposure to highly contagious COVID-19 pathogens during high risk procedures, such as Aerosol Generating Procedure (AGP) and endotracheal intubation. After a trial finalizing all training materials and scenario setting, the first training class titled “Infectious Disease Practice Drill and Refresher Training (Novel Coronavirus)” was organized in early February 2020 (see SupplementA).

Participants

To comply with infection control requirement of HA, HCWs working in high-risk areas for suspected or confirmed cases of COVID-19 were required to attend this training. Target participants included anesthesiologists, Accident and Emergency (A&E) physicians and nurses, intensivists, medical physicians, surgeons, nurses and patient care assistants (PCAs) from isolation wards, A&E, Intensive Care Unit (ICU), and general surgical and medical wards.

Ethical Considerations

Any projects initiated by high management of HA for education purposes are waived from ethical approval by the Research Ethics Committee (REC) of HA. In line with the MDSSC Standard Operation Procedures (SOP), all participants were required to complete a written informed consent on confidentiality issues and use of data, including but not limited to course evaluation, questionnaire, formal and informal feedback in written or verbal format, formative or summative assessment approved by respective steering committee, and audio-visual recording, for internal audit, education and research purposes prior to commencement of training sessions.

Procedures and Measurements

By the end of training sessions, one set of validated questionnaires was distributed to each participant to evaluate training effect. Each item was measured on a 5-point Likert scale ranging from 1 “Strongly disagree” to 5 “Strongly agree”. The questionnaire, with excellent inter-item reliability (Cronbach’s α = .96), was categorized into 5 domains of personal strengths: i) assertiveness, ii) mental preparedness, iii) self-efficacy, iv) internal locus of control, and v) internal locus of responsibility.

Data Analysis

To examine the study hypothesis, participants were stratified into two groups: i) in-situ and ii) lab-based. Independent sample t-tests were used for between-group comparison of population means of different pairs of personal strengths. One-sample t-tests were used to compare overall score of personal strengths with reference point of 3 for the general population of HCWs. All levels of alpha were set at .05 (two-tailed). The data were analyzed using IBM SPSS software (IBM Corp. Released 2017. IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp.).

Theoretical Framework for Training Effect: Kirkpatrick’s Model

Kirkpatrick’s model has been widely adopted to evaluate training processes and effects at different levels in healthcare industries. With further advancement by Philips, the 5-level model consists of:

Level 1: Reaction --- Satisfaction from participants

Level 2: Learning --- Acquisition of skills and knowledge

Level 3: Behavioral change --- Positive change in personal strengths of HCWs

Level 4: Result --- Organizational impacts/ clinical outcomes (e.g., Infection rate of HCWs)

Level 5: Return of Investigation (ROI) --- Cost-effectiveness analysis/ influence on the society

Results of level 1 “Reaction” and 2 “Learning” were retrieved from final report of training record directly; level 5 “ROI” would not be covered in details. This cross-sectional study put great emphasis on level 3 and 4, indicating how simulation training program induced positive changes in personal strengths of HCWs and its translational effect in tackling COVID-19 cases at organizational levels.

Results

Training Deliverables

Of 101 sessions carried out from 05 February 2020 to 18 March 2020, 1-third of simulation classes was held in-situ at respective departments (NA&E = 20; N = 14) and 2-thirds was operated in our Simulation lab (N = 30; N = 36). Within this period under intense time and resources constraint, 1,415 hospital staff members, including 1,167 nurses (82%), 163 doctors (12%) and 85 patient care assistants (6%), were trained (see Table 1). Retrieved from training record, we acknowledged that participants were largely satisfied with the training program and learning objectives were met (see Fig. 1. --- Kirkpatrick’s model, Level 1 & 2; and SupplementA --- Learning objectives).

Table 1

Demographics of COVID-19 Simulation Training Course.

	Number of Session		Number of Attendance		Simulation Setting
Trained Groups	Counts	(%)	Counts	(%)	In-situ		Lab-based
A&E	20	(20)	129	(9)	✓		✗
ICU	15	(14)	116	(8)	✓		✗
Isolation Ward	30	(30)	544	(38)	✗		✓
General Ward	36	(36)	626	(45)	✗		✓
Based on Attendance Record		In-situ			Lab-based
		Counts	(%)		Counts	(%)
Type of HCWs	Doctors	59	(24)		104	(9)
	Nurses	176	(72)		991	(85)
	PCA	10	(4)		75	(6)
Based on Evaluation Forms		In-situ			Lab-based
		Counts	(%)		Counts	(%)
Gender	Female	108	(66)		725	(70)
	Male	56	(34)		311	(30)
Year(s) of Clinical Experience	<3	32	(20)		322	(31)
	3-6	50	(30)		186	(18)
	7-10	54	(33)		318	(31)
	11-15	18	(11)		85	(8)
	>15	10	(6)		124	(12)
Department of Participants	A&E	73	(45)		/	/
	ICU	91	(55)		/	/
	Anaes & OTS	/	/		87	(8)
	MED	/	/		454	(44)
	SURG/ NS	/	/		245	(24)
	Others	/	/		250	(24)

Note. A&E = Accident & Emergency; Anaes & OTS = Anaesthesiology & Operating Theatre Services; HCWs = Healthcare Worker; ICU = Intensive Care Unit; MED = Medicine; PCA = Patient Care Assistant; SURG/NS = Surgery/Neurosurgery; Others included Orthopedics & Traumatology (O&T), Obstetrics & Gynaecology (O&G) and otherwise non-indicated.

Fig. 1

Kirkpatrick’s Model of Evaluation on Impacts of COVID-19 Simulation Training Course.

Significance of Personal Strengths

On volunteer basis, 1,200 out of 1,415 participants completed the questionnaire (Response rate = 85%). All domains of personal strengths, in terms of assertiveness, mental preparedness, self-efficacy, internal locus of control and responsibility, were scored 4.24 or above out of 5 (Excellent). Equal variance assumptions were examined and confirmed (p > .05, for all domains) by Levene’s tests before data processing with t-tests. Regarding between-group comparison of In-situ and Lab-based results, no significant differences were found (p > .05) for all domains. To compare participants’ post-training scores with reference point, statistically significant differences were found (p < .001) for all domains (see Table 2). Fig. 1 summarized impacts of COVID-19 Simulation Training using Kirkpatrick’s model at all levels.19, 20

Table 2

Comparison of In-situ and Lab-based COVID-19 Simulation Training Course on 5 domains of Personal Strengths.

Personal Strengths	In-situ(N = 164)		Lab-based(N = 1,036)		Overall #(N = 1,200)		Between-group P-value*
	M ± SD	Diff	M ± SD	Diff	M ± SD	Diff
Assertiveness	4.21 ± .71	+ 1.21	4.27 ± .65	+ 1.27	4.26 ± .66	+ 1.26	.29
Mental preparedness	4.26 ± .71	+ 1.26	4.25 ± .67	+ 1.25	4.25 ± .68	+ 1.25	.79
Self-efficacy	4.18 ± .71	+1.18	4.24 ± .67	+ 1.24	4.24 ± .68	+ 1.24	.31
Internal locus of control	4.28 ± .66	+1.28	4.26 ± .67	+ 1.26	4.26 ± .66	+ 1.26	.62
Internal locus of responsibility	4.21 ± .68	+1.21	4.24 ± .67	+ 1.24	4.24 ± .67	+ 1.24	.66

Note. Diff = Mean minus Ref; M = Mean; N = Valid number of participants; Ref = Reference point as “3” (Neutral); SD = Standard Deviation.

* [In-situ v.s Lab-based] Independent sample t-test is used to compare between-group performance in different pairs of 5 domains of personal strengths; No significant differences were found at α < .05 levels (two-tailed) for all domains.

# [Overall score] One-sample t-tests were used to compare changes of score in each personal strength with reference point of the population. All domains were found statistically significant (p < .001).

Discussion

In summary, this cross-sectional study showed that COVID-19 specific healthcare simulation training programs, both in-situ and lab-based, enhanced personal strengths of HCWs, in terms of assertiveness, mental preparedness, self-efficacy, internal locus of control and internal locus of responsibility, to a large extent. Since there were lacking in scientific investigation into relevant aspects of COVID-19 for the time being, we expand our discussion to clinical and organization impacts and strategic management approaches aligned with evidence-based medicine.

Organizational Outcomes: Infection Risk of HCWs

The lessons we learned from SARS strengthened HCWs’ resilience and preparedness of the COVID-19 pandemic. Dr. Nguyen and colleagues from Massachusetts General Hospital and Harvard Medical School reported that multivariate-adjusted Hazard Ratio (aHR) of COVID-19 was 11.5 times higher in HCWs in the UK; increased 4.45 times than those in the USA; and additively increased by 400% for those managing confirmed patients of COVID-19. As of 10 June 2020, no HCW was infected with COVID-19; 1,054 out of 1,107 infected patients were discharged from local public hospitals in Hong Kong. Such magnificent figures reflected painstaking efforts of all stakeholders in local healthcare industry during the unprecedented crisis.

Enhancing Personal Strengths by Healthcare Simulation

Assertiveness is one of the non-technical elements in Crew Resources Management (CRM) which has been embedded into our scenario-based training.7, 8, 10 The training provided participants with immersive experience in revisiting proper donning/ doffing procedures of PPE and team-based management of AGP with clear role delineation. Throughout the training, “speak-up” culture formed in error-provoking environment could literally speed up participants’ learning process, resulting in readiness and self-confidence to handle suspected and confirmed cases of COVID-19 in clinical setting.5, 8 Uncertainty of the pathogens and fear of infection and death were detrimental to occupational health conditions of HCWs.6, 14, 21 In addition to placebo-effect, debriefing sessions helped HCWs consolidate their understanding of the pathogens and pertaining practice with “take-home message” as a final trump card for maximizing HCWs’ sense of control and responsibility for patient care in the COVID-19 pandemic.

Effective Management Approach of COVID-19 Simulation

Effective management strategies and validation of training materials are essential drivers of successful healthcare simulation training programs:

Doctor-nurse ratio ≤ 1:5, addressing that nurses had greater training needs and were relatively prone to stress and fatigue

Prioritizing training needs by “High-risk area first” policy (Isolation ward first, general ward the next)

Use of parallel mode of simulation (in-situ and lab-based simulation) in different training venues

Offering quotas for observers to maximize training capacities within short period of time

Alignment with up-to-the-date local and international guidelines of infection control

Participants benefited from effective management approach by flexibility of training modes. For A&E and ICU staff, in-situ simulation could achieve the highest degree of environmental fidelity and provide optimal flexibility for trainers and trainees based on their availability. For training of general ward and isolation ward, simulation training center would be much more appropriate for a balance between risk of infection control and occupancy of clinical operation. There was no evidence showing that levels of fidelity in simulation matter in this study. Regardless of training setting (in-situ or lab-based simulation), observers involved in scenario-based simulation and subsequent debriefing sessions have gained insights from the scenario and enhanced personal strengths as much as their counterparts who participated directly in the scenarios.5, 8

Moreover, all training materials were reviewed by the Taskforce members in multi-disciplinary manner; in particular specialists from Intensive Care Team (ICT) and Quality & Safety (Q&S) department, to ensure that all training components were aligned with hospital policy, algorithm for standard workflow, and international standards of practice (CDC guideline; AHA-BLS guideline regarding COVID-19 for A&E Basic Life Support training).

Conclusion

The study showed promising evidence of how incorporating scenario-based simulation into infection control measures enhanced skills and knowledge acquisition and personal strengths of HCWs. Strengthened assertiveness on team collaboration, mental preparedness, self-efficacy under limited resources, internal locus of control and responsibility on designated clinical duty would probably lead to satisfactory clinical and organization outcomes. Limited empirical studies in related areas have necessitated researchers further investigating inter-factorial association and their long-term impacts on organization across healthcare disciplines. We expect that person-centered principles and strategic approaches we recommend can be translational worldwide, expediting the return to normalcy in near future.

Author contributions

Conceptualization: VC, ES, GN, CS, JL, NC; Methodology: VC, ES, GN, CS, JL, and NC; Software: VC, JL; Validation: VC, CS, JL, and NC; Formal Analysis: VC, ES, GN, JL, and NC; Investigation: VC, ES, GN, CS, JL, and NC; Resources: VC, CS, and JL; Data Curation: VC, and JL; Writing – Original Draft: VC, and ES; Writing – Review & Editing: VC, ES, GN, CS, JL, and NC; Visualization: VC; Supervision: ES, GN, CS, and NC; Project Administration: VC, CS, and JL; Funding Acquisition: CS.

Conflict of interest

The authors declared no conflict of interest.

Funding

This study received no external funding.

Ethical statement

Any projects initiated by high management of Hospital Authority (HA) for education purposes are waived from ethical approval by the Research Ethics Committee (REC) of the Hospital Authority in Hong Kong. Ethical approval was not required. Complied with the Standard Operation Procedures (SOP) of Multi-disciplinary Simulation and Skills Centre (MDSSC), written informed consent was obtained from all participants in the study.

Data availability

The data of this study are available from the corresponding author on reasonable request.