Interprofessional Disaster Simulation During the COVID-19 Pandemic: Adapting to Fully Online Learning.

BACKGROUND: This paper describes the rapid conversion of a face-to-face interprofessional (IP) disaster simulation to an online format in response to COVID-19 campus closures.
METHODS: The online disaster simulation utilized internet-based tools allowing real-time collaboration between IP students. Team exercises involved disaster triage, disease outbreak investigation, and disaster response. Surveys measuring self-assessment of various IP skills and simulation learning outcomes (SLOs) were compared with responses from previous face-to-face simulations.
RESULTS: Results indicated mean scores for IP skills were higher for online students when compared with in-person simulations, and all SLOs were met.
CONCLUSIONS: The online disaster simulation provided an effective, innovative IP educational opportunity.

Rapid conversion of a face-to-face interprofessional (IP) disaster simulation to a fully online format in response to COVID-19 campus closures.

This innovation showed strengthened IP skills in disaster scenarios, with significantly higher scores for online students as compared to in-person simulations.

This simulation has potential for expansion, with increased capacity for simulation program delivery, adaptability for various types of disasters, and new opportunities for diverse partnerships.

The COVID-19 pandemic has underscored the importance of a healthcare workforce prepared for all types of public health emergencies. Training and education regarding disaster health management may prevent excess illness, injury, and death (Walsh et al., 2012). The entire healthcare workforce must be strengthened to improve response to large-scale disasters, including pandemics (Veenema et al., 2020). Disaster health leaders have called for increased efforts in developing interprofessional learning opportunities (Veenema et al., 2017). Various simulation activities have been developed to bolster health professional students’ disaster health management knowledge and skills. Examples described in the literature are often resource-intensive (Jose & Dufrene, 2014). Few articles have described activities that can be conducted fully online focused on fostering collaboration among interprofessional students.

The COVID-19 pandemic resulted in university campus closures nationwide. Health care students were removed from clinical rotations. Classroom education transitioned to fully online instruction, and clinical instruction shifted to virtual simulation learning. Faculty at the University of Hawaii prioritized the conversion of an existing simulation, the Disaster Aftermath Interprofessional Simulation (DAIS) (Glauberman, Wong, Bray, & Katz, 2020), to an online format. The online DAIS was piloted within weeks of the campus closure, providing a timely interprofessional disaster response learning activity for students experiencing a public health disaster in real-time.

The Disaster Aftermath Interprofessional Simulation

The DAIS was originally created as an in-person activity by an interprofessional faculty team composed of nursing and public health professionals. Faculty members were subject matter experts (SMEs) in simulation, disaster response, and epidemiology. The goal was to provide an interactive, immersive, collaborative learning activity where interprofessional students could collectively respond to a public health disaster (Glauberman, Wong, Bray, & Katz, 2020). The activity included three phases: Phase 1 involved conducting disaster triage after a devastating tsunami hits an unspecified community in a developing island nation. Phases 2 and 3 involved coordinating a population-focused response to the disaster aftermath, including managing a disease outbreak in an emergency shelter. Activities for phases two and three were adapted from an existing course developed by the Centers for Disease Control and Prevention (CDC, 2014). Debriefing discussions were led by interprofessional faculty after each phase.

To ensure basic competency, students were required to complete pre-work relating to disaster triage, emergency response, and infectious disease outbreak investigation. Pre-work modules were developed by faculty SMEs in disaster response and epidemiology and incorporated online resources from the Centers for Disease Control (CDC) (2020) and the Federal Emergency Management Association (FEMA) (2021) FEMA, Previous face-to-face versions were conducted in October 2018, April 2019, and October 2019. Evaluations of the in-person DAIS indicated that the activity was an innovative method for preparing interprofessional students to work collaboratively during disaster response situations (Glauberman, Wong, Bray, & Katz, 2020).

Conversion to an Online Format

The modified online DAIS was launched on April 3, 2020 in response to campus closures enacted due to the COVID-19 pandemic. Participants consisted of upper-level pre-licensure nursing students, masters and doctorate public health students, and masters in social work students. This project was approved as exempt by the University of Hawaii Institutional Review Board.

Key elements of the original DAIS were retained when converting it to an online format. This included required pre-work, real-time collaboration in small interprofessional groups, large-group debriefing sessions using a co-facilitation process, and the ability to meet the original DAIS learning objectives. Zoom's web-conferencing platform and Google's online suite of products (Google Docs, Google Sheets) met all requirements to ensure that the integrity of the experience was maintained. Original face-to-face materials were converted to web-based Google documents. For example, the triage activity was adapted from a series of cards into a single Google sheet, which included triage victim descriptions, first aid supplies, and other resources. Students utilized Google Sheets to document the victim's triage status, allocate resources, and transport victims. Background noises of a disaster scene, which increased the fidelity of the in-person simulation, were eliminated because they created audio feedback, interfering with the video conferencing platform. Faculty conducted a beta-test of all online collaborative documents, allowing an opportunity for quality improvement changes before the simulation.

During each phase of the DAIS, all participants were first placed in the main room, where instructions were provided. Short videos were played to illustrate the impact of a tsunami on a community. Students were then moved to pre-assigned breakout rooms. Fifteen Zoom breakout rooms were created for 63 students (4 – 5 per room). Students were given links to Google documents specific to each group that contained information about the scenario and tasks that needed to be completed. Moving students between rooms and various activities required careful coordination by simulation center staff utilizing a detailed agenda (Table 1 ) as a guide.

Table 1

Detailed Agenda for Disaster Aftermath Interprofessional Simulation (DAIS) Simulation, University of Hawaii, 3 April 2020

Time	Subject	Facilitator and Simulation Technician (ST) Notes
15 min	Pre-huddle - Faculty and Simulation Technicians(ST)	Facilitators and ST: Connect to Zoom Meeting.• Last-minute huddle with all room monitors (ST and faculty, each faculty to monitor 3 Breakout Rooms) • Develop contingency communication plans for technology/internet issues.
15 min	Begin Session:Overview of the Program	Main Zoom Room - Screen share of DAIS Powerpoint slides for simulation overview for students:• Introductions • DAIS and Interprofessional Goals • Logistics of the Day • Online etiquette and Zoom tools • Student access & instructions to Google docs and sheets for each DAIS Phase • Upon completion of the icebreaker - Simulation staff send students to pre-assigned breakout rooms using Zoom feature
15 min	Icebreaker Activity	Zoom Breakout Rooms• Objectives for icebreaker (low-stakes) activity:∘ Learn about each other. ∘ Practice team skills - organization, communication, & leadership ∘ Practice use of Google and Zoom tools
40 min. total time for activity10 min:instructions and video30 min: team activity	Phase 1: Triage activity	Main Zoom Room - Introduce Phase 1 Triage Activity• Explain Objectives - the learners will work in teams to:∘ Appropriately assess and apply disaster triage in a disaster scenario ∘ Discuss personal safety during disaster response ∘ Demonstrate ability to communicate effectively with team members ∘ Allocate resources and determine transportation of victims ∘ Discuss ethical decision making during disaster response situations • Play videos - “Setting the scene” & “Tsunami ”:∘ Provide activity instructions and background to the disaster. ∘ “Tsunami” video: news video clips about a tsunami (can be found on internet) • Provide students with access URL to Google document “Phase 1: Disaster Triage” • Review instructions found on the Google document.∘ Identify a leader, scribe and timekeeper and work as a team to complete the following: ∘ Step 1 (Primary Triage): Use victim information to triage appropriately (black, red, yellow, green) ∘ Step 2 (Secondary Triage): Determine transport priority of triaged victims. ∘ Phase 3 (Allocation of limited supplies) ∘ Allocate first aid supplies & resources. • Answer questions and staff send students to breakout rooms using Zoom features to complete activity.
35 min	Debrief Phase 1	Staff call students back from breakout rooms using Zoom featuresMain Zoom Room - Debrief Phase 1• Reflection of student experience and feelings • Review each objective • Summarize lessons learned
30 min5 min:instructions and video25 min: team activity	Phase 2: Response to the Disaster Aftermath	Main Zoom Room - Facilitators introduce objectives for Phase 2: Response to the Disaster Aftermath• Objectives:∘ Identify, analyze, and interpret patterns in data related to a public health emergency. ∘ Investigate and plan a response to the after-effects of an all-hazards disaster. ∘ Discuss methods to mitigate the after-effects of the all-hazard disaster. • Play video of tsunami aftermath • Provide access URL to Google document Phase 2 “Managing the Response”∘ Identify a leader, scribe and timekeeper and work as a team to answer questions focused on initial post-tsunami community assessment ∘ List potential health concerns ∘ List immediate actions to respond to concerns • Answer questions and send students to Zoom Breakout rooms to complete activity
15 min	Debrief Phase 2	Students return from Zoom Breakout RoomsMain Zoom Room - Facilitators select groups to present and debrief
15 min5 min:instructions10 min: team activity	Phase 3a: Management of a Disease Outbreak	Main Zoom Room - Facilitator will discuss Phase 3a: Management of a Disease Outbreak• Provide access URL to Google document with instructions and questions to Phase 3: Shelter Outbreak activity∘ An outbreak of gastrointestinal symptoms occurs in a shelter. Based on data provided, calculate attack rates and attributable risk for a list of food items served. ∘ Discuss and determine the source of the outbreak. • Answer questions and send students to Zoom Breakout rooms to complete activity
15 min	Debrief Phase 3a	Students return from Zoom Breakout RoomsMain Zoom Room- Facilitators select groups to present and debrief
25 min5 min:instructions and video20 min: team activity	Phase 3b: “Epidemic Curve- Pathogen Identification” activity	Main Zoom Room - Facilitator provides report and instructions for Phase 3b: “Epidemic Curve and Pathogen Identification” activity.• Provide access URL to Google document. • Identify outbreak pathogen, and:∘ Create and interpret an Epidemic Curve ∘ Create a list of potential pathogens. ∘ Based on pathogens listed, discuss control measures • Answer questions and send students to Zoom Breakout rooms to complete activity
20 min	Debrief Phase 3b	Main Zoom Room - Facilitators select groups to present and debriefSummarize lessons learned from the program.
15-20 min	Evaluation	Main Zoom Room - Facilitator provide access URL to IPE and simulation evaluations

After each phase, all participants returned to the main room to conduct debriefing in a large group. Faculty employed multiple strategies to facilitate debrief sessions that were engaging for the large number of participants and multiple groups. Faculty were assigned 1-3 debrief questions pertaining to their area of expertise. Some questions were posed to the whole group, and students responded using the chat box. Where more in-depth responses were required, faculty selected one group to answer the question posed, and the group's spokesperson would respond. Group spokespersons were selected by students and alternated during each phase of the exercise.

Evaluation

Continuous quality improvement plans involved careful evaluation to determine if the online format was comparable to the face-to-face delivery. To assess the impact of the online DAIS, all students (N = 63) completed the validated 20-question retrospective pre-post Interprofessional Collaborative Competency Attainment Survey (ICCAS) instrument (Schmitz et al., 2017; Violato & King, 2019). ICCAS is a 20-item survey tool designed for self-assessment of interprofessional team collaboration skills. Validation studies of the tool using exploratory factor analysis supported a single factor structure, with Cronbach's alpha scores ranging from 0.95 to 0.97 (Schmitz et al., 2017; Violato & King, 2019). Schmitz et al. (2017), recommended use of total average scores for analysis. Therefore, the average overall scores were analyzed using paired t-tests. Average overall scores from each ICCAS survey were compared with those pooled from 3 previous in-person DAIS exercises. Results indicated that the average overall mean score for pre- and post-activity evaluation questions demonstrated statistically significant higher scores for the online simulation exercise compared to the in-person simulation exercise (Table 2 ).

Table 2

Disaster Aftermath Interprofessional Simulation 2-Year Summary Results Mean Overall Student Pre and Post ICCAS* Scores: Comparisons Between In-Person (N = 150) and Online (N = 63) Using Paired T-Tests

Questions - Please rate your ability for each of the following statements BEFORE & AFTER: [1-5 Scale: 1=poor, 2=fair, 3=good, 4=very good, 5=excellent]	Mean Overall Student Pre Scores			Mean Overall Student Post Scores
	In-Person	Online	p-value	In-Person	Online	p-value
MEAN OVERALL TOTAL ICCAS* SCORE (Q1-20)	3.73	4.12	.0001	4.25	4.50	.0020

Interprofessional Collaborative Competency Attainment Survey

In addition to completing the ICCAS survey instrument, students were asked three post-activity questions related to perceived ability to collaborate interprofessionally, perceived impact on future practice, and satisfaction with the ability to work through the simulation. A comparison of the responses from earlier in-person exercises with the online exercise provided confirmatory evidence that participation in the online exercise was of equal, if not greater benefit, compared with the in-person exercise. Responses to 2 of the 3 questions showed no significant difference, while responses to “how satisfied were you with your ability to work through the simulations?” showed significantly higher satisfaction scores with the online compared to the in-person exercise (3.99 vs. 4.27, p = .0049).

Students also provided feedback regarding the DAIS by responding to open-ended questionnaire items. Two authors of this study independently coded open-ended responses for themes and discussed discrepancies until they reached 100% agreement. Major themes identified as key strengths and takeaways were teamwork/collaboration, communication, and triage (which included the handling of ethical dilemmas). One student commented, “Teamwork is key. Listen to each other. Take advantage of strength and expertise.” Another said, “It is SO important to use all your people resources. Everyone has knowledge to contribute. I appreciate even more the emotional impact of the ethical dilemmas our front-line healthcare workers are facing. We need some front-line psych support for all of them.”

Discussion

Adapting the DAIS to a fully online format allowed for the delivery of a quality simulation learning experience during the campus closure due to the COVID-19 pandemic. After the campus was reopened, COVID-19 restrictions continued to impact the ability to return to an in-person DAIS simulation. Social distancing requirements limited the simulation center's capacity and posed other logistical issues such as the availability of tabletop space, chairs, and personal protective equipment. However, the online DAIS demonstrated that distance and physical barriers no longer limited program delivery.

The significantly higher scores of the online vs. in-person exercise may have two possible explanations. First, our professional schools have been actively engaged in interprofessional education (IPE) exercises since 2014, and this cohort may have benefited from earlier IPE exposures. Secondly, the exercise was undertaken as the COVID-19 pandemic was unfolding. All in-person classes had transitioned abruptly to online formats 2 weeks earlier. Pandemic awareness was foremost on the students’ minds and they were focused on learning more about disaster management. High engagement during the simulation may have also been a result of the design itself. With little room in the schedule for downtime, students used the time they had to perform tasks, answer questions, and engage with the material and one another in order to complete the simulation successfully.

One of the main limitations was unstable internet connectivity. To address this, simulation technicians stood by to assist students. Simulation facilitators also reminded students to log in using their university credentials to access Google documents. With COVID-19 restrictions still impacting face-to-face education, plans continue for the online format of the DAIS. The methods and tools used for the DAIS can be adapted for other types of disasters, such as large-scale epidemics. Looking ahead, the online DAIS opens up opportunities for collaborative efforts among various local, national, and international partners.

Conclusion

Providing an online DAIS exercise during the COVID-19 pandemic prepared students for future public health emergencies and disasters. Converting the DAIS to an online format provided interprofessional students with an immersive and interactive learning environment that retained the efficacy and quality of the original face-to-face exercise. It provided a foundation for an interprofessional, collaborative response to public health disasters and can serve as a model for other health science programs. Moving forward, faculty plan to expand the simulation to additional professions, such as medicine, pharmacy, and dental hygiene, while also decreasing group sizes to enhance small-group discussions. The simulation will continue to be offered online for logistic reasons. These include increasing capacity for students involved in the exercise, eliminating issues related to transportation, and allowing us to reach a geographically dispersed student population. Furthermore, faculty are exploring the use of validated tools for direct observation of interprofessional team collaboration skills in online settings. Authors suggest that others replicate this format and adapt it to their unique student populations. They encourage the sharing of ideas for ongoing improvement.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of Interest

The Author declares no conflict of interest.