Distance Learning During COVID-19 Mitigates Learning Loss for Interprofessional Education.

SUMMARY STATEMENT: On-site interprofessional education (IPE) simulation is primarily used to teach students teamwork, communication, and crisis resource management. Participants view it as an educational environment in which to acquire and consolidate skills. Virtual IPE simulation is traditionally seen as an opportunity to supplement, complement, and reinforce on-site IPE (OI). We used VI as the sole simulation method during the COVID-19 pandemic to provide IPE because of constraints of social distancing. The VI resulted in substantially achieving similar learning outcomes to OI. This suggests that VI, which has the advantage of being cheaper and more easily scalable than OI, may be an effective remote learning modality for IPE.

We traditionally provided IPE training to our students using OI with mannequins and standardized patients with VI as an adjunct. The COVID-19 pandemic caused our simulation activities to halt because of required social distancing and limiting physical interactions. We, therefore, investigated the provision of VI as an alternative approach to mitigate students' potential learning loss by bridging faculty and students via virtual hospital encounters.

BACKGROUND OF INTERPROFESSIONAL EDUCATION TRAINING AT THE STUDY SITE

We commenced interprofessional education (IPE) training for health professional students in 2010 by using on-site IPE (OI). In 2013, we added virtual IPE (VI) as an adjunct to increase the skills and knowledge of the students. In a crossover study, VI, followed by OI, showed a potential increase in teamwork, communication, and crisis resource management knowledge and students' skills. Therefore, from 2013 to 2018, we provided VI, followed by OI. In 2019, unable to provide VI because of lack of resources, we only provided OI, and in 2020, because of COVID-19 halting the provision of OI, we provided a scaled-up VI.

METHODS

Six sessions of faculty development training were conducted to prepare faculty for delivering VI simulation training to students. The faculty participants consisted of 23 pharmacists, 15 physicians, 9 nurses, and 2 physician assistants. Twenty case scenarios were developed (4 VI and 16 OI) with the objectives of achieving team collaboration in providing patient care and improving effective communication in managing a crisis.

The VI case scenarios included COVID-19 in an ambulatory setting, suspected COVID-19 patient with respiratory failure, pain management, and cardiac arrest in pregnancy. The learners consisted of 66 medical, 59 nursing, 104 pharmacy, and 25 physician assistant students. The students were trained on simulation software Health TeamSpaces (Figure, Supplemental Digital Content 1, which shows VI Simulation Software Health TeamSpaces, http://links.lww.com/SIH/A723).

The virtual workspace had standardized actors who played the roles of patient, relative, or consulting physician. Students had two 30-minute encounters, a 1-hour general debriefing after the 2 sessions, and a 30-minute specialty-specific debriefing in breakout rooms for a total of 2.5 hours (Table, Supplemental Digital Content 2, which compares OI 2019 and VI 2020 simulation, http://links.lww.com/SIH/A724).

ASSESSMENT

We used the following validated tools to assess the efficacy of the IPE training. To increase interrater reliability, faculty were trained as evaluators.

Interdisciplinary education perception (IDEP) surveys completed by students before (“pre”) and after (“post”) the VI session.

Nontechnical and cognitive skills (NTCS) ratings of the students made by faculty during the simulation's live feed.

Standardized patient checklist (SP checklist) rating of students completed by virtual SP/confederates after the virtual encounter.

The NTCS self-reflection ratings, the IPE simulation evaluation, and the faculty debriefing evaluation [Debriefing Assessment for Simulation in Healthcare (DASH)] by students at the end of the session.

Interprofessional education simulation evaluation by the faculty after the session.

For the comparison analysis, the percentage agreement represents agree and strongly agree categories. χ2 tests were performed for pre- and post-IDEP surveys and compared the 2019 and 2020 results of the NTCS, SP checklist, and simulation evaluation by students and faculty. A descriptive analysis was performed for the evaluation of instructors by the students (DASH). A P value of less than 0.05 represents a statistically significant difference.

RESULTS

In the NTCS rating by faculty (Figure, Supplemental Digital Content 3, which compares NTCS faculty rating on OI vs. VI simulation, http://links.lww.com/SIH/A725), there were statistically significant decreases in the following 3 items between 2019 (OI only) and 2020 (VI only): executing established protocol sequence/timing correctly (6%), initiating critical treatments in a timely manner (9%), and exercising good diagnostic decision-making skill (16%). There was no statistically significant difference in the scores of the other items.

In the SP checklist (Figure, Supplemental Digital Content 4, which demonstrates SP rating on the OI vs. the VI simulation, http://links.lww.com/SIH/A726), there was a statistically significant increase in only 1 item, “the students involved you in making decisions about treatment/care.” All other items remained the same from 2019 to 2020.

In the IDEP (Figure, Supplemental Digital content 5, which shows pre- and post-IDEP survey results of OI and VI simulation, http://links.lww.com/SIH/A727), the agreement percentage increased from pre- to post-VI with the “understanding and collaboration” items, reaching statistical significance.

In the self-reflection (Figure, Supplemental Digital Content 6, which compares the student self-reflection score between OI and VI simulation, http://links.lww.com/SIH/A728), there was a statistically significant increase in all items, except for “requesting and receiving confirmation from team members when tasks were completed.”

In the evaluation of IPE simulation by learner (Figure, Supplemental Digital Content 7, which compares student evaluation of their OI vs. VI simulation experience, http://links.lww.com/SIH/A729), only the item “The IPE simulation experience was worth my time” demonstrated a statistically significant decrease (8%) between 2019 and 2020.

In the evaluation of IPE simulation by faculty (Figure, Supplemental Digital Content 8, which compares faculty evaluation of their OI vs. VI simulation experience, http://links.lww.com/SIH/A730), all ratings remained the same from 2019 to 2020 except for the item “The student group size worked well,” which showed a statistically significant improvement.

In the evaluation of faculty debriefing by students who used the DASH (Figure, Supplemental Digital Content 9, which shows students' evaluation of faculty debriefing, http://links.lww.com/SIH/A731), 90% of the students rated the faculty's debriefing as effective to extremely effective.

DISCUSSION

Online education has the advantage of being cheaper and could be scaled up or down quickly. However, it requires extensive training for trainers and students and investing in information technology. Across most assessments, our findings did not demonstrate any statistical difference in student learning outcomes between OI (2019) and VI (2020). Among the items that showed a statistically significant reduction between 2019 and 2020, the difference was only 8% in absolute percentage point in the item “IPE simulation experience was worth my time.” It is stated that “Scientific conclusions and business or policy decisions should not be based only on whether a P value passes a specific threshold.” We, therefore, do not think that the items that did show a statistical decrease are clinically significant. Of importance is that the reduction in the fidelity of VI and Internet-related issues could explain such a decline when VI is compared with OI.

Of note was our inability to provide the same numbers of case scenarios and exposure length between 2019 and 2020. In addition, our findings are limited regarding the transferability of IPE knowledge to clinical practice. Increasing the fidelity of virtual simulation needs further study. However, we mitigated the effects of COVID-19 on IPE training by opting for a full VI. We provided VI for a larger group of students at approximately half the cost of OI, and the results are promising.