Interinstitutional simulation of patients with COVID-19 during a remote acute-care advanced pharmacy practice experience.

Introduction: Faculty and staff from Duquesne University and the University of Pittsburgh Schools of Pharmacy created a simulation activity focused on the care of critically ill patients with coronavirus disease 2019 (COVID-19). Students on remote, short-term-care advanced pharmacy practice experiences (APPE) rotations from both universities worked in comingled teams and completed two educational electronic health record reviews, complex simulation sessions, and debriefs. Individually, students completed two educational electronic health record reviews and verbal patient presentations before and after the simulation sessions.
Objectives: Evaluate the effects of a simulation activity during a remote short-term-care APPE on student confidence and knowledge surrounding the care of a critically ill patient with COVID-19.
Methods: Student knowledge surrounding COVID-19 short-term-care treatment principles was assessed through pre-/postcase-based multiple-choice examinations and an intermittent clinical examination (ICE). Student confidence and perceptions were gathered through anonymous pre-/postsurveys. The written examination and patient presentation recordings were compared from baseline to the final assessment using the Wilcoxon signed-rank test.
Results: In total, 92 students participated in the activity. There was a statistically significant improvement from baseline to the final assessment (preassessment median [interquartile range (IQR)]: 55.3% [50%-60.5%]; postassessment median [IQR]: 68.4 [60.5%-73.7%]; P < .001) on the written examination. ICE total scores improved from baseline (preassessment median [range]: 33 [28-36] vs postassessment median [range]: 36.5 [29.5-43.52]; P = .004) as well as the objective (P < .001), plan (P < .001), and monitoring (P < .001) subdomain scores. Student confidence reported on surveys improved from baseline in all domains.
Conclusion: Remote simulation sessions improve student knowledge and confidence and provide an opportunity for students to experience caring for patients with COVID-19 in a safe environment. Collaboration between schools of pharmacy can be successfully employed to leverage resources and expertise to expand opportunities for students.

INTRODUCTION

Pharmacy education has recently undergone a forced shift in teaching and educational approach due to the novel coronavirus disease 2019 (COVID‐19) pandemic, which has affected not only pharmacy students' in‐person courses but experiential learning activities as well. Specifically, advanced pharmacy practice experiences (APPEs), during which final‐year Doctor of Pharmacy (Pharm.D.) students gain critical hands‐on experience in various pharmacy settings, required prompt adaptation to ensure student and patient safety during the pandemic. Pharmacy schools quickly looked to offer APPEs remotely, but without students reporting on‐site, accessing real‐world electronic medical record (EMR) data quickly became difficult.

Simulation‐based learning has gained momentum in the last decade as an emerging teaching modality in pharmacy education since the American College of Pharmaceutical Education approved the use of such simulations in introductory pharmacy practice experiences for up to 60 of the 300 experiential education hours requirement. The fidelity of the simulation experiences can vary from low to high (Figure 1). High‐fidelity simulation allows for standardization across student experiences although retaining a high degree of real‐world likeness. As simulation laboratory tests utilize technologically advanced patient mannequins that can speak, demonstrate physiologic qualities, and respond in real time, elements of bedside learning and clinical experience are retained to a more thorough degree than with other low‐fidelity approaches such as retrospective EMR review and discussion, where these dynamic, real‐time patient qualities are often lost.

FIGURE 1

Simulation definitions

The University of Pittsburgh School of Pharmacy has been delivering education through simulation since the 1990s. Over time, simulation‐based learning has been integrated into every year of the Pharm.D. curriculum due to its effectiveness and student favorability. High‐fidelity simulation is utilized at this institution to effectively teach a variety of pharmacotherapy topics such as cardiovascular diseases, seizure disorders, infectious diseases, and special topics in short‐term care. , This technology is also effectively utilized to teach clinical skills such as blood pressure assessment, physical assessment of a patient, drug counseling, patient interviews, and interprofessional communication. , ,

During the onset of the COVID‐19 pandemic when APPE rotation sites across the city of Pittsburgh were unable to host student learners on‐site due to safety precautions and the state of Pennsylvania's emergency declaration, the University of Pittsburgh and Duquesne University identified this challenge as an opportunity to combine resources and collaborate in a unique way to deliver APPE‐caliber rotation experiences remotely. The result of this collaboration was a remote patient simulation centered on interdisciplinary communication regarding care of an intensive care unit (ICU) patient diagnosed with COVID‐19 pneumonia. This experience aimed to augment ongoing APPE rotations, as this opportunity allowed students to gain simulated experience providing care to critically ill patients with COVID‐19 without putting students and patients at risk.

METHODS

Study design

This study was a retrospective review of existing educational records to evaluate the effectiveness of the interinstitutional COVID‐19 simulation activity.

Activity description

The interinstitutional COVID‐19 simulation activity involved the use of several different technology platforms. Videoconferencing software (Zoom Video Communications, San Jose, California) was utilized to conduct meetings and interactions between students and faculty. At the time of the activity, both universities utilized Blackboard Inc. (Washington, District of Columbia) learning management systems to administer baseline and postactivity knowledge tests. Qualtrics (Provo, Utah) was used to administer the pre‐ and poststudent perception surveys. EHR Go (Archetype Innovations, LLC, Duluth, Minnesota) is a customizable web‐based simulated EHR where the cases of patient with COVID‐19 were built and deployed. Flipgrid (Minneapolis, Minnesota) is a video discussion platform that allows students to create 10‐minute patient presentation videos, which were graded at a later time by facilitators. Patient simulation software (Laerdal SimMan Software Version 3.5.0, Laerdal Medical AS, Stavanger, Norway) was utilized without a mannequin to provide students a live view of a simulated patient monitor via screen sharing during the videoconference. The intricate merging of these technologies supported the entire learning experience.

In order to provide as close to a real clinical experience as possible, the two cases of patient with COVID‐19 were based on an amalgam of real‐life patients seen by clinical faculty in an ICU setting. The clinical cases were constructed in the EHR Go platform to provide an in‐depth medical record experience. Participants were able to view laboratory values, drug administration, clinical notes, vital signs, drug orders, and so forth, over a period of 2 to 4 days, depending on the patient case. Appendix S1 provides snapshots of prompts, laboratory tests, drugs, and so forth. Patient case 1 (“Garth Merchant” case) involved a male patient admitted to the hospital from a skilled nursing facility with COVID‐19, respiratory distress, and fever. Patient case 2 (“Monica Atchinson” case) involved a female patient admitted from a skilled nursing facility with COVID‐19, respiratory distress, and fever. Part 1 of each of the patient cases required the students to make recommendations shortly after the patient decompensated and was subsequently intubated. Part 2 of the patient cases required the students to make adjustments to the pharmacotherapy based on clinical status changes approximately 12 hours after the first interaction with the patient. Students evaluated each patient twice (ie, part 1 followed by part 2), which provided the opportunity to complete two rotations through the Pharmacists' Patient Care Process. The flow of activities is outlined in Figure 2, which details when the students were working individually and in groups.

FIGURE 2

Schedule of individual student and group activities

The interinstitutional COVID‐19 simulation activity required students to work in assigned groups. The students were assigned into groups of four to seven depending on the total number of participating students. The groups were purposefully a mix of students from both Schools of Pharmacy, which allowed students from different curriculums to collaborate on the patient cases. The students were encouraged to have an icebreaker meeting before the first simulation session via Zoom. After the first simulation, the students were provided literature and local institution guidelines regarding the care and treatment of patients with COVID‐19. They were instructed to meet with their groups to review the material before the second simulation session. Overall, the student groups were able to meet and collaborate together four times during this experience (Figure 2).

Student interaction with patient case 1 (“Garth Merchant”) was an individual effort. Garth Merchant part 1 was assigned 1 day before the first simulation session. Each student had 30 minutes to review the patient records and formulate their recommendations. Afterward, the students utilized Flipgrid to record their recommendations as if they were presenting the patient to a preceptor or another health care provider. One week later, after completion of the second simulation session and debrief, the students were assigned Garth Merchant part 2 to evaluate and make recommendations via Flipgrid. The Flipgrid videos for parts 1 and 2 were evaluated using the intermittent clinical examinations (ICE) rubric. Rubrics for both cases are available in the supplementary materials (Appendix S4 and Appendix S5). The students worked independently on the Garth patient case and uploaded individual videos twice during this activity (Figure 2).

Student interactions with patient case 2 (Monica Atchinson) were a group effort. One hour before the simulation session, the case was released to the students via EHR Go. Students were given 30 minutes to review the medical record on their own and then were instructed to meet with their assigned group via Zoom to discuss and finalize the group's recommendations. Next, the entire group would enter into the simulation session to present their recommendations and participate in the simulation and subsequent debriefing session. One week later, the students completed part 2 of the Monica Atchinson case in the same format as part 1.

The group simulation sessions were facilitated by two clinical faculties who specialize in critical care pharmacotherapy and an instructional development specialist to operate the simulation software. At the beginning of each simulation session, students would provide their recommendations to the facilitators. The clinical faculty followed a guidance document to cue changes to the patient vital signs based on recommendations made by the student groups. Students were required to react to the changes in patient clinical status with new recommendations which were conducted in an accelerated manner due to time constraints of the simulation. At the end of the simulation, the students and clinical faculty engaged in a debriefing session to guide students through the positive and negative outcomes of the session. By the end of the interinstitutional COVID‐19 simulation activity, students would have participated in two group simulation sessions.

The simulation activity was evaluated by comparing the pre‐ and postactivity scores on a clinical knowledge examination and a video‐recorded patient presentation. The clinical knowledge examination was a 38‐question, multiple‐choice clinical examination administered before and after student participation in the simulation activity. Questions were complex case‐based questions that mirrored the question format commonly found in Pharmacotherapy Self‐Assessment Program. The focus of the questions was short‐term‐care concepts associated with a patient diagnosed with COVID‐19. Students evaluated the patient medical record and submitted a video recording regarding the identification of pertinent subjective and objective information as well as the formulation of a pharmacotherapy treatment and monitoring plan. Student communication and the evidence used to develop the treatment plan were also assessed. Grading of the assessment was based on a predefined criterion using the ICE rubric. The results were de‐identified and shared with the study investigators.

Student perception and opinions were solicited with a self‐reported survey (Appendix S2) that was conducted before the first collaboration and upon completion of the final session. The initial content for each survey was developed by the study authors who are critical care pharmacists and educational experts with educational survey research experience. Each survey was then reviewed for content validity specifically focusing on ease of use, structure, clarity of questions, and validity of the questions. Appropriate modifications were made to both surveys based on feedback. The final surveys were pilot‐tested by the study authors, and final modifications were made before dissemination to students. All data were recorded and retained electronically in a de‐identified fashion through Qualtrics (Qualtrics LLC, Provo, Utah), a web‐based tool, by the University of Pittsburgh. Students were not forced to answer any survey question, and their identity or identifiers were not requested. This survey gauged student confidence regarding care of a critically ill patient with COVID‐19, the role of the pharmacist in this setting, and their overall communication skills.

This study was approved as exempt under educational research by both the University of Pittsburgh and Duquesne University. In accordance with The Family Educational Rights and Privacy Act, all student data were de‐identified before analysis.

Statistical analysis

Data were summarized using descriptive statistics including median and range. The clinical, written examination as well as the components of the ICE were compared from baseline to the final assessment using the Wilcoxon signed‐rank test as the data were paired and nonparametric. The survey results were summarized using descriptive statistics including percentages as well as medians and ranges. Results were compared using the Mann–Whitney U test and the Fisher's Exact test where appropriate.

RESULTS

The entire simulation activity described above was completed three times, each with a separate cohort of students who were on rotation at the time. Ninety‐two students in total made up the three cohorts that participated in this activity (Table 1). For the clinical, written examination, 82 students completed the knowledge assessment. There was a statistically significant improvement from baseline to the final assessment (preassessment median [interquartile range (IQR)]: 55.3% [50%‐60.5%]; postassessment median [IQR]: 68.4 [60.5%‐73.7%]; P < .001). Seventy‐nine students who completed both video presentations were evaluated using the ICE rubrics. ICE rubric scores improved on the total assessment (preassessment median [range]: 33/62 [28‐36] vs postassessment median [range]: 36.5/62 [29.5‐43.52]; P = .004). The additional components of the ICE assessment (subjective, objective, plan, monitoring, and communication) are described in Table 2. The student self‐assessment survey results displayed in Table 3 demonstrate improvements in student perception concerning their confidence in several categories associated with caring for a critically ill patient with COVID‐19. In regard to student‐perceived communication skills, of the students who completed the self‐assessment survey, 31% believed their communication was proficient at the conclusion as compared with 23% at the beginning of the activity (P = .25).

TABLE 1

Breakdown of student participation by cohort, school, and group

Cohort	Total students	Total students by school	Group 1	Group 2	Group 3	Group 4	Group 5	Group 6	Group 7	Group 8
1. May/June 2020	48	Pitt: 31 Duq: 17	4 2	4 2	4 2	4 2	4 2	4 2	4 2	3 3
2. July 2020	20	Pitt: 11 Duq: 9	4 3	4 3	3 3
3. August 2020	24	Pitt: 6 Duq: 18	2 6	2 6	2 6

Note: Each cohort represents a different set of rotation students completing remote rotations.

Abbreviations: Duq = Duquesne University School of Pharmacy; Pitt = University of Pittsburgh School of Pharmacy.

TABLE 2

ICE patient presentation results by assessment category

Assessment category	N = 79 students
	Baseline score: Median (range)	Final score: Median (range)	P value
Patient presentation total score (62 possible points)	33 (28‐36)	36.5 (29.5‐43.5)	.004
Subjective score (15 possible points)	7 (4‐10)	6 (3‐9)	.087
Objective score (7 possible points)	3 (1‐5)	5.5 (2‐7)	<.001
Pharmaceutical plan (20 possible points)	14.5 (12‐16.25)	15.5 (13‐18)	<.001
Monitoring (9 possible points)	3 (3‐4)	4 (3‐5)	<.001
Communication (5 possible points)	5 (5–5)	5 (5‐5)	N/A

Abbreviation: N/A = Not available.

TABLE 3

Student self‐assessment survey results

Assessment	Median (IQR) a	P value
Level of confidence in caring for an ARDS patient
Baseline n = 76 Exit n = 96	4 (3‐6) 6 (5‐7)	<.001
Level of confidence in managing sedation in a critically ill patient
Baseline n = 77 Exit n = 96	6 (4‐7) 7 (5‐8)	.001
Level of confidence in caring for a critically ill patient
Baseline n = 76 Exit n = 96	3 (2‐4) 6 (5‐7.5)	<.001
Level of confidence in caring for a short‐term ill patient with COVID‐19
Baseline n = 70 Exit n = 95	3 (2–4) 6 (4‐7)	<.001
Level of understanding of challenges caring for a patient with COVID‐19
Baseline n = 77 Exit n = 96	4 (2‐5) 7 (5‐7.5)	<.001
Level of understanding of the current therapeutic options for a patient with COVID‐19
Baseline n = 77 Exit n = 96	4 (2‐5) 7 (5–7.5)	<.001
Level of understanding of the role of a pharmacist in regard to treating a patient with COVID‐19
Baseline n = 73 Exit n = 95	4 (3‐6) 7 (5‐8)	<.001

Note: Communication skills—reported separately in results.

Abbreviation: ARDS = Acute respiratory distress syndrome.

Scale of 1 to 10 (1 being not confident at all, 10 being completely confident).

DISCUSSION

The interinstitutional COVID‐19 simulation activity showcased the talents and ingenuity of both Schools of Pharmacy. Initially, the faculty met to discuss ways to share resources during virtual remote rotations by inviting students from both programs to participate in virtual topic discussions, journal clubs, and activities. These meetings led to the realization that our students were precluded from participating in the direct care of patients with COVID‐19 due to local institution rules and that they were missing out on a valuable learning opportunity. The resulting rotation activity was designed to optimize student and faculty time so as to not take away from other rotation activities. Overall, faced with emergency remote experiential learning, the authors came together to pool expertise and resources to provide rotation students with a rich learning activity.

In this particular activity, students utilized information collected from EHR Go to develop an individualized plan for the patient and then create a pharmacotherapeutic implementation plan. The communication of that plan was verbalized, and the effects of the drugs recommended were represented on the patient simulation software monitor in an accelerated time frame. The patient monitor was visualized by the students via the Zoom screen‐sharing function. Due to being from different schools of pharmacy, the students were instructed to meet informally as a group via Zoom before the start of the activity. Each student group decided on a spokesperson to present the recommendations, and there was an effort to maintain smaller group sizes to encourage and ensure student participation. Even though one to two students acted as a spokesperson, there was discussion between all members when diverging treatment options were available. The simulation activities provided the students with the opportunity to present the patient to facilitators and make recommendations in real time, which allowed them to see the effects of their choices. These experiences appear to have translated over to their performance on the written examination and the graded video recordings.

The results of the written examinations, student surveys, and video recordings demonstrated that overall student knowledge and confidence improved regarding COVID‐19 and critical care–related topics. The examination questions on the written examination were based around issues and content areas pertinent to the care of a critically ill patient such as analgesia, sedation, ICU preventive measures, and COVID‐19‐related topics. Students scored higher on the written examination after completing the simulation activity when compared with the score on the preactivity written examination. The student perception surveys revealed that the student self‐assessed confidence improved in all domains assessed. The video recordings graded using the ICE rubric demonstrated that all components of the ICE improved from baseline, except the subjective score and communication categories. We believe that an improvement was not seen in the subjective score due to the fact that the pharmacy students were instructed to focus on the assessment and plan portion of their recommendations. Therefore, students likely would not have focused on communicating subjective data in their presentation. The subjective portion of the rubric is also worth approximately 24% of the total points, so skipping this section likely led to an overall poor score on the ICE rubrics. The authors would like to note that although the communication subdomain of the ICE rubric did not change from baseline, we did not expect this to change over a 1‐week time period and two simulation sessions.

The current study is not without limitations. One notable limitation of this study was the timing and number of activities due to the immediate and emerging constraints of the COVID‐19 pandemic. The University of Pittsburgh and Duquesne University have differing APPE rotation schedules, which limited the available time frame for the interinstitutional group activities. Had the rotation calendars matched more closely, an increased number of activities would have been preferred. Another limitation due to offset APPE rotations was the different experience levels, specifically between the first cohort of students. The first cohort of students consisted of Duquesne University students who were in their final APPE rotation, and University of Pittsburgh students were in their first APPE rotation. This limitation was mitigated for the second and third cohorts where the students from both institutions were closer in experience levels. This study is limited by the absence of a direct comparison of pedagogic approaches (ie, simulation, traditional didactic lecture, live APPE patient workup/presentation). This study cannot ascertain the relative effectiveness of this approach to traditional models or methods. Another limitation of the study was the fidelity of the simulations. Due to the remote nature of the simulations, they were limited to utilizing monitoring applications rather than the mannequin and in‐person monitors. The interactive nature of the simulation activities precludes from defining them as low‐fidelity, but the remote nature likely does not meet the high‐fidelity definition of “extremely realistic”. Though the field lacks a standardized term for this type of simulation activity, the study authors suggest “medium” or “intermediate” fidelity would be most appropriate. The absence of a high‐fidelity simulation could have led to a possible loss of spontaneous ad hoc learning that occurs when at the patient's bedside or while on rounds with the multidisciplinary team. Although the simulations were conducted in this manner for the safety of the students and administrators during the COVID‐19 pandemic, the opportunity will arise for high‐fidelity simulations to resume and student experiences to be enhanced further. Due to the retrospective nature of this study, not all the students were required to complete both pre‐ and postsurveys. Notably, there were students who were either added to, or dropped from, the APPE and were unable to complete both surveys and all activities. In addition, this study did not seek to formally assess the students' attitudes toward group member participation or the interinstitutional group dynamics. However, informal student feedback discussed at the debrief following the second simulation suggested overall positive collaborative experiences.

The success of the simulation activity in the virtual rotation setting may lead to consideration of its use during in‐person simulated environments or practicums during the didactic curriculum for an entire pharmacy class year. The virtual format is enticing, as it may alleviate some of the effort required to align the class schedules of two pharmacy classes from separate schools and allow for scaling to include the entire class at each school. Additional resources would be required to develop, facilitate, and assess the student activities. Appendix S3 provides an example of scheduling four student groups, two faculties, and a stimulation coordinator. Multiple sessions would be required to handle large class sizes. Faculty replicating this with an entire class would need to budget 40 minutes per group of students and may need two to three additional facilitators to lead discussions with student teams. A simulation coordinator in each session is necessary to make adjustments to the simulator settings based on student recommendations. Academic integrity would also be a concern if student groups were meeting for simulation sessions throughout the course of a week. This could be alleviated with modified cases and requiring students to sign academic integrity agreements.

CONCLUSION

In lieu of in‐person rotations, remote simulation sessions during virtual rotations can be utilized to improve student knowledge and confidence. The simulation activity described in this study provided an opportunity for students to experience caring for patients with COVID‐19 in a safe environment during a global pandemic. Collaboration between schools of pharmacy can be successfully employed to leverage resources and expertise to expand opportunities for students during and beyond the pandemic.

CONFLICT OF INTEREST

Amy Seybert serves on the board of directors of the Accreditation Council for Pharmacy Education (ACPE). This article does not represent ACPE's or the board of directors' opinions or views. She also receives consulting fees in her role as a consultant for Pfizer. All other authors have no relevant conflicts of interest.

APPENDIX S1 Supporting Information

Click here for additional data file.

APPENDIX S2 Supporting Information

Click here for additional data file.

APPENDIX S3 Supporting Information

Click here for additional data file.

APPENDIX S4 Supporting Information

Click here for additional data file.

APPENDIX S5 Supporting Information

Click here for additional data file.