Medical Student Education During the COVID-19 Pandemic: Initial Experiences Implementing a Virtual Interventional Radiology Elective Course.

RATIONALE AND
OBJECTIVES: In response to the COVID-19 pandemic reducing medical student presence on clinical services and in classrooms, academic institutions are utilizing a virtual format to continue medical student education. We describe a successful initial experience implementing a virtual elective in interventional radiology (IR) and provide the course framework, student feedback, and potential improvements.
MATERIALS AND METHODS: A 2-week virtual IR elective curriculum was created utilizing a combination of synchronous and asynchronous learning and the "flipped" classroom educational model. Students virtually participated in daily IR resident education conferences, resident-led case review sessions, and dedicated lectures. Asynchronous prelearning material consisted of text and video correlating to lecture topics. Anonymous precourse and postcourse surveys were sent to all participating students (n = 10).
RESULTS: Ten students (100%) completed precourse and seven (70%) completed postcourse surveys. Enrolled students were considering residencies in surgery (50%), internal medicine (40%), interventional radiology (30%), and/or diagnostic radiology (30%). Students' understanding of what IRs do and the procedures they perform (p < 0.001), when to consult IR for assistance in patient management (p = 0.005), and the number of IR procedures students could recall (p = 0.015) improved after the course. Case-review sessions and virtual lectures ranked as having the highest education value. Students recommended additional small-group case workshops.
CONCLUSION: This successful virtual IR elective provides a framework for others to continue IR medical student education during the pandemic and grow the specialty's presence within an increasingly virtual medical school curriculum. The described model may be modified to improve IR education beyond the COVID-19 era.

INTRODUCTION

While the COVID-19 pandemic has disrupted medical education at all levels of training, its effect on medical student education has been especially profound. On March 17th, 2020, the Association of American Medical Colleges recommended the suspension of medical student clinical rotations and has reaffirmed this position in subsequent statements (1). In addition to the suspension of clinical rotations, preclinical curricula have migrated to a virtual format to maintain distancing of students (2).

The current disruptions to medical student education pose unique challenges to interventional radiology (IR). Currently, IR is not well represented within the preclinical curriculum and this status quo is unlikely to change during the transition to a virtual curriculum (3, 4, 5, 6). The lack of preclinical IR experience results in students relying on clinical IR electives and/or IR exposure integrated into diagnostic radiology or surgical clerkships to gain exposure to the field (7). Decreased clinical time during the pandemic has therefore disproportionately reduced medical student exposure to IR. This proves a disservice to students interested in pursuing IR training as well as those bound for other specialties who will interact with IR while caring for their patients.

IR educators must adapt to virtual learning and find new ways to educate medical students, as has been done in diagnostic radiology, surgery, and other specialties (4,8,9). This article describes an initial experience in creating and facilitating a virtual IR elective at an academic tertiary care medical center and includes a description of the course framework, student feedback, and suggestions for future improvements.

MATERIALS AND METHODS

Institutional review board approval was obtained for this survey study. The virtual IR elective consisted of a remotely-attended two-week course offered to third and fourth year medical students on a recurring basis. The course was one of over 30 virtual electives offered by the medical school, which ranged across disciplines including various medical, surgical, and subspecialty areas. Students were registered for the course based on their ranking of elective preferences. The course was capped at a maximum 10 students per iteration to allow for interactive virtual group sessions.

Course Curriculum

The course curriculum was reviewed and approved by the medical school, which required virtual courses to include at least 40 hours of “student effort” per week in the form of synchronous and asynchronous learning activities. Synchronous learning refers to learning in which students and instructors engage in real-time, typically utilizing videoconferencing and/or chat software to allow for live interaction, while asynchronous learning refers to learning that occurs at different times for each student, without real-time interaction, utilizing resources such as assigned readings or prerecorded videos provided by the instructor. The virtual elective consisted of approximately 50% synchronous and 50% asynchronous content. A “flipped-classroom” model, in which students were provided asynchronous educational material to review prior to participating in a synchronous lecture on the same topic, was employed throughout the course (5,10).

Synchronous Learning Activities

Synchronous activities occurred in real-time through institutionally-licensed and Health Insurance Portability and Accountability Act compliant videoconferencing software (BlueJeans, Mountainview, California; Microsoft Teams, Redmond, Washington). Each morning, students virtually attended the daily IR resident education conferences (part of the ongoing IR resident training curriculum). Resident-led case-review sessions modeled after morning “film rounds” (Table 1 ), during which a resident would review the previous day's cases and discuss patients’ medical history, work-up, imaging, procedural details, and follow-up, were held with students at various times depending upon resident availability. Students also attended between one and three daily virtual lectures. These were predominantly attending-led and delivered specifically for medical students enrolled in the course (Table 2 ). Attending lectures were scheduled according to academic availability. Resident-led sessions were scheduled during resident academic time when able, otherwise, residents were excused from clinical service to participate in the course. At the conclusion of the course, students virtually participated in an end-of-course student presentation session, in which each student presented on a topic of interest within IR that was not directly covered in the course.

Table 1

Daily Interventional Radiology (IR) Resident Education Conferences (Part of the Ongoing IR Resident Training Curriculum) Were Being Held Virtually Prior to the Start of the Medical Student Elective

IR Resident Education Conferences	Description
Chief Rounds	Bi-weekly case-based conference led by the IR division chief during which trainees work through the pre-, intra-, and postprocedural aspects of cases
Resident Lecture	Traditional didactic lectures provided on a weekly basis by IR faculty and based on the interventional radiology residency didactic curricula
Journal Club	Topic-focused attending-moderated sessions during which residents present 3–4 articles, followed by discussions of study design, methods, validity, and role in the practice of evidence-based IR
Liver Tumor Board	Multidisciplinary conference run by the transplant department which discusses diagnostic and treatment options for patients
Morbidity and Mortality Conference	Resident-led discussion of procedural and patient-care complications attended by IR faulty and trainees, with a focus on safety and quality improvement
Film Rounds*	Daily review of each case performed the day prior – residents typically present the cases they specifically performed, highlighting anatomy, procedural technique, thought process, and educational highlights

Medical students attended these resident conferences as part of the virtual elective.

For the virtual elective, separate case-based reviews (film rounds) were held specifically for medical students every 1–2 days, during which an IR resident reviewed selected cases with the students, ensuring the discussion was appropriate to their education level and allowing more time for questions.

Table 2

Virtual Medical Student Lecture Topics and Associated Asynchronous Prelearning Text Materials

Category	Lecture Topic		Asynchronous Text Material
Intro	IR basics		• Interventional Radiology: A half century of Innovation (Radiology 2014, Baum and Baum) • IR Playbook: Tools of the Trade (Ch.4) • The inaccessible or undrainable abscess: How to drain it (Radiographics 2004, Maher et al)
Arterial interventions	Peripheral arterial disease		• IR Playbook: Aortiliac disease and infrainguinal disease (Ch. 30 and 31) • Recent trends in setting and specialty for endovascular peripheral arterial disease (JVIR 2020, Schramm et al)
	Acute limb ischemia		• Acute limb ischemia (TechVIR 2009, Walker) • Acute limb ischemia interventions (Interventional Cardiology Clinics 2020, Khan and Hawkins) • TOPAS trial (NEJM 1998, Ouriel et al)
	Trauma		• The evolving role of interventional radiology in trauma care (Journal of trauma injury, infection, and critical care 2005, Pryor et al) • IR in the ER (IR Quarterly feature, Fall 2017, Rose)
Venous interventions	Venous disease		• Catheter-directed thrombolysis in management of iliofemoral DVT (Radiographics 2016, Chen et al) • May-Thurner syndrome (Vascular Medicine 2018, Liddel and Evans) • Abstract review: CaVenT, ATTRACT, and Sox trials
	IVC filters		• IR playbook: IVC filters (Ch. 11) • IVC filters: Guidelines, best practice, and expanding indications (Seminars in IR 2016, DeYoung and Minocha)
Interventional oncology	Ablation	NA*	• Tumor ablation: common modalities and general practices (TechVIR 2013, Knavel and Brace)* • Keeping out of trouble in ablation techniques (TechVIR 2018, Smirniotopoulos et al)* • Immuno-oncology and its opportunities for IRs (JVIR 2017, Hickey et al)*
		Lung	• IR Playbook: Lung, kidney, and bone ablation (Ch. 37) • Percutaneous lung tumor ablation (TechVIR 2013, Sheu and Hong)
		Kidney	• IR Playbook: Lung, kidney, and bone ablation (Ch. 37) • Renal tumor ablation (TechVIR 2013, Georgiades and Rodriguez
		Liver	• IR playbook: Liver ablation (Ch. 34)
	Transarterial chemoembolization		• IR Playbook: Transarterial chemoembolization (Ch. 37) • Optimizing transarterial therapies for hepatic malignancy (TechVIR 2018, Kouri) • Abstract review: Combined TACE and ablation
Hepato-biliary	TIPS		• IR playbook: TIPS (Ch. 38)
	Biliary drainage		• Clinical presentation, Imaging, and management of acute cholecystitis (TechVIR 2015, Katabathina et al) • Percutaneous transhepatic biliary drainage (TechVIR 2008, Covey and Brown) • Benign biliary strictures: Evaluation and approaches to treatment (TechVIR 2015, Fidelman) • Percutaneous management of malignant biliary obstruction (TechVIR 2015, Sutter and Ryu)
Additional topics	Uterine artery embolization and varicocele treatment		• IR playbook: Uterine artery embolization (Ch. 28) • Uterine fibroid embolization: technical aspects (TechVIR 2002, Worthington-Kirsch et al) • IR playbook: Varicocele (Ch. 14)
	PAVM embolization		• PAVM embolization: An update (AJR 2010, Trerotola and Pyeritz)
	Lymphatics and thoracic duct embolization		• Modern techniques of lymphangiography and intervention (CVIR 2018, Itkin and Nadolski) • Lymphatic interventions for various types of lymphorrhea (Radiographics 2016, Inoue et al)

AJR, American Journal of Radiology; CVIR, Cardiovascular and Interventional Radiology; IR, interventional radiology; IVC, inferior vena cava; JVIR, Journal of Vascular and Interventional Radiology; NA, not applicable; NEJM, New England Journal of Medicine; PAVM, pulmonary arteriovenous malformation; TechVIR, Techniques in Vascular and Interventional Radiology; TIPS, transjugular intrahepatic portosystemic shunt.

Abbreviated citations are provided for articles and the complete citation for IR playbook is as follows: Keefe, Nicole A., et al, eds. IR Playbook: A Comprehensive Introduction to Interventional Radiology. Springer, 2018.

These materials were provided with the ablation lecture (lung, kidney, liver) that was delivered first.

Asynchronous Learning Activities

Asynchronous learning consisted of prelearning materials in text and video formats to provide background knowledge on topics prior to their corresponding lecture (Table 2) (3, 4, 5,10). Text material was available to students through the institution's library. Videos predominantly consisted of narrated IR procedures available online. Students were also given access to optional IR webinars organized by other institutions.

Course Organization and Evaluation

Content was organized online via course management software used by the medical school (Canvas by Instructure, Salt Lake City, Utah). Students were sent a daily email with the following day's schedule, assigned prelearning content, and videoconferencing links. An example daily schedule is provided in Table 3 . Performance was assessed using an open-book mid-elective quiz focusing on key teaching points covered in the case-review sessions and the end-of-course student presentations. Lecture attendance was also monitored. Completion of asynchronous prelearning material was not directly monitored. The course had a pass and/or fail grading system.

Table 3

Example Generic Daily Schedule

	Educational Activity	Facilitator	Attendees	Time Commitment
Morning	IR Resident Education Conference	IR attending	Enrolled medical students and IR residents	1 hour
	Virtual Medical Student Elective Lecture #1	IR attending*	Enrolled medical students	1 hour
	Asynchronous Prelearning	-	-	2 hours
Break
Afternoon	Virtual Medical Student Elective Lecture #2	IR attending*	Enrolled medical students	1 hour
	Case-review Session(“film rounds”)	IR resident	Enrolled medical students	1 hour
	Asynchronous Prelearning	-	-	2 hours

IR, interventional radiology.

The 8-hour daily commitment ensured approximately 40 hours per week of “student effort,” as required by the medical school.

Lectures were predominantly given by attending IR physicians, with the exception of “Basics of IR” and “Trauma,” given by residents.

Surveys and Statistical Analysis

Anonymous web-based surveys consisting of multiple choice, Likert scales, and free response were sent to all enrolled students before and after the course. Survey participation was optional. Both surveys had questions regarding knowledge and interest in IR. The pre-elective survey had additional questions regarding prior exposure to IR. The postelective survey had questions regarding the effectiveness of different course elements. Survey responses were analyzed for trends and compared using the Student's t-test, when applicable.

RESULTS

Ten students had participated in two consecutive iterations of the course (five students per iteration) at the time of manuscript preparation. All students passed the course.

Ten students completed the precourse survey (100%). Seven completed the postcourse survey (70%).

Interest in IR

Participants reported first learning about or developing interest in IR during a prior medical school lecture (n = 4), IR research experience (n = 2), discussion with peers (n = 1) or family (n = 1), or during review of virtual elective course offerings (n = 2). Three students were Society of Interventional Radiology members and one had previously attended an IR national conference. When asked to provide up to three specialties students were considering for residency, students primarily reported surgery or surgical subspecialties (n = 5), internal medicine or medical subspecialties (n = 4), interventional radiology (n = 3), and diagnostic radiology (n = 3) (Fig 1 ).

Figure 1

Pre-elective and postelective survey response data. Questions marked by an asterisk (*) were asked only during the pre-elective survey. IR, interventional radiology.

IR Knowledge

Regarding IR knowledge prior to the elective, 50% of students (n = 5) felt they “had barely been introduced to IR,” 40% (n = 4) felt “as familiar with IR as any other specialty,” and one student was “slightly more familiar with IR than any other specialty.” After the course, responding students felt they were either “as familiar with IR as any other specialty” (57%, n = 4) or “slightly more familiar with IR than any other specialty” (43%, n = 3). A statistically significant improvement was found in students’ understanding of what interventional radiologists do and the procedures they perform (p < 0.001), when to consult IR for assistance in patient management (p = 0.005), and the number of IR procedures students could recall (p = 0.015) (Fig 1). No difference was found in interest in IR as a specialty choice (p = 0.89).

Student and Educator Feedback

When asked to rank different course elements in order of educational value, case-review sessions, and virtual medical student lectures ranked highest, each with three number one rankings and mean rankings of 2.6 and 2.7, respectively. These were followed by student presentations, IR resident education conference, and asynchronous content, with mean ranks of 2.9, 3.6, and 3.7, respectively. Students provided positive feedback regarding case-review sessions, lectures, course organization, and efforts made to tailor the course to specific student interests. Suggestions for improvement included developing additional small-group case workshops (Table 4 ). IR residents and faculty participating in the course reported entirely positive experiences, with some faculty volunteering to give multiple additional lectures. Faculty reported returning to medical student education in the midst of the pandemic as a “return to normalcy” despite its virtual format and multiple faculty were impressed by students level of interest in IR and the quality of their prelecture preparation, evidenced by student knowledge of lecture topics, and a high level of participation during lectures.

Table 4

Anonymous Student Feedback

Anonymous Student Feedback

• I really enjoyed the course and thought it was a great introduction to the variety of procedures done in IR. I appreciate that the residents and/or faculty tried to sculpt the lecture based on our individual specialty interests. In particular, I thought the film rounds were particularly interesting. • Overall, I am very glad I took this course and am certainly more knowledgeable with understanding how IR plays a role in minimally invasive treatment and am more comfortable with interreacting with IR when taking care of patients in the future. • Really appreciate all the content and how organized it's been. Super interesting lectures, great didactic content on canvas, and film rounds are engaging. • Awesome elective! • It's been an honor to hear directly from IR experts and for you to bring your expertise to film rounds just for us, and I've learned a lot. • One area for improvement could be developing a small group case workshop where we work through patient scenarios, work ups, differentials, treatment options, etc.

IR, interventional radiology.

Feedback was minimally edited for spelling, grammar, and readability.

Discussion

The COVID-19 pandemic and resulting limitations on in-person medical student education has forced educators to rapidly adapt to the “new normal” of remote online learning. While the core IR clinical experience comprised of hands-on procedures and patient-care cannot be replaced, the challenges of finding new ways to engage learners and showcase the scope of IR offers opportunities to positively reshape the future of IR medical student education. The described experience demonstrates that a highly effective virtual IR elective can be created and implemented.

Simulating the clinical experience in a virtual format was one of the foremost goals of the course design. Prior to the COVID-19 pandemic, rotating medical students and residents attended daily IR resident education conference each morning. These conferences were being held virtually to maintain resident education during the pandemic, and students in the elective joined them in virtual attendance, maintaining this aspect of their educational experience. Resident involvement in medical student education was a crucial aspect of the virtual elective. Since residents and medical students work closely during clinical rotations, residents as teachers within virtual electives restore normalcy for both parties, offer future IR applicants’ opportunities to interact with IR residents, and allow residents to hone teaching skills (11). Case-review sessions, which simulate clinical patient-care scenarios more effectively than didactic lectures, allow for increased student interaction and are an effective means of virtual education (4). Students highlighted the resident-led case-review sessions as the best aspect of the virtual elective. In addition to holding case-review every 1-2 days, student feedback recommended the addition of problem-based small group learning opportunities (Table 4). Problem-based learning (PBL), in which groups of students guide themselves through different case-based scenarios while applying their medical knowledge and clinical reasoning skills, provides a student-centered educational approach (12). The creation of successful small-group sessions utilizing PBL is labor-intensive, limiting their inclusion in such a rapidly implemented course, and potential difficulties facilitating multiple groups of students in the virtual setting must be considered. However, PBL is often considered superior to more traditional teaching methods and is well-suited to teaching IR clinical scenarios (12). As a result, virtual PBL sessions are currently being developed for use in IR education at our institution, including within the preclinical curriculum.

The elective applied the “flipped-classroom” model to lectures throughout the course, which encourages asynchronous self-learning followed by synchronous sessions focusing on the application of knowledge (3, 4, 5,10). The number of virtual lectures was much greater than the typical number of lectures in traditional IR elective and represented one of the largest differences between the two experiences. Providing students with asynchronous prelearning reading and video materials to establish background knowledge on a subject allowed facilitators to focus on clarifying difficult concepts, sharing clinical pearls, and engaging with students during virtual lectures. Lectures ranked as one of the course elements with the highest educational-value, likely due in part to the benefits of the “flipped-classroom” model.

The lack of student involvement in clinical patient care and procedures was the largest difference between the traditional IR elective and virtual lR elective. Exposure to procedures could be improved with the creation of new high-quality video content available as prelearning and/or live streaming of cases to students, as has been reported in virtual surgical rotations (13). As students return to clinical service and participate in cases, a “blended-learning” model may be considered to encourage physical-distancing whenever possible. Blended-learning combines online education, either in the form of virtual synchronous or asynchronous learning, with in-person place-based teaching methods (14). This would allow students to be physically present for IR clinical activities and procedures, an irreplaceable aspect of traditional IR clinical rotations, while participating in morning conference, didactic lectures, and other learning activities virtually. The success of the virtual IR elective is encouraging in that students found virtual learning to be effective and would likely be open to its inclusion in future electives utilizing a blended-learning model.

Engagement was further encouraged by tailoring sessions to individual interests. For example, many students in the first iteration of the course were interested in applying to IR and/or diagnostic radiology residencies, thus a residency information session was held. In the second iteration, several students were interested in urology, thus additional genitourinary cases and concepts were covered in the case-review sessions. In fact, many students who planned to apply to a specialty other than IR specifically enrolled in the IR virtual elective, rather than one of many other available electives, with the goal of becoming more familiar with IR's role in patient care as it will relate to their future careers. This unexpected finding highlights the need for more accessible IR rotations for upper-year medical students pursuing nonradiologic careers.

Prior to the pandemic, few practical options regarding IR education existed for students at our institution who did not wish to pursue the traditional 4-week IR elective. Prior studies have shown that medical students are eager to learn about IR and that interest in a 2-week IR elective is high, regardless of career choice (14, 15, 16). Students pursuing non-radiologic specialties may prefer a shorter IR experience tailored towards their specific educational needs. However, only a minority of programs currently offer a 2-week IR elective option (7). In the post-COVID era, offering a 2-week elective targeting students bound for nonradiologic specialties who are unlikely to commit to a 4-week clinical IR elective may allow IR physicians to more effectively educate future colleagues and more broadly increase awareness of IR throughout the medical community. A blended-learning model could also be employed in this setting, in which students spend part of their time on the IR clinical service and the remainder of their time utilizing synchronous and/or asynchronous resources geared towards those entering nonradiologic specialties. Alternatively, a 2-week IR selective can be offered as a subspecialty option within the core surgical clerkship, as is currently done at some institutions (7). Previously, the IR elective at our institution was limited to only the students who had completed the core clerkships, however, the virtual IR elective was offered more broadly, as core clerkships had been suspended This transiently increased IR exposure to students early in their clinical training and may hopefully lead to more opportunities for IR within the core clerkship curriculum. Additional future considerations include offering an abridged course to first and second year medical students to increase the presence of IR within the preclinical curriculum and offering an “away” virtual course to students from other institutions.

Limitations

This study is limited by the small size of the surveyed cohort and the inherent limitations and biases of a survey study. The generalizability of our institutional experience may be limited by the availability of adequate videoconferencing software, ability of IR attendings and residents to participate in synchronous teaching sessions, and access to previously created educational content at other institutions. How participation in a virtual elective impacts student's abilities to obtain letters of recommendation or impacts their application to IR residency was not evaluated. Requests to participate in the elective were received from students from outside institutions, however, no institution-approved mechanism for participation of such students were in place during this initial experience.

CONCLUSION

Interventional radiologists must leverage their ability to embrace technology in order to adapt to the increased emphasis on virtual learning in the wake of COVID-19. This successful initial experience with a virtual IR elective provides a framework for continuing IR medical student education during the pandemic and growing the specialty's presence within an increasingly virtual medical school curriculum.

Disclosures

Scott O. Trerotola is a paid consultant for BD Bard (Covington, Georgia), B Braun Melsungen AG (Melsungen, Germany), W.L. Gore & Associates (Flagstaff, Arizona), Adrenas Therapeutics, Inc (Palo Alto, California), MedComp (Harleysville, Pennsylvania) Cook, Inc (Bloomington Indiana) and Teleflex (Wayne, Pennsylvania) and receives royalties from Cook, Inc, and Teleflex.

Funding

None.

Acknowledgments

The authors thank Dr. Soulen, Dr. Shlansky-Goldberg, Dr. Stavropolous, Dr. Itkin, Dr. Mondschein, Dr. Hunt, Dr. Gade, Dr. Reddy, and Dr. Khaddash (Department of Radiology, Division of Interventional Radiology, Perelman School of Medicine at the University of Pennsylvania) for participating in the course and Dr. Vairavamurthy (Department of Radiology, Division of Interventional Radiology, Keck School of Medicine at the University of Southern California) for inviting students to participate in their COVID-19 webinar series.