Implementation and evaluation of eight virtual surgical electives for medical students during the COVID-19 pandemic.

BACKGROUND: Eight novel virtual surgery electives (VSEs) were developed and implemented in April-May 2020 for medical students forced to continue their education remotely due to COVID-19.
METHODS: Each VSE was 1-2 weeks long, contained specialty-specific course objectives, and included a variety of teaching modalities. Students completed a post-course survey to assess changes in their interest and understanding of the specialty. Quantitative methods were employed to analyze the results.
RESULTS: Eighty-three students participated in the electives and 67 (80.7%) completed the post-course survey. Forty-six (68.7%) respondents reported "increased" or "greatly increased" interest in the course specialty completed. Survey respondents' post-course understanding of each specialty increased by a statistically significant amount (p-value = <0.0001).
CONCLUSION: This initial effort demonstrated that VSEs can be an effective tool for increasing medical students' interest in and understanding of surgical specialties. They should be studied further with more rigorous methods in a larger population.

Introduction

The COVID-19 pandemic caused an unprecedented interruption in medical education for students across the country. Beginning in March 2020, concerns over safety and the limited availability of personal protective equipment forced students to leave their clinical rotations, threatening their opportunities to gain exposure to medical specialties, complete graduation requirements on time, and acquire the skills and knowledge necessary for residency. Concerns rose that student training in surgical specialties would suffer disproportionately from the cancelation of clinical activities due to their procedural nature. In response, surgical educators began quickly devising virtual curricula to meet the needs of their students.

At Emory University School of Medicine, faculty, residents, and senior medical students collaborated to create remote learning opportunities for junior students who had just begun their clinical clerkships. The newly formed Virtual Surgery Education Group (ViSEG) created one- and two-week virtual surgery electives (VSEs) in eight surgical specialties that were each delivered via videoconference two to three times over the course of a four-week period in April–May 2020. While each course had its own set of learning goals and objectives, the overall purpose of the initiative was to introduce students to surgical fields, teach the foundations of diagnosis and management of surgical disease, and offer opportunities for students to develop basic technical skills during virtual laboratory sessions.

In this study, we present the outcomes of our VSEs as measured by the results of a survey administered to students at the conclusion of each course. While recognizing the limitations inherent in remote learning and the limitations of creating novel curricula in a compressed timeline, we aimed to establish the utility of virtual technologies in medical school education and inspire other institutions to investigate their potential during this difficult period.

Materials and methods

Development process

Participating specialties included General Surgery, Cardiothoracic (CT) Surgery, Neurosurgery, Orthopedic Surgery, Otolaryngology (ENT), Plastic Surgery, Urology, and Vascular Surgery. Working in conjunction with faculty and residents, senior medical students designed one-week long courses for each specialty with the exception of General Surgery, which was extended to two weeks because of its relative weight in the in-person surgical clerkship. Learning objectives were individualized for each course based on guidelines for medical school curricula published by professional associations affiliated with each specialty. Core topics covered by the United States Medical Licensing Examination (USMLE) Step 1 and Step 2 board examinations were also reviewed. All sessions were held over Zoom® videoconferencing software. For hands-on skills labs, students picked up kits (Fig. 1 ) with sanitized or disposable instruments from the Department of Surgery offices prior to the course start date. Methods of student evaluation were tailored to each subspecialty by the team administering the course, resulting in distinct forms of quantitative and qualitative evaluations. Regardless, all participating students received a pass/fail grade and could apply course credit towards graduation requirements as an elective but not as a substitute for the surgical clerkship. A summary of resources utilized, teaching modalities employed, and methods of assessment are outlined in Table 1 .

Fig. 1

Supplies delivered to students for remote suture skills workshops. Students mount personal cellphone on tripod and aim to allow instructor view of their hands. Students view instructor demonstration on their laptop.

Table 1

A summary of resources utilized, teaching modalities employed, and methods of assessment for offered electives. CT: cardiothoracic; ENT: otolaryngology.

	General Surgery	CT Surgery	Neurosurgery	ENT	Orthopedic Surgery	Plastic Surgery	Urology	Vascular Surgery
Resources Utilized
Textbook	X		X	X	X	X	X
Journal articles	X		X	X	X	X	X	X
Surgical video	X	X	X	X	X	X	X	X
Teaching Modalities
Student-led lectures	X	X	X	X	X	X	X	X
Faculty-led lectures	X	X	X	X	X	X	X	X
Dept./national lectures	X	X		X		X	X
Case-based learning	X		X	X	X	X	X	X
Surgical video narration	X	X	X	X	X	X	X
Skills labs	X				X	X		X
Telemedicine			X
Methods of Assessment
Attendance/participation	X	X	X	X	X	X	X	X
Daily quizzes				X	X	X		X
Pre-test		X		X		X	X	X
Post-test		X		X	X	X	X	X
5-min oral presentation			X	X		X	X	X

Curricula

General Surgery

Content: This elective aimed to teach pathophysiology and management of common general surgical conditions, elements of perioperative care, and comprehensive care of the trauma patient. Topics covered included biliary disease, abdominal, inguinal, and hiatal hernia, appendicitis, diverticular disease, bowel obstruction, bariatric surgery, and perioperative trauma management, similar to the learning objectives of the third-year clerkship.

Teaching modalities: Senior medical students held introductory lectures covering presentation, management, and complications of the specified surgical condition. Students were assigned readings and operative videos prior to lecture day. Group sessions were held in which faculty and/or residents guided students through cases. Faculty led students through pre-recorded operations highlighting technique and critical steps of the procedure. A surgical skills lab was held to teach basic techniques in suturing and knot tying. Finally, students attended Morbidity and Mortality and Grand Rounds conferences via videoconference.

Student Evaluation: Students were expected to actively participate throughout the week but were not required to complete any graded assignments.

Cardiothoracic Surgery

Content: The week began with a brief anatomy and pathophysiology review before transitioning to disease-specific processes and surgical procedures. Topics covered included endocarditis, lung and pleural space infections, coronary artery disease and coronary artery bypass grafting, medical and surgical management of pulmonary embolism, lung cancer staging and surgical management, hiatal hernia and surgical management of gastroesophageal reflux disease, and surgical management of advanced lung disease. The week culminated in an overview of extracorporeal membrane oxygenation and its role in the management of COVID-19.1, 2, 3, 4

Teaching modalities: Faculty and senior medical students led daily lectures and periodic review sessions. Faculty also guided students through surgical case videos. Students were also invited to attend residency didactics via videoconference.

Student Evaluation: A pre- and post-course quiz of five multiple-choice questions (MCQs) was administered.

Neurosurgery

Content: The goals of the course were to introduce neurosurgical topics to junior medical students, reorient neuroanatomy in a clinical context through surgical videos, and exercise clinical skills through attendance at telemedicine clinics. Student needs were assessed through a combination of board preparation materials and a recently published supplement in Operative Neurosurgery titled “Essential Neurosurgery for Medical Students.” Topics covered included neuroradiology, neurocritical care, trauma, vascular, tumor, spine, and functional neurosurgery.

Teaching modalities: Content was conveyed through a combination of lectures and reading assignments. Residents and faculty narrated surgical video content of select neurosurgical procedures. In addition, several sessions were dedicated to the discussion of career development, mentorship, and work-life balance in neurosurgery.

Student Evaluation: Students were required to deliver a short presentation on a topic or patient of interest but were not asked to complete a pre- and post-course quiz.

Orthopedic Surgery

Content: A range of introductory orthopedic content was covered including residency structure, tips for working as a consulting orthopedic resident, fellowship choices, a systematic approach to radiograph reading, high yield upper and lower extremity trauma cases, casting and splinting techniques, and surgical knot tying and suturing.

Teaching modalities: Teaching was accomplished primarily via lectures and required readings. A suture and knot-tying skills lab was held using pre-recorded videos of a senior medical student demonstrating one and two-handed knot tying. On the final day of the course, students participated in a Jeopardy®-style game to review content covered throughout the week.

Student Evaluation: Students completed a pre- and post-course quiz consisting of four short-answer response questions graded using a rubric created by the supervising faculty. Practice MCQs were incorporated during the second iteration of the course to help students keep up with lecture content but did not contribute to their grade.

Otolaryngology

Content: The course centered around the American Academy of Otolaryngology-Head and Neck Surgery (AAO-HNS) Foundation’s publication, Primary Care Otolaryngology, 4th Ed. Topics included the ENT physical exam, rhinology, otology, facial plastic and reconstructive surgery, laryngology, pediatric otolaryngology, head and neck imaging, and otolaryngology emergencies. Additional readings and course information were provided in consultation with senior otolaryngology faculty.

Teaching modalities: Daily lectures, case-based learning, and surgical video content were supplemented with guided reading questions consisting of otolaryngology essentials. Reserved time offered the option to virtually attend the University of Kentucky’s daily otolaryngology lectures or another consortium.7, 8, 9 Additionally, students attended Grand Rounds and a virtual Q&A session with residents and faculty designed to answer student questions regarding ENT residency.

Student Evaluation: Students completed a pre- and post-course quiz consisting of 12 MCQs. Students also completed ungraded daily quizzes of five MCQs each to assess knowledge gained from the required readings. In addition, each student was required to deliver a 5-min presentation on a topic of their choice.

Plastic Surgery

Content: The basics of both reconstructive and cosmetic surgery were included in the course. Topics included the pathophysiology of wound healing, introduction to flaps and grafts, lower extremity and breast reconstruction, rhinoplasty and facial rejuvenation, hand surgery, microsurgery, and craniofacial surgery.

Teaching modalities: Content was conveyed through a combination of assigned readings, “case-of-the-day” presentations, surgical case videos, practice MCQs, and Q&A sessions with residents and faculty about the residency application process and life as a plastic surgeon. Surgical skills labs included lessons on interrupted and subcuticular suturing and one- and two-handed knot tying. In addition, students attended Grand Rounds and the American Society of Plastic Surgery National Conferences.

Student Evaluation: Students completed a pre- and post-test quiz consisting of 16 MCQs. Student assessment also included level of participation, end-of-course presentations, and improvement in technical skills.

Urology

Content: Content was derived from a curriculum published by the American Urological Association highlighting essential topics for all medical students to learn before they graduate. Additional sources included Campbell-Walsh-Wein Urology, the AUA Core Curriculum for residents, and Pocket Guide to Urology.11, 12, 13 Topics included urinary drainage, urinary tract infection, kidney stones, urological emergencies and imaging, incontinence, benign prostatic hyperplasia, cancer, hematuria, and men’s health.

Teaching modalities: Lectures and case-based discussions were led by senior medical students, residents, and faculty. Additionally, faculty presented videos of urologic procedures.

Evaluation: Students completed a pre- and post-course quiz consisting of 18 MCQs. Assessment also included level of participation and 5-min presentations on a topic of choice.

Vascular Surgery

Content: Following review of the American College of Surgeons Graduate Medical Education recommendations and USMLE content outlines, and consultation with senior vascular surgery faculty, the following topics were included in the course: aortic disease, peripheral arterial disease, carotid disease, venous disease, and vascular trauma. ,

Teaching modalities: Daily lectures and assigned surgical video content were supplemented with reading assignments consisting of landmark vascular surgery trials.16, 17, 18, 19, 20 Daily quizzes based on assigned reading were administered and a vascular anastomosis suturing lab was held mid-week (Fig. 1).

Evaluation: Students completed a pre-course and post-course quiz consisting of 25 MCQs. In addition, students were required to prepare an oral presentation based on an assigned vascular surgery patient vignette.

Course evaluation

On the final day of each course, students were invited to complete a survey delivered via Google Forms®. Survey response was entirely voluntary and refusal to participate did not affect the students’ course grade. Students were invited to complete the survey at the end of each course they completed, even if they had already previously filled out the survey for a different course, and students were not permitted to repeat courses. The survey collected baseline demographic information, including year in medical school, comfort using Zoom® videoconferencing software, and expected residency specialty, if known. Students were asked to rate how well the course met its learning objectives on a four-point Likert scale: “poorly,” “satisfactorily,” “well,” and “very well.” Students were asked to rate their baseline level of interest in the course subject and how it changed after completion of the course using a four-point Likert scale: “decreased interest,” “no change,” “increased interest,” and “greatly increased interest.” Students also reported their understanding of the surgical specialty as compared subjectively to peers at their level of training and whether the course improved their understanding. A four-point Likert scale was used to collect this data (“poor,” “fair,” “good,” and “very good”) and a numerical value was assigned to each rating (1, 2, 3, and 4, respectively). For each course, the mean scores for baseline and post-course understanding were calculated and compared using Student’s t-test for statistical significance. Quantitative data from the various pre- and post-course quizzes was not analyzed for significance because of the heterogeneity of the methods used. Statistical analysis was completed using JMP Pro (version 15.1, ©SAS Institute Inc., Cary, NC). The study was submitted to the Emory University Institutional Review Board and met the criteria for exemption.

Results

Between April 20 and May 15, our institution conducted 18 VSEs, including at least two iterations of each surgical specialty. The post-course survey was sent 83 times and 67 responses were returned, resulting in a survey response rate of 80.7%. Sixty-four (95.5%) respondents were in their third year of medical school, while three (4.5%) were in their fourth. Notably, due to the adjusted academic calendar utilized by our institution that begins the third year of medical school in March, none of the junior students had completed any clerkships prior to completing one or more virtual electives and could thus be expected to have a similar baseline level of clinical knowledge. Concerning the level of comfort with Zoom® videoconference software, 45 (67.2%) reported feeling “very comfortable,” 17 (25.4%) reported feeling “comfortable,” and four (6.0%) reported feeling “somewhat comfortable.” Thirty-five (52.2%) respondents reported “undecided” when asked about their expected residency specialty. Of the remaining 32 respondents, 19 (59.4%) completed courses outside of their expected residency specialty. Sixty-six (98.5%) respondents reported that the course objectives were met “well” or “very well.”

Survey respondents’ change in interest in pursuing the completed course specialty for residency is shown in Fig. 2 . A majority of respondents indicated “increased” or “greatly increased” interest in all but two specialties. Of the 67 survey respondents, only four (6.0%) reported decreased interest in the specialty after completion, and 17 (25.4%) reported no change in interest. Survey respondents’ baseline and post-course understanding of each specialty is shown in Table 2 . Understanding of each specialty increased by a statistically significant amount. When data was combined for all respondents, students’ subjective understanding increased from a mean of 2.0 (corresponding to “fair” understanding) to a mean of 3.3 (corresponding to understanding between “good” and “very good”), with a p-value of <0.0001.

Fig. 2

Survey respondents’ change in interest in surgical specialties after course completion.

Table 2

Participants were asked to rate their pre- and post-course understanding of the surgical subspecialty course they had just completed on a four-point likert scale (“poor,” “fair,” “good,” and “very good”) which was then extrapolated out to a four-point number scale (1, 2, 3, and 4, respectively). Scores were averaged and are reported as mean (standard deviation, SD). On average, all courses accomplished a statistically significant increase in self-reported student understanding of the according surgical subspecialty. CT: cardiothoracic.

Mean (SD)	All Specialties	General Surgery	CT Surgery		Neurosurgery	Otolaryngology	Orthopedic Surgery	Plastic Surgery	Urology	Vascular Surgery
	(n = 67)	(n = 11)	(n = 6)	(n = 4)		(n = 12)	(n = 11)	(n = 8)	(n = 9)	(n = 5)
Baseline understanding	2.0 (0.8)	1.9 (0.8)	2.3 (1.0)	1.8 (0.5)		2.3 (0.6)	2.0 (0.8)	2.4 (0.7)	2.1 (0.6)	1.0 (0.0)
Post-course understanding	3.3 (0.5)	3.1 (0.3)	3.5 (0.5)	2.8 (0.5)		3.6 (0.5)	3.1 (0.5)	3.6 (0.5)	3.4 (0.5)	3.3 (0.5)
P-value	<0.0001	0.0004	0.021	0.015		<0.0001	0.0006	0.001	<0.0001	<0.0001

All six courses that chose to assess their students utilizing pre- and post-course quizzes found improvements in student performance (Table 3 ).

Table 3

Summary of pre- and post-course quiz percentage scores with corresponding percent change. All courses that elected to evaluate their students using a pre- and post-course quiz observed an increase in scores. The General Surgery and Neurosurgery courses did not administer a pre- and post-course quiz. Data not analyzed for significance due to heterogeneity of methods. CT = cardiothoracic.

Course Specialty	Pre-course (%)	Post-course (%)	% Change
CT Surgery (n = 10)	62	80	+18
Otolaryngology (n = 5)	58	87	+29
Orthopedic Surgery (n = 11)	52	82	+30
Urology (n = 9)	50	89	+39
Plastic Surgery (n = 5)	43	64	+21
Vascular Surgery (n = 7)	46	66	+20

Discussion

The COVID-19 pandemic has forced physicians to rapidly adapt and integrate new technologies into their daily practices. Medical student education has not been exempt from this challenge, as medical schools have been tasked with swiftly transitioning curricula to a virtual format to maintain graduation timelines of students across the country. Drawing on their experience incorporating simulation into residency education over the last several decades, surgical educators have been at the forefront of these changes; however, few studies have so far been published on their reception by students.21, 22, 23

This study demonstrates that some surgical education for medical students is feasible in a virtual format. Almost all participating students reported comfort using videoconference technologies including Zoom®; these technologies may be welcomed or even preferred amongst newer generations of medical students in the coming years. Videoconferencing technology allowed for intimate interactions between students, residents, and faculty, with many of the courses including Q&A sessions on the residency application process and work-life balance as a surgeon. Students rated each course highly on its ability to meet its learning objectives. Subjectively, self-reported understanding of the course material increased significantly across all subspecialties by the end of each course, and objective data from the pre- and post-course quizzes trends toward that conclusion.

This study highlights the enhanced exposure to surgical subspecialties that students can obtain through virtual learning. Exposure to the variety of surgical subspecialties is not possible in the traditional clerkship environment where the number of students on each service is limited. In some institutions, training in smaller subspecialties is not offered at all and students must pursue externships in order to determine whether that subspecialty is the right fit for them. In our study, however, participants were able to engage in intensive exposure in up to four subspecialties over the course of the month. Many students chose to enroll in courses outside of their subspecialty of interest, and students in all VSEs on average indicated an increased interest in their course specialty.

This study also showcased the ability of senior medical students to contribute to the education of their junior counterparts. Senior students were an integral aspect of each course, including development and administration of the course, lecture preparation and delivery, and participant feedback and assessment. As frequent users of videoconferencing technology, senior students were able to troubleshoot technological errors and ensure that faculty could focus on the content during their sessions. They recorded each session and created a library of material that is now available to all medical students at our institution at any time. As an adjunct to their efforts, senior students also enrolled in a new virtual Surgical Education elective led by the Director of Medical Student Surgical Education that explored the theory and methods underlying their work.

This study had several limitations. These courses consisted primarily of small numbers of students, typically no more than 8–10 at a time. While small classes facilitated student-lecturer interaction, the sample size limits the generalizability of our results. Sixteen students (19.3%) elected not to respond to the post-course survey, potentially introducing bias into the results. The curricula for these electives were developed in less than four weeks in order to prevent gaps in student education caused by the sudden cancelation of clinical activities. This left little time to conduct needs assessments of the participants: curriculum developers relied instead on published curricula and topics considered essential by national associations and the supervising faculty. It also left little time to develop validated instructional methods and assessments or to ensure that each VSE was uniformly evaluating student progress. Finally, the efficacy of virtual learning from the perspective of the instructors was not assessed and could provide important feedback for development of future courses.

Notably, we did not attempt to compare the students’ experience in the VSEs to the in-person clerkship. The junior students, 95% of our participants, had not completed any clinical clerkships so we could not ask them for a direct qualitative comparison in the end-of-course survey. In addition, the small sample sizes and heterogeneity of the pre- and post-course quizzes precluded meaningful comparison with test results from past cohorts of students in the clerkship. However, the VSEs also had distinct characteristics from the clerkship that complicate direct comparison. The courses were much briefer than the clerkship, but because elective surgeries had largely been postponed, faculty had more time than usual to dedicate solely to medical student education. Lectures were more frequent and predictable than is possible during a clinical rotation, when patient care duties often interrupt formal didactic activities. Regardless, future iterations of these courses should standardize student assessment and feedback to ensure students receive a similar quality of education to the surgical clerkship.

The longevity of the COVID-19 pandemic through the 2020–2021 academic calendar year increases the likelihood that virtual surgical education for medical students will continue to be utilized. In these pilot efforts, we learned much from the constant verbal and written feedback from our consortium members and our learners, such as curating pre-session reading to the level of the learner and avoiding “Zoom fatigue” by careful scheduling. For those attempting virtual education implementation, curricular content creation was facilitated by utilizing specialty-specific, already published guidelines for medical student education. Finally, creation of a consortium of all of the specialties involved in these electives, with weekly videoconference meetings to share ideas, contributed to the overall sense of shared mission and enthusiasm. Having this consortium in place will ease future efforts to incorporate virtual learning into the surgical clerkship. Improvements include the addition of telemedicine in both the clinic and operative setting, where feasible; virtual patient interviews and virtual standardized patient exams; accessing and building upon the library of virtual content created during this study; and individualizing curricula for senior students who have already completed their third-year clerkship.24, 25, 26 Enrollment in these courses could also be opened to senior students wishing to complete an audition rotation at our institution.

Conclusion

Virtual surgical electives can enhance medical students’ exposure to surgical specialties and increase their interest and understanding of these fields. While virtual teaching modalities cannot replace the experience of learning in the clinical setting, further research should be conducted to examine their potential as a permanent adjunct to medical students’ education in surgery.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of competing interest

The authors attest that they have no financial or personal relationships with other people or organizations that could inappropriately influence the work.