Efficacy of Virtual Case-Based General Surgery Clerkship Curriculum During COVID-19 Distancing.

OBJECTIVE: The COVID-19 pandemic created a paradigm shift in medical education with a reliance upon alternative teaching methods to deliver meaningful surgery clerkship content. This study examines the efficacy of a novel, case-based virtual surgery clerkship curriculum to determine its impact on student experience during quarantine. STUDY
DESIGN: Sixteen third-year medical students enrolled in the General Surgery clerkship between April through June 2020 during COVID-19 distancing at a quaternary medical center (Cleveland Clinic, Cleveland, OH) participated in this study. Course surveys, including a 10-question curriculum-based multiple-choice assessment, were administered before and after the clerkship. Analyses include student self-perception of readiness to see a surgical consult independently, students' interest in pursuing a General Surgery residency, and improvement of surgical knowledge. RESULTS AND
CONCLUSION: On a 5-point Likert scale, students felt significantly more assured in their ability to independently assess a surgical consult by the end of the course. Five (31%) students reported an influence of the curriculum on their personal interest in a career in General Surgery. Mean scores on the curriculum-based knowledge assessment increased. These findings highlight that a virtual platform can be a reliable alternative adjunct that delivers surgical content and positively impacts student experience. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s40670-020-01126-5. © International Association of Medical Science Educators 2020.

Introduction

In recent years, a significant effort has been pursued to optimize medical student learning with regard to structure, content, goals, and delivery mechanisms. [1, 2] While pre-clerkship curricula lay the groundworks for foundational scientific knowledge, clinical clerkships create opportunities for medical students to exercise the application of this foundation. The core clerkships are critical in the formative development of medical students from classroom exposure to clinical practice and its intersection with experiential learning. [3] Moreover, evidence suggests that interaction with current surgical residents in a clinical setting positively affects a student’s perception of a surgical career and thus increases their propensity to pursue a General Surgery residency. [4, 5] The opportunities that surgery clerkships offer are crucial for students to determine their postgraduate training and future career trajectories beyond medical school.

With the advent of the novel coronavirus/SARS-CoV-2 (COVID-19), resource allocation and containment efforts have drastically changed the landscape of clinical practice. In General Surgery, the repercussions of the response were significant, resulting in the canceling of non-emergent surgeries, limiting personal protective equipment for invasive procedures, and as with many other specialties, a restructuring of the practice paradigm. [6, 7] For the General Surgery clerkship, the pandemic was particularly impactful given that the key components of learning, clinical exposure, and student mentorship occur in an operating room or procedural setting that is not as easily replicated as in other elements of medical practice. [8, 9] This study examines the efficacy of a remote curriculum in lieu of an in-person General Surgery clerkship. Rapidly, a proprietary case-based virtual curriculum was developed with simulated consult and outpatient clinic patient encounters, integrated with didactics, discussions, pager simulations, and table rounds to approximate the daily learning environment of an in-house General Surgery clerkship. This survey study demonstrates as proof-of-concept that a virtual simulation of surgical patients and case-based learning is indeed effective in positively influencing student perceptions of General Surgery, engendering confidence in general clinical skills, and providing reinforcement of content retention and learning.

Methods

Study Setting

This study was completed using surveys capturing subjective sentiment regarding the General Surgery clerkship during implementation of COVID-19 social distancing policies prohibiting medical student clinical participation on the Cleveland Clinic campus during April and May 2020. A total of 16 third-year medical students enrolled in the General Surgery clerkship at Cleveland Clinic were directly impacted by social distancing regulations prohibiting in-person medical student clinical participation. Primary endpoint of this study was the student’s self-perception of readiness to see a surgical consult patient independently. Secondary endpoints included changes in students’ interest in pursuing General Surgery for residency and improvement of knowledge-based question scores. Participants were provided surveys prior to and after completion of a proprietarily developed case-based interactive virtual patient curriculum (Supplemental Materials 1 and 2). Surveys evaluated students’ concerns about, perceptions of, and interest in General Surgery as well as assessment question topics were selected based on diagnoses frequently encountered on the surgery clerkship by the third-year medical students at our institution. Questions were curated and reviewed by residents and faculty on the study team. Survey data was collected anonymously without any associated student-specific identifier. The virtual curriculum was implemented as a part of the surgery clerkship for the third-year medical students of Case Western Reserve University School of Medicine and the Cleveland Clinic Lerner College of Medicine. This study was approved by the Cleveland Clinic Foundation Institutional Review Board (IRB#20-503) and need for consent was waived (Fig. 1).

Fig. 1

Representative results from student participant responses to pre- and post-course surveys. a Subjective feeling of readiness to see surgical consults in the Emergency Department independently on a 5-point Likert scale. b Percentage correct on content-based knowledge assessment. c Post-course self-assessment of whether virtual clerkship curriculum has influenced participant’s interest in General Surgery. d Pre-versus post-course assessment of participant’s interest in pursuing General Surgery residency

Virtual Curriculum Development and Design

A virtual third-year medical student surgery clerkship curriculum was developed using the Kern six-step method. [10] Clerkship leadership was tasked to engage third-year medical students in virtual educational activities that would meet the learning objectives of the typical surgical clerkship rotation. General needs assessment was performed by reviewing available virtual curricula including WISE-MD modules, ACS/ASE Medical Student Core Curriculum, and shelf exam questions. [11, 12] In addition, clerkship director, subject matter experts, and general surgery residents were consulted on commonly encountered patient scenarios by third-year medical students in outpatient clinic, inpatient wards, operating room, and emergency room settings as well as learning objectives covering clinical skills obtained from managing postoperative patients in the inpatient ward, scrubbing into operative cases, presenting on patient rounds, and writing history and physical exam notes. Subsequently, targeted needs assessment using a questionnaire survey was distributed to the third-year medical students (Supplemental Material 1). Questionnaire evaluated students’ concerns about, perceptions of, and interest in General Surgery as well as knowledge-based questions centered around key teaching points from ten select topics. Assessment question topics were selected based on diagnoses frequently encountered during the typical third-year clerkship at our institution. Questions were curated and reviewed by residents and faculty on the study team. Survey data was collected anonymously without any associated student-specific identifier. This questionnaire was also designed to serve as pre-assessment before implementing the virtual curriculum. Based on the needs assessments, the overarching goal of the virtual curriculum was to utilize patient scenarios to teach third-year medical students to assess patients presenting with common surgical pathologies and peri-operative management of surgical patients. The following learning objectives were provided: (1) identify common surgical etiologies, diagnostics steps, surgical management of disease, and postoperative management; (2) understand key components of evaluating patients with surgical etiologies; (3) elicit pertinent information from simulated patient encounters through patient interview and physical exam; (4) collaborate with peers in team-based setting to develop diagnostic plans for each patient encounter; (5) formulate a treatment plan for each patient encounter based on sound clinical reasoning skills; and (6) demonstrate clear verbal and written abilities to effectively communicate key findings from each patient encounter.

A mix of educational methods was utilized for the virtual curriculum based on traditional lectured based learning method and problem-based learned method based constructivism, social, and humanism learning theories. [13] Students and residents convened in a virtual classroom using Zoom. At the beginning of each week, the students were provided with a patient scenario and resident leading the discussion role-played as the patient from whom the students obtained history gathering and requested focused physical exam findings. The facilitator role playing the patient would leave the virtual classroom; then, the students brainstormed as a team differential diagnoses, and “orders” as a part of their diagnostic workup. The “orders” for the diagnostic work up were emailed by a student representative to the session facilitator. The session facilitator emailed back the students with results of the labs and imaging ordered by the students and changes in clinical picture of the patient in the scenario. After the session adjourned, students then individually wrote a complete history and physical exam note based on the history, physical exam findings, labs, and imaging. The completed note was submitted to the residents for feedback. Prior to the subsequent session, students were “paged” via e-mail about patient updates and changes in clinical course. At the subsequent Zoom sessions, students and resident facilitators discussed the evolving treatment plans, brief didactics on relevant diagnoses, reviewed online videos of relevant operations and procedures, and practiced patient presentations in a virtual table rounding format. This process was repeated for each patient scenario. Evaluation of students and feedback from the students were obtained through the post-assessment survey.

Statistical Analysis

Primary and secondary endpoints were analyzed using either Mann-Whitney’s U or Student’s t tests as indicated. Statistical significance was defined by a two-tailed p value less than 0.05. Analysis was completed using Stata/SE (version 13.0, StataCorp, TX).

Results

Student Participation and Pre-course Perspectives

All 16 third-year medical students enrolled in the General Surgery virtual clerkship successfully completed the virtual curriculum. Participation for both pre- and post-course surveys was 100%. Prior to participating in our study, students had completed a variety of clerkships in other specialties including medicine, obstetrics-gynecology, neurology, psychiatry, pediatrics, and other surgical subspecialties. As the students were withdrawn from clinical responsibilities by the school in mid-March, 43.8% of the participants had participated in at least a partial virtual curriculum during the prior rotation. Based on the pre-surgery clerkship questionnaire, the majority of students expressed anxiety surrounding their readiness to begin their Surgery clerkship based on a 5-point Likert scale ranging from anxious/scared to confident with scores 1 to 5 respectively (median score 2, IQR 2–3; Table 1). When asked about their top concerns about starting their Surgery clerkship, most students reported apprehension about their lack of exposure to the operating room and the weakness of their surgical knowledge, followed by missed opportunities for rounding and patient care (Table 1). Three-fourths of participating students did not express any desire to pursue a career in General Surgery prior to starting the clerkship, followed by 19% and 6% expressing possible and definite postgraduate interest in General Surgery, respectively.

Table 1

Pre-course survey results

	Participants (n = 16)
Previous clerkships
Internal medicine	2 (12.5%)
Family medicine	16 (100%)
Obstetrics/Gynecology	16 (100%)
Psychiatry	16 (100%)
Surgical subspecialty	2 (12.5%)
Pediatrics	16 (100%)
Neurology	16 (100%)
Completed virtual curriculum in previous clerkship	7 (43.8%)
Greatest concern entering General Surgery clerkship (top 2)
Competency in operating room	9 (28.1%)
Surgery knowledge base	9 (28.1%)
Interaction with surgical attending and residents	3 (9.38%)
Rounding and patient care	7 (21.9%)
Surgical shelf exam	4 (12.5%)
Feeling of readiness for General Surgery clerkship	2 (2–3)a
Interest in pursuing General Surgery residency
Yes	1 (6.25%)
Maybe	3 (18.8%)
No	12 (75.0%)

Data shown reflects number of students with percentage of study population

aLikert scale of 1 to 5 reflecting scale of anxious (1) to confident (5), displaying median with interquartile range

Positive Reception of Virtual Patient Case-Based Curriculum

Participants were asked to complete a similar post-course survey at the conclusion of the 11 cases outlined in Table 2. Based on responses on a 5-point Likert scale where 1 to 5 represent not to definitely worthwhile respectively, students were overall very satisfied with the course and recommended continuation of the curriculum for future iterations of the General Surgery clerkship (median score 5, IQR 4–5; Table 3). When asked to evaluate how effective the course met its learning objectives outlined in the “Methods” section, students unanimously agreed that the course definitely met all the learning objectives (Table 3). Moreover, the course went above expectations in terms of simulating the key components of clinical work that are normally expected of third-year clerkship students, including patient interaction, development of treatment plans, inpatient floor work, exposure to the operating room, and regular patient-centric teaching by residents (Table 3). Similarly, participants further reported that the course was fairly effective in addressing their initial concerns about starting their General Surgery clerkship (Likert scale 1 or not effective to 5 or very effective; median score 4, IQR 4–5; Table 3).

Table 2

Virtual patient curriculum outline

Case	Topics covered in patient case format, with sub-topics
1	Acute cholecystitis Ultrasound imaging Introduction to the laparoscope
2	Lower gastrointestinal bleeding Colorectal cancer Postoperative pain and management
3	Pancreatitis Acute versus chronic pancreatitis “Step-up” management of necrotic pancreatitis
4	Small bowel obstruction Inguinal hernias – incarcerated versus strangulated Postoperative acute kidney injury
5	Breast mass Benign versus malignant breast pathology Breast exam
6	Acute appendicitis Preoperative orders and considerations Special populations – pregnant and pediatric patients
7	Peripheral arterial disease – threatened lower limb Endovascular versus open vascular surgeries Introduction to carotid disease
8	Painless jaundice – pancreatic mass Pancreatic cancer workup: endoscopy, imaging Nutrition: types and delivery of parenteral and enteral nutrition
9	Trauma – splenectomy “ABCDE’s” Primary versus secondary survey
10	Gastroesophageal reflux Barrett’s esophagus Bariatrics – Roux-en-Y
11	Trauma – pneumothorax Chest tube management and water seal chamber mechanics Interventions for persistent pneumothorax
12	Resident-student “hangout” session Choosing a specialty and selecting a residency program Mindfulness and wellness during residency and COVID-19

Table 3

Post-course survey results

	Participants (n = 16)
Overall recommendation of curriculum for future students	5 (4–5)
Course achieved learning objectives
Practicing clinical reasoning and patient interaction	5 (5–5)
Generating surgical assessment and plan	5 (5–5)
Understanding management of surgical diseases	5 (5–5)
Utilizing team-based learning	5 (4–5)
Course efficacy of simulating components of clinical work
Patient interaction	4 (4–5)
Generating treatment plan	5 (4–5)
Floor work	4 (3–5)
Operating room exposure	4 (3–4)
Resident didactics/teaching	4 (4–5)
Course efficacy of addressing initial concerns about starting clerkship	4 (4–5)
Interest in pursuing General Surgery residency post-course
Yes	2 (12.5%)a
Maybe	5 (31.3%)a
No	9 (56.3%)a
Course influence on personal interest in General Surgery post-course
Yes	5 (71.4%)a
No	2 (28.6%)a

Data shown reflects median with interquartile range of Likert scale 1 to 5

aMedian number of students with percentage of study population

Students Demonstrate Increased Clinical Confidence After Completing Virtual Curriculum

One of the key criteria used to determine student performance in the third-year clerkship at our institution is their ability to independently complete an initial assessment of a surgical consult patient under resident supervision (Fig. 1). Thus, the primary point of interest in this study was student self-perception and level of confidence to complete this task. Based on a 5-point Likert scale ranging from not prepared to independent scored 1 to 5 respectively, students felt significantly more comfortable and confident about completing an initial assessment of a surgical consult in the emergency room by the end of the course (pre-course median score 2, IQR 1–3; post-course median score 4, IQR 4–4, p = 0.0001; Table 4).

Table 4

Course endpoints of interest

	Pre-course (n = 16)	Post-course (n = 16)	p
Confidence in ability to independently see and complete initial assessment of surgical consult patient in emergency departmenta	2 (1–3)	4 (4–4)	0.0001
Content-based knowledge check questionsb
Overall	55 (3.54)	72.5 (2.14)	0.0002
Acute cholecystitis	93.75 (6.25)	100 (0)	0.3253
Lower gastrointestinal bleed	68.75 (12.0)	62.5 (12.5)	0.7205
Pancreatic mass	6.25 (6.25)	12.5 (8.54)	0.5592
Breast mass	31.25 (12.0)	12.5 (8.54)	0.2120
Small bowel obstruction	68.75 (12.0)	100 (0)	0.0140
Inguinal hernia	43.75 (12.8)	81.25 (10.1)	0.0285
GERD/bariatrics	43.75 (12.8)	87.5 (8.54)	0.0080
Ischemic limb	87.5 (8.54)	100 (0)	0.1536
Trauma	50 (12.9)	68.75 (12.0)	0.2953
Peri-operative management	56.25 (12.8)	100 (0)	0.0018

GERD, gastroesophageal reflux disease

aMedian with interquartile range of Likert scale 1 to 5; Mann Whitney U test

bPercentage scored correct across all students with standard error; Student’s t test

Post-course Survey Demonstrates Positive Impact of Virtual Curriculum on General Surgery Knowledge Retention and Career Interest

The study surveys also presented the same 10 content-based questions at the beginning and conclusion of the course to assess knowledge retention (Fig. 1). Overall mean scores significantly improved, suggesting the course was effective in delivering and teaching content (pre-course mean score 55%, SE 3.54; post-course mean score 72.5%, SE 2.14, p = 0.0002; Table 4). The post-course survey also demonstrated a positive shift in student interest in General Surgery residency: one additional student reporting “yes” (6.25 to 12.5%), two additional students reporting “maybe” (18.8 to 31.3%), and a corresponding decrease in students stating “no” (75.0 to 56.3%). Of the seven students who changed their perception of General Surgery residency, five students (71.4%) stated that their change in opinion was influenced by the virtual patient curriculum (Fig. 1).

Discussion

The COVID-19 pandemic has stressed the healthcare system in many ways, and medical student education has not been spared from these effects. The role of students during this time has been uncertain and has ranged from an expedited graduation and immediate placement onto the wards, to complete separation from the hospital with remote learning. [14-19] In either situation, medical student education during a global pandemic had to evolve rapidly and drastically, with little or no experience to guide medical schools and hospitals as to the best way of providing alternate forms of clinical education. Some programs have incorporated students into the system during the pandemic by having them remotely interview and screen patients in the growing number of virtual clinic visits. [20] However, evaluating acutely ill patients in the ED or trauma bay and assisting in the operating room have been the hallmark of the General Surgery clerkship. The concern surrounding the impact of social distancing on surgical clerkships is that above-mentioned experiences cannot be performed remotely nor be recreated virtually. For third-year medical students, the formative experiences can significantly influence medical students’ postgraduate career decision-making. [21] Although no remote learning environment can replace the trauma bay or operating room, we attempted to develop a remote clinical surgical curriculum that allowed students to care for simulated patients in a similar capacity that would be expected of them under normal conditions.

The results from the present study support that the remote surgical rotation implemented at our institution was a positive experience for medical students, accompanied by a significant improvement in both the knowledge of common surgical diseases and the confidence in evaluation patients with surgical diseases. Additionally, the study assessments demonstrate a significant increase in surgical knowledge retention as assessed by a content-based pre- and post-clerkship question set. The course also had a positive impact on student interest in General Surgery as reflected by the proportional change in number of students interested in a General Surgery residency. In addition to these evaluated metrics, there was a clear improvement in students’ ability to write adequate H&Ps and progress notes, and to present surgical patients as the course progressed. While operating room experience could not be recreated virtually for the students, the virtual curriculum focused on evaluating patients with potential surgical pathologies, clinical and residents played video recordings of pertinent surgeries, and explained key aspects of the procedure and relevant anatomy. The implemented virtual surgery curriculum achieved the patient scenarios covering common surgical etiologies and peri-operative management of surgical patients.

In addition to the positive impact on the medical students, there are also a number of benefits derived from having a virtual curriculum with only resident facilitators. Even more so than the attendings, the residents are perceived as teachers and advisors by most medical students, and the interactions with the residents have a significant impact on student career choice. [22] Residents routinely act as mentors for medical students who utilize their experiences on their clerkship rotations to make career choices. Therefore, these interactions and relationships that the students typically build with the residents during their clerkship are even more important during a time when students have limited exposure to the unique experiences on the surgical clerkship such as scrubbing in surgeries. [23] Subsequently, additional sessions were intentionally allocated for informal student-resident “hangouts” to discuss questions surrounding careers, lifestyles, and advice related to surgery and career in medicine in general. These dedicated sessions may have also played a significant role in increasing the student interest in pursuing a career in a surgical field.

The virtual curriculum also had the unexpected benefit of improving the teaching skills of both junior and senior residents engaged in the endeavor. This curriculum demonstrated the significance of and appreciation for the residents’ role in surgical education as the implementation of this virtual curriculum was only possible through participation of invested residents. When the virtual curriculum was implemented at our institution, a modified residency schedule was in effect with some residents working remotely from home due to decreased surgical volume. This allowed residents interested in teaching students to participate in the virtual curriculum. However, some residents also had direct patient care responsibilities. To encourage resident involvement, relieving residents from clinical duties or a modified schedule to allow residents to dedicate their time to teach the virtual curriculum needs to be considered. Buy-in from the clerkship director, the program director, and administrative chief residents who created the modified resident schedule was imperative and could be a limiting factor for implementation at other institutions.

There are some issues specific to a virtual curriculum for surgery clerkship that need to be addressed as well. Medical students that routinely “shine” on surgery rotations are often very self-motivated, active learners who score highly in traits such as grit. [24] These attributes are difficult to assess through a remote learning platform where opportunities to engage in self-directed learning then report back to group were limited. This may prevent medical students from challenging themselves and from getting a sense of how they would perform as a surgical resident. Similarly, clerkship directors may struggle to assess student performance outside of the metrics assessed in the course. These challenges may be addressed by giving students more autonomy in their virtual curriculum through self-directed learning such asking students to lead discussions on surgical topics or narrate a short surgical video for their classmates with residents present. Additionally, student assessments were performed using a proprietary question set derived from the study content rather than from a standardized assessment—the surgery shelf examination, which was not available at the time of the virtual curriculum given changes impacting shelf exam and clerkship rotation scheduling.

The most significant limitation is the lack of any operating room experience in the remote curriculum which is difficult to adequately simulate in a virtual setting. However, new technology is already helping to bridge this gap. [25] While students did receive suture material and knot-tying practice kits as a part of their overall surgical clerkship, recreating the operating room environment would require an enormous undertaking. For medical students, learning in the operating room occurs by observing the key steps of the surgery and the pertinent anatomy, and demonstrating level-appropriate technical skills through situated learning. [26] Therefore, outside the context of the operating room, observing and participating in team dynamics through scrubbing into an operation cannot able to be replicated virtually. [13] In addition, the situational awareness that students may develop through their experiences in the operating room falls under the “hidden agenda” or informal learning that only result from learning in the operating room. [27] Lastly, the General Surgery clerkship often serves as a gateway into a student’s exploration into the surgical disciplines. In light of this, expansion of the curriculum design into other surgical subspecialties may be an additional area of interest and curriculum development in the future.

Conclusion

Although current times are unprecedented, it is likely that medical school rotations will again be disrupted by unforeseen circumstances or needs for social distancing in the future. This possibility underscores the need for an effective, practicable remote surgical curriculum that can be readily implemented in response. While no virtual experience will ever fully replace the in-person clerkship experience, a modified surgery clerkship can be delivered through a virtual case-based curriculum to utilize patient scenarios to teach medical students to assess patients presenting with common surgical pathologies, to describe appropriate surgical treatments and peri-operative management, and to engage with surgical residents who can provide both feedback and career guidance—all without setting foot in the hospital.

(DOCX 20 kb)