Development and emergency implementation of an online surgical education curriculum for a General Surgery program during a global pandemic: The University of Southern California experience.

BACKGROUND: Physical distancing required by coronavirus disease 2019 (COVID-19) has limited traditional in-person resident education. We present our novel online curriculum for incorporation into traditional surgical educational programs.
METHODS: The online curriculum utilized weekly sub-specialty themed faculty and resident created lectures, ABSITE practice questions, and weekly sub-specialty synchronized readings. Attendance, resident and faculty surveys, and completed ABSITE practice questions evaluated for curriculum success. Curriculum was adapted as COVID-19 clinical restructuring ended.
RESULTS: 77% and 80% of clinical residents attended faculty lectures and resident led topic discussions as compared to 66% and 48% attending traditional in-person grand rounds and SCORE curriculum (both p > 0.05). 71.9% of residents and 16.6% of faculty reported improved resident participation while none reported decreased levels of participation (p < 0.001). 87.1% of residents and 66.7% of faculty preferred the online curriculum (p = 0.374). Completed ABSITE practice questions per resident increased from 21 to 31 questions/week (p = 0.541).
CONCLUSION: Our online educational curriculum demonstrates success and can serve as a model for online restructuring of resident education. Published by Elsevier Inc.

Introduction

In response to the coronavirus disease 2019 (COVID-19) pandemic, surgical programs worldwide have restructured their clinical and educational activities to prepare for the anticipated surge of patients and have physically distanced their residents, thus decreasing the potential for cross-contamination. , To maintain continuity of education at our institution, we developed and implemented a multimodal curriculum over the course of 3 days in compliance with the Accreditation Council for Graduate Medical Education (ACGME) mandate on March 13th, 2020 (Fig. 1 ). This mandate emphasized the need for continuing educational activities while decreasing inter-trainee contact , The goal of our educational curriculum was to facilitate knowledge retention through the incorporation of a multimodal educational platform. Home readings coupled with online lectures were designed to facilitate knowledge acquisition while an online question bank and online question review sessions designed in a flipped classroom setting facilitated active knowledge recall and integration. This substituted for the traditional in-person didactic model. Here we describe the rapid restructuring of our educational curriculum, provide a roadmap for recreation at other institutions and discuss the incorporation of this curriculum into training programs post-pandemic.

Fig. 1

Resident education COVID-19 restructuring timeline.

Material and methods

Curriculum development and implementation

Thirty-five General Surgery residents from an ACGME Accredited program participated in the online curriculum which began March 23rd, 2020. We utilized our institutional Health Insurance Portability and Accountability Act (HIPAA) compliant version of Zoom as the foundation our interactive e-curriculum (Zoom Video Communications, Inc., San Jose, California). We chose this because online e-learning has the capacity to satisfy three major pillars of knowledge acquisition: interaction with content, interaction with instructors, and interaction among peers.

We felt it was critical to provide organized structure for daily activities. Our daily educational activities incorporated a surgical sub-specialty weekly theme which focused discussion at specific specialties (Fig. 2 ). Each day, a topic was selected, and two text chapters were assigned in preparation for the daily online activity (Fig. 3 ). Faculty lectures were scheduled during previously protected educational time facilitating the greatest number of resident attendance while the resident led lectures were scheduled for after work hours to facilitate resident attendance. Each day, residents participated in 1–2 online activities to allow for interaction. All sessions were developed for the curriculum with novel content. The content was created to stimulate both passive and active learning in a flipped classroom setting with educational activities every day. Educational activities were structured to stimulate not only superior knowledge acquisition, but also active recall and utilization of knowledge acquired. Specific educational modules were created to practice test taking strategies, conquer test anxiety, and practice critical thinking. These educational activities included:

Fig. 2

Weekly Curriculum Overview. A sub-specialty theme is assigned to each week. Each day covers topics within the week’s theme.

Fig. 3

Online Curriculum Details. Each day covers a topic with assigned readings, practice questions and online sessions.

Faculty lectures

Faculty preference guided selection of lecture topics with readings of textbook chapters and relevant clinical guidelines selected as preparation for the residents (Fig. 4 ). These topics, chosen by faculty, were scheduled three times per week. We found little difficulty filling the spots because of faculty interest in their chosen topic. Two faculty lectures were directed at core surgical content and the third lecture was directed at an increasingly complex topic. Our online platform allowed for faculty members to interact electronically by asking and fielding questions directly from the residents.

Fig. 4

Faculty lecture with online textbook readings.

Resident topic discussion

Two resident-led sessions were scheduled for the remaining days of the week. These sessions fostered resident educational independence through presentation development and delivery, and improvement of cognitive thought processes. Each surgical topic was split into components with an accompanying list of questions to be addressed by a single resident (Table 1 ).

Table 1

Topic discussion structure and questions.

∙ Surgical anatomy￮ What are the relevant organs involved in the disease process? ￮ What is the blood supply and innervation to those organs? ￮ What are the adjacent anatomical structures or fascial planes that will be encountered during surgery?

∙ Patient presentation￮ What is the prevalence and incidence of the disease? ￮ What are the common risk factors? ￮ What are the signs and symptoms of the disease process? ￮ What are the important laboratory and imaging findings to formally diagnose the disease process?

∙ Patient workup, pre-operative optimization, medical management￮ After a diagnosis is made what studies and/or interventions are needed prior to going to the operating room? ￮ What are the indications for intervening surgically? ￮ If the disease can be managed medically, what are the indications and what will signify failure of medical management?

∙ Surgical options￮ What operative options exist to manage this disease process? ￮ What are the considerations for selecting each option?

∙ Critical steps of commonly performed procedures￮ How are the commonly performed procedures accomplished? ￮ What critical anatomical structures are encountered during the operation?

∙ Post-operative complications￮ What are the most common complications following the operation?￮ What are the frequencies associated with these complications? ￮ What diagnostic workup is required for each diagnosis? ￮ What are the principles of management including need to return to the operating room?

Journal club

Weekly journal club sessions discussing two publications were included as resident-generated content. First, a landmark study within the weekly sub-specialty theme was included to understand the foundation for management decisions considered standard of care. The second study consisted of a contemporary publication examining an unsettled issue in the literature. The discussion focused on study design critiques and considerations for integration into current surgical practices. Each assigned article was presented by one senior resident (PGY4-5) and one junior resident (PGY1-3) and guidelines were provided with specific questions to be addressed (Table 2 ).

Table 2

Journal club structure and questions.

• Background (presented by senior resident)￮ What are the aspects of the disease process necessary to understand the investigative question? ￮ What are the relevant findings from previously published literature? ￮ Each background should conclude with: What is the main research question asked by the study?

• Methods (presented by junior resident)￮ What was the type of study performed? ￮ What were the inclusion/exclusion criteria? ￮ What were the primary and secondary outcomes?￮ How were they measured? ￮ What was the statistical analysis plan?

• Results (presented by junior resident)￮ What were the major findings of the study? ￮ What did the statistical analysis say?

• Discussion (presented by senior resident)￮ What were the most significant findings of the study in relation to changes in patient care? ￮ What were the limitations of the study? ￮ For the landmark paper: How did this study inform future decision making including current practices? Are there current treatment guidelines incorporating these findings? ￮ For the contemporary paper: How does this study add to current knowledge of the subject? How should we incorporate this new information and should it change our practices? Would you incorporate these findings into your practice?

Question review session

Based on the previous week’s sub-specialty theme senior residents led moderated review sessions of ABSITE exam type questions. Participants were encouraged to read through and answer the test questions vocalizing their critical thinking process. This exercise in articulation of critical thinking is directly applicable to oral board scenarios. Test taking strategies were discussed in the context of answer elimination, question stem analysis and logical guessing with higher-level reasoning questions stemming from the original clinical scenario available for more senior residents.

All sessions were recorded and accompanying presentation materials were uploaded to our institution’s cloud-based HIPAA file sharing educational repository. The educational repository was organized by the week’s sub-specialty theme providing an opportunity for those unable to attend the live online session to watch the content later providing the ability for residents on clinical duties to learn on their own schedule.

Active recall and integration of previous sessions and readings was accomplished with the aid of our institutionally subscribed ABSITE preparation question bank, TrueLearn (TrueLearn LLC ©). This system allowed for the creation of tests based on the weekly sub-specialty theme. Daily assignments were created on each resident’s account related to the daily topic. This amounted to 10 questions per day during weekdays and 30 questions on the weekend totaling 80 questions per week.

Evaluation

An IRB approved electronically administered survey focusing on resident and faculty opinion regarding the online educational curriculum was administered as the temporary restructuring of our clinical rotations ended and our clinical residents returned to their original rotations. Surveys were reviewed by two senior residents and one senior faculty member to decrease bias and capture the full scope of curriculum experience. The surveys included questions assessing user satisfaction, resident engagement, and ultimate preference between the novel online platform and the traditional in-person didactic sessions. The survey was distributed amongst 34 clinical residents, excluding the resident who created the survey, and 15 faculty who had given presentations in this time.

The resident survey contained questions aimed at identifying the location and environment where the online platform where utilized. The change in each surveyed residents hours dedicated to education before and after the transition to the online platform was delineated to observe the effect of structuring the education program. Strengths and weaknesses of the online educational curriculum and the traditional in-person didactic educational curriculum were directly compared. A copy of the resident survey can be found in Fig. 5 .

Fig. 5

Resident survey.

The faculty survey was similarly structured and contained questions aimed at identifying the location and environment the online platform was utilized from. It also contained questions aimed at assessing the time commitment needed to prepare for the online presentation. Audience engagement and strengths and weaknesses of the online educational platform were assessed with the survey and the final questions were aimed at directly comparing the traditional in-person didactic sessions to the online education platform. A copy of the faculty survey can be found in Fig. 6 .

Fig. 6

Presenting faculty survey.

Attendance levels between the 11-weeks of the online platform and the pre-COVID in-person didactic sessions were compared. Traditional in-person didactic session attendance for SCORE curriculum, morbidity and mortality, and grand rounds were averaged over the months of November 2019 to January 2020 from attendance logs kept by our administrative staff. The resident attendance level for the online educational curriculum also derived from attendance logs kept by our administrative staff who acted as the host for the online educational sessions. Online educational session attendance was subdivided into faculty lectures, resident led topic discussions, resident led journal clubs, and resident led online question sessions. The percent of residents attending the resident led online question sessions were calculated from the total number of clinical PGY1-3 as the denominator as these sessions were created for PGY1-3 attendance (21 clinical residents). Pre-COVID in-person SCORE curriculum attendance, mandatory for PGY1-3 (21 clinical residents), was directly compared to the online resident driven topic discussions, resident driven journal clubs, and question sessions. The traditional in-person grand rounds attendance was compared to online faculty lectures. The average weekly true learn questions completed during the online educational curriculum was compared to the average weekly true learn questions completed during the 2019–2020 ABSITE study year (January 2019–January 2020). This average was derived by review of the administrative TrueLearn account for our institution.

Results

Attendance

When evaluated, faculty lectures were attended by 77% (n = 27) of clinical residents as compared to pre-COVID in-person grand rounds which averaged an attendance of 66% (n = 23) (p = 0.289). Resident driven topic discussions, journal clubs, and question sessions were attended by 80% (n = 28), 54% (n = 19), and 62% (n = 13) of clinical residents, respectively. This compared favorably to the average attendance of 48% (n = 10) for the pre-COVID in-person SCORE educational sessions. Attendance at resident driven topic discussions was statistically higher than in-person SCORE educational sessions (p = 0.012) while journal clubs, and question sessions were not (p = 0.628 and p = 0.352, respectively). Our faculty lecture recorded lectures have also been re-watched on average an additional 24 times which contributes more to their viewership.

TrueLearn questions

With the assignment of daily TrueLearn questions our clinical residents completed on average 39% of their assigned questions per week, 31 out of 80 questions/week (SD 5.43). This was compared to the TrueLearn question completion rate of 21 questions/week (SD 37.19) over the 2019–2020 ABSITE study year (January 2019–January 2020) (p = 0.541).

Resident survey

The resident survey was completed by 94% (n = 32) of clinical residents. The majority of the residents reported an increase in the weekly time devoted to their education, from <2 h/week (37.5% of respondents, n = 12) before the implementation of the online educational platform to 6–8 h/week (40.0% of respondents, n = 13) after its implementation. 56.2% of clinical residents (n = 15) reported completing around 40–60% of their assigned readings per week while only 12.5% (n = 4) reported failure to complete any assigned readings. When asked why they chose to participate 90.6% (n = 29) cited their desire to support their colleagues while presenting and 96.1% (n = 30) cited its important educational benefit. 71.9% (n = 23) stated their level of participation in the sessions was improved from pre-COVID in-person didactic sessions. Of the 23 clinical residents who reported more participation 52.2% (n = 12) cited increased comfort with the informal educational environment and greater comfort asking questions through the online platform’s chat features as opposed to the in-person sessions.

When asked as to the weaknesses of the online educational platform, 66.7% (n = 14) stated that too much content was assigned per week. 85.7% (n = 12) of these respondents indicated that the readings were over assigned. When asked to state the weaknesses of the pre-COVID in-person didactic educational curriculum 75.0% (n = 24) responded that they were too busy with clinical duties to attend the in-person sessions and 56.3% (n = 18) stated that the in-person didactic educational curriculum usually fell on their day off or their post-call day causing them to miss because of duty hour restrictions. When asked to score the pre-COVID in-person didactic educational curriculum and the online educational platform on a scale of 1–10 (10 being the best), the online educational platform had higher scores with an average score of 7.7 (SD 1.94) as opposed to the in-person didactic educational curriculum which had a score of 4.8 (SD 2.38) (p < 0.001). Finally, 87.1% (n = 27) of residents expressed a preference for the online educational platform over the traditional in-person didactic educational model.

Presenting faculty survey

The presenting faculty survey was completed by 80% (n = 12) of faculty who participated in the online educational platform. 66.7% (n = 8) of the presenting faculty reported that they spent under 2 h preparing for their online educational session while only 16.7% (n = 2) reported it took them over 4 h to prepare. 83.3% (n = 10) of presenting faculty reported that they perceived resident participation to be the same as the pre-COVID in-person didactic educational sessions. None reported a decreased level of resident participation and 16.6% (n = 2) reported increased resident participation. Faculty reporting increased resident participation was statistically lower from the 71.9% (n = 23) of residents who self-reported increased participation (p < 0.001). 50% (n = 6) reported giving their presentation from their home and 50% (n = 6) reported that the online format was more convenient to give their presentation opposed to the traditional in-personal didactic educational sessions. Free text answers regarding the reason for the increased convenience included an ability to present from their own home/office and more resident participation. None of the presenting faculty reported that the online format was less convenient and 75% (n = 9) of presenting faculty rated their willingness to present on the online format as a 10 out of 10. 41.7% (n = 5) reported that they would be likely to present using the online educational platform at another institution. 91.7% (n = 11) of presenting faculty reported the online platform was easy to use.

When asked the strengths of the online educational platform 75.0% (n = 9) of faculty reported that the online platform provided educational benefit for the residents as opposed to only 25% (n = 3) when asked the same question about the pre-COVID in-person didactic model (p = 0.014). 83.3% (n = 10) of presenting faculty reported that it was easier to engage the audience in the pre-COVID traditional in-person setting. 66.7% (n = 6) reported a weakness of the online educational platform was an inability to tell when the residents had questions or assess audience engagement during their lecture. When asked about the weaknesses of the in-person didactic educational format 30.0% (n = 3) reported that it took too much time to travel to the teaching location and 30% reported resident attendance was less with the in-person format. Finally, 66.7% (n = 8) reported that they preferred teaching with the online educational platform over the traditional in-person lecture format. This statistically mirrored the resident preference for the online educational platform (p = 0.374).

Discussion

The initial reception of the multimodal online curriculum has been overwhelmingly positive from both faculty and residents with the majority of respondents voicing their preference for the online educational model. Creation of our online e-curriculum provided several unexpected benefits. Perhaps the greatest being the enthusiasm residents expressed being involved in their own education. Residents on clinical duty watched and even gave live presentations from call rooms and work areas speaking to the versatility of the online platform. Our resident trainees generated new educational content, expanding their synthesis and assimilation of relevant information. Resident presentations rapidly improved as residents adapted to specified time limits and developed new ways to organize their thoughts.

Impressively, resident attendance for these resident driven presentations mirrored faculty lead sessions (80% vs. 77%, respectively). This was hypothesized to be due to an accountability felt to their fellow resident presenter with over 90% of reporting a strong desire to support their presenting colleagues. This not only increased participation but also had the benefit of fostering a sense of solidarity in a time of physical isolation. Interaction with peers is increasingly becoming a recognized protective factor in resident burnout. Last but certainly not least, was the ability to record our educational sessions for future viewing. Successful creation and utilization of this online repository of lectures was demonstrated by our faculty lectures having been re-watched on average an additional 24 times at the time of transition back to pre-COVID-19 clinical duties.

Arguments against incorporating an online educational platform stem from its inability to foster traditional interpersonal interactions. Body language expressed by the audience can be an important tool for the presenter to assess audience engagement and adapt the pace and audience interaction of the lecture in real time. An interesting observation that was made between our resident and faculty surveys was the discordance between self-reported resident participation and presenting faculty’s perception of resident participation. While 71.9% of residents reported that their level of participation was increased only 16.6% of presenting faculty reported the same increase in resident participation. While it is important to note that none of the residents or faculty surveyed reported less participation, the discordance in opinion can be explained by two factors; a lack resident comfort asking questions and presenting viewpoints on the online platform, and the inability of the presenting faculty to assess audience engagement over the online platform. As our residents and faculty’s experience with Zoom increases, their ability to utilize this technology to facilitate interaction will only grow. The presenter’s ability to assess audience engagement can be improved increased experience checking the chat feature of Zoom for questions posed by residents, utilizing the poll feature to gauge audience comprehension of a specific topic, and utilizing the multiscreen audience view to observe engagement. Tools for audience member to better express their comprehension and participate in the topic discussion include the annotate feature and the raise hand feature of Zoom.

Initially, approximately 20 h of structured education time per week was provided to our residents including assigned self-study readings and practice questions. With the return to a normal schedule and the resulting increase in operative volume, our online educational platform must adapt to remain a viable adjunct to our traditional educational model. Based on feedback provided by our surveys the online educational platform was modified to accommodate the change in resident and faculty workload. Our education model was scaled back from 5 days of online lectures, readings, and assigned questions to three days with one day incorporating faculty lecture, morbidity and mortality conference and grand rounds. The remaining days were composed of resident driven education sessions. These will alternate between resident driven journal clubs, topic discussions, and question review sessions. All sessions will be augmented with assigned TrueLearn questions and suggested readings. To date we have completed four weeks of our modified curriculum with initial success.

Conclusion

While it is difficult to see the silver lining of changes forced upon our healthcare system, our online surgical education curriculum serves as an example of how a crisis can bring about meaningful change. The documentation of our online surgical education curriculum can easily be broken down into its individual components for incorporation by other institutions looking to expand their educational program. Ultimately the success of our online educational curriculum will be measured by continued resident participation after the return to the pre-COVID-19 clinical duties. Continued review and adaptation of our educational curriculum and assessment of its success will be an area of future research at our institution.

Declaration of competing interest COI

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors and the listed authors have no conflicts of interest.