The Impact of COVID-19 on Surgical Education.

BACKGROUND: At the onset of social distancing, our general surgery residency transitioned its educational curriculum to an entirely virtual format with no gaps in conference offerings. The aim of this study is to examine the feasibility of our evolution to a virtual format and report program attitudes toward the changes.
METHODS: On March 15, 2020, due to the coronavirus disease (COVID-19) our institution restricted mass gatherings. We immediately transitioned all lectures to a virtual platform. The cancellation of elective surgeries in April 2020 then created the need for augmented resident education opportunities. We responded by creating additional lectures and implementing a daily conference itinerary. To evaluate the success of the changes and inform the development of future curriculum, we surveyed residents and faculty regarding the changes. Classes and faculty answers were compared for perception of value of the online format.
RESULTS: Pre-COVID-19, residency-wide educational offerings were concentrated to one half-day per week. Once restrictions were in place, our educational opportunities were expanded to a daily schedule and averaged 16.5 hours/week during April. Overall, 41/63 residents and 25/94 faculty completed the survey. The majority of residents reported an increased ability (56%) or similar ability (34.1%) to attend virtual conferences while 9.9% indicated a decrease. Faculty responses indicated similar effects (64% increased, 32% similar, 4% decreased). PGY-1 residents rated the changes negatively compared to other trainees and faculty. PGY-2 residents reported neutral views and all other trainees and faculty believed the changes positively affected educational value. Comments from PGY1 and 2 residents revealed they could not focus on virtual conferences as it was not "protected time" in a classroom and that they felt responsible for patient care during virtual lectures. A majority of both residents (61%) and faculty (84%) reported they would prefer to continue virtual conferences in the future.
CONCLUSIONS: The necessity for adapting our academic offerings during the COVID-19 era has afforded our program the opportunity to recognize the feasibility of virtual platforms and expand our educational offerings. The majority of participants report stable to improved attendance and educational value. Virtual lectures should still be considered protected time in order to maximize the experience for junior residents.

Introduction

First recognized in China in December 2019, the coronavirus disease (COVID-19) has since had a devastating impact on medicine and the medical community as a whole. Despite being a year out, numbers of new exposures remain high and the medical community is still grappling with no end in sight. To limit potential exposures, many institutions implemented virtual learning curricula for all levels of medical learners. This created a large demand for virtual platform media with immediate adoption without time to assess efficacy or satisfaction.

Despite the urgency of this evolution, virtual learning has been a well-studied concept with documented benefits including increased participant engagement, satisfaction and knowledge retention. , However, its full implementation into modern practice has been hindered prior to the COVID-19 era due to reasons such as a lack of personnel skilled in technology-oriented learning, limited financial or software resources, poor results or buy-in, and resistant attitudes to changing traditional education. O'Doherty et al. summarized that a virtual education platform must have the “Skills, Resources, Institutional Strategies, and Attitude” to be effective. With the rapid and unavoidable adoption of virtual curricula due to COVID-19, multiple specialties now recognize those limitations in real-time. Trainees in surgical specialties suffer the loss of intraoperative exposure, difficulty in learning anatomy remotely, and the halt of elective surgical cases. , Along with the swift restructuring of the surgical team and the strict limitations of the number of residents on service, these changes have led many to voice fears of inexperience.6, 7, 8, 9

Thus, the continued prevalence of disease and ongoing social distancing recommendations must now shift the focus from maintaining educational opportunities to enforcing educational standards and evaluating efficacy. In order to maintain quality and emotional wellbeing during COVID-19, some recommend that traditional didactics can be supplemented by increased laparoscopic case review and breadth of lecture content to include nontechnical teaching and wellness. , Early studies have shown that residents and faculty had an overall “positive” impression of the adapted virtual curricula and some noted benefits of increased accessibility and attendance. , 11, 12, 13 However, the detailed retrospective examination of resident and faculty satisfaction within the general surgery community is still lacking. Furthermore, to our knowledge the impressions by persons from different postgraduate years (PGY) and years in practice has been largely unstudied. The aim of this study is to describe the feasibility of our evolution to a virtual format and report both program attitudes as well as changes in curriculum offerings. We hypothesized that both residents and faculty would be supportive of the virtual format transition and that overall curriculum offerings would increase in number and spread of content and time of day from pre-COVID to SURGE and remains stable post-SURGE.

Methods

Pool of Residents

The University of Texas at Southwestern (UTSW) Medical Center IRB considered this study exempt and approved the waiver of consent. UTSW is one of the largest surgical programs in the country supporting approximately thirteen incoming categorical residents in each class. Residents range from Post Graduate-Year (PGY) 1-7, with some extending training years with time for research or additional degrees (e.g., M.B.A, M.P.H).

Clinical schedule and curriculum

Standard

Educational conferences for the majority of the 2019-2020 year were mandatory half-day protected sessions every Wednesday morning. These consisted of once monthly departmental Morbidity and Mortality (M&M) conferences followed by an hour Grand Rounds with a guest lecturer and case scenarios discussions to follow. The remaining three Wednesdays of the month consisted of a “Chief's Conference” lecture which is our format for an oral board preparation focused on upper levels with mock cases, followed by a didactic lecture and intermittently a professionalism lecture. Faculty would attend the monthly M&M and Grand Rounds and individual faculty would be asked by our resident Academic Chairs to facilitate Chiefs’ or didactic conferences.

Outside of the protected time, each department would individually host tumor boards, journal clubs, and weekly to monthly specialty department conferences scheduled at their discretion. These were required attendance for the residents on service at that time but not available to off-service residents. Residents rotate from service to service on a monthly basis between four sites: Parkland Memorial Hospital (PMH; public county hospital), Clements University Hospital (CUH; private hospital), Children's Hospital, and the Dallas Veterans Affairs (VA) Hospital. All are located within the Dallas Medical district except for the Dallas VA, which is off-site, and approximately 20-miles away.

COVID-19 and surge

With the onset of COVID-19 and social distancing recommendations from the Centers for Disease Control (CDC), many departments made individual decisions to cancel conferences and limit groups. However, on March 15, 2020 an institution wide policy became official and mass gatherings above twenty-five participants were restricted. Given our program size, this carried an obvious implication for educational conferences. With the joint efforts of the Academic Chairs and the surgical department, conferences were converted immediately to a virtual format-using Zoom©. Shortly thereafter, the cancellation of elective cases in April 2020 and the drive to limit resident exposure and time in the hospital led to the adaptation of a SURGE curriculum. This consisted of having only half the residents in the hospital at any given time, to maintain a back-up workforce in times of illness. This workforce rotated on a weekly basis. This schedule created the need for augmented resident education opportunities for those who were at home in order to maintain their involvement and learning. Weekly conferences were now expanded to a daily conference structure (Fig. 1 ). Wednesday morning structure remained the same, bolstered by individual department tumor boards, conferences and journal clubs opened to all residents through virtual platforms. Additionally, new lectures and operative video reviews were created based on resident requests. Residency coordinators were responsible for monitoring resident attendance at conferences to ensure adequate educational involvement however due to our size and connectivity issues with virtual platforms, we recognized discrepancies in the attendance data that prohibited accurate interpretation. With the restarting of elective cases, on May 25, 2020 the residents resumed the previous clinical schedule. However, the institutional policy for mass gatherings remained in place, requiring the institution to continue virtual learning.

Fig. 1

Example of a weekly curriculum offering schedule.

Study design

Sample

We administered a cross-sectional electronic climate survey to evaluate the opinions of the residents and faculty within the UTSW Department of Surgery. The general surgery residents and the general surgery faculty listservs were used to email links to the surveys. Of the surgery residents used for analysis the recently graduated chiefs and new interns who were not present for both the standard and new curriculum were excluded. Surveys were sent out early July and responses were collected for approximately a week.

Survey design

Two self-administered surveys were created on Survey Monkey. Both consisted of five questions and a free-text comment section, differing only in the first question, which asked PGY year (1-7) for residents or number years from training (<5 years, 5-10 years, 10-15 years, and >15 years) for faculty. The next three questions asked (1) how likely one is to attend conferences when virtual rather than in-person (2) how the educational value of the conferences was affected by transitioning to a virtual platform and (3) how the change to a virtual platform affected conference attendance? The questions were to be responded to in relation to the new virtual curriculum only. Answers were graded in a five-point Likert scale from 1 (very negatively) to 5 (very positively) and a three-point Likert scale from 1 (decreased) to 3 (increased) respectively. The final question asked if the person would like to continue virtual conferences in the future (Yes or No; See Appendix). Responses were anonymous and required the entire survey to be filled out (except free-text answers) prior to submission, thereby limiting missing data.

Curriculum analysis

Curriculum offering data were aggregated between January 8, 2020 and June 30, 2020. This was organized into contiguous 5-week phases based on the resident SURGE scheduling. The pre-COVID curriculum phase was defined as between February 5, 2020 and March 11, 2020, the SURGE curriculum phase was defined as between March 18, 2020 to April 22, 2020 and the post-SURGE phase was defined as April 29, 2020 to June 3, 2020. The lectures were then further divided into categories by time of day (7 AM-10 AM, 10 AM-2 PM, and 2 PM-10 PM slots), type of lecture content (clinical, non-clinical, technical, and research), invited audience (full surgery department, specialty department, or multidisciplinary) and finally if resident level specific (senior residents PGY3-5 or junior residents PGY1-2). Chi-squared tests were performed and statistical significance deemed when P value < 0.05.

Statistical and free-text comment analysis

Descriptive statistics were performed on resident and faculty survey results as a whole. Non-parametric statistical analyses were performed for residents and faculty as a whole using Kruskal-Wallis tests through R-studio. When the Kruskal-Wallis test indicated statistical significance, pairwise within-group Mann-Whitney-U tests were performed in order to analyze differences amongst sub groups such as postgraduate year or years of practice experience. Statistical significance was deemed a P value < 0.05.

Free-text comments from the survey were analyzed with an inductive thematic approach. Two reviewers independently reviewed all comments from faculty and residents and derived representative themes. Related themes were then consolidated and organized into positive and constructive concepts. The reviewers then jointly reviewed the comments again in entirety with the thematic codes and assigned codes to all statements. Descriptive data regarding the variety of themes, frequency, and preferred format are reported.

Results

Residents responses

65.1% of residents responded to the survey (41/63), majority of responses from PGY3 and PGY4 classes. Regarding “how likely they were to attend virtual conference” the median response (median; IQR) was more likely (4; 3-4). Regarding “the effect of virtual platforms on conference educational value,” residents felt it remained the same (3; 2-4). And regarding “the effect of virtual platforms on conference attendance,” residents reported it remained the same (3; 2-3). Analysis of resident responses by PGY yielded statistically significant values for all questions: “how likely they were to attend virtual conference” (P value = 0.015), “the effect of virtual platforms on conference educational value” (P value = 0.004), “the effect of virtual platforms on conference attendance” (P value = 0.005; Table 1 ). Secondary analysis showed PGY1 residents most often had discordant responses showing statistical significance: “how likely they were to attend virtual conference” PGY-1 versusPGY-3 (P value = 0.003), “the effect of virtual platforms on conference educational value” (PGY-1 versus PGY-3 (P value = 0.001) and “the effect of virtual platforms on conference attendance” PGY-1 versusPGY-3 (P value = 0.001; Table 1).

Table 1

Granular resident survey response analysis.

	How likely to attend†	Conference value†	Actually attended*
Resident (n = 41)   PGY-1 (6)   PGY-2 (7)PGY-3 (12)PGY-4 (9)PGY-5 (3)PGY-6 (4)	2.5 (2-3)3 (3-4)4 (4-4.25)4 (4-5)5 (3.5-4)4 (3.5-5)	2 (2-2)3 (2-3.5)4 (3-4)3 (3-4)3 (3-3.5)4 (3.5-4)	1.5 (1-2)2 (2-2.5)3 (2.75-3)3 (2-3)3 (2.5-3)3 (2.5-3)

Results are reported in median and interquartile ranges.

P value < 0.05.

P value < 0.01.

Faculty responses

Of all Deparment of Surgery faculty, 26.6% responded to the survey (25/94), with a largely equal distribution of respondents from various categories of post training experience. Regarding “how likely they were to attend virtual conference” the median response (median; IQR) was more likely (4; 3-4). Regarding “the effect of virtual platforms on conference educational value”, faculty felt it remained the same (3; 3-4). And regarding “the effect of virtual platforms on conference attendance”, faculty reported it remained the same (3; 1-3; Table 2 ). When comparing different levels of post training experience, faculty responses were statistically significant only for “the effect of virtual platforms on conference attendance,” (P value of 0.015; Table 2). Secondary analysis showed the difference was between faculty with practice experience 5-10 years and practice experience 15+ years (Pvalue = 0.009; Table 2).

Table 2

Granular faculty survey response analysis.

	How likely to attend	Conference value	Actually attended*
Faculty (n = 25)Practice <5 years (7)Practice 5-10 years (7)Practice 10-15 years (4)Practice 15+ years (7)	4 (3.5-4.5)4 (3.5-4)4 (3.75-4)3 (3-3)	4 (3-4)4 (3-4)3 (2.75-3.25)3 (2.25-3)	2 (2-3)2 (2-3)3 (2.75-3)2 (2-3)

Results are reported in median and interquartile ranges.

P value < 0.05.

Comparison between residents and faculty

When comparing responses between residents and faculty as a whole, there were no statistical differences.

Free-text comments analysis

Optional comments were left by 76% (31/41) of the residents and 80% (20/25) of the faculty. Primary thematic analysis identified 18 total themes (7 positive, 11 constructive; Table 3 ). The most common positive themes were preference for a hybrid format (44%), convenience and increased attendance (18%), accessibility for off-site/research residents (16%), and flexibility in scheduling/number of lecture offerings (9%). Statements reflective of positive themes include: “it's the future. Embrace it,” “we need to keep streaming once we start meeting for many reasons,” “less time wasted in walking to the conference…makes morning rounds less hectic. Saves time.” and “research resident working away from Dallas. I love virtual format.” The most common constructive themes were pro-in-person format (22%), less interactive (13%) and anonymity/inability to gauge audience (13%), lack of “protected time” (11%), loss of camaraderie (11%) (Table 3). Statements reflective of constructive themes include: “I think I am more likely to "attend" conference virtually but be less present during them. When it's a big group of us in the room sharing one log-in through a single computer, there's much more multitasking going on,” “Virtual is hard for interactive sessions, to involve the audience effectively, and to gauge interest and understanding” and “one important element missing from these virtual conferences is the natural camaraderie that comes from a casual interaction…the conversation outside in the hallway is sometimes a refreshing reminder of the collegiality of our specialty.” Comments regarding faculty engagement were reported as both positive and constructive elements. Additionally, of the “lack of protected time” that was referenced by residents 60% was in junior residents, and 40% in senior residents. Overall, 82% of the free-text comments had positive or mixed comments, only 18% of comments were entirely constructive. Comments supporting a hybrid format were 2.5x more frequent than comments supporting in-person format only (Table 3).

Table 3

Thematic analysis of free-text comments.

Positive themes	Residents	Faculty	Total
Preference for a Hybrid Format	18	7	25
Convenience and Attendance Improvement	4	6	10
Accessible for Off-Site/Research Residents	8	1	9
Flexibility in Scheduling/Number of Offerings	3	2	5
Material Available for Later Reference	3	1	4
Improved Faculty Engagement	2	1	3
Decrease Time Wasted	2	1	3
Socialization During Social Distancing	1	0	1
Retained Educational Value	1	0	1
Constructive themes
Preference for In-Person Format	9	1	10
Less Interactive	3	3	6
Anonymity/Cannot Gauge Audience	1	5	6
Lack of “Protected Time”	5	0	5
Loss of Camaraderie	4	1	5
Less Faculty Engagement	4	0	4
Less Preparation	3	0	3
Technical Difficulties	2	1	3
Distractability/Inattention	1	0	1
Taken Less Seriously	0	1	1
Harder to Communicate/Ask Questions	1	0	1

Conference data

Each phase consisted of 35 days. Pre-COVID had an estimated 4.2 lectures/week, SURGE had a total of 22.2 lectures/week and post-SURGE had a total of 12.2 lectures/week. There was 5.3x growth in curriculum offerings per week seen from pre-COVID to SURGE time that remained elevated at 2.9x in the post-SURGE phase. After the initiation of virtual curricula, we see a wider spread in the time of day of curriculum offerings that maintains post-SURGE although this did not reach statistical significance (Table 4 ). There was, however, a significant increase in the variety of lectures and the type of invited audience (P value < 0.01). Clinical lectures increased by a factor of 7.4x and 4.6x from pre-COVID to SURGE and post-SURGE. Nonclinical lectures, technical lectures, and research lectures also increased significantly during SURGE and then returned to baseline levels post-SURGE (Table 5 ). Regarding the invited audience, there was a 2.7x increase in curriculum offerings to the entire department with the introduction of invitations to previous or newly developed department-specific and multidisciplinary lectures (Table 6 ). Additionally, the development of resident level specific lectures led to a 5.1x increase in junior resident-specific lectures and 2.6x increase in senior resident-specific lectures from pre-COVID to SURGE although this too did not reach statistical significance (Table 7 ).

Table 4

Number of curriculum offerings by time of day.

Time	Pre-COVID	SURGE	Post-SURGE
7 AM-10 AM	16	48	32
10 AM-2 PM	5	47	23
2 PM-10 PM	0	16	6
X2 (4, N = 193) = 8.99, P= 0.06

Table 5

Number of curriculum offerings by type of lecture.

Topic	Pre-COVID	SURGE	Post-SURGE
Clinical	12	89	55
Nonclinical	3	8	4
Technical	6	8	0
Research	0	6	2
X2 (6, N = 193) = 22.42, P< 0.01

Table 6

Number of curriculum offerings by invited audience.

Audience	Pre-COVID	SURGE	Post-SURGE
Full Surgery	21	56	21
Departmental	0	18	12
Multidisciplinary	0	37	28
X2 (4, N = 193) = 26.98, P< 0.01

Table 7

Number of curriculum offerings by dedicated resident audience.

Resident audience	Pre-COVID	SURGE	Post-SURGE
Junior (PGY1-2)	16	82	50
Senior (PGY3-5)	5	13	7
X2 (2, N = 173) = 1.75, P= 0.42

Discussion

Our study supports previously recognized major barriers to adopting virtual education in medicine such as the lack of “skills, resources, institutional strategies, support and attitude.” However the lack of support and attitudes were not from the institution or faculty as previously reported in literature but rather junior residents, which was an unexpected finding. Specifically, PGY1 residents had significantly lower responses in all three survey questions. Interns more acutely felt that the virtual format would affect their attendance and the quality content of the lectures. Indeed they also most often reported a “lack of protected time” in the free-text comments. The majority of comments otherwise featured positive themes supporting a hybrid format, accessibility, and flexibility and constructive themes such as the lack of interaction and the inability to gauge audience engagement. We also found by evaluating our curriculum content that we were able to increase lecture offerings substantially in overall number, time of day, content, and focused audience with relative consistency post-SURGE. While some of these did not reach statistical significance the overall growth of the numbers is evident and impactful nonetheless. Thus we able to support our hypothesis that virtual curriculums are indeed feasible, maintaining overall attendance and conference value, while additionally providing increased learner opportunities in a sustainable manner.

Junior faculty were more supportive of virtual education as hypothesized but this did not reach statistical significance. Adaptation to new technology has been a limitation often cited in regards to faculty buy-in, however, we did not find this to be the case. In fact, both residents and faculty were largely supportive in continuing virtual education post SURGE.

Time, being an often-cited resource limitation, was actually not a major barrier in our population. Most felt that attendance was not affected or even positively affected due to more universal access to lectures. Residents or faculty that were prohibitively limited from conference pre-COVID due to factors such as remote working locations, being off-site for research years, or limited time in the mornings prior to clinical duties, were now able to join easily from their respective locations. Some however, commented that while attendance increased, true presence or engagement may have been negatively affected.

Curriculum offerings additionally improved during the SURGE schedule and continue to maintain numbers post-SURGE. Invitations to pre-existing departmental lectures such as tumor boards, research conferences or critical care lecture series now offer entirely new multidisciplinary education and exposure when off-service. Time constraints, which previously consolidated learning to a weekly Wednesday morning conference, were less restrictive, facilitating distributed learning with a wider spread of offerings throughout the day and week and the creation of new lectures. These included lectures focused primarily on junior resident or senior resident education, evening lectures dedicated to technical skill development via analysis of recorded operations, and a multitude of often under-prioritized non-technical skill lectures such as ergonomics and interview skills. Furthermore, the ability to easily record lectures for later review is a novel benefit that was often commented on as a major positive aspect of virtual lectures and a goal for continued learning.

A major finding of our study documenting a critical deficit in virtual curricula is not necessarily the lack of institutional buy-in but rather resident buy-in. Despite 33 education-hours dedicated to junior residents, PGY1 residents most negatively differed when responding to survey questions such as conference value and affected attendance. The most common reason reported was the impact of multiple distractions that combated the concept of “protected educational time.” While we do not have data to show how prevalent this problem was pre-COVID, we feel that responses and free text to the survey can act as a surrogate to document the decline in “protected educational time” due to the virtual platform. Examples of resident free text include, “I think I am more likely to attend conference virtually but be less present during them” and “virtual conference has increased attendance but has diminished active participation.”

While it is difficult to address the concerns of all parties in any type of curriculum development and to accurately compare the changes in perception, the fact that the virtual platform seems to further widen that gap in junior resident perceived involvement and education protection is a critical finding. Thus, while learner engagement will always be a forefront problem, it seems to be a more difficult and pervasive issue with virtual format teaching. Despite the development of a vaccine, there is no clear end to the need for social distancing and virtual lectures remain a large part of continued education. Thus the perceived lack of “protection” of junior residents must be further investigated and addressed so that it does not have lingering impacts on educational value.

Overall we do recognize certain limitations to our study. We did not have an accurate way of monitoring attendance which is a large component of understanding supply and demand of virtual lecture offerings. Given the size of our program and department, and the rapid switch, we were unable to create and maintain accurate early attendance information and thereby excluded the data due to numerous inconsistencies that made interpretation of actual attendance very difficult. Additionally, our faculty attendance is not tracked to such a granular extent. Furthermore, the survey used was not validated due to the rapid change and need for urgent assessment and had limited faculty response. Despite this, we feel that our data still provide valuable information regarding the need for continued protected resident education with a virtual curricula.

Conclusion

With the advent of COVID-19 and the unprecedented requirement for a virtual education platform, a seamless change to virtual weekly educational conferences was enacted. Analyzing lecture offerings data and survey results from residents and faculty, we were able to find that the virtual format is feasible, largely supported, and provides benefits such as more interdepartmental collaboration, multiple new lecture time and topic offerings, the ability to record for future review and increased accessibility with subsequent improved faculty involvement. However, we are also able to discern that not all lectures are conducive for virtual learning and that not all residents feel the value equally. Junior trainees remain a vulnerable population that requires focused educational protection when there is decreased oversight with a virtual platform.

Disclosure

None.