Virtual Radiation Oncology Clerkship During the COVID-19 Pandemic and Beyond.

PURPOSE: We evaluated the impact of a virtual radiation oncology clerkship. METHODS AND MATERIALS: We developed a 2-week virtual radiation oncology clerkship that launched on April 27, 2020. Clerkship components included a virtual clinic with radiation oncology faculty and residents, didactic lectures, student talks, and supplemental sessions such as tumor boards and chart rounds. Medical students completed pre- and post-clerkship self-assessments. Faculty and resident participants also completed surveys on their experience with virtual lectures and clinics. Pre- and post-clerkship results were compared using a 2-sided paired t test. An analysis of variance model was used to analyze the clerkship components.
RESULTS: Twenty-six medical students, including 4 visiting students, enrolled over 2 clerkship periods (4 weeks). All students completed the pre- and post-clerkship self-assessments and agreed that the clerkship improved their understanding of radiation oncology. Compared with 3 (11.5%) students who agreed that they understood the daily responsibilities of a radiation oncologist before the clerkship, 22 (84.6%) students agreed and 3 (11.5%) strongly agreed that they understood the daily responsibilities of a radiation oncologist after the clerkship (P < .0001). Although 15 students (57.7%) reported an increased interest in radiation oncology because of the clerkship, the mean level of interest in radiation oncology as a career remained the same, with pre- and post-clerkship scores of 3.0 (±0.9) and 3.0 (±1.1) on a 5-point scale, respectively (P = .7). Students found virtual clinic and didactic lectures to be the most valuable components of the clerkship. Most respondents agreed (30.8%) or strongly agreed (65.4%) to recommend the clerkship to their classmates.
CONCLUSIONS: Our virtual clerkship was effective in increasing medical student interest in and knowledge about radiation oncology. These data will help optimize a new paradigm of virtual radiation oncology education for medical students during COVID-19 and beyond.

Introduction

Medical students generally have limited exposure to radiation oncology during their preclinical and clinical years and are required to independently seek educational opportunities in this specialty. For many students, the clerkship is their first formal experience with radiation oncology and is therefore a critical tool for educating and recruiting physicians-in-training.1, 2, 3 At our institution, we have offered 2- and 4-week clerkship electives to internal and visiting medical students.

However, with the onset of the COVID-19 pandemic in our state in early 2020 and subsequent recommendations from the Association of American Medical Colleges, on-site clerkships were suspended. To maintain an educational opportunity for students, we rapidly developed and implemented a virtual radiation oncology clerkship using telemedicine tools. The virtual clerkship was met with enthusiasm at our medical school and allowed us to not only maintain but also to broaden one of the few avenues that connect medical students with radiation oncology. We report here student and educator feedback and impact of the clerkship on student interest in and knowledge about radiation oncology.

Methods and Materials

Clerkship design

The virtual clerkship curriculum was designed by an advisory committee of key stakeholders, as described previously.

Canvas (Instructure Inc, Salt Lake City, UT), a cloud-based learning management system, was used to host our clerkship. We used Canvas’s (Instructure Inc) integrated calendaring system, communication stream, built-in web conferencing functionality, and assignment modules to organize clerkship activities. Synchronous (“live”) didactic lectures with faculty and residents were held on Zoom (Zoom Video Communications, San Jose, CA), a cloud-based video conferencing tool. All Zoom (Zoom Video Communications) sessions were recorded and password protected and required registration of attendees for attendance tracking. Zoom (Zoom Video Communications) features such as poll questions and live chat moderated by resident cohosts were used to help with student engagement. The virtual clinic was conducted on our institution’s Epic Systems (Epic Systems Corporation, Verona, WI) telemedicine portal. Students could log into video visits via a web browser using the VidyoConnect (Vidyo Inc, Hackensack, NJ) application. Multidisciplinary tumor boards and chart rounds were held on WebEx (Cisco Webex, Milpitas, CA), another video conferencing tool. In addition, a journal club, during which students chose a relevant and recent oncology paper in the literature to present, was hosted via Zoom (Zoom Video Communications).

Although our institution did not allow visiting students to formally rotate during the COVID-19 school closure, we allowed interested students from outside institutions to audit the course and participate in the didactic sessions and student journal club presentations.

This study was exempted by our institutional review board.

Data collection and analysis

Student pre- and post-clerkship self-assessments and faculty and resident surveys (Figs. E1-4; available online at https://doi.org/10.1016/j.ijrobp.2020.06.050) were collected using Research Electronic Data Capture (REDCap), a web-based application for capturing clinical research data hosted by our institution. Residents and faculty who gave lectures and/or worked with students in the clinic received surveys at the end of the clerkship on their experience with teaching in the virtual format. To pass the clerkship, students were required to complete the preclerkship assessment on the first day of the rotation, before participating in any clerkship activity, and the post-clerkship assessment on the last day of the clerkship, after the final clerkship activity. For the questions that were identical on the pre- and post-clerkship assessments, the order of questions and answer choices were altered to decrease recall bias. Although pre- and post-assessments were linked by respondent, responses were anonymized and aggregated, and we encouraged students to be honest in their feedback.

Paired t tests were used to compare the pre- and post-clerkship assessment results. Clerkship components were assessed using an analysis of variance model. Statistical analyses were performed using SAS software (version 9.4; SAS Institute, Cary, NC). All tests were 2-sided with an alpha level of 0.05.

Results

Student demographics

At the time of this analysis, 26 students had enrolled in our virtual clerkship. Twelve home-institution students were enrolled in the first offering of the clerkship (period 1) from April 27 through May 10, 2020. Ten home-institution students and 4 students from outside institutions were enrolled in the second offering (period 2) from May 27 through June 7, 2020.

All students reported the clerkship as their first radiation oncology rotation. Half of the students were female. Although the 4 visiting students ranked radiation oncology as their top choice for specialty, the home-institution students ranked oncology (n = 4), internal medicine (n = 3), surgery (n = 3), neurology (n = 2), dermatology (n = 2), otolaryngology (n = 2), and other (n = 6; urology, pediatrics, physical medicine and rehabilitation, anesthesia, cardiology, and ophthalmology) as their top choice for specialties. Additional student characteristics are provided in Table 1 .

Table 1

Student demographics and prior radiation oncology experiences

	Whole cohort (n = 26)	Cohort 1 (n = 12)	Cohort 2 (n = 14)
Age (median, range), y	27 (23-36)	27 (23-31)	28 (25-36)
Sex
Female	13 (50)	7 (58.3)	6 (42.9)
Male	13 (50)	5 (41.7)	8 (57.1)
Race
Asian	9 (34.6)	4 (33.3)	5 (35.8)
Caucasian	10 (38.5)	6 (50.1)	4 (28.6)
Black or African American	4 (15.4)	1 (8.3)	3 (21.4)
Latino, or of Spanish origin	2 (7.7)	1 (8.3)	1 (7.1)
Other	1 (3.8)	-	1 (7.1)
Clinical experience
First clinical year	21 (80.8)	12 (100.0)	9 (64.3)
Second clinical year	5 (19.2)	-	5 (35.7)
Degree track
MD	18 (69.3)	8 (66.7)	10 (71.4)
MD/PhD	5 (19.2)	3 (25.0)	2 (14.3)
Other	3 (11.5)	1 (8.3)	2 (14.3)
First radiation oncology rotation	26 (100.0)	12 (100.0)	14 (100.0)
Had prior exposure to radiation oncology	7 (26.9)	1 (8.3)	6 (42.9)
Research	4 (57.1)	-	4 (66.7)
Shadowed	2 (28.6)	-	2 (33.3)
Attended lectures	3 (42.9)	-	3 (50.0)
Other experiences	4 (57.1)	1 (100.0)	-
Motivations for enrolling in virtual clerkship
Interest in field	18 (69.2)	7 (58.3)	11 (78.6)
Interest in learning with new technologies	18 (69.2)	7 (58.3)	11 (78.6)
COVID-19 restrictions	23 (88.5)	12 (100.0)	11 (78.6)
Visiting student from outside institution	4 (15.4)	-	4 (28.6)

Data are median (range) or n (%).

Clerkship revision

After clerkship period 1, a 2-week break was used to improve the clerkship based on feedback from students, residents, and faculty.

Students from clerkship period 1 requested that didactics cover a broader range of disease sites (“having additional didactics on treating specific types of cancer would be informative”), more virtual clinic experience, and that they be required to enable their camera for all of the lectures [“Force students to show faces on Zoom (Zoom Video Communications) for lectures; this forces us to pay more attention”].

Faculty and resident responses reflected similar themes: “The availability of more interactive options (poll question variety, live Q&A) would be helpful”; “Being able to see all participants (encouraging them to be seen rather than disabling video camera) was also helpful”; and “It would be better for students to see more patients in virtual clinic.”

Students also requested to have more time allotted for their end-of-clerkship talks (“It would be nice to have a little more time to present the papers. Some of them have a lot to discuss, and for students new to radiation oncology, it’s helpful to discuss even the more basic things.”) and for attendance in chart rounds to not be mandatory (“I did not find chart rounds useful as it was fast paced and often above my level of understanding making it difficult to follow.”).

Based on the feedback, we revised the clerkship to include didactics for every disease site using a combination of prerecorded and synchronous lectures. We requested that students have their cameras on for synchronous lectures and encouraged all speakers to use interactive polls in their lectures. We also incorporated more virtual clinic into the student schedules. Given the number of students and time constraints, we did not increase the allotted time for student talks; however, we assigned students faculty mentors to provide guidance for their journal club papers and talks. We also continued to require attendance at our weekly chart rounds, which we believed encapsulated important aspects of our specialty. The core components of the revised curriculum are summarized in Table 2 .

Table 2

Curriculum components

1. Didactics (a combination of prerecorded and synchronous during the first week of rotation)	Topics • History of radiation oncology • Introduction to radiation oncology∗ • Radiation physics∗ • Radiation biology • Radiation therapy for breast cancer • Radiation treatment planning session • Approach to clinic notes • Basics of prostate cancer/brachytherapy (prerecorded version used for period 2) • Head and neck cancer treatment planning • Radiation and radiosurgery for CNS tumors • Gynecologic cancer • Hodgkin lymphoma • Lung cancer treatment planning • Pediatric oncology • Virtual department tour (prerecorded version used for period 2)
2. Virtual clinic (during the second week of rotation)	Clinical services • CNS • GI • GU • Lung • Lymphoma • Pediatrics
3. Additional educational sessions (throughout clerkship)	Sessions • Multidisciplinary tumor boards • Chart rounds • Resident education session
4. Student talks (last 2 d of rotation)	Journal club format • Recent oncology publication of their choice

Abbreviations: CNS = central nervous system; GI = gastrointestinal; GU = genitourinary.

Adapted from Radiation Oncology Education Collaborative Study Group lecture material.

Clerkship impact on knowledge about radiation oncology and learner satisfaction

Student response rate for the postclerkship assessment was 100%. Students ranked the curriculum components on a scale from 1 (most valuable) to 5 (least valuable). Students found the clinic to be most valuable (mean, 1.3; standard deviation [SD], ±0.7), followed by didactic lectures (2.5; ±1.1), journal club presentations (3.4; ±1.0), treatment planning sessions (3.5; ±1.1), and chart rounds (4.3; ±1.0) (P < .0001; Fig. 1 ). Students from period 2 reaffirmed the value of the virtual clinic: “Virtual clinic was the most valuable in showing the day-to-day decision making involved in radiation oncology”; “Particularly appreciated attending teaching during clinic.” Although many students commented that the virtual curriculum was conducive to learning (“Great mix of didactics, clinic and journal club presentations”; “The clerkship was very well-organized, and I appreciated the diverse opportunities despite us interacting with the course virtually”), 1 student found the virtual lecture format challenging [“I personally find the lectures hard to follow, especially over Zoom (Zoom Video Communications).”]

Fig. 1

Mean rank value and standard deviation of each clerkship component by medical students (n = 26). Respondents rated each curriculum component on a 5-point ranking scale of most valuable (1) to least valuable (5).

All students agreed that the clerkship improved their understanding of radiation oncology. Compared with 3 (11.5%) students who agreed that they understood the daily responsibilities of a radiation oncologist before the clerkship, 22 (84.6%) students agreed and 3 (11.5%) strongly agreed that they understood the daily responsibilities of a radiation oncologist after the clerkship (P < .0001; Fig. 2 A).

Fig. 2

Prerotation versus postrotation student responses to knowledge of and interest in radiation oncology.

Most respondents agreed (30.8%) or strongly agreed (65.4%) to recommend the clerkship to their classmates.

Clerkship impact on interest in radiation oncology

Sixteen (61.5%) students agreed or strongly agreed that the clerkship helped inform their decision to choose a specialty; 6 (23.1%) remained neutral, and 4 (15.3%) disagreed or strongly disagreed. One student commented, “I wish I had had more exposure earlier in my medical school career when I might’ve considered radiation oncology.”

Fifteen students (57.7%) reported an increase in their interest in radiation oncology after this clerkship; 34.6% reported their interest stayed the same, and 7.7% reported a decrease in their interest in radiation oncology.

Reasons for increased interest in radiation oncology included learning about the number of diverse therapies in our field (“There are more therapies in radiation oncology than I thought”; “didn’t previously know that radiation oncology does procedures, like brachytherapy”; “nice balance of clinic and procedural work”), the technical nature of our specialty (“There is a real tangible relationship to physics and technology in ways that other fields may not have”; “I have really enjoyed learning about the technology used in radiation oncology”; “This clerkship invalidated preconceived notions that radiation is a nonspecific treatment.”), the important role our specialty plays in multidisciplinary cancer care (“Much more applicable to specialties I am interested in than I previously thought”; “gained an appreciation of how central radiation oncology is in the treatment of cancer today and how integral it will remain despite much of the headline attention to immunotherapy”; “I learned about the coordination among teams for oncology treatment”), and the people in our department (“Great and supportive residents and faculty”; “I liked the people I met in this clerkship”).

Reasons given for decreased interest included “The day-to-day work was less interesting than I thought” and “My interest in pursuing the field personally is lower than prior, although I do have a new appreciation for the field and will be a better clinician for understanding/knowing more about radiation oncology.”

Mean level of interest in radiation oncology as a career remained the same with pre- and post-scores of 3.0 (±0.9) and 3.0 (±1.1) on a 5-point scale, respectively (P = .7). The 5-point Likert scale was defined as (1) not interested at all; (2) would consider oncology but not necessarily radiation oncology; (3) consider learning more about radiation oncology; (4) considering applying to radiation oncology residency; and (5) likely to apply to radiation oncology residency (Fig. 2B). However, after the clerkship, 5 (19%) home-institution students had radiation oncology as 1 of their top 3 specialty choices (compared with only 1 home-institution student before the clerkship).

Faculty and resident experience

Thirteen faculty and 9 residents participated in the virtual clerkship through giving lectures or working with students in the virtual clinic over the 2 clerkship periods. Their survey response rate was 95%. Overall, they thought virtual clinics and lectures yielded an educational benefit to the students and found ease of educating students via the virtual format acceptable (Table 3 ).

Table 3

Resident and faculty response to survey questions assessing virtual lectures and virtual clinics (n = 22)

	Mean (±SD)
Virtual lectures yielded an educational benefit to the students∗	4.4 (± 1.1)
Ease of educating students with the virtual lecture format†	4.6 (± 0.6)
Virtual clinic yielded an educational benefit to the students∗	4.4 (± 0.5)
Ease of educating students with the virtual clinic format†	3.7 (± 0.9)

Questions were rated on a scale of 1 to 5:

(1) Strongly disagree; (2) Disagree; (3) Neither agree nor disagree; (4) Agree; (5) Strongly agree.

(1) Very Challenging; (2) Challenging; (3) Average; (4) Easy; (5) Very Easy.

Discussion

The COVID-19 pandemic and resulting shelter-in-place orders both required and allowed us to redesign medical student education. By successfully implementing a virtual radiation oncology clerkship, we broadened our specialty’s reach to medical students. Over 2 virtual clerkship periods, we had 22 home-institution students participate, representing a substantial increase compared with the 4 total home-institution students who enrolled from July 2018 to March 2020.

This increased exposure to radiation oncology for medical students is important given recent trends of declining radiation oncology residency applications.6, 7, 8 In the 2019 match, 30 residency positions across 22 US radiation oncology residency programs were unfilled, with the unmatched rate (14.5%) nearly doubled from the previous year. Furthermore, racial and ethnic groups underrepresented in medicine and women continue to be disproportionately low in our field.9, 10, 11 Deville et al found that radiation oncology had less women, Black, and Hispanic/Latinx representation compared with the 20 largest training specialties. Chapman showed an ominous trend of declining Black representation in radiation oncology trainees in recent years. However, half of our virtual clerkship students were female, and more than one-fifth were Black and/or Latinx. Thus, the virtual clerkship can be a feasible instrument to help recruit the best students and a more diverse workforce to radiation oncology.

However, most medical students will not pursue radiation oncology as a career. Thus, the clerkship is also important for educating students about radiation therapy and the roles of the radiation oncology care team. Because cancer remains the second leading cause of death with estimates of 17 million new cases and 9.5 million deaths globally, , these future physicians, regardless of specialty, will undoubtedly care for patients with cancer. More than 50% of patients with cancer will undergo radiation at some point during their disease course, and medical students need to be aware of our treatment modality and how it affects their patients. Despite this, students are rarely exposed to formal radiation oncology curricula during their medical education. , Indeed, the majority of our home-institution students taking our virtual clerkship (86%) had no prior exposure to our field. Before the clerkship, 90% did not understand the daily responsibilities of a radiation oncologist. We were able to significantly increase students’ overall knowledge of radiation oncology over the 2-week virtual clerkship. Students commented that through the clerkship, they became aware of the broad range of treatments available in radiation oncology, were able to correct preconceived notions they had about our field, and wished that they had earlier exposure to radiation oncology during their medical training.

We designed the virtual curriculum to reproduce key components of our in-person clerkship, including faculty-led didactic sessions, clinic, student talks, and supplemental sessions, such as quality assurance rounds and multidisciplinary tumor boards. Didactics and student talks were hosted over Zoom (Zoom Video Communications). To re-create the interpersonal aspects of these typically in-person activities, most lectures were synchronous, and all participants were required to turn on their videos. We further encouraged the use of poll questions and chat to help engage students. Residents were crucial to faculty-led didactics; each session was cohosted by a resident moderator who helped address chat questions while faculty spoke. Although 1 student found the virtual lecture format challenging for learning, there were several advantages with the virtual format. There was increased participation by faculty in giving lectures to students and increased departmental attendance to student talks, potentially facilitated by the increased flexibility with the virtual format.

Of all the clerkship components, students rated virtual clinics as the most valuable. The clinic allows students to develop their skills in assessing and presenting a cancer patient’s history and pertinent findings and allows faculty to evaluate students’ clinical performance. Although integrating medical students into our evolving virtual clinic workflow posed additional logistical challenges, we considered the clinic as an indispensable component of the clerkship. Several factors allowed us to accommodate these students into the virtual clinic workflow. First, our institution quickly implemented a robust telehealth platform, with more than 70% of clinic visits in our department being over video as early as March 2020. Second, many faculty in our department had developed mostly virtual workflows that could accommodate students, with implementation of personal Zoom (Zoom Video Communications) “workrooms” with their residents, nurse coordinators, and nurse practitioners to hold patient presentations and team discussions. Finally, the residents again played a key role in facilitating the virtual clinic experience for medical students; they connected with their assigned students before the clinic on Zoom (Zoom Video Communications) to introduce students to the video visit platform and their team’s specific virtual workflow. This helped streamline clinical workflow with respect to student integration and reduce burden on faculty and their teams. Table 4 lists key pointers for implementing effective virtual clerkship didactics and clinic.

Table 4

Key pointers for implementation of virtual clerkship components

Virtual lectures (synchronous)	• For larger groups, assign resident comoderators to help field chat questions during session • Incorporate poll questions to encourage student engagement • Require students’ videos to be turned on to further encourage engagement
Virtual clinic	• Develop virtual clinic workflows with teams before clerkship initiation • Residents lead introductory sessions with assigned students to introduce telehealth tools and clinic workflow
Mentorship (future directions)	• Assign students to faculty/resident teams to facilitate more one-on-one interaction • Encourage resident and faculty check-in with assigned students by email or phone call after clerkship is complete for establishing longitudinal relationship • Zoom (Zoom Video Communications) happy hour with faculty and residents • Host additional sessions for students on work–life balance, leadership, and research opportunities

It is important to note that most home-institution students taking this virtual clerkship were not interested in applying to radiation oncology residency. Thus, their feedback about the clerkship would be expected to differ from that of students who are interested in matching into a radiation oncology residency and take our clerkship as an “audition” elective. Prior research found that medical students applying into radiation oncology considered the following educational activities as moderately to extremely important to include in a clerkship curriculum: an opportunity to perform an unsupervised history and physical examination; an opportunity to give a formal lecture; a didactic hands-on session on radiation contouring and planning; a formal case presentation to faculty; and lectures on treatment planning, radiobiology, physics, and evidence-based medicine. Our students found didactic sessions with faculty and residents to be second most valuable after the virtual clinic. Interestingly, our students reported treatment planning sessions and quality assurance rounds (chart rounds) to be least valuable. This likely reflects the background of the students, who largely were without prior experience or interest in radiation oncology as a career. Thus, they might not derive educational value in the more technical and specialized nature of these sessions. These data from our students are valuable to enhance the incorporation of radiation oncology into the general medical school curriculum.

A key question is how medical education will evolve moving forward. For our institution, we hosted 1 more 2-week virtual rotation from June 8 through June 21, 2020 (period 3), before resuming in-person clerkship activities for the 2020 to 2021 academic year. However, our institution will resume in-person clerkships only for internal students and will not be hosting visiting clerkships. This policy will be reviewed later this year but is currently in line with Association of American Medical Colleges recommendations and the policies adopted by many medical schools and residency training programs. The cessation of the visiting clerkship will have important implications for the residency application process; many programs and students use the away elective as an opportunity to evaluate one another. Students without radiation oncology programs at their home institutions will likely be most affected by these policy changes.

As telehealth reduces the burden on patients while increasing access to care, virtual clerkships can similarly improve access to our specialty for students. In the COVID-19 era, virtual clerkships allow reduced department crowding and reduced exposure risk for students. After the COVID-19 pandemic, virtual clerkships may continue to have a role in overcoming geographic and financial barriers associated with away rotations. We plan to continue our virtual clerkship for the near future given that social distancing measures will remain in place in our institution. Furthermore, cancer center providers at our institution anticipate that 50% of visits will be virtual moving forward, with 75% agreeing or strongly agreeing to use video visits to a greater extent after the COVID-19 restrictions are lifted (internal data). Thus, virtual clerkship components, such as virtual clinics, should naturally continue to train future physicians in the practice of outpatient medicine. Although we have welcomed visiting students to audit our 2-week clerkship thus far, we plan to formally host visiting students for credit in the summer and fall so that visiting students can participate in patient-facing activities, such as the virtual clinic. These clinical activities were rated by students as the most valuable component of our virtual clerkship and are necessary for the full educational experience.

We acknowledge that our study is based on the small and unique experience of a single tertiary academic medical center and may be challenging to generalize to other settings. We benefited from a quickly implemented and comprehensive telehealth infrastructure and a patient population largely capable of participation in video visits. Also critical to the success of our virtual clerkship was the support of faculty and residents who were engaged in the education of our medical students and receptive to hosting them in virtual clinic and leading lectures.

Finally, although our virtual clerkship was initially created to primarily address the educational needs of medical students during this pandemic, a critical need still exists to address the loss of mentorship, which is the other key purpose of visiting clerkships. Medical students often select institutions for their away clerkships for a number of reasons, including but not limited to (1) learning more about the field of radiation oncology, (2) learning more about a program they wish to consider for training, (3) participating in research, (4) establishing a mentorship or personal connection with faculty in specific areas of interest, and (5) obtaining a referee letter. To continue encouraging and developing the next generation of entrants into radiation oncology, it will be critical to overcome these gaps, given the already decreasing applicant pool. Although this broader goal will require creativity and participation from all training institutions, some suggested proposals are included in Table 4. We hope that our efforts are simply the beginning of the development and optimization of a new paradigm for virtual radiation oncology education for medical students.

Conclusions

The implementation of a 2-week virtual clerkship in radiation oncology at our institution was received positively by medical students, faculty, and resident educators in our department and holds promise as a tool to potentially mitigate inequities in access by expanding exposure to our field. Based on the impact of this program, we will continue to offer the virtual rotation to home-institution students and visiting students in the 2020 to 2021 academic year.