Implementation and Effectiveness of a Completely Virtual Pathology Rotation for Visiting Medical Students.

OBJECTIVES: Coronavirus disease 2019 (COVID-19) upended in-person medical education, relocating many activities online. We designed a completely virtual pathology rotation to replace our traditional visiting rotation.
METHODS: The virtual away rotation was listed in the Visiting Student Application Service (VSAS) and advertised on social media as well as various medical student mailing lists. Nine students were selected to participate in three month-long rotations. The virtual curriculum mirrored our typical in-person clerkship with didactic lectures and daily signout but also included activities exclusive to the virtual rotation such as digitally scanned slide trays and small-group problem-based learning. Anonymous surveys were conducted in which both participants and instructors rated their experiences.
RESULTS: Postrotation feedback was overwhelmingly positive from both participants and instructors. Students considered virtual slide sessions as the most effective teaching tool and did not feel hindered overall by lack of in-person experiences. Volunteer trainee instructors indicated the experience improved their teaching and diagnostic skills and expressed interest in teaching future virtual courses.
CONCLUSIONS: The success of the virtual away rotation raises consideration for applications beyond the pandemic era and may provide a more level playing field for medical students from underrepresented groups. © American Society for Clinical Pathology, 2021. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.

Our virtual away rotation provided a means to continue visiting medical student education during coronavirus disease 2019–related furloughs from in-person training.

The success of our virtual-only rotation indicates utility beyond the pandemic era, with ability to extend opportunities for those facing financial or geographic barriers to in-person away rotations.

Virtual-only pathology rotations can be well liked and considered effective by both students and instructors, as well as provide benefits exclusive to the virtual experience.

Introduction

Visiting, or “away,” rotations provide the opportunity for a medical student, usually in their fourth year, to complete a rotation in their specialty of interest at an institution other than their own. They are typically done at programs where the student is interested in applying for residency, leading to the alternative title “audition rotation.” Visiting rotations have become increasingly important, with program directors citing performance during these rotations as a major factor in consideration for interviewing and ranking. According to the 2021 National Board of Medical Examiners survey, 50% of responding pathology program directors cited a rotation within the department as a factor in ranking applicants, rating its importance at 3.8 on a scale of 1 to 5.[1] In line with the increasing emphasis on away rotations, they are becoming increasingly common. In 2015, approximately 60% of US medical students did at least one away rotation.[2]

The coronavirus disease 2019 (COVID-19) pandemic and resultant efforts to maintain 6 feet of interpersonal distance (“social distancing”) have had a profound impact on medical education. At our institution, medical students were furloughed from in-person classes and clerkships following the Association of American Medical Colleges (AAMC) guidance in the spring of 2020.[3] As the hospital canceled elective procedures, leading to a significant reduction in pathology specimens, the Department of Laboratory Medicine and Pathology (DLMP) sent most of our residents and fellows home, as well. The department adapted quickly and moved our graduate and undergraduate medical educational programs online.[4,5] When the AAMC also discouraged visiting rotations,[6] we used the virtual tools and strategies we had developed to create an entirely virtual away rotation. The purpose of this project was to develop and demonstrate methods to both minimize the limitations of virtual medical education and leverage features that are either enhanced by or exclusive to virtual learning.

Materials and Methods

The virtual elective was modeled to closely mimic the already existing in-person elective for visiting medical students. Course objectives are listed in . A comparison of the two types of electives is shown in . The course was listed in the Visiting Student Application Service (VSAS) and advertised on Twitter, the DLMP residency website,[7] and medical student mailing lists including the AAMC Group on Student Affairs, the Student National Medical Association, and the AAMC Group on Diversity and Inclusion. As for in-person rotations, eligible students were US medical students (MD or DO) in the third or fourth year of training in good standing with their home institution. They underwent the same onboarding process, including completion of a Health Insurance Portability and Accountability Act self-study; a Privacy, Confidentiality, and Information Security Agreement; a clean background check; and acknowledging and agreeing to our institution’s Medicine Policy on Professional Conduct.

All residents and fellows were given the opportunity to participate by overseeing daily virtual slide sessions. Faculty were invited to participate in daily signout based on interest in teaching, student interest in their service, and access to a microscope camera. Participation was voluntary for all educators, and no formal incentives were given.

Teaching materials were adapted from those developed for a prior virtual medical student pathology course.[4] Specifically, course-specific lectures were designed to address the course objectives and covered normal histology, pathology processing, neoplasia, and autopsy and death certificates. Virtual slide trays, hosted on PathPresenter, comprised 10 to 12 whole-slide images (WSIs) each. Slides were selected to represent the most commonly encountered neoplastic and nonneoplastic entities in each organ system. Small-group (“detective”) cases were adapted from autopsy cases, with deidentified laboratory and imaging results as well as representative WSIs on PathPresenter.

Anonymous rotation evaluations were conducted at the end of the rotation for both students and resident and fellow educators . Students rated each faculty member on a variety of measures using a Likert scale of 1 to 5. Respondents also rated the different components of the course and were asked for input on the positive and negative aspects of the course using free-text answers. To ensure anonymity, survey responses were not reviewed until completion of all three sessions. Students were evaluated according to their home institution’s form, in the same manner as traditional visiting rotations.

We used the Zoom (Zoom Video Communications) platform for all lectures, slide sessions, and signouts. Microscope camera software is CellSens (Olympus) or SPOT (SPOT Imaging). Slide sessions were conducted using the whole-slide imaging platform PathPresenter. Communication and organization were done via Microsoft Teams (Microsoft Corporation). Our learning management system (LMS) is Canvas (Instructure). We used the slide scanning system Aperio (Aperio Group) and TeamViewer remote access software (TeamViewer AG) for real-time frozen-section evaluation.

Results

Design

We modeled the rotation after our typical in-person clerkships, aiming to achieve the same goals: educate medical students about the practice of pathology, give prospective residency applicants the chance to evaluate our department, and provide opportunities for our trainees and faculty to evaluate the students as prospective residency applicants.

For in-person rotations, we offer pathology clerkships over four different sites, usually accepting one student at a time. Students who do a pathology rotation tend to be those who are interested in pursuing a pathology residency, either internal medical students or visiting students. Internal students are guaranteed a rotation. For visiting students, we receive an average of 21 applicants/year, and about 11 students complete a visiting rotation yearly.

Our typical clerkship pairs medical students with a resident. They attend morning didactic sessions and then observe and eventually assist with the service work assigned to the resident, which includes a mix of surgical pathology services and autopsy. At our site, they also spend a week with our laboratory medicine colleagues, attending signouts for microbiology, immunology, and coagulation, as well as lectures, grand rounds, and call rounds. At the end of the rotation, they give a 10-minute presentation about an interesting case or test they encountered during their rotation.

The virtual rotation was structured similarly to the traditional rotation, with adjustments as required by the virtual format. Participants attended regularly scheduled morning resident didactics, which were given virtually to reach residents at different sites. Each week, one faculty member was assigned to conduct daily sessions in the form of virtual signout and/or formal teaching between 9 am and 2 pm. The session included routine daily signout with residents and fellows, with the slide images transmitted via microscope camera. If the faculty was involved in a frozen section during that time, frozen-section slides were scanned on the Aperio system after the diagnosis had been given to the surgeon, and the students were given remote access to the scanner via TeamViewer, allowing them to attempt frozen-section diagnosis in close to real time. At 2 pm, the students met virtually with a volunteer resident or fellow to go over a virtual slide tray of WSIs hosted on PathPresenter.net. Each day was devoted to basic pathologic processes in a single organ system, and students were assigned three slides, representative of three common entities, to preview the day before, coming to the slide session prepared to describe the slide and make a guess at the diagnosis. Trainee educators gave feedback and emphasized preset teaching points. One of the 4 weeks was devoted to laboratory medicine, when the students joined the first week of the virtual course offered by our laboratory medicine colleagues.[5] A sample daily schedule of the anatomic pathology weeks is shown in . Activities scheduled during signout time or after 3 pm were optional to give flexibility to faculty during the day as well as to students located in a later time zone. More detailed descriptions of each component are listed in .

An additional component of the virtual rotation is a group activity we termed “detective cases,” in which the students worked together to determine the diagnosis in a difficult case. They received a case presentation each Monday, met to discuss, and then ordered tests, receiving the results in “real time.” This process continued throughout the week until a second meeting on Friday, when they presented their conclusions from the case. Each case was taken from a real patient whose diagnosis hinged on a histologic diagnosis in combination with laboratory findings. Examples include Wilson disease, lymphoma, and metastatic carcinoma of two primary sites. Students could order physical examination, history, test results, imaging, or biopsies (WSIs) to arrive at their diagnosis. Orders and results were posted on a dedicated discussion board on the LMS site, monitored and administered by the course director (L.K.K.).

Visiting Medical Student Recruitment/Selection

We received 24 applications via VSAS. To ensure we could give adequate attention to each student, we set an initial limit of three students per rotation. Students were selected according to usual protocol for in-person away rotations, including review of curriculum vitae and a letter of interest. We prioritized students who showed an interest in pathology, as indicated in their letter as well as activities while in medical school, such as previous pathology rotations and participation in pathology interest groups. We accepted nine students, comprising six DO students and three MD students, all in their fourth year of training and all but one of whom indicated an interest in pursuing a pathology residency. They were located in all time zones in the continental United States; we scheduled activities between 8 am and 3 pm Pacific Standard Time (PST) to minimize early mornings or late nights for everyone involved. Three 4-week rotations were held in succession during the fall and winter of 2020.

Virtual Pathology Elective Educators

Seven faculty members participated as attending of the day. On services in which one attending was on service for an entire week, students joined the same attending every day. During some rotations, students indicated special interest in certain areas, such as neuropathology or dermatopathology, and a few days of signout in those areas were arranged. Time differences became an issue in this setting, as some of our services regularly sign out later in the day, pushing those sessions into the evening hours for students in eastern time zones. We therefore prioritized services that typically sign out starting at 9 or 10 am PST. Volunteer trainee educators included six second-, third-, and fourth-year residents and one fellow, each of whom delivered between one and three slide sessions per rotation; they signed up for sessions based on interest in the topic and availability. Each slide set was accompanied by a list of brief clinical information, diagnosis, and key teaching points. Faculty were available to review slides as needed prior to the sessions.

Evaluation of the Virtual Elective

Postrotation survey and in-person feedback, completed by all nine students, was overwhelmingly positive. On a 1 to 5 scale from strongly disagree to strongly agree, every student rated every faculty member a 5 on every metric (eg, “This instructor encouraged discussions and responded to questions”). Individual components of the course were also rated highly, each getting a rating of 4 (very good) or 5 (excellent) from every student, with the exception of the laboratory medicine week and the resident didactic lectures . Three students rated the laboratory medicine week at a 3 (good). The other 3 rating was given to the resident didactic lectures by one student.

As part of an anonymous survey, students were asked to rate each course component on a scale of 1 (poor) to 5 (excellent). Average ratings are shown. CP, clinical pathology.

When asked, “What is helping your learning in this course?” seven of nine students noted the slide sessions with the residents as the most or one of the most effective tools. Crucially, many of the responses included comments about how these sessions allowed them to ask questions without judgment or fear of looking “silly.” They appreciated the opportunity to try out their developing skills away from the (real or perceived) critical eye of the attendings. Three students identified the detective cases as the best or one of the best things about the course, with several asking for additional cases during the rotation.

When asked, “What can your instructor do to improve your learning in this course?” students suggested one-on-one signout sessions with attendings, additional normal histology slide sessions, more clinical information included with the virtual slide sets, and the possibility of using technology to participate in an autopsy. Only one mentioned the downside of not being able to gross. Overall, the students did not feel the virtual nature of the rotation hindered their learning. In fact, one student remarked, “This is the best rotation I’ve ever done!”

Feedback from the trainees who participated in slide sessions (five of seven responding) was also overwhelmingly positive Figure 1 . While no formal incentive was provided, the educators found the experience to have benefited them. On a scale of 1 to 5, in which 1 is not at all and 5 is extremely, the respondents answered 4 or 5 when asked if the experience improved their teaching skills. They specifically pointed out it helped them identify the most pertinent points and focus on correlating pathologic findings to clinical information. All but one answered 4 or 5 when asked if the experience improved their diagnostic skills. One noted they had chosen topics in which they felt they needed more review in order to benefit their own knowledge. Every respondent thought the virtual rotation was extremely beneficial for the residency program, and all indicated they were “likely” or “very likely” to participate in future sessions. One specifically urged the program to continue the rotations after the pandemic ends. Interestingly, the participating trainees preferred the virtual rotation to in-person rotations. When asked for suggestions to improve the course, one trainee suggested each rotation could support up to five or six students.

Figure 1

Trainee feedback. As part of an anonymous survey, residents and fellows who led slide sessions were asked to rate how much they agreed with the above statements from 1 (not at all) to 5 (extremely).

Technological Difficulties

The faculty administering this course offered informal feedback about the importance of communication and organization. Specifically, real-time communication was essential; if a student was having trouble accessing the Zoom room, did not know where they were supposed to be, or had a time-sensitive question, email was not an efficient way to communicate. We experienced this early on, when a miscommunication about the first-day schedule led one student to access the wrong room at the wrong time, missing the orientation lecture. We found the chat function of Microsoft Teams to be an efficient means of communicating between all three students and the attending or trainee assigned to that day. The students also used Teams to discuss the detective cases among themselves. “One-click access” to meeting rooms was crucial. We ensured the students had multiple ways to access the daily calendar, where the schedule was clear and each meeting link was clearly labeled Figure 2 .

Figure 2

A sample daily schedule of the anatomic pathology (AP) weeks. BST, bone and soft tissue; CP, clinical pathology; GI, gastrointestinal.

Discussion

The COVID-19 pandemic has had profound impacts on medical education. Pathologists all over the world have developed innovative solutions to deliver education to students, including via social media,[8,9] web-based training such as PathElective.com,[10] and individual online courses.[4,5] We sought to address the specific need for visiting pathology rotations, in which students participate for the purposes of education as well as evaluation of an individual residency program. This rotation was received with extremely positive feedback from everyone involved.

Our in-person rotation is a fairly traditional apprenticeship model that relies heavily on peer-assisted learning, an approach that has been shown to be an effective educational tool for both the learner and the teacher.[11] In our experience, this model has a relatively small effect on overall daily workflow, with most of the effort borne by the trainees. Drawbacks include the time required of the assigned resident, the limited capacity for students, the lack of consistent engagement with individual faculty (who are often asked for letters of recommendation and who give input for residency interviews and rank lists), and the lack of level-appropriate didactic instruction. Importantly, a visiting rotation also requires the time and money for a student to leave their home institution for 4 weeks.

Our virtual rotation maintains many of the advantages of in-person rotations and, we believe, enhances them Table 2 . The focus on peer-assisted learning is present in the form of the afternoon slide sessions, which were one of the most highly rated components of the course. In addition, the trainees who participated overwhelmingly felt this benefited both them and the residency program, even preferring the virtual format to in-person rotations. As traditional rotations pair a resident with a medical student for a week at a time, we expect this result is likely due to the decreased time required of them, which they estimated at 1 to 5 hours for the virtual course (data not shown).

Table 2

Comparison of Traditional In-Person Rotations and Virtual Rotations

Characteristic	Traditional Rotation	Virtual Rotation
Main contact person	Resident	Attending
Number of students per rotation	1	3
Trainee involvement	1 second-year resident per week, 3-4 residents over a 4-week rotation	3-5 residents and fellows per week, 5-8 trainees over a 4-week rotation
	Student shadows trainee, and trainee often spends extra time doing didactic teaching during slide preview time	Prepare for and oversee 1-hour slide session on assigned days
Attending involvement	2-3 faculty per week, 6-12 faculty over a 4-week rotation	1-2 faculty per week, 4-6 faculty over a 4-week rotation
	Daily signout as usual, with one additional student	Daily signout as usual with three additional students plus broadcasting via microscope camera
Services experienced	GYN, BST, HNL, GI, Derm, Autopsy, CP topics	Participating attendings included GI, GU, Derm, GYN, Neuropath, and Breast, but all were offered CP topics
Daily activities	Resident didactic, grossing (mostly observation), previewing with the resident, signout, tumor board as available	Resident didactic, signout, dedicated lectures, frozen section, slide sessions, small-group work/“detective case,” tumor board as available

BST, bone and soft tissue; CP, clinical pathology; Derm, dermatopathology; GI, gastrointestinal; GU, genitourinary; GYN, gynecologic; HNL, head, neck, and lung.

The virtual format allowed us to incorporate additional activities, which we feel improved the educational value of the rotation. The Zoom/microscope camera signouts allowed more students to attend and increased interaction with the material, as students could annotate the screen or look up clinical data in real time. As students were not limited to one physical site, we were able to offer a wider selection of services. In addition, the sessions included an attending, a trainee, and multiple medical students, fostering a sense of community, stimulating discussion, and allowing all parties to assess the compatibility of the students and the program.

The detective cases were another highly rated component. These cases allow students to exercise their clinical reasoning skills to order tests and make diagnoses with minimal guidance, receiving feedback in almost real time. It was also an opportunity for the students to work together, a welcome opportunity in this socially distanced setting. The popularity of this component has led us to develop more cases with a higher level of complexity, which we plan to incorporate in future in-person rotations as well.

The lower ratings given to the laboratory medicine component may be related to the distinct nature of that course, which was not designed to complement this course. The two courses have different structures, schedules, and faculty. Future iterations of our away rotation will include more laboratory medicine topics within the framework we designed, including more participation from laboratory medicine faculty, additional lectures, and laboratory medicine components included in detective cases.

The other lowest-rated component (although still rated over 4 on average) was the resident didactics. We believe this reflects the importance of the course-specific and level-appropriate lectures, which was also seen in the higher overall rating of the course lectures compared with the resident didactics (4.9 vs 4.3), as well as the free-text responses, in which two students noted the course lectures to be one of the most useful parts of the course. In fact, two students requested more formal courses of this nature.

The major drawback of the virtual visiting rotation is the amount of time required of the participating faculty, which was also noted in a similar virtual elective implemented at Johns Hopkins.[12] Daily signout required the use of microscope cameras and Zoom as well as time spent communicating and scheduling with the students. Because of the pandemic, many services were conducting signout virtually already, making these changes less noticeable. Faculty who volunteered for teaching noted an extra hour or two, including increased time at signout due to teaching. However, we expect that in the future, as practices return to “normal,” this additional layer may discourage some faculty from participating. In addition, some signout sessions routinely take place at multiheaded microscopes or other locations that are not currently equipped with a microscope camera and/or appropriate software.

For faculty administering the course (L.K.K.), delivering rotation-specific lectures and discussion sessions as well as management of the detective cases and responding to student questions all required significantly more effort than an in-person rotation, estimated at an extra 10 hours per week. However, the positive reception of these components has induced us to consider incorporating them into traditional in-person rotations. Moreover, the increased interaction between faculty and students allowed for more in-depth assessment of students’ learning and therefore tailoring of teaching sessions. As in many settings, the limitations of social distancing have illuminated creative solutions that will persist long after the pandemic.

In that vein, the success, and possibly even superiority, of this virtual visiting rotation has led us to consider its application beyond COVID-19. This is especially important given the inequities of the visiting rotation system. The growing importance of visiting rotations creates a burden for medical students in time, money, and stress. This burden is not evenly applied, as students with lower socioeconomic status, extracurricular responsibilities, or lack of fluency in the system are less able to participate in visiting rotations. According to a 2016 study, the estimated cost of each visiting rotation was close to $1,000.[2] In short, the more important a visiting rotation is viewed, the more barriers there are to success for medical students from underrepresented groups. We believe virtual visiting rotations are an effective alternative that will allow us to reach, educate, and recruit students from all backgrounds.

Limitations of this study include the relatively small sample size of both students and educators. The authors were heavily involved in administration of the course. This allowed us to quickly identify and address any issues that arose, as well as to improve processes (communication, materials, organization, timelines) based on real-time feedback. However, this was a significant time investment, and we do not know whether this model is transferable to other groups of educators. Therefore, we do not have data on whether this rotation will be sustainable in the future or if there will be demand for it. We plan to take the information learned from this experience to build a more standalone course infrastructure, which will hopefully decrease the amount of administrative time in future iterations.

Evaluation of the students, as for in-person rotations, was largely informal and unstructured. Compared with an in-person rotation, the students had much more interaction with faculty, allowing for more direct observation than is typical. Therefore, while the overwhelming consensus from the educators involved was that the students demonstrated good progress in their pathology skills and reasoning during signout, discussions, and slide sessions, no objective data (test scores, etc) were available. Given that this was a highly motivated group of students who sought out this course due to difficulties caused by a pandemic, their high level of engagement and performance is not surprising. In the future, more standardized student evaluations will be useful to ensure effectiveness of the course, as well as to monitor consistency over time, location, and educators.

Medical education was significantly shaken by the COVID-19 pandemic, and visiting rotations were brought to an abrupt halt. Virtual solutions have since arisen and sustained critical student learning in the wake of mandated and precautionary measures, proving its value beyond a supplementary role. We believe virtual education will persist and flourish beyond the current era of social distancing, with applications in areas of outreach, accessibility, diversity, and recruitment.

Table 1

Goals and Objectives as Listed for the Rotation

By the end of this course, students will be able to:

- Describe how a biopsy or surgical specimen is processed, from procurement to glass slides.

- Define the terms neoplasia, benign, malignant, and carcinoma.

- Recognize tissue of origin under the microscope.

- Recognize basic pathologic entities.

- Identify and describe the components of a pathology report.

- Understand how pathologic diagnoses affect patient care.

- Describe the utility and appropriate use of frozen section and immunohistochemistry.

- Identify reasons to pursue pathology as a career.

- Understand the structure and culture of our institution’s pathology residency.

Table 3

Descriptions of Course Components and Technology Employed

Component	Description	Technology Used
Resident didactics	Resident-level didactic lectures given from 8 to 9 am daily. Preexisting part of a yearlong curriculum. Wednesdays are combined AP/CP topics.	Zoom, PowerPoint, PathPresenter
Virtual signout	Faculty of the week, resident, and/or fellow conduct signout via Zoom and microscope camera with medical students observing. Participants can draw on screen via annotate function, observe report writing, and help with chart review on their home computers.	CellSens or SPOT, Zoom
Slide sessions	Organ system-based slide trays created in PathPresenter and shared as unknowns with students. Each student assigned three slides to preview, go over with volunteer resident or fellow daily.	PathPresenter, Zoom
Detective case	Group activity that starts with a case presentation. Students discuss and decide on orders and receive results in real time until they reach a diagnosis. Check-ins on Mondays and Fridays to discuss each case. One case per week, based on real cases.	Zoom and Teams for discussion, Canvas for posting results, PathPresenter for pathology results
Course-specific lectures	Basic lectures on normal histology, pathology language, pathology processing, autopsy, and death certificates, given by course faculty during signout time.	Zoom
Intraoperative consultation	After frozen section is performed, slides are scanned into Aperio. Students log in to TeamViewer to view slides and form diagnoses. Gross photos are taken via phone and transmitted via Teams for student diagnosis.	Aperio, TeamViewer, Microsoft Teams for communication
Optional activities	Tumor boards, laboratory medicine call rounds, autopsy brain signout, readings, grand rounds.	Zoom

AP, anatomic pathology; CP, clinical pathology.