Virtual Interviews for the Complex General Surgical Oncology Fellowship: The Dana-Farber/Partners Experience.

Introduction

The SARS-CoV-2 virus has resulted in unprecedented disruptions in academic medicine. This has been especially true when interviewing applicants for training programs, including surgical residencies and fellowships. Before the COVID-19 pandemic, in-person interviews had been criticized for their inefficiency and associated expense to applicants.1,2 Recently, the Association of American Medical Colleges strongly encouraged all interviews with students, residents, and faculty to be conducted virtually.3 Given this new reality, training programs have been forced to adapt and adjust. At the Dana-Farber/Partners Complex General Surgical Oncology (CGSO) Fellowship Program, we implemented lessons learned from other institutions4 and created a virtual interview experience that simulated the pre-COVID-19 in-person interview process. We describe our experience below.

Logistics of the Day

Our overarching objective in developing a virtual fellowship interview process was to simulate the in-person interview day on a virtual platform. We created a seamless, automatic, and internally controlled interview experience with all applicants and faculty that was conducted using the Zoom platform (Zoom Video Communications, San Jose, CA). Participants received a single invitation prior to the interview day. We used Zoom’s breakout room function to create a virtual “main room”, five themed “interview rooms”, and a breakout room that we designated as the “hallway”. We provided two interview sessions, a morning and an afternoon session, which were strategically designed to accommodate applicants from different time zones. All applicants in each of the two interview sessions began their interview day in the main room where the program director provided a 30-min general overview of the fellowship and contributing three institutions (Dana-Farber Cancer Institute, Brigham and Women’s Hospital, and Massachusetts General Hospital). After the program director’s presentation, the main room became the fellow’s room where fellows had a semi-structured agenda, which we outline below.

Each of the two interview sessions (A and B) had a total of 10 applicants, and these were split into 2 groups (A-1 and A-2; B-1 and B-2). After logging into the Zoom session, all applicants entered the waiting room. The program coordinator confirmed the arrival of the applicants who were then transferred to the main room. At the start of each session, groups A-1 and B-1, respectively, began their day with interviews and groups A-2 and B-2, respectively, began their day in the fellow’s room. Our program coordinator moved applicants and faculty to their designated location by utilizing the functionality of breakout rooms in Zoom. Groups A-1 and B-1 were placed in the hallway where they waited to be placed into their first interview room. Use of a “hallway” prevented overlap of applicants in interview rooms and ensured that applicants did not inappropriately return to the main/fellows room. All interviews were scheduled for 15 min and faculty and applicants were notified when there was 1 min remaining using a broadcast messaging feature that presented as a banner at the top of each viewer’s Zoom screen. At the conclusion of each interview, applicants were moved back to the hallway where they waited to be moved to their next interview. This proceeded in the same fashion until all applicants completed a total of 5 interviews (Fig. 1). Faculty had 5 min in between interviews to score each applicant.

Fig. 1

Conceptual model of the virtual interview day and illustration of a single applicant’s schedule through the course of the virtual interview day (applicant in the A-1 group)

While groups A-1 and B-1 were being interviewed as outlined above, groups A-2 and B-2 were in the main room with the current fellows. The current fellows had a semi-structured agenda that included a pre-recorded virtual tour of the hospitals in our program, a presentation highlighting research opportunities available to fellows, and prepared talking points about topics of interest to prospective fellows (Fig. 1).

During the second half of each interview session, groups A-1 and B-1, respectively, were moved back to the main room (i.e., fellow’s room) and groups A-2 and B-2, respectively, were moved to the hallway so they could begin their scheduled interviews.

Safety Measures

All faculty and applicants participated virtually from the comfort of their offices or homes. Each interview room consisted of two to four faculty members, with at least one faculty member from each hospital in each room.

Faculty Preparation

Faculty who were scheduled to interview applicants were sent two PDF documents for each session. These PDF documents allowed for easy identification and navigation of each candidate’s application using bookmarks (i.e., application, curriculum vitae, letters of recommendation). Faculty were provided with access to a centralized and electronic scoring document using Google Docs (Google LLC, Mountain View, CA) to score applicants, and the results were blinded in real-time, to prevent influencing of applicant scoring. Applicant scores were then compiled into a Microsoft PowerPoint (Microsoft, Redmond, WA) presentation with each applicant represented by an individual slide. After the morning session, the first 10 applicants were ranked in a discussion conference that took place in a separate breakout room. The exact process was then repeated for the afternoon session. Following the afternoon session, the entire group of 20 applicants was ranked. This allowed the program director to move applicants within the rank order using the slide sorter view within Microsoft PowerPoint, which simulated the ranking process of applicants that normally happened during the pre-COVID-19 era.

Impression by Applicants and Faculty

Every year, a survey is conducted of all applicants for internal quality improvement. Applicants reported a significantly more favorable perception of the flow of the interview schedule in 2020 compared with 2019 (91% versus 33% scored the flow of the interview as “excellent”, P = 0.02). There were no statistically significant differences in how applicants scored the length and setup of the interviews; the tour; the helpfulness of faculty, current fellows, and other personnel; or overall impression of our facilities. This suggested that the virtual interview had served to match the in-person interview of the prior year, while maintaining the recommended objectives of social distancing.

On surveying the participating faculty, the majority of the faculty reported an excellent overall impression of the interview day.

Strengths of the Virtual Interview Day

One of the biggest strengths of the virtual interview experience was the ability to maximize the number of faculty members participating in the interview process. Bringing multiple faculty members together has always been a challenge, which is especially true during the COVID-19 era. Additionally, a virtual interview experience eliminated the burden of associated costs for the program itself with regards to food and drink but, more importantly, the cost and inconvenience of travel for applicants.

Areas for Improvement and Recommendations

Although there are many aspects of a virtual interview experience that are appealing to program leadership, faculty, and applicants, there are some limitations. The inability of applicants to tour the city and hospital(s) can be a significant deficit of a virtual interview experience, although the true utility of such tours has been questioned in recent years. However, in order to partially address this, our fellows created a video tour of the three hospitals in our fellowship and had prepared talking points about living in the Boston area. Applicants may have a difficult time making a fully informed decision without being able to explore the hospital(s) and city that they may end up calling their place of work and their home. There is risk that attrition rates within training programs may increase due to these limitations when the reality of a training program does not mirror the virtual experience. As it is unclear when we will resume in-person interviews, we recommend that training programs and/or their respective hospitals invest in creating a professional virtual tour of their hospital(s) and city.

Feedback from applicants identified that the 1-min alert during the 15-min interview led to an unnatural, and at times awkward, conclusion to the conversation when applicants were abruptly removed from the interview room into the hallway. We recommend that at the 1-min warning the faculty and applicants conclude their discussion and have both the applicants and faculty mute their microphones and turn off the video capability to allow a more natural end to the interview.

Programs should allow time for practice sessions with all key administrative staff and faculty (Table 1). We conducted five practice sessions so that we could test out features on Zoom to see how effective they could be on the interview day, and to troubleshoot moving applicants between the main room, interview rooms, and the hallway. The administrative staff and program coordinator were also available to practice with any faculty member who requested it.

Table 1

Key features of the Dana-Farber/Partners virtual interview experience

Key Features of Pre-Planning

• Practice sessions with key administrative staff.

• Clear communication with all faculty regarding logistics of the day, and time allowed for practice sessions with interviewing faculty

• Communicate to all faculty members that they should refrain from sharing their screens so that they do not inadvertently share any of the applicant or ranking documents

• Send faculty easy to navigate electronic applicant packs that allow for quick identification of each applicant

• Centralized, electronic scoring that allows faculty only to see their own score, so they are not influenced by other faculty

Key Features of the Fellow’s Main Room

• Semi-structured agenda featuring a pre-recorded virtual tour of the hospital and short presentations addressing topics of interest to applicants (e.g., research opportunities)

• Fellows running the room should be prepared to discuss topics of interest to applicants to avoid awkward periods of silence (e.g., housing options, operative volume, didactics, mentorship, support during the job search, graduated autonomy, support during dedicated research time, etc.)

Key Features of the Overall Virtual Interview Experience

• One virtual session with the same invitation link for all applicants (e.g., one Zoom session)

• The program coordinator should have a phone number for every applicant and faculty in case they are accidentally dropped from the Zoom session or if any questions arise

• Seamless, automatic, and internally controlled (all applicants and faculty are moved from the main room to the interview rooms and to the hallway by the administrative staff)

• Simulation of the real interview day: main room, interview rooms, and hallway

• Keep groups small (e.g., 5 applicants per group that are being interviewed at any given time)

• Morning and afternoon interview sessions to accommodate applicants from different time zones so that they are not being interviewed in the early hours according to their local time

• The fellows should keep the fellow’s room light but informative. Of note, we did not do a social cocktail hour due to feedback received from other programs as well as input from our own internal candidates; instead, we optimized the time spent with fellows in the main room

• Fellows leading the fellow’s room should have a semi-structured agenda featuring a pre-recorded virtual tour of the hospital and short presentations addressing topics of interest to applicants (e.g., research opportunities, housing options, operative volume, didactics, mentorship, support during the job search, graduated autonomy, support during dedicated research time, etc.)

Conclusion

The COVID-19 pandemic has dramatically altered how we recruit and interview applicants. The practice of inviting applicants to our hospital for interviews may now compromise the health and safety of our faculty, applicants, and patients. Furthermore, non-essential travel has been prohibited by our institution and similar policies have been implemented at other institutions in the United States and Canada. This new reality has forced us to adapt our interview process into a completely virtual interview experience. It is imperative that the virtual interview experience simulate the in-person interview process as much as possible. The virtual interview experience we have outlined may prove useful for future interviews for resident and fellowship training programs.