Brave New World: Challenges and Opportunities in the COVID-19 Virtual Interview Season.

The COVID-19 pandemic has forced the transition of the traditional residency interview to a virtual format. This new interview format creates additional challenges and opportunities for both programs and applicants. The specific challenges of the virtual interview format are described, as well as means to mitigate those challenges. In addition, opportunities to improve residency selection from the program end are described.

INTRODUCTION

The COVID-19 pandemic has had a great impact on all facets of health care in the United States, and the long-term impacts of the pandemic have yet to be realized (1,2). What is well established is that the pandemic has disrupted professional and personal lives of trainees (3, 4, 5). In addition, the clinical, educational, and scholarly missions of residency programs have been upended; COVID-19 has changed staffing, postponed patient interactions, changed interpretations from on-site to off-site, and required redeployment of a significant number of residents (3, 4, 5, 6).

Also deeply impacted have been US medical schools and their students, given the widespread uncertainty in all facets of medical education (7,8). The Association of American Medical Colleges (AAMC) stated in March 2020 that “unless there is a critical healthcare workforce need locally” students’ clinical rotations should be suspended (8). Temporary suspension of administrations of the United States Medical Licensing Examination (USMLE) occurred (9). These changes resulted in delay of required clinical rotations and licensure examinations. Consequently, the AAMC, which administers the Electronic Residency Application Service (ERAS), delayed the timeline for residency application submission and availability of documents to programs (10).

In addition to the delayed timeline, a work group within the Coalition for Physician Accountability, to which the AAMC and the Accreditation Council on Graduate Medical Education are party, recommended that the ERAS 2020 timeline be shifted, away rotations “be discouraged” except a few exceptional circumstances, and “all programs commit to online interviews and virtual visits for all applicants, including local students, for the entire [2020-2021] cycle and that the medical education community commit to creating a robust digital environment and set of tools to yield the best experiences for programs and applicants” (11). Further, on May 21, 2020, the Association of Program Directors in Radiology and Association of Program Directors in Interventional Radiology issued a position statement endorsing virtual interviews for all residency applicants to diagnostic radiology and interventional residency programs (12).

As these are uncharted waters, the forthcoming online interviews and virtual visits raise additional issues to resolve and questions to answer:

How can the applicant present themselves in the best possible light?

How can the program assess the candidate?

How can the virtual applicant best learn about the residency program and its environment?

How can the residency program best portray its opportunities and optimize the interview experience?

We seek to answer these questions and provide guidance for applicants and programs in advance of the forthcoming residency application cycle in the following sections.

HOW CAN THE APPLICANT PRESENT THEMSELVES IN THE BEST POSSIBLE LIGHT?

Each year, beyond the usual ERAS application completion and interview experiences comes the need for radiology residency applicants to compete against a particularly talented group of medical students in extremely competitive fields. Diagnostic radiology and interventional radiology are consistently among the most competitive of specialties (13).

To complete ERAS applications in the best manner each year, each applicant must present themselves in many domains; one extremely helpful tool is the AAMC fillable PDF version of the MyERAS Application, an interview season tool intended to be used by applicants to prepare their applications (14).

This year brings the added stressors of the pandemic-required virtual interview season. A number of additional factors will have an impact. USMLE Step 2 scores will not be as readily available for many students based on exam cancellations beyond their control (9). Both students with lower Step 1 scores and international medical graduates may be particularly impacted in this setting. Faculty advisers must advise applicants differently this year, to include specific recommendations outlined in Table 1 .

Table 1

Recommendations for Applicants

Build and submit a robust ERAS application, to include a well-crafted personal statement and interesting hobbies and interests (14). Individualized personal statements should be considered for programs of keen interest.

Curate and cull existing virtual footprints across all digital platforms to present a professional demeanor.

Research online desired and targeted radiology residencies.

Apply to an appropriate, not excessive, number of programs. Anecdotally, medical schools are concerned that the decreased travel costs and time will result in increased application numbers.

Check programs’ interview schedules and assure these do not conflict with other commitments.

Assure available technology is adequate (e.g., internet connectivity, hardware, and software should include all needed videoconference downloads).

Practice virtual interviews by:

1. Anticipating questions based on typical queries and based on details of an individual's ERAS application and preparing and practicing those questions. 2. Practicing with the likely software platforms. It is helpful to download apps and learn the nuances of each platform. 3. Preparing program-specific questions to ask interviewers.

Although there is little refereed literature on virtual applicant presentation, many tips are provided in online resources (15, 16, 17, 18, 19). Chief among the tips for applicants in these online videos are the items in Table 1 and added simple strategies provided to improve virtual interview performance; sound, location, lighting, and applicant appearance tips are specifically outlined. Among the provided tips are to select a quiet location where there will be no disturbances by family, friends, or pets; to turn off all device notifications; to avoid back lighted or distracting backgrounds, or use a neutral “virtual background”; and to dress similarly to a live interview setting. The AAMC has also provided a newly published helpful virtual interview application guide for applicants (20).

HOW CAN THE PROGRAM ASSESS THE CANDIDATE?

The need for residency programs to reconstruct their interview processes to accommodate a virtual platform offers unique opportunities for improvements; the virtual platform can be leveraged to create unique experiences for radiology residency applicants. Yet, there remain challenges for residencies to overcome.

Traditional assessment of radiology candidates is both objective and subjective. Objective criteria include USMLE board exam scores, clerkship grades, class rank, faculty letters of recommendation, and the dean's letter. Programs must be mindful that ERAS entries are likely to be impacted this year because fourth year students have not had the opportunity to complete the home and away rotations typically crucial in obtaining letters of recommendation. These should be taken into account when evaluating portfolios as recommended by the AAMC, including reducing the number of recommendation letters required (21). USMLE Step 2 scores may not be available due to exam cancellations and delays (9). Beyond these, if the number of applications increases as anticipated, programs face additional challenges in screening applicants to interview and determining if, in fact, candidates are genuinely interested in the program. Residencies are likely to require a new rubric for both screening applications and scoring applicants in this interview cycle.

Subjective assessment criteria that programs traditionally rely on in an in-person interview setting are changed, specifically interpersonal and communication skills, personality, emotional intelligence and overall “fit” (21). Modifications are required to permit subjective assessment of applicants in the remote interview setting. However, if done appropriately, much of the subjective assessment can resemble and potentially improve the in-person experience.

Conducting interviews over web conferencing allows departments to expand the number and type of individuals who participate in the interview day. While in-person interviews are limited to immediate faculty and their availability within their clinic schedules, academic time and vacation, virtual interviews are able to leverage a larger pool of interviewers with residency alumni, retired faculty, and offsite residents and faculty. Many of these may even have more experience in conducting interviews than members of the immediate faculty, and higher quality candidate assessments may ensue. In addition, candidates may feel a larger sense of community as they connect with members outside the department and all around the country.

There are more opportunities for “second look” interviews when employing a virtual platform. Both the program and the candidate can schedule follow-up remote interviews to gather additional information, answer questions that arise thereafter, or reschedule those interviewers unavailable on the initial interview day.

The residency program must acknowledge, and hazard against, the potential for unconscious bias due to the virtual interview setting. There are inherent differences among candidates in videoconferencing capabilities. Internet access and speeds, web conferencing technology, background setting appearance, and uncontrollable distractions are just a few of the ways in which candidates could unfairly be judged in the virtual interview process. Not all candidates will have reliable, high speed internet, state-of-the-art videoconferencing tools, and a perfectly lit room absent of distractions. In addition, some candidates attend medical schools whose resources can better prepare students for virtual interviewing, some of which may offer optimized facilities to conduct interviews. These applicants must not be viewed more favorably than those with lesser resources. Care must be taken to prevent unfairly influenced judgment in assessment of candidates.

A highly valued aspect of the interview season are social events (22). Social events provide applicants and residents with an informal environment to get to know one another, and for applicants to learn more about the department. Additionally, social events give both parties an opportunity to assess likeability and overall cultural fit. These social events can attempt to be recreated online with videoconferencing “virtual happy hours” now commonplace during the stay-at-home COVID-19 era. The online format potentially allows for larger resident attendance to meet interviewees by answering questions, and in turn, more robust resident feedback for applicant assessment. Zoom (Zoom Video Communications Inc, San Jose, California) is a commonly used videoconferencing platform and utilizing Zoom tools such as “breakout rooms” to put candidates into smaller groups with residents who have similar interests allows for more personalized experiences per applicant.

Virtual interviews allow for unique social events, a greater number and more experienced interviewers, and more opportunities for follow-up discussions. Program directors need not feel anxious about interviews being held virtually this year but, rather, focus on leveraging the strengths of the virtual platform to create unique experiences for radiology residency applicants.

HOW CAN THE VIRTUAL APPLICANT BEST LEARN ABOUT THE RESIDENCY PROGRAM AND ITS ENVIRONMENT?

It is imperative that applicants are offered high quality online access to program information before, during, and after the interview.

Larger academic programs typically have “brand appeal” and will appear on many students' applications and rank lists likely without these programs making significant additional effort.  These programs are found regularly on the US News & World Report Best Medical Schools Radiology list and are easily identified by students (23). However, for smaller and lesser known programs, this year may be significantly more challenging. These are the programs that students typically may rank due to their interpersonal interactions on the interview day and the general “vibe” of the program (24). Students may know little in advance about the surrounding area and opportunities for partner employment, schooling, housing and leisure activities. Interactions with faculty and residents in these programs prove deeply impactful and sway students. Conversely, some applicants may be unaware of program geographic and environment detractors without physically visiting. For example, one of the authors has had students report “I didn't know that there was so much snow/trees/cold in New Hampshire” (24).

Arguably, the measure of a residency program's recruiting success in ERAS 2020 and NRMP Match 2021 may be its department's virtual alacrity and access. Successful programs may be those able to provide more and better information on their websites.

Optimal virtual program information should be readily accessible and clearly identified on the website to include concise information about the curriculum and unique aspects of the program. This information should be presented in an attractive and welcoming format. Multiple short, less formal videos that engage the viewer are one example. These videos may include: short testimonials from prior residents, especially those who have distinguished themselves thereafter; residents and faculty highlighting what brought them to the program and personal stories interspersed with images of residents and faculty at work and play; local amenities and attractions to illustrate residents in enjoyable downtime personal time; a virtual light-hearted department tour to include any great resident lounges or educational amenities; a day-in-the-life of a resident video and short welcome videos by the program director and department chair.

This information should demonstrate the strengths of the program such as educational opportunities, elective time and away rotations, teaching and research opportunities, diversity and inclusion initiatives, and mentorship programs. It may include both program-unique (personalized) and institution public relations department-produced segments but should authentically represent the institution, program, faculty, residents, and area. Only by being honest and realistic will a program ensure matched residents are well informed and more like to be a good fit.

Other considerations are to leverage millennial-favored social media tools. The residency, program director, and individual resident leaders should maintain and active social media presence (25). Consider allowing students access to resident educational conferences, ideally outside of the interview day when they may be more relaxed and receptive.

HOW CAN THE RESIDENCY PROGRAM BEST OPTIMIZE THE INTERVIEW EXPERIENCE?

Significant preparation will be needed to optimize the applicant interview day. There have been concerns voiced among program directors during informal national discussions that the number of applicants needed to interview in order to fill during the Match will need to increase relative to prior years which may involve more interview days or more candidates per day.

To assure as optimal an experience as possible, programs may turn to successful large group interview experiences (26). Key points to emphasize include:

A carefully coordinated interview day. The interview day must be very carefully coordinated with strict scheduling with applicable web-conferencing links clearly identified. Frequent scheduled breaks must be included in each applicant's itinerary to avoid videoconference fatigue. To facilitate multiple concurrent virtual interviews within a department, an institutional or departmental subscription to a web-conferencing service is key; this prevents restrictions placed on the number of concurrent meetings. The use of “breakout rooms” permits a quick transition to applicant-faculty interviews following an initial group meeting. Each residency program should assign a central administrator to schedule and proctor the interview day grid. Measures to minimize the potential for security compromises (e.g., “Zoom bombing”) might include using the “waiting room” function of some software programs and rapidly rejecting and blocking uninvited attendees. It is recommended that all interviewers be familiar with how to do this or that conferences are monitored by an administrator who is. The videoconference platform of choice must be well-tested in advance, with simulation scheduling and practice interviews. This encourages faculty, graduate medical education (GME) personnel, and resident leader training, to include IT support preparation and availability on both practice and actual interview days (27). If programs are considering recording interviews for future review, permission must be given by both faculty and students. Students may find this very intimidating as it is not part of the usual interview process and generally recording is not recommended. Programs may wish to emphasize to students that they are NOT recording interviews.

Individualized applicant experience. The program being tolerant of student technical issues and allowing buffer time is certain to be noticed and appreciated by the applicants. One program offers the opportunity for each applicant to test and troubleshoot connectivity prior to the interview, and approximately 15% of applicants avail themselves of the offer (26). One way to further individualize the interview day is to offer small group sessions led by different faculty as an engaging manner of introduction. Individual student interviews are a must, which may be performed by one or two faculty in a session. Interviews should be no longer than 20 minutes and interviewers should strive to be friendly, open, genuinely interested in the student, diligent about making an effort to relax the applicant, and remain engaging despite interruptions in video feeds and incongruities between video and audio feeds (28). This benefits programs and applicants alike. The current extensive COVID-19 experiences of virtual teaching and social interactions is likely to help both students and faculty feel more comfortable during these interviews. If a student expresses specific interests, consider offering access to relevant faculty in a follow-up virtual session. Residency social media groups could be used as a means of students asking follow-up questions of faculty or residents.

The preinterview dinner, revisited. The preinterview dinner plays an important role in many applicant's recruitment as an opportunity for residents and students to interact informally. It often provides useful information for all parties. To replicate, consider a virtual informal “get to know you” party (social hour), and assign small student groups to small resident groups. Alternatively, senior residents may reach out to the applicants prior to the interview day. In each instance, the program must reassure applicants that faculty do not have virtual access, and that the social hour is indeed intended as strictly social. These social interactions may be better after the interview sessions rather than the typical scheduling to enable students to ask questions that arose during interviews.

Postinterview feedback. Seeking and receiving feedback following Match Day is likely to prove key in planning future years. It is possible that virtual interviews will become the new normal; an evaluation of what did and did not work in 2020 should prove beneficial to each program. An additional resource to explore is input from residents engaged in fellowship interviews, as well as those fellowship program directors, faculty and GME personnel who staged virtual interviews in March 2020 in the face of the pandemic.

CONCLUSION AND STEPS FORWARD

Arguably, the key component for success in the upcoming ERAS/NRMP residency recruitment cycle will be virtual readiness. Applicants programs must be ready. This has broad implications. In the coming months, all fourth-year medical students must become facile with professional live online communication. Their advisors and medical schools must provide resources to help them. Programs must update and modernize their online presence to provide candidates with as much information as possible about their educational system and clinical environment. Program faculty, residents and coordinators must also become facile with developing an online interview system that allows applicants and program personnel to interact and learn about each other. Much of the organizational burden for the virtual interview process is likely to fall to program coordinators. Programs and sponsoring institutions must provide support for program coordinators to become facile with the use of online platforms for conferencing.

Developing and implementing a sustainable online system for residency recruitment will require a tremendous amount of investment by all stakeholders, including medical schools, students, GME-sponsoring institutions, and GME programs. It will require an investment of time, effort, and talent. It will also necessitate an investment of money, because, while costs may be saved in some areas such as travel and food, costs will increase in others such as videography, script writing, and video editing.

Success in developing virtual readiness will also require open communication and collaboration in a variety of channels:

All stakeholders in residency recruitment must invest and collaborate in the coming months. They must all be willing to make mistakes and keep going and be open and willing to learn from each other. For the health and success of GME recruitment moving forward, all stakeholders must be willing to share experiences with one another so all may benefit from the change.