Applicant Preferences for Virtual Interviews: Insights from the 2020-21 Integrated Plastic Surgery Application Cycle.

BACKGROUND: The COVID-19 pandemic created an unprecedented residency application cycle that required the use of virtual interviews. The inaugural class of applicants participating in virtual interviews were surveyed concerning their preferences and suggestions for future application cycles.
METHODS: A survey was distributed to 349 individuals who applied for an integrated plastic surgery residency position at three institutions during the 2020-21 application cycle. Responses were analyzed to determine the most popular attributes of virtual interviews.
RESULTS: Response rate was 44%. Eighty-six percent of applicants valued having time to consider their interview offers before scheduling; however, nearly one-third felt the instructions provided by programs lacked clarity and did not abide by the American Council of Academic Plastic Surgeons guidelines. Eighty-two percent of applicants valued having a preinterview social, and most preferred smaller breakout rooms organized by themes. A short interview day (less than three hours) was associated with less applicant familiarity with the program, its people, and its location when compared with medium (three to seven hours) and long (more than seven hours) interview days (P < 0.001 for all). Eighty percent of applicants preferred an interview day where they were placed into interview rooms by a program coordinator.
CONCLUSIONS: The virtual interview format can be optimized in many ways to benefit both applicants and programs. The data presented in this article provide suggestions for future iterations of virtual interviews.

Takeaways

Question:

What is the ideal virtual interview format for plastic surgery applicants?

Findings:

Applicants from the 2020-21 Integrated Plastic Surgery application cycle were surveyed. Eighty-six percent of applicants valued having the standardized interview release date, but nearly a third felt that programs did not adhere to guidelines. Applicants preferred less than 1-hour-long pre-interview socials. Interview days lasting more than 3 hours were better at promoting applicant familiarity with the program, its people, and its location than short (less than 3 hours) days.

Meaning:

The views of the inaugural class of plastic surgery applicants partaking in virtual interviews should be considered when implementing virtual interviews in the future.

INTRODUCTION

The coronavirus disease 2019 (COVID-19) pandemic has dramatically altered the process of applying to integrated plastic surgery residency. Sixty percent of integrated plastic surgery applicants during the 2020–21 cycle felt that COVID-19 had significantly impacted their application, and nearly 20% had considered a career change.[1] In response to the pandemic, the Association of American Medical Colleges and the American Council of Academic Plastic Surgeons (ACAPS) announced that residency interviews would be held virtually for the 2020–21 application cycle.[2,3]

Given the unprecedented circumstances, programs adapted to find novel mechanisms to mimic in-person interviews and provide a seamless experience for both applicants and interviewers.[4] Applicants navigated these changes while prioritizing their needs, which were to perform well during the interview, find accurate information about the program, and most importantly, determine if they were a good “fit.” This nebulous perception of “fit” has historically been the most important factor in integrated plastic surgery applicants’ ranking of programs.[5] Programs have also previously relied heavily on interview day: a 2018 survey of plastic surgery program directors showed that the three highest rated factors in programs’ ranking of applicants were interactions with faculty during the interview, interpersonal skills, and interactions with residents on interview day.[6] Another survey of program directors suggests that interview performance was indicative of performance during residency, which is an important goal during the residency selection process.[7] As such, both parties benefit from optimizing the virtual interview format and preserving any modicum of in-person interaction that was previously so valuable.

In prior application cycles, applicants reported spending an average of $531 per in-person interview and a cumulative $6690 on interviews during a traditional application cycle.[8] In addition to saving money, applicants benefit from virtual interviews in other ways: overall convenience, minimal loss of clinical training, and limited use of vacation time.[9] However, virtual interviews impede direct, in-person communication, which led to applicants having less knowledge about the program, faculty, and residents when compared with applicants who interviewed in person.[10]

Although new, the virtual interview format has the potential to replace in-person interviews even after the COVID-19 pandemic has passed. To maximize the utility of this format, understanding which aspects of the virtual interview correlated best with applicant satisfaction is vital because this can provide useful guidance for future application cycles.

METHODS

The primary aim of this study was to assess how the virtual interview format impacted applicant familiarity with the residency program, faculty, and residents. Secondary aims were to identify virtual interview strategies that were most ideal from an applicant’s perspective and provide comments on how the format could be improved in future years. To do so, an anonymous, electronic questionnaire with branching logic and a maximum of 48 questions was administered (www.surveymonkey.com, Palo Alto, Calif.). The institutional review board at The Ohio State University approved the study with exemption (IRB # 2021E0187).

Applicants to integrated plastic surgery residencies at The Ohio State University, University of California San Diego, and Wake Forest University during the 2020–21 application cycle were recruited as participants. Three hundred and forty-nine potential participants were identified. An email including the purpose of the study and a link to the survey was distributed to all potential participants on February 24, 2021. The survey was then open for a period of 9 days and a reminder e-mail was sent on February 27, 2021.

Participants were first asked about their experiences with the interview invitation and scheduling process, and to what degree the aforementioned abided with the ACAPS policy of a standardized interview offer release date (December 4, 2020) and scheduling date (December 7, 2020).[2] Next, information on which virtual interview formats the applicants encountered and preferred was collected. Familiarity with the program, its people, and its location was assessed on a Likert scale [1 (not at all familiar) to 5 (extremely familiar)] after attending a pre-interview social (PIS) and after attending interview days of varying length. Lastly, participants were asked for their technical preferences and use of outside resources. No demographic information was collected to allow for participant anonymity.

For technical preferences, a “static” model referred to a virtual interview where applicants rotate through several virtual faculty rooms, each with unique meeting identifications (IDs). Before the interview day, applicants were provided with a list of “static” meeting IDs to join at pre-specified times. In a “dynamic” model, applicants and interviewers all joined one meeting and the PC “dynamically” paired faculty and applicants together at prespecified times using breakout rooms. At the end of the interview, the PC ended the breakout rooms and returned applicants to a larger meeting room for a short break before assigning them to the next interviewer’s breakout room. In a “reverse dynamic” model, the PC moved faculty interviewers around to applicant rooms.

Responses were analyzed to determine familiarity by virtual interview attribute using Kruskal-Wallis tests. Analyses were performed using R Statistical Software (version 4.0.2; R Foundation for Statistical Computing, Vienna, Austria) and a P value of less than 0.05 was considered significant.

RESULTS

A total of 155 responses were recorded for a response rate of 44%.

Interview Scheduling

The most commonly used methods for scheduling interviews were e-mailing the program coordinator (PC), followed by the Electronic Residency Application System (ERAS) scheduler, Thalamus, and Interview Broker (Fig. 1). Of these, e-mailing the PC and ERAS were the preferred modalities. Eighty-six percent of applicants valued having the weekend after the uniform interview release date on Friday, December 4 to consider their options before scheduling on Monday, December 7 (Table 1). However, 27% of applicants felt that the instructions they received from programs on the uniform interview release date did not adhere to ACAPS guidelines. When asked whether programs prevented applicants from scheduling interviews before the uniform scheduling date on Monday, December 7, 28% of applicants disagreed (Table 1).

Fig. 1.

Preferences for interview scheduling software. IVB, interview broker.

Table 1.

Applicant Experiences with Interview Scheduling

Variable	n (%)
I valued having the weekend to think about  interview options before the uniform scheduling date on Monday, 12/7
Yes	131 (86%)
No	4 (3%)
No preference	13 (9%)
Not applicable	4 (3%)
The instructions I received from the majority of programs  on Friday, 12/4 about the scheduling process adhered to the ACAPS guidelines for scheduling interviews
No	41 (27%)
Yes	109 (72%)
Not applicable	2 (1%)
The majority of programs prevented applicants from  scheduling interview dates/times before the ACAPS listed date of Monday, 12/7
Strongly disagree	8 (5%)
Disagree	35 (23%)
Neutral	18 (12%)
Agree	67 (44%)
Strongly agree	25 (16%)

Pre-interview Socials

Ninety-six percent of applicants reported attending a PIS the night before an interview. When asked whether PIS were valuable, 82% of applicants agreed (Fig. 2). After attending a PIS, applicants reported being most familiar with the people in a program. Applicants were least likely to be familiar with the program’s location when compared with the program and its people (P < 0.001, Fig. 2).

Fig. 2.

The value of preinterview socials.

During a PIS, 90% of applicants preferred smaller breakout rooms, 6% preferred one large breakout room, and the remaining 4% had no preference. Regarding content of PIS, 55% of applicants preferred rooms organized by a specific topic such as the program’s location, resident work–life balance, or rotations; 20% preferred rooms organized by resident PGY-level, 20% had no preference, and the remaining 23% preferred no organization at all. Although not initially queried, applicants suggested in free-text responses that PIS be no longer than one hour.

When asked which gift they preferred to receive before an interview, 49% preferred a care package containing snacks or program merchandise, 37% preferred a voucher for food, and 14% had no preference. Forty-six percent of applicants reported being paired with a resident contact or buddy before interview day. Among those who were paired with a buddy, 63% of applicants thought it was helpful, while 11% did not, and 26% had no preference.

Interview Day

Interview days ranged in length from short (less than three hours) to medium (three to seven hours) to long (more than seven hours). The majority of applicants attended interview days that were medium length (99%), followed by long (53%) and short (51%) (Table 2). Compared with medium and long interview days, short interview days resulted in applicants being less familiar with the program, its people, and its location (P < 0.001 for all; Fig. 3). Eighty-six percent of applicants preferred to have a shorter day with back-to-back interviews, whereas 8% preferred a longer day with interviews interspersed throughout the day (Fig. 4). Individual interviews lasted a median of 14.8 minutes, and applicants’ preferred length for interviews was similar, at 15.7 minutes (Fig. 4).

Table 2.

Applicant Experiences and Preferences for Interview Day

Variable	n (%)
Which interview format did you prefer for faculty  interviews?
1-on-1 with faculty	83 (58%)
Interviewing with multiple faculty at once	31 (22%)
No preference	30 (21%)
Overall during the interview day, did you prefer  to interview with all program faculty or a subset of program faculty?
All faculty	87 (60%)
Subset of faculty	24 (17%)
No preference	33 (23%)
If there were multiple faculty in one interview  room, which did you prefer?
Individual computers	74 (54%)
Conference room	29 (21%)
No preference	34 (25%)
When you had breaks between interviews, what  did you do during the majority of the breaks?
Breakout room with applicants	5 (4%)
Breakout room with residents and applicants	111 (77%)
Individual/personal unscheduled time	28 (19%)
When you had breaks between interviews, what  was your preferred activity during breaks?
Breakout room with applicants	12 (8%)
Breakout room with residents and applicants	50 (35%)
Individual/personal unscheduled time	74 (51%)
No preference	8 (6%)

Fig. 3.

Applicant familiarity with program, people, and location by interview day length. Short interview days lasted less than three hours; medium, between three and seven hours; and long, more than seven hours.

Fig. 4.

Preferences for length of interview day.

Sixty percent of applicants preferred to meet all faculty on interview day, while 17% preferred meeting a subset (Table 2). While interviewing with faculty, 58% of applicants preferred meeting one-on-one with faculty and 22% preferred multiple faculty members at once (Table 2). If there were multiple faculty members in an interview room, 54% of applicants preferred to have faculty at individual computers showing up on separate screens, whereas 21% preferred to have all faculty in one screen (Table 2).

The majority of applicants reported spending their breaks between individual interviews in breakout rooms with residents and applicants (77%, Table 2). However, 51% of applicants preferred to have breaks that allowed for personal or free time, and only 35% preferred breakout rooms with applicants and residents (Table 2).

Participants were also asked how they preferred to receive information about a program. Fifty-eight percent of applicants selected a live information session on the day of the interview, 33% preferred a prerecorded session provided in advance, and 8% had no preference.

Technical Preferences

The overwhelming majority of applicants preferred to use Zoom (93%), followed by no preference (4%), Thalamus (2%), and Webex (1%). The modality used to place an applicant in an interview room was also assessed. Eighty percent of applicants preferred the dynamic model, 8% a static, 8% a reverse dynamic, and the remaining 4% had no preference.

Resources

Ninety-seven percent of applicants reported having a reliable internet connection and interview room: 85% at home, 8% at medical school, and 4% at other locations such as a parent’s home or office. Forty percent of applicants were disconnected from a virtual interview, and 86% of them reported having a contingency plan such as contacting the PC or calling faculty interviewers at their pre-specified phone numbers.

All applicants reported using at least one resource to supplement their knowledge of programs. Most often, applicants used the 2020–21 plastic surgery applicant spreadsheet—an anonymous forum for applicants housed on Google spreadsheet—followed by home faculty, residents at other programs, residents at home program, podcasts, ACAPS-sponsored events, and other (Fig. 5). When asked about resources that would be helpful for applicants interviewing virtually in the future, most applicants reported wanting a video tour of the facilities, a map of the city with hospital locations, an interactive virtual tour of facilities or the city, and map of the hospital. In free-text responses, applicants appreciated the programs that provided a pre-recorded virtual tour of the city and hospital with a resident “tour guide.”

Fig. 5.

Additional resources. Fac, faculty; pod, podcasts; res, residents; sheet, Google Sheet for 2020–21 plastic surgery applicants.

DISCUSSION

Residency interviews are among the most important factors in resident selection.[6,11,12] For plastic surgery applicants, the interview day experience is the second most important factor in deciding where to rank a program.[5] Traditionally, casual and formal interactions during in-person interviews contribute to both the program and applicant’s sense of “fit.”[5-7,13,14] These highly valued in-person interactions were jeopardized in the wake of the COVID-19 pandemic and replaced by virtual interviews. This format represents a seismic shift in the way interviews are conducted, and although new, it presents an opportunity to rethink and optimize the residency selection process, which has historically been time- and cost-prohibitive.[8,15] The aim of this study was to provide programs with data on the inaugural class of applicants’ opinions on the virtual interview format and offer recommendations for how it can be optimized in future cycles.

The ACAPS policy of a standardized interview offer release date and scheduling date was used during this application cycle.[2] Since its inception, the uniform interview release and scheduling dates were widely appreciated by plastic surgery applicants.[16] In our survey, nearly nine in 10 applicants appreciated having the weekend to consider their invitations and create a schedule for interviews. However, almost one third of applicants felt that the programs’ instructions in the interview offer e-mail lacked clarity and did not adhere to ACAPS guidelines. Additionally, roughly one third of applicants reported that programs allowed scheduling to occur before the uniform scheduling date. This is consistent with data from last year, which demonstrated that only 13% of applicants felt the scheduling process was straightforward.[16] In free-text responses, many applicants reported that programs still used “first-come, first-serve” language in their initial invitation which set up a “prisoner’s dilemma” among applicants. Many respondents reported adhering to the ACAPS guidelines by waiting to schedule interviews on December 7, 2020, but they were left with sub-optimal options as other applicants had acted on the “first-come, first-serve” language and scheduled over the weekend. Identifying and providing ways to enforce guidelines and ameliorate the stress of scheduling interviews should be a focus for programs in future application cycles.

Prior studies of plastic surgery applicants have shown the quality of interaction with program directors and residents are pivotal in an applicant’s decision to rank a program.[7,13,14] In traditional application cycles, these interactions take place at an in-person PIS the night before an interview. During this virtual application cycle, nearly all applicants attended a virtual PIS, and 82% thought they were valuable. Despite the virtual format, PIS allowed applicants to become familiar with people and the program, but less familiar with the program’s location. To preserve some semblance of an in-person PIS, an overwhelming majority of applicants preferred to have smaller breakout rooms organized by a theme ostensibly allowing for more candid and organic interactions between applicants and residents.

Regarding the interview day itself, applicants were less likely to be familiar with the program, its people, and its location if interview day lasted less than three hours. However, most applicants also preferred to have a shorter interview day with back-to-back interviews. The ideal interview day length is long enough to allow applicants to acquaint with the program and its people, but short enough to preclude long periods between interviews, which cumulate to excessive screen time. The median preferred length of an individual interview was 16 minutes, which is marginally shorter than the 20 minutes reported in a survey of plastic surgery applicants during a traditional in-person application cycle.[13] To get to interview rooms, applicants overwhelmingly preferred the “dynamic” model wherein a third party, such as the PC, placed applicants in interview rooms. This preference was also observed among neurosurgery applicants who interviewed virtually: 80% of respondents reported that the dynamic model ran more seamlessly and was more time efficient.[17] The dynamic model allows applicants to focus on the interview itself, rather than on the logistics of getting in between rooms, which may lead to this preference. In between interviews, applicants prefer to have personal, free time or be in breakout rooms with co-applicants and residents. Nearly half of all applicants were disconnected during an interview, and a contingency plan was in place in 86% of cases.

All applicants used resources outside of interview day to learn more about programs. In the future, applicants believed having a video or virtual interactive tour of facilities is important. Maps of the hospital and the surrounding city were also deemed useful. Our results indicate that regardless of interview day length, applicants were least familiar with a program’s location. Curating resources that showcase a program’s location will be important in filling the gap in an applicant’s appraisal of a program because geography is an important component in an applicant’s decision of where to rank a program.[5,13]

There are several limitations to our study. First, we distributed this survey to an amalgamated list of applicants to three programs across the United States (one on the west coast, one in the midwest, and one on the east coast), and this may not represent the entire applicant pool. Another limitation is responder bias: we achieved a 44% response rate; however, due to the anonymity of the survey responses, we were unable to perform a nonresponder analysis.

Our survey was administered after the last virtual interview was conducted, but before rank lists were submitted by both programs and applicants. Although it is possible respondents were obscuring their true opinions due to concern that a negative answer could alter their match outcome, demographic data was not collected to preserve anonymity.

Furthermore, our study did not comment on the impact of virtual interviews on other aspects of resident recruitment, such as, the experience of faculty interviewers, money and time saved by applicants, and applicant perceptions of how participating in virtual interviews impacted their performance in the match. Further studies are needed to elucidate the impact of virtual interviews on these important topics.

CONCLUSIONS

Based on the opinions of the inaugural class of plastic surgery applicants interviewing virtually, we propose the following recommendations for future application cycles:

Programs should provide a clear and uniform interview invitation offer on the standardized ACAPS interview release date. Offers should provide the relevant information such as date and times of all interview options and PIS information. One scheduling method such as e-mailing with the PC or the ERAS scheduler should be used by all programs on the standardized ACAPS scheduling date. The scheduler (PC or ERAS) should not accept responses prior to the scheduling date. Programs still using “first-come, first-serve” language in their interview offer e-mails should be reported and required to reset their scheduling to adhere to the ACAPS scheduling date.

Pre-interview socials are most useful to applicants when they are no longer than an hour and have smaller breakout rooms organized by a theme such as a program’s location, resident work–life balance, or PGY level.

Interview days lasting more than three hours are best for promoting applicant familiarity with the program, its people, and its location. Having a shorter interview day with back-to-back interview day is preferred, as is having a “dynamic” model wherein the PC orchestrates the placement of applicants in interview rooms. A contingency plan should also be in place in the event that an applicant is disconnected from an interview.

Programs should consider providing a virtual or interactive tour of the hospital including any physical spaces that plastic surgery residents use daily, such as lounge/workspaces, cafeteria, wards, clinics, operating rooms, etc. Videos, maps, or images of nearby neighborhoods and the day-to-day commute may also help applicants envision themselves at a program.