Perception of the Virtual Interview Format in Hand Surgery Fellowship Applicants.

PURPOSE: The Coronavirus disease 2019 pandemic occurred during the interview period for numerous surgical fellowships, resulting in most programs transitioning to a virtual interview format during the 2020-2021 application cycle. This study investigated modifications adopted by fellowship programs and perceptions of the virtual interview format among hand surgery fellowship applicants.
METHODS: Voluntary, anonymous online surveys were emailed to all applicants to the Johns Hopkins hand surgery fellowship during the 2020-2021 interview cycle. The surveys were released after the rank order list certification deadline on May 6, 2021, and closed on May 18, 2021, before the match results were released. Descriptive statistics based on the overall cohort and primary outcome of an applicant's willingness to recommend virtual interviews in the future were conducted.
RESULTS: Thirty-four of 112 (30.4%) applicants completed their surveys. Twenty-seven (79.4%) survey respondents recommended the virtual interview format in the future and 7 (20.6%) did not. Applicants who recommended virtual interviews were similar to those who did not on the basis of the number of interviews received and taken, information provided by programs, and self-rated competency with the virtual interview format. Those who recommended virtual interviews rated the effectiveness of self-advocacy higher compared with those who did not. All respondents agreed that cost savings and scheduling were more effective with virtual interviews. Perceived weaknesses differed between the 2 groups and included the lack of physical tour, difficulty with self-advocacy, and technical difficulties. The majority of survey respondents preferred in-person interviews before the interview cycle (n = 32, 94.1%), whereas nearly half of survey respondents preferred virtual interviews after the interview cycle (n = 16, 47.1%).
CONCLUSIONS: Nearly 80% of survey respondents recommended virtual interviews in the future. Major benefits included effectiveness of scheduling and cost savings. CLINICAL RELEVANCE: Virtual interviews may be considered as an alternative or adjunct to in-person hand surgery fellowship interviews in the future.

In response to the COVID-19 pandemic, the Association of American Medical Colleges strongly encouraged teaching hospital faculty to conduct interviews by telephone or virtually for the 2020–2021 residency application cycle. In addition to local travel restrictions, residency and fellowship program directors were forced to use web- and video-based platforms to conduct applicant interviews.

Interviews represent one of the most important aspects of the application process from the perspective of program directors and applicants in creating their rank list. , Traditionally, residency and fellowship training programs host in-person, on-site interviews to allow prospective trainees to interact with residents and staff, tour the facility, and visit the surrounding area. For these reasons, a recent survey administered to medical students and residents demonstrated that both groups favor the in-person interview format compared with the virtual interview format under normal circumstances. In particular, trainees have expressed concern that virtual interviews do not facilitate an accurate representation of themselves compared with in-person interviews. ,

Before this application cycle, a limited number of graduate medical education residency and fellowship programs had published their experience with the virtual interview format.7, 8, 9, 10 For urology residency applicants, a single-site cross-sectional study demonstrated that applicants perceived web-based interviews as less effective than traditional on-site interviews. However, an adult orthopedic reconstruction fellowship program reported that 85% of survey respondents believed that videoconference interviews provided a satisfactory understanding of the fellowship program. Given the conflicting existing literature regarding the efficacy of virtual interviews from the applicant’s perspective, additional research is needed to understand their role in the hand surgery fellowship application process.

The purpose of this study was to understand the modifications adopted by fellowship programs, identify perceived strengths and weaknesses of the virtual interview format during the 2020–2021 hand surgery fellowship application cycle, and compare differences between those who would recommend the virtual interview platform to prospective applicants with those who would not. We hypothesized that applicants who are willing to recommend virtual interviews in the future would differ in their perceptions of the strengths and weaknesses of the virtual format compared with those who did not.

Materials and Methods

All 112 applicants to the Johns Hopkins hand surgery fellowship were identified and invited to participate in a survey regarding their perspectives on the virtual interview format using Qualtrics software. The study was reviewed and exempted from the university’s institutional review board. The survey was distributed via email on May 6, 2021 after the rank order list certification deadline. A reminder email was sent on May 17, 2021, and the survey closed on May 18, 2021, the day before match results were released. These dates were chosen to minimize bias related to rank list creation and match results. The full-length survey is available in Appendix 1 (available online on the Journal's website at www.jhandsurg.org).

The survey gathered information on applicant demographics including how many programs applicants applied to, the number of interviews they received and attended, and the format of those interviews. Applicants were asked to rate their competence with virtual video communication platforms and how program information was distributed before the interview date. Information was gathered on virtual interview logistics including the number of interviews at each program, days of the week they occurred, and the virtual platform used most frequently. Information about the interview experience was gathered through questions about any technical difficulties that were encountered, including video issues, audio issues, difficulty navigating the platform, connection issues, and scheduling issues. Survey respondents were asked to rate the perceived detriment of not being able to meet residents/fellows and support staff, not being able to tour the facilities physically, and not being able to see the surrounding area. Applicants were asked questions regarding the perceived effectiveness of certain elements of the virtual interview process, as well as its perceived strengths and weaknesses compared with an in-person interview process. Finally, applicants were asked about their preferred interview format before and after the interview cycle and whether they would recommend virtual interviews for future application cycles. Open text questions related to strengths, weaknesses, and recommended formats also were included.

Descriptive statistics were conducted including mean, median, and interquartile range for continuous variables and percentages were used to summarize categorical variables with respect to the overall cohort and the primary outcome.

Results

Demographic data

Of the 112 applicants emailed, 34 (30.4%) completed the survey. Summary statistics for all survey questions are presented in Tables 1 and 2 for continuous and categorical variables, respectively. Applicants reported applying to a mean of 41.8 programs, receiving a mean of 19.4 interviews, and completing a mean of 17.6 interviews, nearly all of which were virtual. The majority of applicants attended medical school in coastal cities (n = 24, 70.6%) compared with Midwest and international cities. Over 85% of applicants reported interviewing at multiple programs in a single day. Additionally, 15 (44.1%) of survey respondents reported interviewing at programs they were not considering seriously, of which 4 (26.7%) indicated using those interviews as practice.

Table 1

Summary Statistics for Applicants for Continuous Variables (n = 34)

Variable	Mean ± SD	Median (Q1, Q3)	Min, Max
No. of programs applied	41.76 ± 16.61	39.5 (30, 50)	12, 100
No. of interviews offered	19.41 ± 8.34	18.5 (12.25, 25)	6, 39
No. of interviews taken	17.62 ± 6.67	19 (12.25, 22.75)	5, 34
No.of virtual interviews	17.38 ± 7.07	18.5 (12.25, 22.75)	0, 34
Money spent during cycle	747.35 ± 787.71	390 (100, 1275)	0, 3000
Percentage of programs with a centralized meeting space	77.62 ± 24.80	83 (70, 100)	17, 100
Percentage of programs with a virtual tour	35.94 ± 24.70	30 (17.5, 50)	5, 100
Percentage of programs that provided virtual information before interview day	60.59 ± 24.53	54 (42, 77.75)	20, 100
Percentage of programs that provided physical information/ material before interview day	23.03 ± 17.95	20 (11, 25)	1, 80
Percentage of programs that provided any information session during the interview day	83.44 ± 19.22	85.5 (75, 100)	30, 100
Expectation of cost for in-person interview cycle	6097.06 ± 3357.80	5000 (3250, 9750)	0, 12000

Table 2

Summary Statistics of Applicants for Categorical Variables

Variable	Total (%) n = 34
Location of medical school by region
Coastal cities (east and west)	24 (70.59)
Midwest and outside of United States	10 (29.41)
Interviewed at programs they were not seriously considering
No	19 (55.88)
Yes	15 (44.12)
Reason for interviewing at programs not seriously considered
No disadvantage in taking more	7 (46.67)
Additional practice	4 (26.67)
Other reason	4 (26.67)
Total
Competence navigating virtual communication platforms before interview season
Below average	2 (5.89)
Average	18 (52.94)
Above average	14 (41.18)
Average number of dates available to schedule an interview
1	1 (2.94)
2	21 (61.76)
3	10 (29.41)
≥4	2 (5.88)
Average number of days the interview was conducted
1	18 (52.94)
2	10 (29.41)
3	5 (14.71)
≥4	1 (2.94)
Average number of interviews per program∗
1	1 (3.03)
2	0
3	5 (15.15)
≥4	27 (81.82)
Part of the week most interviews occurred
Weekday	4 (11.76)
Weekend	10 (29.41)
Equal	20 (58.82)
Most frequent interview structure
Single interviewer per interview	21 (61.76)
Multiple interviewers per interview	13 (38.24)
Most effective interview structure
Single interviewer per interview	22 (64.71)
Multiple interviewers per interview	12 (35.29)
Interviewed at multiple programs in a single day
No	15 (44.12)
Yes	19 (55.88)
Clinical patient images shown during virtual interview
No	11 (32.35)
Yes	23 (67.65)
Disclaimers made to protect patient privacy if patient images were shown
No	7 (30.43)
Yes	16 (69.57)
Most frequently required interview attire
Business	11 (32.35)
Business casual	0
Did not specify	23 (67.65)
Types of technical issues encountered
No issues	3 (8.82)
Connection	20 (58.82)
Platform navigation	6 (17.65)
Video	18 (52.94)
Audio	16 (47.06)
Scheduling	10 (29.41)
Other	3 (8.82)
Amount of technical difficulty experienced
None	3 (8.82)
A little bit	30 (88.24)
A moderate amount	1 (2.94)
A great deal	0
Where were interviews conducted∗
Home	26 (78.79)
Hospital	5 (15.15)
Other	2 (6.06)
Detriment of not being able to meet residents/ fellows and support staff
No detriment	0
A little bit	11 (32.35)
A moderate amount	15 (44.12)
A great deal	8 (23.53)
Detriment of not having a physical tour
No detriment	6 (17.65)
A little bit	16 (47.06)
A moderate amount	5 (14.71)
A great deal	7 (20.59)
Detriment of not seeing the surrounding area
No detriment	1 (2.94)
A little bit	22 (64.71)
A moderate amount	6 (17.65)
A great deal	5 (14.71)
Effectiveness at advocating/ conveying yourself virtually compared with in-person
Significantly less effective	0
Less effective	13 (38.24)
No difference	10 (29.41)
More effective	10 (29.41)
Significantly more effective	1 (2.94)
Effectiveness of cost savings virtually compared with in-person
Significantly less effective	0
Less effective	0
No difference	0
More effective	2 (5.88)
Significantly more effective	32 (94.12)
Effectiveness of scheduling virtually compared with in-person
Significantly less effective	0
Less effective	0
No difference	0
More effective	10 (29.41)
Significantly more effective	24 (70.59)
Effectiveness of learning about program virtually compared with in-person
Significantly less effective	0
Less effective	21 (61.76)
No difference	7 (20.59)
More effective	6 (17.65)
Significantly more effective	0
Effectiveness of learning about surrounding area virtually compared with in-person
Significantly less effective	10 (29.41)
Less effective	18 (52.94)
No difference	6 (17.65)
More effective	0
Significantly more effective	0
Effectiveness of meeting current residents/ fellows virtually compared with in-person
Significantly less effective	5 (14.71)
Less effective	22 (64.71)
No difference	4 (11.76)
More effective	3 (8.82)
Significantly more effective	0
Effectiveness overall of virtual format compared with in-person
Significantly less effective	0
Less effective	11 (32.35)
No difference	11 (32.35)
More effective	12 (35.29)
Significantly more effective	0
Strengths of virtual interview
Ability to participate in more interviews	29 (85.29)
Less formal	5 (14.71)
Ease of scheduling	30 (88.24)
Cost	34 (100)
Other	1 (2.94)
Weakness of virtual interview
Lack of physical tour	13 (38.24)
Difficult to advocate self or convey things effectively	13 (38.24)
Less intimate/ personal	29 (85.29)
Technical difficulties	16 (47.06)
Other	4 (11.76)
Preferred interview format before interview season
In-person	32 (94.12)
Virtual	1 (2.94)
Other	1 (2.94)
Preferred interview format after interview season
In-person	15 (44.12)
Virtual	16 (47.06)
Other	3 (8.82)

Denotes question in which total n = 33 due to a missing response.

Perceived strengths and weaknesses of the virtual interview platform for all respondents

When querying all survey respondents, the majority preferred in-person interviews before the interview cycle (n = 32, 94.1%), whereas nearly half of survey respondents preferred virtual interviews after the interview cycle (n = 16, 47.1%). In terms of strengths of the virtual interview, 34 (100%) of respondents selected cost, 30 (88.2%) selected ease of scheduling, and 29 (85.3%) selected ability to participate in more interviews. In terms of weaknesses of the virtual interview, 29 (85.3%) of respondents selected less intimate/personal, 16 (47.1%) selected technical difficulties, and 13 (38.2%) of respondents selected a lack of physical tour and difficulty with self-advocacy. No survey respondent felt that the virtual interview format was more effective than in-person interviews in learning about the surrounding area. On average, survey respondents expected to pay over $6,000 for the in-person interview cycle, but reported actually spending $747.35 during the virtual cycle.

Modifications adopted by hand surgery fellowship programs

From the program perspective, 33 (97.1%) of programs had ≥2 dates available to schedule an interview. The most frequent interview structure was a single interviewer (n = 21, 61.8% of respondents) and the majority of respondents (n = 22, 64.7%) felt the single interviewer structure was most effective. On the day of the interview, only 35.9% of programs provided a virtual tour and 83.4% of programs provided an information session.

Grouped analysis based on primary outcome

Twenty-seven (79.4%) survey respondents recommended the virtual interview format for prospective applicants and 7 (20.6%) did not. These 2 groups were similar based on the number of interviews received and taken, information provided by programs, and self-rated competency with the virtual interview format. Those who recommended virtual interviews in the future rated the detriment of not being able to meet residents/fellows and support staff lower compared with those unwilling to recommend virtual interviews. Those unwilling to recommend virtual interviews rated the effectiveness of advocating/conveying themselves virtually lower compared with the willing to recommend group. The perceived weakness of the virtual interview format varied between the 2 groups with those not willing to recommend virtual interviews most commonly rating a lack of a physical tour and difficulty in advocating for themselves as weaknesses, whereas technical difficulties were the most cited weakness among those who did recommend the virtual interview format in the future (Table 3 ).

Table 3

Summary Statistics by Applications Who Would Recommend Virtual Interviews

Variable	NoN = 7	YesN = 27
Types of technical issues encountered
No issues	1 (14.29)	2 (7.41)
Connection	2 (28.57)	18 (66.67)
Platform navigation	3 (42.86)	3 (11.11)
Video	5 (71.43)	13 (48.15)
Audio	4 (57.14)	12 (44.44)
Scheduling	3 (42.86)	7 (25.93)
Other	0	3 (11.11)
Detriment of not being able to meet residents/fellows and support staff
No detriment	0	0
A little bit	0	11 (40.74)
A moderate amount	6 (85.71)	9 (33.33)
A great deal	1 (14.29)	7 (25.93)
Effectiveness at advocating/conveying yourself virtually compared with in-person
Significantly less effective	0	0
Less effective	5 (71.43)	8 (29.63)
No difference	2 (28.57)	8 (29.63)
More effective	0	10 (37.04)
Significantly more effective	0	1 (3.70)
Effectiveness overall of virtual format compared with in-person
Significantly less effective	0	0
Less effective	4 (57.14)	7 (25.93)
No difference	2 (28.57)	9 (3.70)
More effective	1 (14.29)	11 (40.74)
Significantly more effective	0	0
Weaknesses of virtual interview
Lack of physical tour	6 (85.71)	7 (25.93)
Difficult to advocate self or convey things effectively	6 (85.71)	7 (25.93)
Less intimate/personal	6 (85.71)	23 (85.19)
Technical difficulties	4 (57.14)	26 (96.30)
Other	0	4 (14.81)
Preferred interview format after interview season
In-person	6 (85.71)	9 (33.33)
Virtual	1 (14.29)	15 (55.56)
Other	0	3 (11.11)

Other

Free responses regarding ways to change the virtual interviews in the future were mixed but most felt more information about the program and surrounding city as well as smaller group question sessions would have improved the experience. When asked in a free response question what the most important factors are when forming a rank list, 8 of the 34 (23.5%) survey respondents listed “fit” as the most important factor.

Technical issues were common, with 30 (88.2%) of the respondents stating that they experienced a small degree of technical difficulty during the interview process. The top 3 technical issues encountered by applicants included connection issues (n = 20, 58.8%), video issues (n = 18, 52.9%), and audio issues (n = 16, 47.1%).

Discussion

This study evaluated the perception of virtual hand surgery fellowship interviews among applicants during the COVID-19 pandemic. While 94.1% of applicants preferred the in-person interview process before the interview cycle, 44.1% preferred the in-person interview after the interview cycle. With further optimization, virtual interviews may become a mainstay in the surgical fellowship interview process.

The increased flexibility of virtual interviews has highlighted an issue many programs were facing before the pandemic; that is, the number of applicants has continued to increase despite a limited number of interview spots. According to our survey, the average number of interviews offered and accepted by survey respondents were 19.4 and 17.6, respectively. While the number of interviews offered is consistent with previous literature, the number of interviews taken by our survey respondents is higher than reported previously. A study from 2015 found that the majority of hand fellowship applicants accepted 12 interviews and often had to cancel an interview because of another interview conflict. Previous studies have demonstrated that competitive applicants hoard interviews, using some as practice.12, 13, 14 Our survey respondents unanimously rated virtual interviews as more effective based on ease of scheduling and allowed 19 (55.9%) of survey respondents to attend >1 program interview in a day. Nearly half of our survey respondents endorsed interviewing at programs they were not considering seriously, of which over a quarter reported using those interviews as practice. The virtual interview platform may worsen interview hoarding by competitive applicants by decreasing financial and scheduling constraints. Several groups have suggested ideas to decrease the practice of interview hoarding, including encouraging programs to open more interview slots and capping the number of programs an applicant can apply to and attend. , Further research is likely needed to better understand how to create a fair and diverse applicant pool.

When stratifying survey respondent results based on their willingness to recommend virtual interviews in the future, we found similarities between the groups based on demographic information or interview logistics, and differences in the perceived strengths and weaknesses of the virtual interview process. The perceived effectiveness of virtual interviews in advocating and conveying oneself was lower in the group unwilling to recommend virtual interviews. This is consistent with the literature demonstrating trainee concerns about conveying themselves virtually. In addition, the unwilling to recommend group rated virtual interviews as having a greater detriment on being able to meet a program’s residents and fellows. Finally, there were differences in perceived weaknesses between the 2 groups; a lack of tour and difficulty with self-advocacy were rated as weaknesses more often in the unwilling to recommend group, whereas those willing to recommend virtual interviews cited technical difficulties as a weakness more often. Perceiving lack of a tour and difficulties with advocating for oneself as a weakness may speak to applicants’ desire to find a professional and social fit at a program. Although “fit” remains a poorly defined term, previous studies have associated happiness, program collegiality, and faculty relationships with the term and may represent areas that are particularly difficult to assess in a virtual interview process. ,

Cost savings was a perceived strength of the virtual interview process across all survey respondents. Before the COVID-19 pandemic, the economic and clinical burden associated with the surgical fellowship interview process was being examined. In 2014, a survey of 129 orthopedic surgery residents demonstrated an average cost of $5,875 for travel and 11 missed residency training days to attend fellowship interviews. Similarly, in 2017 a survey of general surgery fellowship applicants demonstrated that 57.7% of residents missed ≥7 days of clinical training to attend interviews, 62.3% spent over $4,000 on the interview process, and 57.3% of residents were in favor of change in the interview process. Consistent with other groups that have published on the benefits of virtual interviews, we found candidates spent on average $747 for the entirety of the virtual interview cycle, lower than the average in-person costs reported in the literature. The perceived weaknesses associated with the virtual interview may be tempered by their cost savings and ability to reduce disruption to applicants’ clinical duties.

There were several limitations to this current study, including its single-site nature, although applicants to the Johns Hopkins hand surgery fellowship program were professionally and geographically diverse. With a response rate of 30.4%, most participants in the interview cycle were not captured in our survey results, which introduces concerns regarding response bias and the overall generalizability of findings. Additionally, this study may be limited by recall bias because survey respondents had just completed virtual interviews and their in-person interview experiences were likely more remote. However, the logistical constraints created by the COVID-19 pandemic created a situation in which virtual and in-person interviews could not be compared more directly. Finally, this survey was sent intentionally to applicants after the rank order list certification deadline and closed before match day to minimize bias related to match results. Therefore, this study does not capture perceptions of the virtual interview format based on match results of applicants. It is possible that perceptions would change based on match outcomes.

While COVID-19 travel restrictions have lifted gradually, the pandemic forced a radical change in the graduate medical education residency and fellowship interview process. It is unclear whether interviews in upcoming application cycles will be in-person or virtual, but one alternative is a hybrid approach. Based on the findings of a survey sent to maternal fetal medicine applicants, 58.7% preferred a hybrid approach combining in-person with virtual interviewing. Similarly, a survey sent to medical student applicants revealed that 71% of them believed that virtual interviews should be an option for future applicants. The results of this study may be used to further strengthen the role of virtual interviews as an adjunct or alternative to in-person interviews, particularly with optimizing the interviewee’s experience to help identify fit, improve virtual tours, and allow candidates to advocate for themselves. Mailing brochures and program information before the interview day, creating digital content for applicants to view during interviews, and increasing the amount of time current fellows and residents spend interacting with prospective applicants are the possible first steps to addressing applicant concerns about virtual interviews and optimizing the process for future years.