Virtual Interviews for Sports Medicine Fellowship Positions Save Time and Money but Don't Replace In-Person Meetings.

Purpose: To understand the perceptions of program directors (PDs) and fellowship applicants regarding the virtual interview process for orthopaedic surgery sports medicine fellowship programs.
Methods: An anonymous online survey was distributed through the American Orthopaedic Society for Sports Medicine (AOSSM) to applicants and PDs of orthopaedic surgery sports medicine fellowship programs following the 2020-2021 fellowship application cycle.
Results: A total of 40 responses were received from PDs for a response rate of 47% (40 of 85) and 72 responses were received from applicants for a response rate of 27% (72 of 271). All of the surveyed PDs (40/40, 100%) agreed/strongly agreed that the applicant's interview carries significant weight in determining where an applicant is ranked on the match list. Fifty-eight percent (23 of 40) of PDs agreed/strongly agreed that virtual interviews negatively affected their personal connection with the fellowship interviewee. The presence of virtual interviews allowed 80% (57 of 71) of applicants to go on more interviews. Seventy-three percent (51 of 70) of applicants were able to save greater than $5,000 on travel expenses and 63% (25 of 40) of fellowship programs were able to save greater than $2,500 by conducting virtual interviews. Conclusions: Virtual interviews allowed fellowship programs and applicants to complete more interviews, but both PDs and applicants stated that interviewing in-person was important for applicants to meet faculty and tour the institution where they may be spending a year. In contrast, significant financial savings resulted due to the transition to virtual interviews. Finally, both PDs and applicants were in favor of having the option of interviewing virtually, suggesting that virtual interviews may continue to play a role in future application cycles. Clinical Relevance: This study may be valuable to fellowship programs that will continue to implement virtual interviews into future application cycles.

Orthopaedic surgery is becoming increasingly subspecialized, as the number of fellowship-trained orthopaedic surgeons has increased from 76% in 2003 to 90% in 2013. Commonly, orthopaedic surgery residents apply for fellowship during their postgraduate year 4 (PGY-4) and attend in-person interviews at their fellowship programs of interest. However, the coronavirus disease 2019 (COVID-19) (severe acute respiratory syndrome coronavirus 2) pandemic has disrupted the medical, educational, and professional landscape for medical students, residents, and fellows.2, 3, 4, 5 As a result, interviews for residency programs and clinical fellowships have transitioned from a traditional in-person format to an online or virtual format.6, 7, 8 This led to programs quickly developing their own interview process, materials, and expectations while applicants involuntarily adapted to an entirely new interview format in addition to dealing with the stress of fellowship interviews.

Previous studies have investigated the most important factors that programs search for when ranking an applicant., Specifically, Haislup et al. surveyed orthopaedic surgery sports medicine fellowship program directors (PDs) to better understand the factors that PDs consider when selecting prospective fellows. The authors found that the most important factor taken into account by PDs was the quality of the applicant interview. In a similar study, Baweja et al. also surveyed orthopaedic sports medicine fellowship PDs and found that the fellowship interview was the most important factor in ranking fellowship applicants. Thus, the change from a traditional in-person interview to a virtual interview may significantly affect the match process for both fellowship programs and applicants.

Although recent literature has evaluated the virtual fellowship interview process,,12, 13, 14, 15 there have been few studies that have specifically focused on the virtual interview process for orthopaedic surgery sports medicine fellowship programs. The purpose of this study was to understand the perceptions of both PDs and fellowship applicants regarding the virtual interview process for orthopaedic surgery sports medicine fellowship programs. The authors hypothesized that both PDs and fellowship applicants would prefer in-person interviews despite the considerable cost savings of virtual interviews.

Methods

An exemption was obtained from the Tulane University Biomedical Institutional Review Board (2021-001). Following the 2020-2021 fellowship application cycle, online surveys using Qualtrics (Seattle, WA; Provo, UT) were distributed through the American Orthopaedic Society for Sports Medicine to all orthopaedic surgery sports medicine fellowship applicants and PDs participating in the San Francisco Match (Appendix Tables 1 and 2, available at www.arthroscopyjournal.org). Follow-up e-mails were sent 2 and 4 weeks after the initial e-mail to encourage more participation.

Appendix Table 1

Orthopaedic Sports Medicine Fellowship Applicants’ Survey

1. Do you prefer in-person or virtual interviews?○ In-person ○ Virtual ○ No preference 2. How important is it for you to interview in-person?○ Very Important ○ Important ○ Neutral ○ Unimportant ○ Very Unimportant 3. How important is it that you meet the program’s faculty members in-person?○ Very Important ○ Important ○ Neutral ○ Unimportant ○ Very Unimportant 4. How important is it that you tour the program’s institution and facilities in-person?○ Very Important ○ Important ○ Neutral ○ Unimportant ○ Very Unimportant 5. Virtual interviews negatively affected my personal connection with the fellowship program.○ Strongly Agree ○ Agree ○ Neutral ○ Disagree ○ Strongly Disagree 6. Virtual interviews resulted in meeting fewer faculty members.○ Strongly Agree ○ Agree ○ Neutral ○ Disagree ○ Strongly Disagree 7. Once the COVID-19 pandemic is over, programs should give applicants the choice between either in-person or virtual interviews.○ Strongly Agree ○ Agree ○ Neutral ○ Disagree ○ Strongly Disagree 8. I would prefer to do a preliminary virtual interview at a program followed by an in-person interview.○ Strongly Agree ○ Agree ○ Neutral ○ Disagree ○ Strongly Disagree 9. Virtual interviews will negatively affect my ability to match at desired programs.○ Strongly Agree ○ Agree ○ Neutral ○ Disagree ○ Strongly Disagree 10. I was worried about computer technical difficulties during a virtual interview.○ Strongly Agree ○ Agree ○ Neutral ○ Disagree ○ Strongly Disagree 11. How much money did you save in total travel expenses by not having to go to in-person interviews?○ $1-$2,500 ○ $2,501-$5,000 ○ $5,001-$7,500 ○ $7,501-$10,000 ○ >$10,000 12. How many fellowship programs did you apply to this year?○ 1-20 ○ 21-40 ○ 41-60 ○ 61-80 ○ 81-100 13. How many interview offers did you receive?○ 0 ○ 1-5 ○ 6-10 ○ 11-15 ○ 16-20 ○ 21-25 ○ ≥26 14. How many virtual interviews did you attend?○ 0-5 ○ 6-10 ○ 11-15 ○ 16-20 ○ 21-25 ○ ≥26 15. Did the presence of virtual interviews cause you to apply to more programs?○ Yes ○ No ○ Maybe 16. Did the presence of virtual interviews allow you to go on more interviews?○ Yes ○ No ○ Maybe 17. Were you required by your residency program to take vacation days to attend virtual interviews?○ Yes ○ No ○ Not Sure 18. In which region of the country is your residency program located?○ Northeast – CT, MA, NH, NJ, NY, PA, RI, VT ○ Midwest – IN, IL, IA, KS, MI, MN, MO, NE, ND, OH, WI ○ South – AL, AR, DC, KY, LA, MD, MS, NC, OK, SC, TN, TX, VA, WV ○ West – AZ, CA, CO, HI, NV, NM, OR, UT, WA 19. What is your age?○ <25 ○ 26-30 ○ 31-35 ○ 36-40 ○ 41-45 ○ 46-50 ○ >50 20. Optional: With which gender do you most identify?○ Male ○ Female ○ Transgender Male ○ Transgender Female ○ Gender Binary Non-Conforming ○ Other ○ Prefer Not to Say 21. Optional: Are you of Hispanic, Latino, or Spanish origin?○ Yes ○ No ○ Prefer Not to Say 22. Optional: How would you describe yourself? (Check all that apply)○ White ○ Black or African American ○ American Indian or Alaska Native ○ Asian ○ Native Hawaiian or Pacific Islander ○ Other

Appendix Table 2

Orthopaedic Sports Medicine Program Directors’ Survey

1. Do you prefer in-person or virtual interviews?○ In-person ○ Virtual ○ No preference 2. How difficult was your program’s transition from in-person interviews to virtual interviews?○ Very Difficult ○ Difficult ○ Neutral ○ Easy ○ Very Easy 3. How important is it for you to interview applicants in-person?○ Very Important ○ Important ○ Neutral ○ Unimportant ○ Very Unimportant 4. How important is it that your applicants meet your faculty members in-person?○ Very Important ○ Important ○ Neutral ○ Unimportant ○ Very Unimportant 5. Did your program change the structure of the interview day given the virtual format? Select all that apply.○ More interviewees per interview date ○ Less interviewees per interview date ○ More time per individual interview ○ Less time per individual interview ○ More interview dates offered ○ Less interview dates offered ○ Structure did not change 6. How important is it that your applicants tour your institution and facility?○ Very Important ○ Important ○ Neutral ○ Unimportant ○ Very Unimportant 7. Conducting virtual interviews negatively affected my personal connection with the fellowship interviewee.○ Strongly Agree ○ Agree ○ Neutral ○ Disagree ○ Strongly Disagree 8. Once the COVID-19 pandemic is over, programs should offer both in-person and virtual interviews.○ Strongly Agree ○ Agree ○ Neutral ○ Disagree ○ Strongly Disagree 9. I would prefer to do a preliminary virtual interview with an applicant followed by an in-person interview.○ Strongly Agree ○ Agree ○ Neutral ○ Disagree ○ Strongly Disagree 10. If both in-person and virtual interview opportunities are available for applicants and an applicant chooses a virtual interview over an in-person interview, this would make the fellowship interviewee look less dedicated to the program.○ Strongly Agree ○ Agree ○ Neutral ○ Disagree ○ Strongly Disagree 11. The applicant’s interview carries significant weight for where they are ranked on the match list.○ Strongly Agree ○ Agree ○ Neutral ○ Disagree ○ Strongly Disagree 12. I am concerned that the virtual interview process will affect the quality of applicant that matches at my program.○ Strongly Agree ○ Agree ○ Neutral ○ Disagree ○ Strongly Disagree 13. I was worried about computer technical difficulties while conducting a virtual interview.○ Strongly Agree ○ Agree ○ Neutral ○ Disagree ○ Strongly Disagree 14. How much money did your program save in the interview season by offering virtual interviews?○ $1-$2,500 ○ $2,501-$5,000 ○ $5,001-$7,500 ○ $7,501-$10,000 ○ >$10,000 ○ $0, or virtual interviews increased cost 15. How many applicants does your program typically interview in a given year?○ 1-20 ○ 21-40 ○ 41-60 ○ 61-80 ○ 81-100 ○ >100 16. Virtual interviews have allowed you to interview more applicants.○ Yes ○ No ○ Maybe 17 What interview software/platform did your program use? (Free Response) 18. How long have you been the program director of your orthopaedic surgery sports medicine fellowship program?○ 0-4 years ○ 5-9 years ○ 10-14 years ○ 15-19 years ○ ≥20 years 19. How many orthopaedic sports medicine faculty members (including yourself) are in your orthopaedic surgery department?○ 1-5 ○ 6-10 ○ 11-15 ○ 16-20 ○ 21-25 ○ 26-30 ○ >30 20. In which region of the country is your program located?○ Northeast – CT, MA, NY, PA, RI ○ Midwest – IL, IA, KS, MI, MN, MO, OH, WI ○ South – AL, FL, GA, KY, LA, MD, MS, NC, SC, TN, TX, VA ○ West – AZ, CA, CO, NV, NM, UT 21. What is your age?○ ≤40 ○ 41-45 ○ 46-50 ○ 51-55 ○ 56-60 ○ 61-65 ○ 66-70 ○ >70 22. Optional: With which gender do you most identify?○ Male ○ Female ○ Transgender Male ○ Transgender Female ○ Gender Binary Non-Conforming ○ Other ○ Prefer Not to Say 23. Optional: Are you of Hispanic, Latino, or Spanish origin?○ Yes ○ No ○ Prefer Not to Say 24. Optional: How would you describe yourself? (Check all that apply)○ White ○ Black or African American ○ American Indian or Alaska Native ○ Asian ○ Native Hawaiian or Pacific Islander ○ Other ○ Prefer Not to Say

Many items in the surveys were rated on a 5-point Likert scale. Questions in the survey distributed to the fellowship applicants included whether virtual interviews negatively affected their personal connection with the fellowship program, whether they would rather complete a preliminary virtual interview at a program followed by an in-person interview, and how much money they saved in total travel expenses as a result of the virtual format. PDs were asked if the structure of their interview day changed given the virtual format and whether programs should offer both in-person and virtual interviews once the COVID-19 pandemic ends.

Results

A total of 40 responses were received from PDs, for a response rate of 47% (40 of 85), and 72 responses were received from applicants for a response rate of 27% (72 of 271). One hundred percent of PDs (40 of 40) who responded agreed/strongly agreed that the applicant’s interview carries significant weight in determining where the applicant is ranked on the match list. Seventy-seven percent of PDs (30 of 39) and 65% (47 of 72) of applicants stated that they preferred in-person interviews, whereas 15% (6 of 39) of PDs and 29% (21 of 72) of applicants preferred virtual interviews (Fig 1). Seventy-five percent (30 of 40) of PDs and 68% (49 of 72) of applicants said it was important/very important to interview in-person. Eighty-five percent (34 of 40) of PDs and 67% (48 of 72) of applicants thought it was important/very important to meet their faculty members in-person. Eighty percent (32 of 40) of PDs and 67% of applicants (48 of 72) thought it was important/very important that applicants tour the institutions and facilities in-person.

Fig 1

Preferences of sports medicine program directors (A) and applicants (B) for in-person and virtual interviews.

Fifty-eight percent (23 of 40) of PDs agreed/strongly agreed that conducting virtual interviews negatively affected their personal connection with the fellowship interviewee, whereas 43% (31 of 72) of applicants agreed/strongly agreed that virtual interviews negatively affected their personal connection with the fellowship program. Forty percent (16 of 40) of PDs agreed/strongly agreed that they were concerned that the virtual interview process will affect the quality of applicant that matches at their program. When applicants were asked if virtual interviews will negatively affect their ability to match at their desired programs, only 29% (21 of 72) of applicants agreed/strongly agreed.

PDs were asked if the structure of their interview day changed, given the virtual format. Fifty-three percent (21 of 40) had more interviewees per interview day, 20% (8 of 40) had less time per individual interview, 18% (7 of 40) offered more interview dates, and 30% (12 of 40) did not change their structure (Fig 2). Sixty percent of PDs (24 of 40) stated that virtual interviews resulted in their respective program interviewing more applicants than usual, whereas the presence of virtual interviews allowed 80% (57 of 71) of applicants to go on more interviews. Fifty-two percent (37 of 71) of applicants stated that the presence of virtual interviews did not cause them to apply to more programs, whereas 45% (32 of 71) stated that it did.

Fig 2

Changes made by sports medicine fellowship programs to their interview process as a result of the coronavirus disease 2019 pandemic.

Sixty percent of PDs (24 of 40) agreed/strongly agreed that once the COVID-19 pandemic ends, programs should offer both in-person and virtual interviews. If both in-person and virtual interview opportunities are available for applicants and an applicant chooses a virtual over an in-person interview, 58% (23 of 40) of PDs agreed/strongly agreed that this would make the fellowship interviewee appear less dedicated to the program. Sixty-five percent of applicants (47 of 72) agreed/strongly agreed that programs should give applicants the choice between in-person or virtual interviews. When asked if they would prefer to do a preliminary virtual interview followed by an in-person interview, 20% (8 of 40) of PDs and 43% (31 of 72) of applicants agreed/strongly agreed.

When asked how much money their fellowship program saved in the interview season by offering virtual interviews, 33% of PDs (13 of 40) saved between $1 and $2,500, 28% (11 of 40) between $2,501 and $5,000, and 20% (8 of 40) between $5,001 and $7,500 (Fig 3A). When asked how much money applicants saved in total travel expenses through the virtual interview process, 17% (12 of 70) saved between $2,501 and $5,000, 27% (19 of 70) saved between $5,001 and $7,500, 26% (18 of 70) between $7,500 and $10,000, and 20% (14 of 70) more than $10,000 (Fig 3B).

Fig 3

Costs saved by orthopaedic sports medicine fellowship programs (A) and applicants (B) as a result of transitioning to virtual interviews.

Fifty-five percent (22 of 40) of PDs thought the transition from in-person to virtual interviews was easy/very easy and 84% (27 of 32) used Zoom as their interview software/platform. Thirty-three percent (14 of 40) of PDs and 58% of applicants (41 of 71) agreed/strongly agreed that they were concerned about computer technical difficulties while conducting a virtual interview.

Discussion

Based on the results of this study, the interview process for applicants of orthopaedic sports medicine fellowship programs is critical, as 100% of the surveyed PDs agreed/strongly agreed that the applicant’s interview carries significant weight in determining where the applicant is ranked on the match list. This study was performed to reflect on the recent transition of fellowship interviews from in-person to virtual due to the COVID-19 pandemic. Interviewing virtually may have significantly impacted the match process as 58% of PDs agreed/strongly agreed that virtual interviews negatively affected their personal connection with the fellowship interviewee and 40% agreed/strongly agreed that they were concerned that the virtual interview process will affect the quality of applicant that matches to their program. This may have led fellowship programs to rely more heavily on letters of recommendation or personal phone calls as previous studies have shown the importance of personal connections in the match process., Yong et al. assessed the perspectives of fellowship directors and applicants regarding the virtual interview process for orthopaedic surgery trauma fellowships. The authors found that 75% of fellowship directors stated that virtual interviews limited their ability to familiarize themselves with an applicant and only 50% reported that they were comfortable ranking an applicant after a virtual interview. In contrast, Vadi et al. compared match rates at an anesthesiology residency program between applicants who completed in-person interviews versus those who completed virtual interviews. The authors found no significant difference in match rate between the groups.

In this study, a large proportion of PDs and applicants stated that it was important/very important to interview in-person, to meet the faculty members in-person, and to tour the institution in-person. Without being able to visit the program in-person, the applicant may not have a sense of the program culture, resources available, and whether or not they will enjoy living in the area for a year., Similarly, Chandler et al. surveyed residents applying to a pediatric surgery fellowship at a single institution and found that only 35% believed that virtual interviews allowed them to decide if the program was the right fit for them. Bamba et al. surveyed applicants who interviewed at a plastic surgery residency program and found that those who interviewed in-person felt more acquainted with the program, faculty, and residents more than those who interviewed virtually.

Regarding future fellowship application cycles, 60% of PDs agreed/strongly agreed that programs should offer both in-person and virtual interviews and 65% of applicants agreed/strongly agreed that applicants should be given the choice between in-person and virtual interviews. Robinson et al. found that 79% of cardiothoracic surgery fellowship PDs and 55% of applicants agreed/strongly agreed that virtual interviews should be offered in the future, yet 85% of PDs and 80% of applicants agreed/strongly agreed that virtual interviews should be offered with the option of an in-person interview. Interestingly, if both in-person and virtual interviews were offered and an applicant chose a virtual over an in-person interview, 58% of PDs agreed/strongly agreed that this would make the interviewee appear less dedicated to the program. This may create a bias in favor of applicants who can afford to travel to in-person interviews. Although PDs and applicants believe that both in-person and virtual interviews should be offered in the future, only 20% of PDs and 44% of applicants agreed/strongly agreed that they would prefer to complete a preliminary virtual interview followed by an in-person interview.

One benefit of the virtual interview process is cost savings. More than 80% of applicants were able to save greater than $5,000, whereas 60% of fellowship programs were able to save greater than $2,500 (Fig 3). Before the global pandemic, a study by Oladeji et al. surveyed orthopaedic surgery residency PDs and residents regarding the in-person fellowship interview process. The authors found that residents spent an average of 11 days away from their residency programs and $5,875 on total travel costs. A separate study performed a literature review of the costs of attending surgical fellowship interviews and found that applicants saved close to $6,000 in travel expenses with virtual interviews. Finally, Vining et al. surveyed residents applying to surgical oncology fellowships and found that respondents favored the cost and time savings, increased efficiency, and decreased stress related to travel as benefits of virtual interviews.

Efficiency is another benefit of the virtual interview process, as 53% (21 of 40) of fellowship programs had more interviewees per interview day and 60% (24 of 40) interviewed more applicants than usual. In addition, interviewing virtually allowed 80% (57 of 71) of applicants to go on more interviews. Even though virtual interviews may affect the personal connection between the interviewee and interviewer, and the ability of the applicant to get a feel for the program, it allows applicants to interview at more programs and programs to interview more applicants. This may benefit the more qualified applicants, as they may be able to attend more interviews and potentially take away interview opportunities from applicants who are less qualified. Regardless, 207 of 222 (93%) positions were filled across the participating sports medicine fellowship programs, which was similar to previous years. Bates et al. provided perspectives on the virtual interview process for radiation oncology residency programs and stated that virtual interviews led to increased efficiency by allowing residency programs to interview more applicants per day while allowing faculty to maintain their clinical duties. In addition, the reduced costs of the virtual interviews allowed applicants to attend more interviews and fellowship programs to interview more total applicants.

Limitations

There are limitations to this study, including the relatively low response rate from both PDs and applicants. Thus, this study may not reflect the perceptions of all orthopaedic sports medicine fellowship PDs and applicants. In addition, the sample size was small, as this was a cross-sectional study and PDs and applicants were surveyed following only one application cycle. Recall bias may have played a role in survey responses, as surveys were completed after the fellowship match rather than in the middle of the interview season. Finally, the results are subject to responder bias, as fellowship programs’ and applicants’ perceptions may have been based on their match results.

Conclusions

Virtual interviews allowed fellowship programs and applicants to complete more interviews, but both PDs and applicants stated that interviewing in-person was important for applicants to meet faculty and tour the institution where they may be spending a year. In contrast, significant financial savings resulted due to the transition to virtual interviews. Finally, both PDs and applicants were in favor of having the option of interviewing virtually, suggesting that virtual interviews may continue to play a role in future application cycles.