The Impact of COVID-19 on Radiation Oncology Residency Applicant Away Rotations, Interviews, and Rank Lists: A Comparison Between the 2020 Match and 2021 Match.

PURPOSE: The COVID-19 pandemic modified the Residency Match process for fourth-year medical students. In-person away rotations were discouraged, interviews were virtual, and traditional factors used to rank programs were absent. Here, we compare survey results administered to both the 2020 and 2021 Match applicants to assess the influence of the pandemic on the radiation oncology (RO) Match process.
METHODS: An institutional review board-approved prospective cross-sectional study was conducted. The 2020 and 2021 RO Match applicants at a large RO program were invited to participate. Descriptive summary statistics were assessed.
RESULTS: The 2020 and 2021 Matches each had 76 applicants complete the survey with response rates of 54% and 57%, respectively. The 2 groups were predominantly white, cisgender male, single, and without children. Whereas 11% of 2020 applicants did not complete away rotations, 45% of 2021 applicants did not. For 2021 Match applicants, 65% of away rotations were performed virtually, whereas 51% were not for medical school credit. Of the applicants, 84% were satisfied with virtual interviews and 72% felt cost savings were worth not having in-person interviews. Whereas 49% of Match 2020 applicants spent >$5000 in interview costs, 0% of the Match 2021 applicants did so, with 45% spending <$100. Postinterview communications from programs increased during the pandemic from 36% to 42% in 2020 Match and 2021 Match, respectively. Although program culture was the most common factor influencing 2021 Match applicants program rankings, half of applicants did not gain a sense of program culture during virtual interviews.
CONCLUSIONS: We found 2021 Match applicants completed fewer away rotations, were satisfied with virtual interviews/reduced costs, and did not gain a sense of program culture through virtual rotations/interviews despite it being the most important ranking factor reported. This study supports further exploration of virtual away rotations and virtual interviews moving forward beyond the pandemic.

Owing to the COVID-19 pandemic, the Association of American Medical Colleges discouraged clinical rotations at outside institutions (“away rotations”) while recommending online interviews. Traditionally, in-person away rotations served as opportunities for applicants to learn about other residency programs. Moreover, residency program directors report away rotation evaluations as the most important metric for offering interviews.

Interviews are traditionally in person and affect programs’ Match rank order lists., Few studies have evaluated online/virtual residency interviews, although some data suggest they may enhance the applicant pool by reducing travel costs, scheduling conflicts, and educational interruptions. Gamesmanship through letters of interest and “rank-to-match” letters complicate the Match process, with postinterview communication providing some applicants unfair advantages.

The drastic changes to the radiation oncology (RO) Match process offer a window both for study of equitable access to virtual away rotations as well as the opportunity afforded by online interviews. This study evaluates COVID-19-related changes in the RO Match process including away rotations, interviews, and program ranking through survey of the 2020 and 2021 Match applicants.

Methods

Survey participants

An institutional review board–approved prospective cross-sectional study was conducted by surveying RO applicants during the 2019 to 2020 (Match 2020) and 2020 to 2021 (Match 2021) cycles. The applicants to a single, large, accredited RO residency program were surveyed after Match Day by email. Consent was obtained electronically, participation was voluntary, and responses were anonymous.

Survey design

Participants were sent a nonvalidated online survey as previously reported, distributed using the REDCap survey tool (2013, Vanderbilt University). Questions sent to both groups focused on applicant demographics, RO rotations, interviews, factors that affect applying or ranking programs, and postinterview communications (Supplementary Table E1). New questions were added to the 2021 survey pertaining to perspectives and satisfaction with virtual away rotations and online interviews.

Statistical analyses

Descriptive summary statistics were tabulated using χ2 and Fisher's exact tests. All tests were 2-tailed with a significant P value threshold of 0.05. All statistical analyses were conducted using R v4.0.3 in RStudio v1.3.1093 (R Foundation for Statistical Computing).

Results

Participants

Fifty-seven percent of 2021 Match applicants (76 of 133) completed the survey. There were no significant demographic differences between the 2020 and 2021 Match cohorts (Table 1). The 2021 survey respondents were primarily white (52%), cisgender male (60%), single (39%), without children (88%), and from the Northeast or South regions. Two-thirds of applicants reported having a radiation oncology program in their medical school.

Table 1

Match 2020 and Match 2021 survey participant characteristics

Characteristic	2020, N = 76*	2021, N = 76†	P value2
Race			0.7
White	44 (60%)	38 (54%)
Asian	18 (25%)	19 (27%)
Black	5 (6.8%)	5 (7.0%)
Hispanic	5 (6.8%)	9 (13%)
NHPI	1 (1.4%)	0 (0%)
Prefer not to answer	3	5
Sex			0.3
Cisgender male	52 (70%)	45 (60%)
Cisgender female	22 (30%)	30 (40%)
Prefer not to answer	2	1
Relationship status			0.8
Single	27 (36%)	30 (39%)
In a serious relationship/engaged	26 (34%)	27 (36%)
Married/civil union/domestic partnership	23 (30%)	19 (25%)
Disadvantaged status			0.2
No	71 (96%)	62 (89%)
Yes	3 (4.1%)	8 (11%)
Prefer not to answer	2	6
Parental status			0.8
No child	66 (87%)	67 (88%)
Has child	10 (13%)	8 (11%)
Pregnant	0 (0%)	1 (1.3%)
Medical school region			0.8
Northeast	19 (25%)	21 (28%)
Midwest	19 (25%)	16 (21%)
South	22 (29%)	21 (28%)
US territory/IMG	7 (9.2%)	11 (14%)
West	9 (12%)	7 (9.2%)
Has home RO program			>0.9
No	24 (32%)	23 (30%)
Yes	52 (68%)	53 (70%)

Abbreviations: IMG = International Medical Graduate; NHPI = Native Hawaiian and Other Pacific Islanders; RO = radiation oncology.

Statistics presented: n (%)

Statistical tests performed: χ2 test of independence; Fisher's exact test

Away rotations and applications

Of 2021 Match survey respondents, 45% did not complete any away rotations, compared with 11% of 2020 applicants (Fig 1). Among 2020 and 2021 Match survey respondents who did not participate in away rotations, 75% and 73%, respectively, expressed desire to do so, and 50% and 52%, respectively, felt not completing an away rotation impacted their rank. Most away rotations were performed virtually (65%) and were not for medical school credit (51%; Table 2). One in 5 applicants did not gain a sense of culture via virtual away rotations. Factors impacting which programs applicants applied to were statistically unchanged: general reputation (88%), location (86%), and mentor recommendation (66%) remained the most influential factors (Supplementary Table E2). Factors impacting 2021 Match applicant ranking of programs also included Twitter (11%), Student Doctor Network (7%), and the anonymous Google Spreadsheet (32%).

Fig. 1

Number of completed away rotations by Match 2020 and Match 2021 applicants.

Table 2

Responses regarding number, credits received, and observations of program culture among 2021 Match applicants who did away rotations

Away Rotations	2021 Match; N = 43
In-person AR
0	28 (65%)
1	11 (26%)
2	4 (9%)
Number of virtual AR received credit
0	22 (51%)
1	8 (19%)
2	5 (12%)
3	6 (14%)
4	2 (4%)
Number virtual AR did NOT receive credit
0	27 (63%)
1	8 (19%)
2	4 (9%)
3	2 (4%)
4	2 (4%)
2021 observed culture via virtual AR
Not at all	1 (2%)
Not much	1 (2%)
Neutral	7 (16%)
Somewhat	13 (30%)
Very much	21 (49%)

Abbreviations: AR = away rotations.

Totals may not equal 100 due to rounding. A total of 44 radiation oncology rotations were for credit and 30 away rotations were not for credit.

Interviews

The 2021 Match survey respondents applied, offered, and accepted a median of 20 (interquartile range [IQR], 13-29), 16 (IQR, 9-27), and 14 (IQR, 9-19) interviews, respectively. Figure 2 summarizes attitudes toward virtual interviews for the 2021 cohort due to the COVID-19 pandemic. Of the applicants, 84% were satisfied with virtual interviews and 72% felt the cost savings were worth it. Only 51% reported gaining a sense of program culture from virtual interviews. Interview costs were significantly reduced for 2021 applicants, with 49% of 2020 Match versus 0% of 2021 Match applicants spending >$5000 in interview costs. Of 2021 Match applicants, 45% spent <$100 on interviews (Fig 3).

Fig. 2

Match 2021 applicant perspectives on virtual interviews. Binary categories were created with responses for either (1) very much or somewhat and (2) neutral, not much, or not at all.

Fig. 3

Interview cost comparison between 2020 Match and 2021 Match.

Postinterview communications and ranking

Postinterview communications by programs to survey respondents increased from 36% in 2020 to 42% in 2021 (Supplementary Table E3); no other significant differences were noted, with 58% of 2021 applicants reporting inappropriate questioning and 33% reporting postinterview “rank-to-match” correspondence, 79% reporting writing a “top choice” letter of interest, 25% reporting submitting more than 1 letter of interest, and 38% reporting mentor-assisted communications (Supplementary Table E3). Most factors impacting how survey respondents rank programs remained unchanged (Fig 4 and Supplementary Table E2), although department facilities and technology impact increased from 36% to 58% (P = .009) and department website impact increased from 13% to 36% (P = .002). For the 2021 Match, 33% also reported virtual away rotations and 63% reported resident achievements as factors impacting how they ranked programs.

Fig. 4

Factors that impact how applicants rank programs during 2020 Match and 2021 Match.

Discussion

This RO Match survey compares perspectives on away rotations, interviews, and ranking practices of the pre-COVID-19 2020 Match and 2021 Match survey respondents. Due to COVID-19 restrictions, many 2021 Match applicants did not complete away rotations. Virtual away rotations were created to provide an alternative, but many applicants did not receive any school credit. Applicants expressed high levels of satisfaction with virtual interviews, which were less costly. Program tours and department websites had an increased impact on how applicants ranked programs during COVID-19. While the interpretation is limited by the response rate of 57%, this is the first paper to prospectively compare RO residency applicant perspectives on away rotations, interviews, and ranking practices before and during COVID-19.

Away rotations

Away rotations are important for both applicants and program leadership, particularly in RO. Survey respondents learn about the program culture, while program leadership learns about residency candidates beyond their application. A third of RO applicants do not have a home program, making away rotations critical for exploring RO, demonstrating interest, and networking. In 1 pre-COVID-19 survey, 89% of applicants completed at least 1 away rotation, with 79% reporting away rotations as extremely important to communicate interest and obtain letters of recommendation.

Away rotation costs are significant, posing a barrier for students with limited financial resources. Underrepresented in medicine students are 40% less likely than non-Hispanic white students to attend medical schools with RO departments and thus depend on away rotations for RO exposure and mentorship. Novel programs demonstrate the potential to increase diversity within RO through increasing accessibility of virtual RO rotations. Away rotations may also be challenging for applicants with children or other familial responsibilities. Normalizing meaningful alternatives to the traditional in-person away rotations may increase equity among applicants. Most virtual away rotations during Match 2021 did not provide applicants medical school credit, likely due to their rapid development. We recommend that programs continue offering virtual away rotations while working with medical schools to provide credit for these experiences.

COVID-19-associated travel restrictions necessitated virtual interviews. Particularly for RO residency applicants, who are completing both internship and residency interviews, average pre-COVID-19 in-person interview costs are upward of $8000. While it has been hypothesized that virtual interviews could mitigate these costs, the extent is largely unknown. During the 2020 Match, 49% of applicants spent >$5000 on interviews (2% spent <$1000), while during the 2021 Match, no applicants spent >$5000 on interviews (45% spent <$100). Despite program culture being the top factor impacting RO program ranking by applicants, half of applicants failed to gain a sense of program culture during virtual interviews. Nonetheless, the majority of 2021 applicants were satisfied with virtual interviews and expressed the cost savings were worth the trade-offs of virtual interviews.

Postinterview communications are pervasive in RO.,,, These correspondences occur bidirectionally, with applicants sending letters of interest to programs and programs informing applicants they are “ranked to match,” violating the National Resident Matching Program communication code of conduct. The 2021 Match demonstrated increases in program-initiated contacts, likely attributable to both poor applicant familiarity without away rotations and fear of going unfilled.

Ranking

Before the pandemic, away rotations, informal conversations with residents, and program culture observations were the 3 most significant factors impacting an applicant's ranking of programs; these remain unchanged. However, 2 factors significantly increased: program websites and the departmental facilities, technology, and machines. The 2021 applicants likely used online information to better understand programs. For example, the “anonymous Google Spreadsheet” is used across residency specialties by applicants to share information about program culture and general impressions, which may ultimately influence how applicants perceive programs. Whether this leads to similar levels of program fit and satisfaction is unknown.

Limitations

Respondents represented applicants to 1 large RO residency program, potentially limiting the generalizability of the study; however, our cohort includes the majority of applicants in the 2020 and 2021 Matches.17 The data interpretation is limited by a 56% response rate. Response bias is also possible, given the response rate and survey dissemination timing. The delayed survey distribution time may impact recall bias, although the survey followed Match Day to allow respondents to answer questions honestly without worry of impacting their Match.

Conclusions

The COVID-19 pandemic introduced new considerations to the RO Match process. The 2021 Match survey respondents completed fewer away rotations (mostly virtual, many without credit), were satisfied with virtual interviews and associated costs savings, reported decreased familiarity with program culture, and engaged in more postinterview communications. This study provides support for exploring virtual away rotations and interviews beyond the pandemic.