Away Rotations, Interviews, and Rank Lists: Radiation Oncology Residency Applicant Perspectives on the 2020 Match Process.

PURPOSE: Using 2020 match applicants, the purpose of this study was to identify baseline applicant perspectives on the match process surveying (1) away rotations, (2) interview/postinterview communications, and (3) factors influencing applicant rank order lists. METHODS AND MATERIALS: Applicants in the 2020 match cycle at a large radiation oncology (RO) residency program received a questionnaire covering demographics and the match process: away rotations, interview/postinterview communications, and ranking. Univariable and multivariable logistic regression analyses were used to identify factors associated with completing fewer away rotations.
RESULTS: Of 141 surveys sent, 76 were completed, for a response rate of 54%. Most applicants were White, male, and matched into RO. One in 3 applicants did not have a home RO program. Most applicants completed 2 RO rotations (ie, a home rotation and an additional away rotation; range, 0-4 total rotations); RO rotations influenced the applicant rank order lists and the ultimate match result for 94% and 79% of applicants, respectively. Forty-seven percent of applicants reported being asked inappropriate questions during the interview (eg, parental or marital status). Applicants did not perceive a consistent message regarding postinterview communications from program directors. Most applicants were contacted postinterview. Interviews cost most applicants more than $5000. Thirty-seven percent of respondents reported submitting a letter of interest after the interview, hoping to improve their rank. When applying to programs, general reputation and location were the most common influential factors mentioned. When ranking programs, informal conversations with residents and program culture observations were the most common influential factors mentioned. Based on multivariable analysis, applicants who completed fewer RO rotations (including away rotations) had greater odds of matching to their home program (odds ratio [OR], 12.05; 95% CI, 1.27-206.69), lower odds of program location influencing where to apply (OR, 0.04; 95% CI, 0.003-0.37), and lower odds of the program's general reputation affecting their rank list (OR, 0.04; 95% CI, 0.001-0.47).
CONCLUSIONS: The results suggest that medical students perceive away rotations as an important influencer of their match process. Although applicants and program directors both participate in postinterview communications, interactions with residents influence rank order lists. These data may serve as an up-to-date baseline to evaluate the influence of the COVID-19 pandemic on the RO match process.

Introduction

After the 2020 match, the imbalanced radiation oncology (RO) applicant-to-position ratio remains concerning.1, 2, 3 Poor RO exposure and medical-student recruitment will likely be exacerbated by the COVID-19 pandemic. Regulations of the Association of American Medical Colleges posed additional challenges to programs seeking to match candidates via essential components of the match: in-person away rotations were discouraged, and interviews went virtual.

Away rotations allow students to broaden their RO experience, show interest in specific programs, and obtain invaluable letters of recommendation. Program directors (PDs) report that RO rotations play an important role in applicant evaluations. Without away rotations, applicants may not gain a true sense of program culture, known to significantly influence applicants’ rank order lists.,

In this article, we describe the interim 2020 match applicant experiences with critical components of the match process: away rotations, interview and postinterview communications, and ranking. Given the significant changes to the 2021 match cycle owing to the COVID-19 pandemic, our data serve as an up-to-date baseline for comparison to evaluate how these modifications may affect perceptions of the 2021 RO match.

Materials and Methods

Survey design

An institutional review board (IRB)–approved, cross-sectional, anonymized, and nonvalidated online survey was generated using REDCap Survey (2013, Vanderbilt University, Nashville, Tennessee). Questions included demographics as well as questions focused on critical components of the match process, including away rotations, interview and postinterview communications, and factors influencing applying to or ranking programs (Table E1). Questions were intentionally selected by recent 2020 match applicants for the planned follow-up comparison with the 2021 match. Interview topics deemed inappropriate were selected as defined in previous literature, and outlined by the National Resident Matching Program's Match Communication Code of Conduct.

Survey participants

Invited survey participants were all applicants who applied to a large, accredited RO residency program during the 2019 to 2020 cycle. Applicants were contacted in summer 2020 via their application email and invited to participate. Electronic consent was obtained (in accordance with IRB approval); participation was voluntary, and responses were anonymous.

Statistical analyses

Descriptive summary statistics were tabulated. Univariable logistic regression analysis was performed to assess covariables associated with participation in 0 to 1 rotations (fewer away rotations) versus 2 to 4 rotations (more away rotations). Effect size was reported as odds ratios (ORs) with 95% confidence intervals (95% CIs). Multivariable logistic regression analysis was conducted using the purposeful-selection method., All tests were 2-tailed with a significant P-value threshold of .05. All statistical analyses were conducted using R statistical software, version 4.0.3, in R Studio, version 1.3.1093 (R Project for Statistical Computing).

Results

Survey distribution and response rate

The survey was distributed after graduation to 141 RO applicants. Current/valid emails were missing for 32 applicants (only the school email was provided and/or an error message was received upon sending the email). Seventy-six applicants completed the survey. Response rates including and excluding out-of-date emails were 76 of 141 (54%) and 76 of 109 (70%), respectively.

Survey respondent characteristics

Table 1 shows respondent demographics. One in 3 applicants reported not having a home RO program. Nearly all those with home programs completed their home RO rotation, with 84% matched to a nonhome institution. Eighty-nine percent of applicants completed more than 1 away rotation (mode, 2 rotations; range, 0-4 rotations).

Table 1

Radiation oncology applicant characteristics

Variable	No. (N = 76)	%
Specialty applied to
RO only	70	92
RO + Rad/IR	3	4
RO + Rad/IR + IM	1	1
RO + IM	1	1
RO + pediatrics	1	1
Specialty matched to
RO	73	96
Rad/IR	2	3
Missing	1	1
Disadvantaged background
No	71	93
Yes	3	4
Do not remember	2	3
Gender identity
Cisgender male	52	68
Transgender male	0	0
Cisgender female	22	29
Transgender female	0	0
Prefer not to answer	2	3
Ethnicity/nationality
White	48	63
Asian	19	25
Black	6	8
Hispanic, Latinx, Spanish	5	7
Native Hawaiian/Pacific Islander	1	1
American Indian/Alaskan Native	0	0
Prefer not to answer	5	7
Medical school region
International	5	7
Midwest	19	25
Northeast	19	25
South	22	29
US Territory	2	3
West	9	12
Relationship status
Single	27	36
Serious relationship/engaged	26	34
Married/CU/DP	23	30
Parental status
No children	66	87
1 Child	4	5
≥2 children	6	8
Pregnant	0	0
Has a home program
Yes	52	68
No	24	32
Number of away rotations
0	8	11
1	9	12
2	30	39
3	22	29
4	7	9

Abbreviations: CU = civil union; DP = domestic partnership; IM = internal medicine; IR = interventional radiology; Rad = radiology; RO = radiation oncology.

Factors influencing applying to and ranking programs

Figure 1 shows factors influencing which programs to apply to and their rank order. Among applicants who completed away rotations, 94% reported that away rotations influenced how they ranked programs. When choosing RO programs to apply to, general program reputation and location were the most significant factors (Fig 1). Half of applicants spent more than $5000 during interviews. Observations of program culture and informal conversations with residents were the most common factors influencing applicants’ rank-order lists (Fig 1); virtual factors (eg, social media or the program's website) were least influential.

Figure 1

Factors influencing a radiation oncology applicant's decision to apply to and rank residency programs.

Interview and postinterview communications

Nearly half of applicants reported inappropriate interview questions (eg, marital or parental status or specific programs applied to; Table 2). Men and women were equally asked questions about these topics. Applicants reported that most PDs did not mention an official postinterview communication policy (Fig E1). Two-thirds of applicants received postinterview communication (Table 3). Two-thirds had mentors network on their behalf. One in 5 applicants were told they were ranked to match. One in 3 applicants submitted a letter of interest, most doing so to improve their match. Most letters were sent to PDs, with 71% of survey participants explicitly stating that the program to which they submitted a letter was their top choice. Four applicants sent multiple letters.

Table 2

Percentage of applicants who were asked inappropriate questions during their interviews

	No. (%)
Topics asked during interview	Overall (N = 74)*	Women(n = 22)	Men(n = 52)	P value†
No. of programs applied to	7 (10)	3 (14)	4 (8)	.4
Specific programs applied to	13 (18)	3 (14)	10 (19)	.7
Whether couples matching	6 (8)	1 (5)	5 (10)	.7
Married, children, expecting Children	21 (28)	6 (27)	15 (29)	>.9
Any RO program's rank	5 (6)	0 (0)	5 (10)	.3
None of these	39 (53)	13 (59)	26 (50)	.6

Abbreviation: RO = radiation oncology.

Two applicants were excluded for responding “Prefer not to answer” to the gender question.

Statistical tests performed included Fisher exact test and χ2 test of independence.

Table 3

Postinterview communication behaviors

Variable	No. (N = 76)	%
Contacted postinterview/before match
Yes	27	36
No	49	64
Applicant told “rank-to-match”
Yes	15	20
No	61	80
Mentor mediated communication to programs
Yes	25	33
No	51	67
Applicant thought LOI would improve rank
Yes	36	47
No	40	53
Who recommended sending LOI
Program chair	2	3
Program PD/APD	8	11
Other program faculty	7	9
Program residents	16	21
Medical school deans	9	12
Medical school faculty	15	20
Other mentors	27	36
No one	30	39
Applicant sent a postinterview LOI
Yes	28	37
No	48	63
Applicants who sent LOI	No. (n = 28)	%
LOI was sent to
Department chair	8	29
PD/APD	26	93
Other faculty	2	7
Other	3	11
LOI mentioned program rank (ie, Number 1)
Yes	20	71
No	8	29
Applicant wrote more than 1 LOI
No	24	86
Yes	4	14

Abbreviations: APD = assistant program director; LOI = letter of interest; PD = program director.

Factors associated with away rotations

Applicants who completed fewer away rotations were more likely to match to their home program on univariable and multivariable logistic regression analysis (univariable OR, 10.18; 95% CI, 2.33-54.42; multivariable OR, 12.05; 95% CI, 1.27-206.69) (Table 4). On multivariable analysis, they also were less likely to report that program location influenced which programs to apply to (OR, 0.04; 95% CI, 0.003-0.37) or that general program reputation influenced their rank list (OR, 0.04; 95% CI, 0.001-0.47).

Table 4

Univariable and multivariable logistic regression analysis for factors associated with participating in fewer away rotations (0-1) versus more away rotations (2-4)

	Univariable			Multivariable
Variable	OR†	95% CI	P value*	OR†	95% CI	P value*
AMCAS disadvantage status‡			.633
No	1.00	[reference]
Yes	1.87	(0.08-20.80)
Gender			.517
Cisgender male	1.00	[reference]
Cisgender female	0.67	(0.17-2.19)
Medical school region			.201			.066
Northeast	1.00	[reference]		1.00	[reference]
Midwest	1.34	(0.30-6.40)		3.72	(0.43-43.36)
South	0.59	(0.10-3.08)		0.97	(0.08-11.68)
US territory/IMG	5.00	(0.80-36.24)		122.85	(2.17-38063.43)
West	0.47	(0.02-3.88)		0.05	(0.0003-2.35)
Relationship status			.550
Single	1.00	[reference]
Serious Relationship	1.05	(0.26-4.28)
Married	1.93	(0.52-7.58)
Ethnicity/race			.540
White	1.00	[reference]
Asian	0.73	(0.15-2.83)
Black	2.59	(0.31-18.07)
Hispanic	2.59	(0.31-18.07)
Parental status			.546
No Children	1.00	[reference]
≥1 Child	1.59	(0.31-6.58)
Has a home RO program			.145			.145
No	1.00	[reference]		1.00	[reference]
Yes	2.58	(0.74-12.11)		42.03	(1.95-5773.06)
Matched to home RO program			.002			.005
No	1.00	[reference]		1.00	[reference]
Yes	10.18	(2.33-54.42)		12.05	(1.27-206.69)
Cost of interview			.879
≤5000	1.00	NA
>5000	0.92	(0.31-2.73)
Was contact before match			.229
No	1.00	[reference]
Yes	0.48	(0.12-1.55)
Wrote a LOI			.466
No	1.00	[reference]
Yes	0.65	(0.19-2.01)
Mentor assisted communication			.114			.124
No	1.00	[reference]		1.00	[reference]
Yes	0.36	(0.08-1.26)		0.06	(0-0.64)
Impacted applying to program§
No impact	1.00	[reference]
General reputation	2.15	(0.35-41.84)	.453	20.89	(0.60-4403.88)	.101
Location	0.32	(0.09-1.25)	.099	0.04	(0.003-0.37)	.033
Home PD recommendation	0.75	(0.22-2.34)	.633
Mentor recommendation	0.85	(0.28-2.64)	.772
Resident/faculty experience	0.33	(0.11-10)	.051	0.25	(0.03-1.66)	.276
Rankings	0.63	(0.20-1.87)	.408
Impact on rank§
No impact	1.00	[reference]
Interview dinner	1.13	(0.38-3.48)	.832
Conversations with residents	0.24	(0.05-1.12)	.068
Program culture	0.30	(0.07-1.36)	.114
Visible diversity	0.98	(0.33-2.92)	.977
Tour	0.70	(0.20-2.17)	.546
General reputation	0.22	(0.05-0.91)	.037	0.04	(0.001-0.47)	.012
Location	0.38	(0.11-1.43)	.146

Abbreviations: AMCAS = American Medical College Application Service; OR = odds ratio; PD = program director.

P value calculated by Pearson χ2.

Odds ratios were calculated with a reference of participating in more rotations compared with less rotations.

Status when applying to medical school from the AMCAS.

Factors compared with the reference of “no impact” for each variable.

Discussion

Our interim 2020 RO match survey provides applicant perspectives on critical components of the match process: away rotations, interview/postinterview communications, and program ranking. Most applicants completed at least 2 away rotations. These were important for the one-third of applicants who reported no home RO program. Adjusted for confounding variables, RO applicants who participated in fewer rotations more often matched at their home program and were less likely to emphasize location or general reputation in their program choice.

For interview/postinterview communications, gamesmanship has been more vocally discouraged after Wu and colleagues’ article “Taking the ‘Game’ Out of the Match.” However, half of RO applicants reported being asked inappropriate interview questions. Despite programs’ discouraging postinterview communications, two-thirds of applicants received them. The “degaming” proposal was directed toward PDs, and applicant-initiated communication guidance is lacking. One in 3 applicants submitted letters of interest, decreased from approximately 70% as reported in a previous study. Seventy-one percent of applicants sent letters of interest stating how the applicant ranked the program (i.e. "Number 1"), hoping this would increase their rank at their top choice.

For applicants’ rank order lists, the most common factor influencing rank included informal conversations with residents and observations of program culture. These 2 factors are benefits of away rotations. One in 5 applicants were told they were ranked to match. These communications also influenced applicants’ final rank order lists.

Based on these findings, the RO match process during the COVID-19 pandemic generated a markedly altered match landscape. With many applicants without a home rotation, we hypothesize that 2021 match applicants without a home program may be disadvantaged and/or may need to use alternative methods of networking without traditional away rotations. Given the continuance of postinterview communication and the lack of personal connections, postinterview communications may be even more tempting in the 2021 match. In addition, many applicants will miss out on resident interactions that inform program fit and otherwise would typically influence program rank.

This survey-based study has several limitations. Respondents represented applicants to only 1 large program; however, this program also captured 93% of applicants in the 2019 to 2020 cycle. The overall response rate left room for potential response bias; however, there was an adequate response to make future comparisons. The delayed interview-to-survey distribution time also portended recall bias, although it may have provided participants more time to reflect and provide thoughtful responses.

In a specialty that is fighting overexpansion while simultaneously struggling with inadequate medical-school exposure and racial/socioeconomic diversification, the current study's data highlight inequities in the RO residency match. Given the aberrant 2021 match cycle, we must ensure all applicants are treated fairly. Program-culture observations and informal conversations with residents, often obtained through away rotations, remain the most common factors influencing how applicants prioritize rank order lists. With virtual interviews and fewer opportunities to experience RO programs firsthand, understanding the influence virtual rotations have on RO applicants is vital. Particularly with the inception of programs such as the Radiation Oncology Intensive Shadowing Experience, there is great potential for increasing exposure in the future among students underrepresented in medicine via virtual rotations that increase opportunities for underrepresented students in RO while reducing costs associated with away rotations. This study's data provide an up-to-date baseline to understand the possible effects COVID-19-related restrictions will have on 2021 match applicants. Our planned follow-up 2021 match survey will provide much needed insight.