Disparate Responses in Match Outcome across Competitive Surgical Subspecialties to Pandemic Era Constraints: An Analysis of Impacts of Minimal Auditions.

OBJECTIVE: The Covid-19 pandemic eliminated nearly all visiting sub-internships. We seek to uncover match rate disparities across plastic surgery, otolaryngology, urology, and neurosurgery subspecialties with respect to in-person appraisals. These data aim to highlight the common practices as well as subtle differences that each subspecialty may be displaying in selecting their respective residency candidates.
DESIGN: We accessed publicly available online spreadsheets between March 24 to 27 specific to the following surgical subspecialties: plastic surgery, otolaryngology, neurosurgery, and urology. We collected available information including the matched applicants' medical school, the institution at which they matched, and whether they had previous communication with their matched program. This data was then used to record whether the applicant matched at their home institution.
SETTING: N/A PARTICIPANTS: N/A
RESULTS: There was a statistically significant increase in the number of plastic surgery and otolaryngology applicants who matched at their home programs during the 2020 to 2021 application cycle. 12.1% and 17.2% of plastic surgery applicants matched at their home program in the 2018 to 2019 and 2019 to 2020 application cycles, compared to 25.0% during the 2020 to 2021 application cycle (p = 0.0345). Overall, 23.4% and 22.2% of otolaryngology applicants matched at their home program in the 2018 to 2019 and 2019 to 2020 application cycles, compared to 31.3% during the 2020 to 2021 application cycle (p = 0.0482). Neurosurgery and urology applicants did not demonstrate statistically significant differences in home match rates during the 2020 to 2021 application cycle (p = 0.164 and p = 0.105, respectively).
CONCLUSIONS: Covid-19 related restrictions in the 2020 to 2021 match cycle led residency programs to utilize novel selection mechanisms to evaluate applicants. Without visiting sub-internships during the 2020 to 2021 match cycle, some programs appear to have intentionally favored candidates with whom they were previously acquainted. The significantly higher number of international medical graduates and non-senior medical graduates among neurosurgery and urology residencies, respectively, likely washed out the home matching effect among these specialties but does not discount the importance of in-person appraisals.

INTRODUCTION

The selection process for matching into surgical subspecialties continues to be stringent each year. In order to match successfully into surgical subspecialties such as plastic surgery, otolaryngology, neurosurgery, or urology, students are expected to be highly competitive with above average USMLE Step 1/2 scores, research publications, and Alpha Omega Alpha Honors Society status , Furthermore, subjective evaluations of student applicants, including letters of recommendation and visiting sub-internships, appraise applicant characteristics beyond those found on CVs and board scores. In particular, while recommendation letters can be highly variable and inconsistent in the eyes of the evaluators, visiting sub-internships offer residency programs the ability to evaluate student performance amongst the residents and faculty at their institution, giving them valuable insight into the student's intangible traits such as work ethic, preparedness for operative cases, and integration into the program should they match there. ,

The importance of the in-person experience for both applicants and residency programs has been well documented in recent literature. A survey of fourth year medical students found that nearly 100% of students applying into plastic surgery, otolaryngology, neurosurgery and urology had performed a visiting sub-internship , Varying interpretations of visiting sub-internships were seen among program directors in these surgical subspecialties. A survey of plastic surgery program directors found that 67% of first year residents in their programs had previously performed a visiting sub-internship at their institution 72% of otolaryngology program directors felt that away rotations were “extremely” or “very” important in evaluating candidates, and 87% of urology program directors noted that special consideration is given to applicants who completed a visiting sub-internship at their institution Interestingly, while neurosurgery applicants reported attending an average of three away rotations, a survey of program directors found that performance on visiting sub-internships was ranked near the bottom of factors influencing resident selection.

The Covid-19 pandemic caused unprecedented changes in the 2020 to 2021 match cycle. The pandemic eliminated nearly all visiting sub-internships due to travel and visitor restrictions, and also shifted the interview cycle to a virtual format In turn, program directors were forced to assess applicants using other measures of success to evaluate applicants’ “fit” in their program. In this study, we aim to examine the impact of Covid-19 on the 2021 “home match” pattern for the following surgical subspecialties: plastic surgery, otolaryngology, neurosurgery and urology. Due to the perceived importance of in-person analysis among several of these subspecialties, we aim to uncover whether applicants during the current 2020 to 2021 match cycle compared to prior years, were more likely to match at their home programs where this facet of in-person relationships and, hence, assessment, was likely preserved.

These four subspecialties are similarly sized, with plastic surgery having 187 spots, otolaryngology having 350 spots, neurosurgery having 234 spots, and urology having 357 spots in the 2021 match cycle , Among these programs, not a single spot went unfilled in the 2021 match , Due to their similarities in size, rigor, and competitiveness, it is appropriate to compare and contrast these surgical subspecialties to one another, versus years prior, in order to decipher the impact of this year's diminished in-person evaluation and therefore to gain a deeper understanding of factors influencing the success of respective applications.

Since the National Resident Match Program no longer releases proprietary data that would allow any unencumbered analysis, we utilized crowd-sourced information collected from previously matched applicants14, 15, 16 The consistency of the information has made it feasible for our current analysis due to the establishment of a publicly available online spreadsheet that is used by applicants for the purpose of communication over the past several years. Each specialty has their own spreadsheet containing a plethora of information ranging from program details to applicant advice columns. Of note, most of these spreadsheets post updated match data detailing applicants’ demographic information, such as the applicant's medical school, and whether they completed an away rotation at the program at which they matched, information that is unfortunately unavailable from sources such as the NRMP. Among the surgical subspecialties we explored, plastic surgery, otolaryngology, neurosurgery, and urology had the most complete set of matched applicant information, allowing us to analyze this self-reported information not only for the 2021 match cycle, but for the past several match cycles as well. This consistency between years allows us to validate comparisons and gives us the ability to highlight the impact of the Covid-19 pandemic on match results among applicants in the 2020 to 2021 residency application cycle.

Herein, we seek to uncover any match rate disparities across plastic surgery, otolaryngology, urology, and neurosurgery subspecialties with respect to in-person appraisals. These data aim to highlight the common practices as well as subtle differences that each subspecialty may be displaying in selecting their respective residency candidates. By publicizing these disparities, we hope to assist 1) student applicants to these subspecialties in focusing and tailoring their applications and 2) each subspecialty in self-reflection of their own process of resident selection, given that all four are competing for the same swath of the top surgically-oriented medical students.

METHODS

On March 19, 2021 match results for the 2021 application cycle were released. We accessed publicly available online spreadsheets between March 24 to 27 specific to the following surgical subspecialties: plastic surgery, otolaryngology, neurosurgery, and urology. We also accessed the spreadsheets for the previous two application cycles (2018 to 2019 and 2019 to 2020). Among each specialty, the programs among the top 50th percentile according to Doximity reputation rankings were selected for inclusion We collected available information including the matched applicants’ medical school, the institution at which they matched, and whether they had previous communication (i.e. visiting sub-internship, research year, virtual sub-internship) with their matched program. This data was then used to record whether the applicant matched at their home institution. Next, applicant information was confirmed on each program's website for application cycles 2018 to 2019 and 2019 to 2020. For the 2020 to 2021 application cycle, applicant information was confirmed on the program's social media account if available.

Data points were compared for statistically significant differences in the number of surgical subspecialty applicants who matched at their home institution and the number of applicants who had previous communication with their matched program using a Chi-square test. All analyses were conducted in the statistical program, JASP, version 0.14.1.

RESULTS

Among the top 50th percentile of programs based on 2021 Doximity reputation rankings, our search over the course of the 2018 to 2019, 2019 to 2020, and 2020 to 2021 application cycles yielded information on 43 plastic surgery programs, 65 otolaryngology programs, 58 neurosurgery programs, and 72 urology programs.

Among plastic surgery applicants, 12.1% (n = 14) and 17.2% (n = 20) matched at their home program in the 2018 to 2019 and 2019 to 2020 application cycles, while 25.0% (n = 30) plastic surgery applicants matched at their home program in the 2020 to 2021 application cycle (p = 0.0345). Among otolaryngology applicants, 23.4% (n = 56) and 22.2% (n = 54) matched at their home program in the 2018 to 2019 and 2019 to 2020 application cycles, while 31.3% (n = 75) otolaryngology applicants matched at their home program in the 2020 to 2021 application cycle (p = 0.0482). Among neurosurgery applicants, 26.8% (n = 41) and 18.2% (n = 28) matched at their home program in the 2018 to 2019 and 2019 to 2020 application cycles, while 20.3% (n = 29) neurosurgery applicants matched at their home program in the 2020 to 2021 application cycle (p = 0.164). Among urology applicants, 12.7% (n = 28) and 19.4% (n = 43) of urology applicants matched at their home program in the 2018 to 2019 and 2019 to 2020 application cycles, while 19.3% (n = 43) urology applicants matched at their home program in the 2020 to 2021 application cycle (p = 0.105; Table 1 ).

TABLE 1

Number of Applicants Matching at Their Home Program

	2021	2020	2019
Plastic Surgery
Yes	30 (25.0%)	20 (17.2%)	14 (12.1%)
No	90 (75.0%)	96(82.8%)	102 (87.9%)
Total	120	116	116
p value	0.0345
ENT
Yes	75 (31.3%)	54 (22.2%)	56 (23.4%)
No	165 (68.8%)	189 (77.8%)	183 (76.6%)
Total	240	243	239
p value	0.0482
Neurosurgery
Yes	29 (20.3%)	28 (18.2%)	41 (26.8%)
No	114 (79.7%)	126 (81.8%)	112 (73.2%)
Total	143	154	153
p value	0.164
Urology
Yes	43 (19.3%)	43 (19.4%)	28 (12.7%)
No	180 (80.7%)	179 (80.6%)	192 (87.3%)
Total	223	222	220
p value	0.105

If an applicant did not match at their home institution, we analyzed self-reported information of previous contact (i.e. visiting sub-internship, research year, or virtual sub-internship) with the program at which they matched (Table 2 ). Between the 2019 to 2020 and 2020 to 2021 match cycles, reported communication between non-home matched applicants decreased from 52% to 22.2% among plastic surgery applicants, 32.8% to 9.7% among otolaryngology applicants, and 34.1% to 10.0% among urology residents (p < 0.001; Table 2). No data was available to analyze for the 2020 to 2021 application cycle among neurosurgery applicants.

TABLE 2

Reported Communication Among Non-Home Program Matched Applicants

	2021	2020	2019
Plastic Surgery
True	20 (22.2%)	50 (52.0%)	-
False	70 (77.8%)	46 (47.9%)	-
Total	90	96	-
p value	<0.001	-
ENT
True	16 (9.7%)	62 (32.8%)	58 (31.7%)
False	149 (90.3%)	127 (67.2%)	125 (68.3%)
Total	165	189	183
p value	<0.001
Neurosurgery
True	-	41 (32.5%)	38 (33.9%)
False	-	85 (67.5%)	74 (66.1%)
Total	-	126	112
p value	-	0.820
Urology
True	18 (10.0%)	61 (34.1%)	57 (29.7%)
False	162 (90.0%)	118 (65.9%)	135 (70.3%)
Total	180	179	192
p value	<0.001

Note: No data is available for communication for Neurosurgery 2021 or Plastics 2019

DISCUSSION

Covid-19 related restrictions on visiting sub-internships demonstrated varying effects on competitive surgical subspecialties during the 2020 to 2021 application cycle. Our findings suggest that plastic surgery and otolaryngology both demonstrated a statistically significant increase in the number of applicants who matched at their home institution during the 2020 to 2021 application cycle as compared to the previous 2 cycles. In contrast, neither neurosurgery nor urology applicants demonstrated a statistically significant increase in their home institution match rates during the 2020 to 2021 application cycle.

Among applicants who did not match at their home institutions, self-reported communication between the applicant and the program at which they matched decreased for plastic surgery, otolaryngology and urology (no data was available for neurosurgery) in the 2020 to 2021 cycle. While the decrease in self-reported communication paralleled an increase in home match rates for plastic surgery and ENT applicants, this trend was not seen among urology residents. Urology home match rates remained stable despite a statistically significant decrease in self-reported communication with their matched program. While self-reported communication for neurosurgery residents was not reported for the 2020 to 2021 match year, we can assume that communication inevitably decreased for these applicants as well due to Covid-19 related restrictions faced by all applicants. In turn, neurosurgery applicants experienced a similar trend to urology applicants, with home match rates remaining fairly stable during the 2020 to 2021 match cycle.

The factors underlying this disparity may be multifaceted. Each specialty has varying relative importance of objective versus subjective evaluation measures. Neurosurgery program directors placed applicant performance on visiting sub-internships as one of the least important factors of their selection process Despite differences in program director emphasis of visiting sub-internships among neurosurgery programs as compared to plastic surgery, otolaryngology and urology programs, the importance of in-person appraisals cannot be understated. Neurosurgery program directors’ low ranking of the value of visiting sub internships may reflect 1) minimal hands-on experience given the nature of their surgery and 2) the overwhelming percentage of international medical graduates (IMGs, which we will further discuss), diluting the perception of such in-person clinical appraisal of graduating seniors. Still, the literature has shown that nearly 100% of medical students applying into neurosurgery perform an away rotation in the field emphasizing a contrasting viewpoint among neurosurgery applicants and program directors On the contrary, evaluation of plastic surgery, otolaryngology, and urology applicants depends highly on their performance on their visiting sub-internship and their “perceived fit” in the program. Program directors have rated interaction with residents or faculty as the perceived objective of visiting sub-internships. , , ,

Several programs attempted to preserve interactions with applicants despite travel restrictions with the creation of virtual sub-internships. These sessions consisted of less formal Zoom program introductions, lectures, and provided an opportunity for applicants to meet faculty and residents. While it allowed applicants to show interest in specific programs, they still lacked the opportunity for programs to assess applicants in-person.

Another potential influence of home match rates may be the differing number of IMGs and non-senior medical graduates who historically are more likely to match in neurosurgery and urology as compared to plastic surgery and otolaryngology. Analysis of NRMP and American Urological Association match data (the urology match does not participate in NRMP) found that neurosurgery programs matched a significantly higher number of IMGs (p < 0.001), while urology residencies matched a significantly higher number of non-seniors (p < 0.001; Table 3 ), compared to plastic surgery and otolaryngology , This discrepancy underscored an important and prevalent pathway likely through dedicated research for IMGs and non-seniors to have a successful match in neurosurgery and urology , Not surprisingly, these non-traditional applicants may have had exposure to programs at which they matched, emphasizing the notion that applicant familiarity (other than through traditional clinical away rotations) may still be held in high regard. We suspect that the high number of IMGs and non-senior medical graduates among neurosurgery and urology residencies may have washed out the home matching effect as seen in plastic surgery and otolaryngology during the 2020 to 21 application cycle.

TABLE 3

ABCD

	IMGs	Non-Seniors	Total Matched Applicants	p value
Plastic Surgery	8	9	187
Otolaryngology	6	17	350
Urology	8	51	357	0.000092
Neurosurgery	17	11	234	0.034995

In January of 2022, Step 1 will become pass/fail and will present a novel manner in which residency programs must evaluate applicants. Step 1 has historically been one of the most important objective measures of evaluation among residency programs, with 94% of program directors using Step 1 scores to evaluate residency candidates With this change, applicant evaluation will likely shift to place even more emphasis on subjective measures of success. This will only magnify the increasing importance of in-person analysis and holistic evaluation of applicants.

As with any review, these results must be interpreted in the context of the study design. Crowd-sourced 2021 match data has yet to be confirmed on individual program websites due to the novelty of the information, even though data was verified on public program social media pages when available. In addition, reassuring cross-verification of such crowd-sourced data against official data from years prior has lent confidence to our analysis. Similarly, available information concerning applicants’ previous contact with their matched program is dependent on their use of the publicly available Google spreadsheet, and cannot be independently confirmed on program websites.

Separately, we chose to analyze the top 50th percentile of programs in all four subspecialties rather than all programs together based on Doximity reputation rankings. While Doximity is by no means an official ranking source, its rankings have been widely popular among graduating medical students and have been referenced in several peer-reviewed published studies23, 24, 25 In our current study, Doximity also offers a uniform source of rankings across all four subspecialties as well as longitudinal consistency. The choice to analyze the top 50th percentile programs is important because 1) top programs are naturally more competitive and generate the most interest for visiting sub-internships in normal years and are, therefore, likely the most impacted by Covid constraints; 2) the bottom half of programs would most likely match the remaining applicant pool by recipricol defaults, thereby washing out any choice impact we tried to discern. While program selection depends on a myriad of factors, we decided to focus on the top 50th percentile for the reasons outlined above. We felt that we would have large enough data points while not being averaged out by bottom half of the numbers that, by virtue of the match, would likely exist in some form of reciprocity.

CONCLUSION

Covid-19 related restrictions on the 2020 to 2021 match cycle might have impacted residency programs differently in regard to their selection mechanisms to evaluate applicants. An increase in home match rates was seen in plastic surgery and otolaryngology, but not in neurosurgery or urology. Familiarity with applicants, particularly after a visiting sub-internship, has been shown to be important in residency candidate selection. Without visiting sub-internships during the 2020 to 2021 match cycle, programs were likely left to select candidates with whom they were previously acquainted. We suspect that the high number of IMGs and non-senior medical graduates among neurosurgery and urology residencies, respectively, likely washed out the home matching effect among these specialties. Taken together, despite the seemingly disparate responses in home match rates between plastic surgery/otolaryngology and urology/neurosurgery, the importance of in-person appraisals, whether in the form of clinical away rotations or perhaps with prior research exposure, cannot be discounted. We hope that our findings can provide additional insights, to students and programs alike, not only in the immediate post-Covid application cycles but also in the years beyond, on how in-person experiences could be valued during the match process across competitive surgical subspecialties.