Postinterview Communication in Orthopaedic Surgery Residency Interviews: A Survey of Match Applicants.

Background: To maintain the integrity of the match, postinterview communication (PIC) from programs to applicants is monitored and discouraged. The most recent report on the prevalence of PIC in orthopaedics found that 64% of surveyed applicants in 2014 and 2015 had received some form of PIC during their match cycle. In July 2019, the American Orthopaedic Association's Council of Orthopaedic Residency Directors (AOA/CORD) released a guideline recommending the elimination of all PIC in any form. The goal of this follow-up study was to determine the current prevalence of PIC with orthopaedic surgery applicants and assess the perspectives of medical students who recently applied for orthopaedic surgery residency positions.
Methods: A 35-question survey was e-mailed to all orthopaedic surgery residency applicants of 4 geographically diverse residency programs in postmatch March 2020. The survey was open for 1 month, and the responses were reported using descriptive statistics.
Results: Of the 229 respondents (21% response rate), 91 (39.7%) received PIC during the 2019 to 2020 residency match cycle. The program director was most commonly identified (80.2%) as the person who communicated with the applicants. At the interview day, 198 respondents (86.5%) were told that programs would not be contacting the applicants with PIC. However, over a quarter of respondents (25.3%) who received PIC answered that those programs contacted applicants after announcing they would not. Nearly half of the respondents (48.5%) agreed or strongly agreed that PIC causes added stress on applicants, and over half (52.9%) agreed or strongly agreed that all programs should stop participating in any form of PIC. Conclusions: Although the prevalence of PIC seems to have decreased since the 2014 and 2015 match, there is still room for improvement toward eliminating PIC. The AOA/CORD position statement from July 2019 should be disseminated to all members of the residency selection team to ensure consistency from all programs. The primary limitation of this study was the 21% response rate. Level of Evidence: Level IV (survey study).

Introduction

The National Resident Matching Program (NRMP) was created to address the competition for residents during a time when there were more available residency spots than applicants to fill them[1,2]. Since the early 1980s, there have been more applicants entering the match than available positions, which has caused it to become a more competitive, complicated, and expensive process for applicants. The goal of the match algorithm was to ensure a fair process for applicants and programs based on rank-ordered lists created by both parties after the interview season[3]. Part of the fairness of the match is limiting postinterview communication (PIC) between applicants and programs. The NRMP specifically discourages PIC with the expectation that both parties follow a specific code of conduct, which includes “respecting an applicant’s right to privacy and confidentiality, accepting responsibility for the actions of recruitment team members, refraining from asking illegal or coercive questions, declining to require second interviews or visiting rotations, and discouraging unnecessary PIC”[4,5]. Reports of PIC have highlighted that the practice frequently results in applicants being pressured to offer assurances of their interest in the communicating program[6,7]. PIC varies by specialty and residency program, and the prevalence and outcomes of PIC have been studied in many specialties, including orthopaedics (64% prevalence)[8], dermatology (32%)[9], urology (60%)[10], and radiation oncology (55%)[11].

Although the NMRP match code of conduct is clear in describing ethically acceptable and unacceptable PIC, orthopaedic residency programs have not strictly adhered to the code of conduct. In the 2014 and 2015 match cycles, Brooks et al. reported that 64% of orthopaedic surgery residency applicants received PIC from 1 or more programs[8]. Because of the prevalence of PIC and the negative impact it can have on applicants, including pressure to alter their rank lists and affirm interest in programs who reach out to them[8], in 2019, the American Orthopaedic Association’s Council of Orthopaedic Residency Directors (AOA/CORD) released a guideline for PIC with residency applicants[12]. Specifically, the guideline states that the “CORD oversight committee recommends that ANY and ALL post-interview communication should be eliminated”[12]. At the time of the publication, 97% of ACGME-accredited orthopaedic residency programs were members of AOA/CORD and all program directors were explicitly made aware of this guideline. As a follow-up to the 2016 Brooks et al.[8] study that showed the PIC prevalence to be 64% in a sample size of 293 applicants (24% response rate), the purpose of this study was to determine the current prevalence of PIC with orthopaedic surgery applicants and assess the perspectives of medical students who recently applied for orthopaedic surgery residency positions.

Methods

After approval by the Institutional Review Board, e-mail addresses of all those who applied to 1 or more of 4 geographically diverse orthopaedic residency programs were chosen based on the faculty interested in collaborating on this project and were pooled. This allowed for 1,090 of the orthopaedic surgery applicants in the 2020 cycle to be contacted for this prospective survey study. Adhering to the Association of American Medical Colleges (AAMC) data policy, all applicants were contacted by the senior author (MK) through e-mail after the 2020 match results were released. On March 25, 2020, an anonymous, web-based survey invitation was sent out, followed by reminder e-mails to complete the survey. Overall, the survey was open for 1 month.

The survey was modeled after the 2014 to 2015 match cycle survey of orthopaedic applicants from the work of Brooks et al.[8] The modifications included removal of questions about second looks and inappropriate questions during interviews. In addition, Likert-style subjective questions regarding the practice of PIC were included, as well as further objective questions regarding applicants’ experiences with PIC. For clarifying purposes, the instructions for the survey included the following: “We are interested in understanding the prevalence and content of post-interview communication as part of the residency application process”. For the purposes of this study, PIC refers to “communication initiated by the program to an applicant after the interview by a faculty member/Program Director/Chairperson/resident at the institution, but before final rank lists are due”. In addition, the introduction to the survey noted that the NRMP guidelines state that “program directors shall not solicit or require PIC from applicants nor shall program directors engage in PIC that is disingenuous for the purpose of influencing applicants’ ranking preferences”. The survey is included in Appendix 1, http://links.lww.com/JBJSOA/A324.

Statistical Analysis

Descriptive statistics including percentages and means with standard deviations through Microsoft Excel were used to report survey results.

Results

Responses were received from 229 of 1,090 (21.0%) contacted applicants. The respondents of the survey were 76.9% male (176/229) and 21.4% female (49/229), which is representative of the distribution of the orthopaedic surgery applicant pool per the 2020 AAMC historical data set (80.3% male and 19.7% female; Table I)[13]. Most were White (77.7%), followed by Asian (14.8%), Hispanic/Latin (5.2%), and Black/African American (4.4%). Over half of the respondents (51.1%) applied to more than 80 orthopaedic surgery residency programs. The average number of completed visiting student orthopaedic elective rotations in this cohort was 3.2 (±1.0) rotations. Ninety-one respondents (39.7%) received PIC (Table I). The most common methods of PIC were e-mail (65.9%), phone call (52.7%), and text message (16.5%). Most often, the program director was selected as the person who communicated with the applicants (73/91 respondents) (Table II).

Of the 91 respondents who received PIC, 80.2% were under the impression that if they ranked the communicating program first, and they were guaranteed to match at the program. This was true for 26.4% of respondents who ranked the communicating program first and matched there. Alternatively, 15.4% of applicants who ranked the communicating program first did not match there. Most of the respondents (58.2%, 53/91) did not rank the communicating program first (Table III). There were 138 responses to the geographic breakdown of PIC, and the most selected regions were the northeast and midwest (Fig. 1). In an optional free response question, 24 different residency programs were disclosed by applicants as having participated in PIC this application cycle. Regarding how receiving PIC made those applicants feel, 75 of 91 selected “happy a program wanted me” and 55 of 91 selected “confident that if I ranked them high, I would match at the program”. Alternatively, 35 of 91 selected “unsure of how I should immediately respond”, 15 of 91 selected “pressured to give an answer that I wasn’t ready to give”, and 14 of 91 selected “unsure if I ranked them high, I would match at the program”.

Fig. 1

Geographic breakdown of PIC in the 2019 to 2020 application cycle.

At the in-person interview day, 198 respondents (86.5%) were told by at least 1 program that they would not be contacting the applicants with PIC. However, over a quarter of respondents (25.3%) who received PIC answered that those programs contacted applicants after announcing they would not. Fifteen of the PIC applicants (16.5%) responded that at least 1 program requested to know where they were ranked on the applicant’s list. Over half of those who received PIC (51.7%) were told by at least 1 program where they were ranked on the program’s list. Twenty-two of the applicants (24.2%) who received PIC stated that the PIC caused them to rank the communicating program higher on their rank lists and 7 (7.7%) ranked the communicating program lower on their rank lists. Nearly half of the respondents (48.5%) agreed or strongly agreed that the PIC causes added stress on applicants. Over half (52.9%) agreed or strongly agreed that all programs should stop participating in any form of PIC. The additional responses to Likert-style questions regarding the perceptions of PIC are shown in Fig. 2.

Fig. 2

Medical student opinions regarding postinterview communication.

Discussion

PIC by residency programs with the intent to elicit an applicant’s rank information or to influence an applicant’s rank list is in violation of the NRMP code of conduct for programs participating in the residency match[4,5]. The primary finding of the 229 surveys completed (21% response rate) was that PIC remains prevalent in the orthopaedic surgery residency process, with 39.7% of respondents receiving communication during the 2020 match cycle. This survey demonstrates that elimination of PIC by orthopaedic surgery residency programs has not yet been achieved.

The most selected geographic regions participating in PIC with this cohort of students were the northeast and midwest geographic regions (84/138). It was not possible to determine whether this result corresponded to multiple programs in each region or a single program in each region contacting multiple applicants. Applicants had the option to disclose the program names, and 62.5% of the named programs (15/24) were in the northeast and midwest geographic regions. A potential reason for this distribution may be that over half of the orthopaedic surgery residency programs are in those 2 geographic areas (56.7% of programs). In our opinion, these findings may serve as a benchmark for the programs in these geographical regions to work toward eliminating PIC in future match cycles.

An encouraging finding from this study was 86.5% of respondents reported that during the interview day, some residency programs announced that they would not be contacting applicants after the interview day. Interestingly, 25.3% of the PIC-recipient applicants reported programs that announced that they would not be participating in PIC proceeded to contact applicants. The reason for this phenomenon is unclear. It is possible that what is interpreted as PIC among applicants is not congruent with the type of communication that the AOA/CORD guidance seeks to eliminate. If many of the applicants understood an e-mail response to one of their questions as PIC, then the prevalence of match violation worthy PIC may be overestimated by this survey study. Regardless of the reason for this discrepancy, this finding highlights the need for the residency interview process to be cohesive and organized to ensure fairness to the applicants and the integrity of the match process. This finding also highlights the need to clarify among applicants and faculty members what type of communication encompasses match violation worthy PIC. Continued education, communication, and on-going assessment regarding PIC remain necessary. The study design introduces a limitation of recall bias. The survey was likely not specific enough to determine when communication may have been more general vs. specifically informational about rank probability because the nuances of these conversations can be very subtle. Although this is a limitation of the study itself, it highlights the very reason that we believe the only way to improve this process is an absolute moratorium on PIC.

It is important to ensure that applicants have an accurate and complete understanding of the NRMP matching algorithm and the terminology that programs typically use in their PIC. In our study, 59.8% of respondents agree or strongly agree that some programs are disingenuous with their PIC. This likely has to do with the terminology used by programs participating in PIC. When an applicant is told they are “ranked to match”, this could be interpreted by an applicant as the assurance that their position on the program’s rank list is within the number of available residency spots. Although this may be the case, programs may also use the historical number of ranks it takes to fill their residency spots when determining the students who will be told they are “ranked to match”. For example, if a program has 5 PGY-1 spots, but they have previously matched an applicant in the 25th spot on the program’s rank list, they may tell the first 24 students on their rank list that they are “ranked to match”. Of the 91 applicants who received any PIC, 80.2% of them were under the impression that they would match at that program if they ranked them highly, and 75 of 91 selected that they felt “happy that a program wanted [them]” when they received PIC. The communication resulted in 22 of the applicants (24.2%) ranking the communicating program higher on their rank lists. Fifteen percent of the applicants who were told they were ranked to match and subsequently ranked that program first on their rank list reported that they did not match there. This result without the understanding of terminology perpetuates the idea that programs are disingenuous with their PIC.

PIC is a controversial topic because of the effect it may have on the applicant. Over half of the respondents (55%) agree or strongly agree that “when applicants are told they are ‘ranked to match’ after their interview, it could cause distress because of pressure to affirm interest in the program”. Jena et al.[14] reported 65.2% of fourth-year medical students applying to various specialties found PIC to be stressful, and Berriochoa et al.[15] similarly reported that 70% of respondents reported distress because it relates to PIC. If an applicant receives PIC from a program that may not have been their top choice, the fear of not matching may cause them to feel pressured to move that program up on their list even if it is not truly where they wanted to match. Medical students rely heavily on online forums to gauge their likelihood of success before and during the match cycle based on information posted by other students[16]. Minutes after a student receives any form of PIC, the program, method of communication, and message will be disseminated to the rest of the applicant pool. If an applicant is told by one of their highly desired programs that there will be no PIC and subsequently finds out that another applicant received PIC from that program, this may cause unnecessary confusion and distress.

Of those students contacted, 25.3% stated that they were told where the student was ranked on the program’s rank list. It is unclear what information was communicated to the remainder of the students who received PIC, but whatever was communicated was not enough for 68% of those students to make changes to their rank lists. With 69.9% of the applicants surveyed strongly disagreeing or disagreeing with PIC being essential to the match process in addition to the 48.5% of applicants who feel that PIC is distressing to the applicant, we should continue to use the AOA/CORD guideline in the effort to eliminate PIC in the orthopaedic surgery residency selection process.

There are several limitations to this study, the first and most important being that it is a survey study with a low response rate. We contacted the pool of applicants who applied to 4 orthopaedic surgery residency programs, which did not capture the entire pool of applicants. Of the 1,090 applicants we were able to contact, only 21% of them submitted responses to the survey. The survey itself investigated a controversial topic, which may have contributed to the low response rate because applicants may not feel comfortable reporting this information for fear of retaliation from programs who only reached out to a small pool of interviewees. It is possible that lack of interest or concern regarding postinterview communication was another reason for the low response rate, and therefore, the responses from those students may have changed the outcome of this study. Alternative methods of contacting applicants or eliciting responses, such as some form of incentive, may need to be used to achieve the highest possible response rate in a future study.

Conclusion

Although the prevalence is less than previously reported, this survey revealed that PIC is still commonplace in the orthopaedic surgery residency selection process. This study suggests that PIC does not change the rank list in most of the applicants, has a negative emotional impact on most of the applicants, and can be misinterpreted. In our opinion, it would be in the best interest of applicants for PIC from programs and from applicants to be eliminated entirely. The AOA/CORD position statement from July 2019 should be disseminated to all members of the residency selection team to ensure that a united effort is made to uphold the integrity of the match and make the process slightly less distressing to applicants. In addition, applicants should be aware of the NRMP reporting procedures for programs violating the match communication code of conduct.

Appendix

Supporting material provided by the authors is posted with the online version of this article as a data supplement at jbjs.org (http://links.lww.com/JBJSOA/A324).

TABLE I

Demographics of Survey Respondents

Characteristic	N (%)	AAMC 2020 Demographics
Total respondents	229	1,699
Sex
Male	176 (76.9%)	1,364 (80.3%)
Female	49 (21.4%)	334 (19.7%)
Prefer not to say	4 (1.7%)	1 (0.06%)
Ethnicity
American Indian/Alaska Native	1 (0.4%)	12 (0.7%)
Asian	34 (14.8%)	269 (15.8%)
Black/African American	10 (4.4%)	142 (8.4%)
White	178 (77.7%)	1,022 (60.2%)
Hispanic/Latin	12 (5.2%)	157 (9.2%)
Native Hawaiian/Pacific Islander	0 (0%)	2 (0.1%)
Others	6 (2.6%)	73 (4.3%)
Number of programs applied
<20	3 (1.3%)
21-40	7 (3.1%)
41-60	29 (12.7%)
61-80	73 (31.9%)
81-100	60 (26.2%)
>100	57 (24.9%)
Number of visiting student orthopaedic electives completed (mean, SD)	3.2 ± 1.0
Postinterview communication received?
Yes	91 (39.7%)
No	136 (59.4%)
N/A	2 (0.9%)
Percentage of programs that mentioned communication from applicants was actively discouraged or would not change the ranking process?
0%	18 (7.9%)
∼10	13 (5.7%)
∼25%	31 (13.5%)
∼50%	60 (26.2%)
∼75%	66 (28.8%)
∼90%	29 (12.7%)
∼100%	12 (5.2%)

TABLE II

Demographic Distribution of Applicants Who Did or Did Not Receive Postinterview Communication

Characteristic*	PICN (%)	No PICN (%)
Total respondents	91 (39.7%)	136 (59.4%)
Sex
Male	68 (38.9%)	107 (61.1%)
Female	22 (44.9%)	27 (55.1%)
Prefer not to say	1 (33.3%)	2 (66.7%)
Ethnicity
American Indian/Alaska Native	0	1
Asian	7	27
Black/African American	5	5
White	78	98
Hispanic/Latin	3	9
Native Hawaiian/Pacific Islander	0	0
Other	2	4
Number of programs applied
<20	2	1
21-40	6	1
41-60	13	15
61-80	33	39
81-100	19	41
>100	18	39
Number of visiting student orthopaedic electives completed (mean, SD)	3.3 (±1.1)	3.1 (±0.9)

TABLE III

Details Postinterview Communication Content

Postinterview communication recipients (N = 91)	N (%)
Number of programs (PIC) received per respondent (N=91)
1	40 (41.2%)
2-4	44 (48.4%)
5-7	7 (7.7%)
Format of PIC received
Phone call	48 (52.7%)
Letter	4 (4.4%)
E-mail	60 (65.9%)
Text message	15 (16.5%)
Other	6 (6.6%)
Person who communicated with applicant
Associate PD	5 (5.5%)
Chairperson	24 (26.4%)
Faculty member	19 (20.9%)
Program director	64 (70.3%)
Resident with specific aim of soliciting response commitment	22 (24.2%)
Secretary/program coordinator	15 (16.5%)
Other	2 (2.2%)
Did the program request to know where they were ranked?
Yes	7 (7.7%)
No	76 (83.5%)
Both (some did, some did not)	8 (8.8%)
Did the program tell you where you were ranked?
Yes	23 (25.3%)
No	44 (48.4%)
Both (some did, some did not)	24 (26.4%)
Did the programs indicate they would rank you at the top of the list if you ranked them 1?
Yes	3 (3.3%)
No	79 (86.8%)
Both (some did, some did not)	9 (9.9%)