Problematic communications during 2016 fellowship recruitment in internal medicine.

Some internal medicine residency program directors have expressed concerns that their third-year residents may have been subjected to inappropriate communication during the 2016 fellowship recruitment season. The authors sought to study applicants' interpersonal communication experiences with fellowship programs. Many respondents indicated that they had been asked questions that would constitute violations of the National Residency Matching Program (NRMP) Communications Code of Conduct agreement, including how they plan to rank specific programs. Moreover, female respondents were more likely to have been asked questions during interview experiences about other programs to which they applied, and about their family plans. Post-interview communication policies were not made clear to most applicants. These results suggest ongoing challenges for the internal medicine community to improve communication with applicants and uniform compliance with the NRMP communications code of conduct during the fellowship recruitment process.

Background

Many legal [1,2] and ethical standards guide appropriate communication between Internal Medicine (IM) fellowship applicants and programs during the recruitment season. For example, programs and applicants participating in the National Residency Matching Program (NRMP) Medical Specialties Matching Program (MSMP) must agree to the Specialties Matching Service Match Participation Agreement and are encouraged to follow the Match Communication Code of Conduct [3]. The agreement forbids programs from requesting applicants to disclose ‘the names, specialties, geographic locations, or other identifying information about programs to which they have or may apply,’ as well as preventing programs from requesting ranking preferences. Aside from the Match agreement, principles outlined in the code further professional standards with respect to post-interview communication, respecting applicants’ right to privacy and confidentiality, and program director responsibilities [3].

In September 2016, several IM​ residency programs began posting on a discussion forum that their residents were subjected to inappropriate communication with fellowship program directors (PDs)​ and faculty during the fellowship recruitment period. The NRMP responded to these and other reports by posting a message for program directors on its program distribution list. It emphasized the terms of the Match Participation Agreement that all applicants and programs sign, which seeks to ensure professional communication free from coercion and undue pressure. To determine the extent of inappropriate communication, the authors developed a questionnaire that was disseminated in May 2017 to residents who participated in the 2016 NRMP MSMP.

Methods

Development and solicitation of resident participation

The authors developed a quantitative and qualitative survey instrument intended to gain insight on the violations most commonly reported on the program director discussion forum, as well as to allow respondents to discuss other areas of concern (Appendix A). Besides basic demographic data such as gender and preferred fellowship specialty, it asked respondents to indicate whether fellowship PDs or other faculty asked them to name other programs to which they applied or how they intended to rank that program. Participants were also asked to specify perceived program director expectations of post-interview communication and whether sensitive questions, including family planning, relationship/marriage status and sexual orientation, were asked. An open-ended question collected applicants’ fellowship interview experiences and suggestions for improving the process.

The instrument was completely anonymous, meaning no responses could be linked to individuals, residency, or fellowship programs. Respondents were discouraged from including any identifying information in the open-ended comments section. The survey was approved by the University of California, San Francisco (UCSF) Committee on Human Research (IRB #17–21,934).

The authors announced the survey and its purpose through the internal medicine residency discussion forum in May 2017. Weekly reminders were sent until mid-June. Most IM residency PDs and their administrators subscribe to this discussion forum. In a separate communication, coordinators were asked to forward the email to all current residents known to have applied to fellowship. The authors did not have access to which program directors forwarded the request to their residents or to which residents completed the survey.

Data analysis

Survey results are reported using frequency distribution. ‘Do not recall’ responses were excluded from analysis. Certain question responses were cross-tabulated with gender and high or low fill-rate specialty, a new variable that characterized a ‘high’ specialty as having at least a 95% fill rate based on the 2016 MSMP. Where ‘fill-rate’ analysis is discussed, specialties not participating in MSMP are excluded from analysis. Resident free text comments were analyzed by conventional content analysis. Coded categories derived directly from the text data were created in order to convey expressed comments [4].

Results

There were 403 responses, of which 387 reported applying to fellowship during the 2016 recruitment season. This convenience sample represents approximately 7% of the 5,285 certified applicants in the 2016 subspecialty match [1]. Most respondents (73%) were in university-based residency programs, with respondents from community programs (27%) constituting the remainder. Eighty-seven percent reported starting fellowship at a university-based fellowship. The five most commonly reported matriculating specialties (% of total respondents) were: cardiology (23%), pulmonary/critical care (17.4%), hematology/medical oncology (12%), gastroenterology (11%), and infectious disease (9%). Of the 358 respondents who answered the gender identity question, 142 (40%) identified as female and 216 (60%) identified as male.

Demographic questions

Table 1 summarizes the results relating to demographic questions, including a gender breakdown. One-half of survey respondents answered affirmatively to whether they had been asked about relationship/marriage status during any interview-related activity (e.g. interview, facility tour, and pre-interview dinner). No gender difference appears except for family planning questioning. The percentage of respondents reporting age- and family planning-related questions were 12% and 15%, respectively. Tweny-seven percent of females and 9% of males reported that they were asked about current or previous plans to have children. Questions about religious affiliation or beliefs and sexual orientation appeared infrequently (<1%).

Table 1.

Questions asked during 2016 interviews or interview-related activities.

	% of ‘Yes’ respondents (number of ‘Yes’ responses)	% of female respondents with ‘Yes’ response (number of female respondents)	% of male respondents with ‘Yes’ response (number of male respondents)
Demographic questions
Age	12% (43)	11% (15)	12% (24)
Religious affiliation or religious beliefs	2% (7)	2% (3)	2% (4)
Sexual orientation	<1% (3)	<1% (1)	<1% (2)
Relationship/marriage status	49% (182)	52% (72)	51% (104)
Current or previous plan to have children	15% (55)	27% (36)	8.7% (18)
Questions about ‘other’ programs
Fellowship program director asked to name other programs to which respondent applied	34% (127)	43% (54)	35% (71)
Faculty, other than fellowship program director, asked to name other programs to which respondent applied	52% (194)	65% (85)	51% (103)
Fellowship program director asked to name how respondent planned to rank program	10% (36)	12% (17)	9% (19)
Faculty, other than fellowship program director, asked to name how respondent planned to rank program	9% (33)	9% (13)	9% (19)
Asked about other specialties to which respondent applied	3% (10)	2% (3)	3% (7)
Asked to return for second visit	7% (24)	6.4% (9)	6.5% (14)
Asked to do a visiting rotation to increase chances of matching at a particular program	2% (7)	4% (5)	<1% (2)

Questions about ‘other’ programs

Table 1 also provides frequency distribution based on questions about the programs to which the respondent applied, plans for ranking a specific program, other specialties to which the respondent applied and requests for a second visit and visiting rotation. The questions garnering the largest overall percentage of ‘yes’ responses include: faculty asking to name other programs of interest (52%), program director asking to name other programs of interest (34%), and program director asking to disclose his/her rank position (10%). Females reported being asked to name other programs of interest by fellowship program director or faculty more often than males.

Illustrative comments detailing interview questions include:

I was directly asked where I was applying, my marriage and family status and how I planned to rank. Sometimes these questions were direct and other times they were subtle. I didn’t rank the program(s) that I thought asked intrusive questions.

Multiple times asked about plans for pregnancy or if I am currently pregnant. Also asked very specifically what other programs I applied [to].

Perceived expectations about post-interview communication

Table 2 provides results of overall and gender-specific data based on responses to perceived expectations about post-interview communication. Overall, a plurality (40%) of respondents indicated that the program director encouraged, but did not require post-interview communication. Thirty-eight percent reported that they did not know how program directors felt about post-interview communication. A small percentage (2%) felt that PDs required post-interview communication. Females were less likely than males to report that PDs did not desire post-interview communication, respectively 13% v 24%, but more likely to report that PDs encouraged post-interview communication, 45% v 37%, respectively.

Table 2.

Perceived program director expectations about post-interview communication – overall and by gender.

	Percentage (number of respondents)*	% of female respondents (number of females)*	% of male respondents (number of males)*
PDs did not desire any post-interview communication	20% (73)	13% (19)	24% (52)
PDs encouraged, but did not require post-interview communication	40% (147)	45% (63)	37% (80)
PDs required post-interview communication	2% (8)	4% (5)	1% (3)
I do not know how program directors felt about post-interview communication	38% (138)	38% (54)	38% (81)

*Not all respondents chose to report gender identification

Several comments demonstrate the complexity surrounding program communication with applicants:

Political process where communication post-interview probably mattered at least as much as the criteria that got me the interview in the first place.

Some programs seemed to focus more on post-interview communication, but even then it seemed they just wanted to know if applicants were truly interested in their program … It seemed common sense that a program would want to know applicants who were very interested in their program, and I freely wanted to tell programs at the top of my list that I was very interested in them.

I was led on by a program director who provided myself and my program director feedback beyond saying they really liked me. They used terms like, ‘We look forward to seeing you in July,’ and, ‘If we are your top choice you’ll end up here.’ I did not end up there despite them being my top choice.

I think it was most helpful when program directors would say specifically what they expected as far as communication went post interview.

Multiple programs made it clear that they would not ‘rank me to match’ unless I clearly communicated to them that I would rank them #1.

There is intense pressure to rank a program number 1 … Programs seem quite fixated on not falling down their rank list and so will prioritize those who designate a program #1. This culture needs to change.

Qualitative analysis of specific resident comments

As previously discussed, residents were asked to provide free text comments regarding their individual fellowship interview experiences and any suggestions they had for improving the process. Comments were provided by 133 residents (32.8%). Since comments were free text, some covered multiple elements in the process and included both negative and positive aspects. Table 3 summarizes positive and negative comments based on category. Other than general comments, the interview garnered the most positive comments. Expenses (16), scheduling (15), and post-interview communication (15) were the categories garnering the most negative comments. Interview-related comments fell into one of three categories based upon qualitative analysis: right to privacy (12), recruitment team (3), and coercive questions (12).

Table 3.

Categorization of resident comments.

Category	Positive comments	Negative comments
Interview; conduct of and lack of time	9	13
Residency program support	1	3
Application	0	7
Expenses	0	16
Scheduling	0	15
Post-interview communication	1	15
General comments	51	2
Total	62	71

Discussion

To our knowledge, this is the first study to examine the fellowship interview experiences of Internal Medicine residents who have participated in the NRMP subspecialty match. Our results document ongoing challenges with uniform alignment with Section 6.0 of the Match Participation Agreement and the NRMP Code of Conduct [3], including coercive interview questions inquiring about where else applicants were applying and how they were planning to rank the program. In addition, a substantial number of applicants reported being asked at some point during the interview process about their relationship status and children/family plans. Variable approaches to post-interview communication policies were also reported. Moreover, women were more likely to report being asked questions about other programs to which they applied and their family plans. Content analysis of qualitative data [4] revealed many concerns such as challenges with scheduling interviews, the considerable expenses incurred with the application and interview process, the conduct of the interviews and ambiguous, sometimes misleading, post-interview communications.

Since at least 2004, other specialties have reported similar results involving inappropriate communication during the recruitment season. Holliday et al. [5] reported that 90% of survey respondents for Radiation Oncology residency positions indicated experiences in conflict with NRMP policies. Sbicca et al. reported [6] that 44% of Dermatology applicants were asked about their marital status and 19% were asked if they had or intended to have children. Similarly, Hessel et al. [7], reported that 32% of female fourth year medical students, compared to only 3% of male students, were asked about family planning during their residency interviews. Hern et al. [8] reported that 65.9% of medical students had been asked at least one illegal question during interviews, with a higher percentage of these questions in surgical, rather than non-surgical, specialties.

Taken together, the respondents in our survey of Internal Medicine resident applicants mirror those previously reported by other specialty applicants. Some faculty and program directors may not be aware of the literature on the subject, or they may have misunderstood or disregarded the NRMP Code of Conduct. Given our and other survey results, it appears likely that asking sensitive and inappropriate questions remains an ongoing, undesired part of the applicant experience.

This study has several limitations. There was a relatively small number of respondents compared to the total pool of subspecialty applicants, making the results not necessarily representative of the experiences of the total applicant pool. In addition, because the survey was anonymous, it could not be determined whether respondents were from a small number of programs, leading to dissemination bias. It is also possible that duplicate surveys could have been completed. In addition, the survey was distributed more than six months after NRMP match lists were submitted, so respondents’ recollections of their interview experiences may have introduced error. Subspecialty analysis was not possible due to the small number of respondents pursuing fellowships in various subspecialties.

Based on these survey data the authors strongly suggest that the internal medicine community implement the following recommendations:

Before the interview season starts, program directors should inform faculty, current trainees, and others who interact with applicants what questions cannot be asked of applicants.

During the interview day, program directors should explicitly state their policy on post-interview communication with applicants.

Program directors, or their designees, should never ask applicants about their rank lists.

A more robust study of applicants’ experiences and perceptions of the fellowship application process should be conducted to better capture problematic behavior and its consequences for both applicants and programs.

An anonymous reporting system, that does not require login, should be established in an effort to promote applicants’ real-time reporting of inappropriate recruitment behavior.