The gynecologic oncology fellowship interview process: Challenges and potential areas for improvement.

The application and interview process for gynecologic oncology fellowship is highly competitive, time-consuming and expensive for applicants. We conducted a survey of successfully matched gynecologic oncology fellowship applicants to assess problems associated with the interview process and identify areas for improvement. All Society of Gynecologic Oncology (SGO) list-serve members who have participated in the match program for gynecologic oncology fellowship were asked to complete an online survey regarding the interview process. Linear regression modeling was used to examine association between year of match, number of programs applied to, cost incurred, and overall satisfaction. Two hundred and sixty-nine eligible participants reported applying to a mean of 20 programs [range 1-45] and were offered a mean of 14 interviews [range 1-43]. They spent an average of $6000 [$0-25,000], using personal savings (54%), credit cards (50%), family support (12%) or personal loans (3%). Seventy percent of respondents identified the match as fair, and 93% were satisfied. Interviewees spent a mean of 15 [0-45] days away from work and 37% reported difficulty arranging coverage. Linear regression showed an increase in number of programs applied to and cost per applicant over time (p < 0.001) between 1993 and 2016. Applicants who applied to all available programs spent more (p < 0.001) than those who applied to programs based on their location or quality. The current fellowship match was identified as fair and satisfying by most respondents despite being time consuming and expensive. Suggested alternative options included clustering interviews geographically or conducting preliminary interviews at the SGO Annual Meeting.

Introduction

The application process to receive training in gynecologic oncology is highly competitive. Fellowship programs accredited by the American Board of Obstetrics and Gynecology (ABOG) participate in the National Residency Match Program (NRMP). The NRMP is an independent, not-for-profit organization that manages both the Main Residency Match as well as a Specialties Match Service that includes 27 Matches for nearly 60 subspecialties (National Resident Matching Program, n.d.). The matching process involves submission of program rank lists by applicants and applicant rank lists by fellowship programs that are matched by a computer system. A successful match constitutes a binding agreement between a program and the matched applicant.

Obstetrics and gynecology residents applying for fellowship in gynecologic oncology apply to programs in December of their third year. Applicants receive invitations to interview from May to August of the following year. Rank lists are due in September and the NRMP reveals match results in early October. In 2016, 70 US graduate applicants and 10 foreign applicants applied for a total of 56 positions in 41 programs. Fifty-five applicants (69%) matched, filling 98% of the available slots (National Resident Matching Program, n.d.).

Applicants may choose how many programs to apply to, and typically invited applicants are given 2 to 3 interview dates to choose from. They must arrange coverage of their clinical services at their home institution and travel and lodging is paid for out-of-pocket by the applicant. A dinner or social gathering the night before typically precedes the interview. The interview day begins sometime in the early morning and concludes in the afternoon. While some programs employ several one-on-one interviews, others prefer a group interview style with 2 to 4 interviewers per applicant. Depending on flight availability, applicants may need to spend another night in a hotel and travel home the next day. Therefore, applicants must arrange to be away from their hospital for 2–3 days per interview.

Prior to the NRMP system, fellowship application was a free-market system where programs actively recruited candidates, and candidates had to apply to each program individually. The NRMP system was designed to level the playing field and to help fellowship program directors reduce the hassle of recruiting employees. However, there are some perceived drawbacks to the NRMP. Fellowship programs require planning for interviewing and ranking a large number of residents. Residency program directors need to arrange coverage for residents away from duties and residents face cost difficulties and time away from service.

To the best of our knowledge, no prior literature has been published regarding how applicants to gynecologic oncology fellowship programs view the match process. Insight into problems encountered by candidates could help prompt change in the way we conduct interviews. The purpose of our study was to conduct a web based survey of successfully matched gynecologic oncologists who participated in the NRMP match system. Our hypothesis was that successfully matched gynecologic oncology applicants will report having spent a great deal of time and money on the application process and will have novel ideas for improvement of the match process.

Materials and methods

After IRB approval by the Albert Einstein College of Medicine Institutional Review Board, all Society of Gynecologic Oncology (SGO) list-serve members who have participated in the National Residency Match Program® (NRMP) for gynecologic oncology fellowship were asked to complete an online survey delivered by Survey Monkey® modeled after surveys used to query participants in surgical and medical subspecialty matches (Appendix I) (Frishman et al., 2016, Meals and Osterman, 2015, Bernacki et al., 2012). Linear regression modeling was used to examine association between year of match, number of programs applied to, cost incurred, and overall satisfaction. Data for a cohort of respondents who matched between 2008 and 2016 was analyzed separately as no information exists regarding how many programs participated in the NRMP prior to 2008. List-serve members were incentivized to respond by pledging to donate $5.00 to the Foundation for Women's Cancer for each response received.

Participants were asked to provide free text responses to the following queries: 1) What would you say are the major benefits of the gynecologic oncology match process? 2) What would you say are the major disadvantages of the gynecologic oncology match process? 3) Please use this space to provide us with suggestions on how you think the gynecologic oncology match process could be improved. Responses were recorded in face valid terms, and were coded to domains by three independent judging reviewers. Each individual response could be coded to one or multiple thematic domains depending upon content appraisal by each judge. Kappa values were calculated to assess consistency in agreement between judges in assignment of domains. For all thematic domains with kappa < 0.70, arbitration was conducted for code assignment amongst the three reviewing judges. The process of arbitration included reading of free text statements and comparison of initial coding. For statements coded to disparate domains, arbitration amongst the three judges decided on the most appropriate code(s). For these statements with disparate coding, manualization of rules for code assignment was conducted and recorded (Appendix II). Manualized rules were then used to check domain assignment. For themes with kappa > 0.7, domains were assigned to statements by majority rule.

Results

A total of 2057 list serve members of the Society of Gynecologic Oncology were identified. Twenty-four (1%) were returned undeliverable. Of the 282 survey respondents (14% of delivered), 269 were eligible (13% of delivered). Thirteen respondents were ineligible as they self-identified as interviewing prior to the NRMP match. Two hundred and eleven (75%) responded to the first request, 27 (10%) to the second request, and 43 (15%) to the third request.

Most applicants (91%) applied during only one match cycle, whereas 8% applied more than once (Table 1). Participants applied to a mean of 20 programs (range: 1–45) and were offered a mean of 14 interviews (range: 1–43). They spent an average of $6000 (range: $0–25,000), using personal savings (54%), credit cards (51%), family support (11%) or personal loans (3%) to cover costs. Interviewees spent a mean of 15 (range: 0–45) days away from work and 37% reported difficulty-arranging coverage. Linear regression showed an increase in number of programs applied to and cost per applicant over time (p < 0.001). Those who matched lower on their list were more likely to be disappointed (p < 0.001) or think the process was unfair (p = 0.03). Applicants who applied to all available programs spent more money ($2473 extra, p < 0.001) than those that applied based on geographical preference or quality of a program. In the cohort of respondents analyzed separately who matched between 2008 and 2016 (n = 149), more money was spent each year after controlling for number of programs participating and number of interviews attended (Appendix III). Nineteen people in this cohort (13%) reported applying to more than the number of programs available. The majority of respondents identified the match as fair (70%) and were satisfied (93%) by Likert scale response. Ninety-three percent matched to his or her first or second ranked program.

Table 1

Results of fellowship applicant questionnaire. Descriptive statistics provided for n = 269.

Survey question	Frequency (%)	Median [range]
Year of fellowship match		2010 [1993–2016]
Number of years applied
1	246 (91)
2	21 (8)
3 or more	1 (0.5)
Unknown	1 (0.5)
Number of programs applied to		20 [1–45]
Number of interviews offered		14 [1–43]
Number of interviews completed		11 [1–24]
Factors for applications:
Geographical preference	108 (40)
Quality of training	152 (57)
Applied to all, then selected	82 (31)
Missing	8 (3)
Time spent away to interview (days)		15 [0–45]
Difficulty in arranging coverage:
No	166 (62)
Yes	100 (37)
Missing	3 (1)
Match system fairness:
Very fair	86 (32)
Somewhat fair	104 (39)
Unsure	51 (19)
Somewhat unfair	20 (7)
Very unfair	6 (2)
Missing	2 (1)
Match choice:
First	133 (49)
Second	51 (19)
Third	22 (8)
Fourth	24 (9)
Higher than fourth	37 (14)
Missing	2 (1)
Satisfaction with Match:
Very satisfied	220 (82)
Somewhat satisfied	32 (12)
Neutral	7 (3)
Disappointed	6 (2)
Very disappointed	4 (2)
Missing	0 (0)
Amount spent on application process (dollars)		$6000 [0–25,000]
How were interviews financed?
Personal savings	146 (54)
Credit Card	137 (51)
Took out personal loan	8 (3)
Borrowed money from family/friends	31 (12)
Supplemental income	11 (4)
Missing/unanswered	4 (2)
Outstanding debt prior to application?
No	71 (26)
Yes	198 (74)
Missing	0 (0)
Amount of prior debt		$172,500 [5000–500,000]

Five hundred and fifty-two free responses by 269 respondents were recorded for three questions: 1) What are the benefits of the interview process? (212 free responses, 79% of respondents), 2) What are the disadvantages of the interview process? (218 free responses, 81% of respondents), and 3) Suggestions for improvement? (122 responses, 45% of respondents). Thematic domains for benefits of the match process with kappa agreement ≥ 0.7 were fairness, networking, other and no response. Domains of the disadvantages of the match process with kappa ≥ 0.7 were cost, time, and no response. Suggestions for improvement that had kappa ≥ 0.7 were better coordination between programs, program cost subsidization, geographical subsidization/SGO interviews, capping interviews, shorter interviews, programs taking internal applicants should withdraw from match, and no response. Free responses regarding benefits the of interview process included domains of fairness of the process, opportunity to network, see new institutions, and explore new opportunities (Table 2, Fig. 1). Free text comments regarding the disadvantages within the current fellowship match process were dominated by themes of monetary and time expenditure (Table 2, Fig. 2). The last free-text survey question asked respondents to comment on how they thought the gynecologic oncology interview process could be improved (Table 2, Fig. 3).

Table 2

Thematic domains regarding the fellowship interview process identified within free text responses by survey respondents. Free text responses were total n = 552, n = 212 for benefits, n = 218 for disadvantages, n = 122 for suggestions for improvements.a

Domain: n (%)
Benefits of the interview process	Fairness	Networking	Visiting institutions	Exploring new options	Organization	Familiarity	Becoming a GYN Oncologist	No benefit	Other	No response
N = 212	58 (27)b	62 (29)b	57 (27)	52 (25)	35 (17)	4 (2)	25 (12)	9 (4)	6 (3)b	57 (21)b
Disadvantages of the interview process	Cost	Time	Unfairness	Connections required	Poor coordination	Uncertainty regarding outcome	Process favors programs	No disadvantages	Other	No response
N = 218	131 (60)b	99 (45)b	57 (26)	40 (18)	27 (12)	17 (8)	5 (2)	4 (2)	15 (7)	51 (19)b
Suggestions for improvement	Better coordination between programs	Programs should subsidize cost	Transparency	Screen applicants before interviews	Centralize interviews geographically or at SGO	Shorter interviews	Cap number of interviews offered	Programs taking internal applicants should withdraw early	Other	No response
N = 122	37 (31)b	14 (12)b	27 (23)	20 (17)	29 (24)b	3 (3)b	3 (3)	11 (9)b	19 (16)	147 (54)b

Each statement was allowed to be coded to more then one thematic domain.

Thematic domains-kappa ≥ 0.7.

Fig. 1

Representative quotes from respondents regarding benefits of the current fellowship matching process.

Fig. 2

Representative quotes from respondents regarding disadvantages of the current fellowship matching process.

Fig. 3

Representative quotes from respondents regarding ways to improve the current fellowship matching process.

Discussion

Our study demonstrates that the gynecologic oncology fellowship application process is time consuming and expensive for applicants. As time progresses, applicants are applying to more programs and the financial burden of interviewing is increasing. Despite these barriers, the majority of respondents reported that the process was fair and satisfying. Participants had many ideas on how to improve the interview process, and suggested clustering interviews geographically or conducting preliminary interviews at the SGO Annual Meeting.

Our study shares similarities with other published evaluations of subspecialty match programs. In 2015, the Committee on Fellowship Training in Obstetrics and Gynecology (COFTOG) issued recommendations from a panel discussion that included allocation of weeks of each month for particular subspecialty interviews, supplemental video or national meeting interviews, regional coordination of interviews, and expansion of social networking between applicants to share expenses (Frishman et al., 2016). Meals et al. survey of 61 hand surgery fellowship applicants resulted in suggestions to centralize or coordinate interviews, universal applications (i.e. eliminating supplemental applications), limitation on numbers of interviews accepted or offered, and alternatives to in-person interviews (Meals and Osterman, 2015). In a survey of 366 pathology residents applying to fellowship, 21% stated they would prefer a matching program over a free-market system (Bernacki et al., 2012).

There were similarities in the out of pocket monetary and time away from residency spent by applicants to gynecologic oncology and other subspecialty fellowships. We found an average cost of $6000 with a higher upper limit range of $25,000. Iqbal et al. reported $5818 spent by matched and $3786 spent by unmatched candidates to obstetrical and gynecologic fellowship programs (Iqbal et al., 2014). An average of $6000 spent per candidate was also similar to the report by COFTOG/CREOG (Frishman et al., 2016). In a survey of 113 neuro-radiology fellowship candidates, cost was listed as the number one disadvantage of the interview process (Hammoud et al., 2005). Niesen et al. surveyed orthopedic surgery candidates who reported an average expense of $4671 +/− $2454 (range $0–12,000), missed an average of 10 days of work (range 1–24) and completed an average of 10 ± 3.4 (range 1–20) interviews (Niesen et al., 2015). Participants in our study cited a slightly higher number of average days out of work, 15 days, and higher maximum number of 45 days. In another survey of pediatric surgery fellows, the average candidate expense was $6974, which represented 14% of the average applicants' total salary (Little et al., 2005).

There are several differences in our study as compared to the existing literature. In our analysis, we demonstrated increased cost and number of programs applied to over time. Our study is unique in that we were able to obtain free-response suggestions from participants, code them using grounded theory methodology and report them in idiographic format. Free response answers explicitly stated that some applicants perceived that programs broke rules, hired candidates outside of the match and exercised nepotism in ranking candidates. Interestingly, there was a correlation with perception of fairness and satisfaction with the ranking of matched position.

Our study had a 14% overall response rate which falls short of results reported by similar studies with response rates of 32%–80% (Meals and Osterman, 2015, Hammoud et al., 2005, Little et al., 2005, Cannada et al., 2015, Shetty et al., 2005). The low response rate may be in part due to the fact that we surveyed the whole SGO community rather than specifically querying recent applicants to gynecologic oncology fellowship. Our study is also limited in that we included only participants who had successfully matched to fellowship positions. It would be very valuable to have data from individuals who have applied to gynecologic oncology fellowships and not been successful. Our study is also limited by significant recall bias. Thirteen percent of respondents who applied between 2008 and 2016 reported applying to more programs than were available to apply to for that particular year (Appendix III), exemplifying recall bias that is inherent in any survey study.

The current fellowship interview process is time consuming and expensive. The cost of spending an average of 15 days away from residency training should not be overlooked, as the institutional burden of this cannot be quantified. Suggested options for improvement include clustering interviews geographically, conducting preliminary interviews at the SGO Annual Meeting, greater effort at prescreening candidates, and retrospective review of results of match practices and results for both candidates and programs. Future publication of match statistics for both programs and candidates might also provide transparency to validate fairness of the NRMP match.

Conflict of interest statement

The authors of this study have reviewed and completed the International Committee of Medical Journal Editors (ICMJE) Form for Disclosure of Potential Conflicts of Interests. None of the authors have any conflicts to disclose.