Factors Considered in Ranking Hip and Knee Arthroplasty Fellowship Applicants: A Survey of Program Directors.

BACKGROUND: Variables considered by hip and knee arthroplasty fellowship program directors (PDs) to select fellowship candidates are not well known.
METHODS: A web-based questionnaire containing 5 questions was developed and sent to all 92 adult reconstruction fellowship PDs via email. Three questions collected program information including the number of positions available, the number of candidates interviewed, and ranked annually. PDs were then given a list of 12 factors and asked to rank them in the order of importance. A weighted score for each factor was calculated using the following scale: 5 points each time a factor was ranked 1st, 4 points each time a factor was ranked 2nd, 3 points for each 3rd place rank, 2 points for each 4th place rank, and 1 point for each 5th place rank. PDs were also allowed to write in other factors they considered important when ranking fellowship candidates.
RESULTS: The overall response rate was 34.8% (32/92). Seventy-five percent of responding programs indicated that they interview between 21 and 40 applicants per year for their fellowship position(s). The interview was ranked as the most important variable in selecting applicants by 53.1% of responding PDs, followed by letters of recommendation (ranked first by 25% of PDs) and personal connections to the applicant and/or letter writer(s) (ranked first by 9% of PDs). A positive correlation was identified between the program size and an applicant's geographical ties to the city/town of the fellowship program (r s  = 0.472; P = .006).
CONCLUSIONS: According to hip and knee arthroplasty fellowship PDs, the interview, letters of recommendation, and personal connections to the applicant and/or letter writers are the most important factors considered in selecting arthroplasty fellowship candidates.

Introduction

Orthopaedic surgery subspecialty fellowship training has become increasingly popular over the last several years [1], with 90% of all physicians taking the American Board of Orthopaedic Surgery part II having completed at least 1 year of fellowship training [2]. Not surprisingly, job advertisements are increasingly targeting fellowship-trained orthopaedic surgeons over general orthopaedists [3]. Furthermore, orthopaedic residents who intend to practice general orthopaedics may still desire additional training in one particular subspecialty.

Over the last decade, hip and knee fellowship training has gained popularity at a more rapid rate than any other orthopaedic surgery subspecialty [4]. This field, also commonly referred to as “adult reconstruction,” is considered the most selective of the orthopaedic subspecialties, with a fellowship match rate of only 68% [5]. In their annual match statistics report, the American Association of Hip and Knee Surgeons (AAHKS) noted that there were 235 total applicants for 183 total positions equating to 1.28 applicants per available position. There were 177 residents who matched (75.3%) consisting of 151 United States allopathic graduates, 20 United States osteopathic graduates, 1 graduate from a Canadian program, and 5 foreign graduates [6]. Given the growing competitiveness of the subspecialty, it is important for well-qualified applicants to focus their application and interview strategies on areas considered important by program directors (PDs). However, little is known about the particular applicant factors adult reconstruction PDs consider most important.

The goal of the present study was to survey adult reconstruction fellowship PDs to identify the factors they considered most important in selecting fellowship candidates. Our hypothesis was that the interview would be the most important variable in selecting fellowship candidates.

Material and methods

This study was determined to be exempt by our local institutional review board. A web-based questionnaire containing 5 questions was developed and sent to all 92 adult reconstruction fellowship PDs via email (SurveyMonkey, San Mateo, CA). The questions were designed by the study team to identify factors deemed important by PDs in selection of fellowship applicants (Appendix 1). Questions 1-3 were multiple-choice questions with the following program details: (1) the number of fellowship positions available at each program; (2) the number of applicants each program interviews annually; and (3) the number of applicants each program ranks annually. The fourth question asked PDs to rank, from 1 to 12 with 1 being the most important, a list of 12 applicant variables that could be used to select applicants. Finally, PDs were given a free-text option allowing them to indicate any additional factors they consider when ranking applicants. The full questionnaire can be found in Appendix 1.

A complete list of adult reconstruction fellowship programs was obtained from the AAHKS website. PDs’ email addresses not listed on the AAHKS website were identified using the San Francisco Match website, individual fellowship program websites, the American Academy of Orthopaedic Surgeons website, or by searching for corresponding authors’ email addresses in PubMed. In the event that the email address for the PD could not be identified by the aforementioned means, the survey link was sent to the email address of the program coordinator as listed on the San Francisco Match website. In addition, the adult reconstruction representative on the American Academy of Orthopaedic Surgeons Match Committee assisted in promoting the survey. The initial email was sent out on June 27, 2019 containing a link to the web-based questionnaire, and 2 additional reminder emails were sent out on July 12, 2019 and July 25, 2019, respectively. Each of the 3 senior authors sent out a single round of emails.

IBM SPSS Statistics® Software v.26 (Armonk, NY) was used for statistical analysis. Counts and percentages were calculated for questions 1-3. For question 4, a weighted score was calculated using the following scale: 5 points each time a factor was ranked 1st, 4 points each time a factor was ranked 2nd, 3 points for each 3rd place rank, 2 points for each 4th place rank, and 1 point for each 5th place rank. The cumulative scores were then aggregated to identify which variables were deemed most important. The number of times a variable was chosen as the “most important” variable was also calculated. Spearman rank-order analysis was performed to identify relationships between program size and the factors ranked.

Results

The overall response rate was 34.8% (32/92). Responding PDs took 3 minutes, on average, to complete the questionnaire. The majority of PDs who responded had either 1 (37.5%) or 2 (28.1%) fellowship positions. Programs most commonly interview either 21-30 (46.9%) or 31-40 (28.1%) applicants for their fellowship positions each year. There was wide variation in the number of applicants ranked each year, with the most common response being 21 to 25 (31.3%). A complete summary of PDs' responses to questions 1-3 can be found in Table 1.

Table 1

Program director responses to questions 1-3.

Response	# Programs	Percent programs
Q1: How many fellowship positions are available in your program?
1	12	37.50%
2	9	28.13%
3	3	9.38%
4	4	12.50%
5	2	6.25%
6	2	6.25%
Total	32
Q2: Approximately how many applicants do you interview each year?
<10	0	0.00%
11 to 20	4	12.50%
21 to 30	15	46.88%
31 to 40	9	28.13%
40 or more	4	12.50%
Total	32
Q3: Approximately how many applicants do you rank each year?
<10	1	3.13%
11 to 15	7	21.88%
16 to 20	7	21.88%
21 to 25	10	31.25%
>26	7	21.88%
Total	32

The interview was ranked as the most important variable in selecting applicants by 53.1% of responding PDs with an average rank (from 1 to 12) of 2.16. A complete listing of the frequency of variables selected as the most important is demonstrated in Figure 1. Results of the weighted score calculation are displayed in Figure 2. The interview (123 points), letters of recommendation (LORs) (104 points), and personal connections to the applicant and/or familiarity with the applicant’s letter writer(s) (70 points) were the 3 highest rated applicant variables. Reputation of the applicant’s medical school (1 point) and extracurricular activities (2 points) had the lowest weighted scores.

Figure 1

Overall score of the 12 factors taken into account when ranking adult reconstruction fellowship applicants.

Figure 2

Frequency of factors identified as most important by adult reconstruction fellowship directors in ranking applicants.

The Spearman rank-order analysis revealed a moderate, positive correlation between the program size and the number of applicants interviewed, demonstrating larger programs interview more applicants (r = 0.377; P = .034). There was a moderate, positive correlation between the program size and the number of applicants ranked, indicating larger programs rank more applicants (r = 0.511; P = .003). A significant positive correlation was also identified between the program size and an applicant’s geographical ties to the city/town of the fellowship program (r = 0.472; P = .006). A complete breakdown of correlation analysis between the program size and applicant variables can be found in Table 2.

Table 2

Spearman rank-order analysis between the fellowship program size and fellowship applicant variablesa.

Applicant variables	Spearman rho (rs)	P-value
The number of applicants interviewed	0.377	.034
The number of applicants ranked	0.511	.003
Strength of the applicant’s residency training in adult reconstruction	−0.148	.419
The applicant’s geographical ties to the city/town of fellowship program	0.472	.006
Interest in a career in academics	−0.192	.292
The interview	0.332	.063
Research experience	−0.150	.412
Letters of recommendation	−0.254	.161
Personal connections to the applicant and/or letter writer(s)	−0.158	.387
Extracurricular work (mission work, service work, etc.)	0.217	.232
Reputation of the applicant’s residency program	−0.133	.468
Reputation of the applicant’s medical school	0.210	.248
Interesting or unusual life experiences	0.092	.618
Comments made regarding the applicant’s technical competence	0.067	.715

All variables were analyzed in correlation to the program size of respondent program directors. The range of the program size was 1-6 positions.

Select PDs (12.5%) answered the final question with a free-text response. Their responses included the following: “looking for [a] well-rounded, solid person who gets along with people… Most important that they are committed to learning”; “performance through medical school and residency—for example, United States Medical Licensing Exam (USMLE) scores, grants, awards, and presentations, and also, health policy, global health, business of medicine, and basic science interests, and essentially, anything that shows continuous excellence and broader interest and capacity above good clinical skills”; “direct feedback from the orthopaedic residency program”; and “interest in value-based care, leadership, and health policy work, which are part of our curriculum.”

Discussion

This study demonstrates that fellowship PDs believe the interview is the most influential factor in ranking prospective hip and knee arthroplasty fellowship applicants. Although that is not surprising, what applicants may find helpful is that LORs, personal connections of the PD to the faculty who write the applicant’s LOR, and the reputation of the applicant’s residency program were also noted as important variables by PDs when ranking applicants. In addition, positive correlations were identified between the fellowship program size and the number of applicants interviewed, the number of applicants ranked, and the geographical ties of an applicant to the city where a fellowship program is located. Ours is the first study to investigate this question among adult reconstruction fellowship PDs [7].

Several previous articles have documented the factors considered important in selecting fellowship candidates for fellowship training in other orthopaedic subspecialties. In sports medicine, Baweja et al. noted the interview, LORs, the applicant’s residency program, and research experiences as the most important factors [8]. In hand surgery, LORs, completion of an orthopaedic surgery residency, comments regarding the applicant’s technical competence, and having an M.D. as opposed to D.O. degree were identified as the most important factors in obtaining an interview invitation [9]. In a comprehensive survey study of all orthopaedic subspecialty fellowship directors in 2013, LORs from subspecialty faculty, quality of the residency program of the applicant, and LORs from the residency PD were identified as the 3 most important criteria in fellow selection [10]. When creating a rank list, the interview was deemed the most important criterion [10]. Technical competence was identified as the fourth most important variable, and anecdotally, this factor is often incorporated into the interview day for surgical residency and fellowship interviews. Our study did not specifically ask PDs to state whether or not they incorporated a practical/technical skill exercise into their interview day. This may be an area of further investigation in future studies. In addition, while sports medicine faculty valued research experiences as the 4th most important candidate variable, our study in adult reconstruction demonstrates that research was the 6th most important variable, of the choices provided [8]. One possible explanation is that sports medicine has been a highly competitive match for over a decade, leading to a large number of applicants for available spots. As DeFroda et al. [11] recently reported, the research productivity is increasing among sports medicine fellowship applicants with programs located in the Northeast region tending to match applicants with a higher number of publications. Research experiences offer an additional set of variables for which to compare and contrast fellowship candidates. The academic productivity of the programs whose PDs responded in the present study is not known. This could have influenced how strongly research was considered. It is unknown if sports medicine fellows, on average, publish more peer-reviewed literature than adult reconstruction fellows; however, it is our belief that as adult reconstruction fellowship continues to grow in competitiveness, research experiences including publications and posters/podium presentations will become increasingly valued in the selection process.

With the growing competitiveness of adult reconstruction fellowship positions and the overall fellowship application match as demonstrated by the mean 1.28 applicants per fellowship position in 2019 [6], it is important for potential applicants to understand the application process, the number of positions available, and the variables deemed important by fellowship PDs. The majority of responding PDs train either one (37.5%) or 2 (28.1%) fellows per year. However, the median range of the number of applicants ranked was 21-25, demonstrating the competitiveness from both an applicant standpoint and from a program perspective competing for quality applicants. In addition, the geographic ties by an applicant may also factor into applicant selection. This study also identified a moderate, positive correlation between the program size and geographical ties of an applicant to the city where a program is located. Although it is not entirely clear why such a correlation exists, a few possible reasons include that the programs with more fellowship positions tend, on average, to be more academically focused, located in large metropolitan areas, and highly competitive.

A common method of identifying program information is to visit individual fellowship program websites, which typically include a list of faculty, a description of fellowship activities, current fellows, alumni, and the number of available positions per year. Unfortunately, as Gu et al. [12] reported in a recent study, not all adult reconstruction fellowship programs have functional websites, and the level of detail provided on existing websites varies widely. For instance, the application deadline was included on only 20% of program websites, only 28% reported a rotation schedule for fellows, and a mere 22% of sites detailed the expected call responsibilities. Furthermore, programs have been, and continue to be, deficient in listing accurate contact information, with only 63% of programs mentioning the PD’s name, 72% listing a phone number to contact about the program, and 75% listing an email address for the PD and/or coordinator [12]. Other orthopaedic subspecialty fellowship program websites have been shown to have similar shortcomings [13]. The challenges associated with identifying accurate contact information was found in the present study, as numerous methods were required to identify direct contact information for either the PD or program coordinator.

Applicants are faced with the challenging decision of how many programs to apply to in an effort to achieve a comfortable quantity of interviews to maximize chances of matching while controlling costs. While there is a paucity of data on costs associated with the fellowship application process which was not addressed in the present study, this is an area that deserves future study.

Limitations

There are several limitations to this study. First, not all PDs’ direct email addresses could be identified despite numerous methods in attempting to do so. This may have contributed to the 34.8% response rate and the resultant sampling bias. The results reported are representative of only those PDs who provided information and may have been different if additional PDs had responded. Second, the survey was designed to be brief to facilitate higher response rates with questions and criteria created based on previous survey studies in orthopaedic surgical subspecialties. As such, it is possible that additional variables that were not included in the survey may be considered important by PDs when selecting fellowship applicants. Although we hope the option of a free-text response would help mitigate this potential limitation, it is still possible that our survey did not completely capture all applicant factors that PDs consider important.

Although continuing improvements are necessary to allow hip and knee arthroplasty fellowship applicants greater transparency in the process of applying to fellowships, the results of the present study provide useful information that applicants can use during the process of preparing their applications and selecting fellowship programs.

Conclusions

According to hip and knee arthroplasty fellowship PDs, the interview, LORs, and personal connections to the applicant and/or letter writers are the most important factors considered in selecting arthroplasty fellowship candidates.

Conflict of interest

The authors declare there are no conflicts of interest.