Determining the Most Important Factors Involved in Ranking Orthopaedic Sports Medicine Fellowship Applicants.

BACKGROUND: Orthopaedic surgery residencies and certain fellowships are becoming increasingly competitive. Several studies have identified important factors to be taken into account when selecting medical students for residency interviews. Similar information for selecting orthopaedic sports medicine fellows does not exist.
PURPOSE: To determine the most important factors that orthopaedic sports medicine fellowship program directors (PDs) take into account when ranking applicants. STUDY
DESIGN: Cross-sectional study.
METHODS: A brief survey was distributed electronically to PDs of the 92 orthopaedic sports medicine fellowship programs that are accredited by the Accreditation Council for Graduate Medical Education (ACGME). Each PD was asked to rank, in order, the 5 most important factors taken into account when ranking applicants based on a total list of 13 factors: the interview, the applicant's residency program, letters of recommendation (LORs), personal connections made through the applicant, research experience, an applicant's geographical ties to the city/town of the fellowship program, United States Medical Licensing Examination (USMLE) scores, Orthopaedic In-Training Examination (OITE) scores, history of being a competitive athlete in college, extracurricular activities/hobbies, volunteer experience, interest in a career in academics, and publications/research/posters. Factors were scored from 1 to 5, with a score of 5 representing the most important factor and 1 representing the fifth-most important factor.
RESULTS: Of the 92 PDs contacted, 57 (62%) responded. Thirty-four PDs (37%) listed the interview as the most important factor in ranking fellowship applicants (overall score, 233). LORs (overall score, 196), an applicant's residency program (overall score, 133), publications/research/posters (overall score, 115), and personal connections (overall score, 90) were reported as the second- through fifth-most important factors, respectively.
CONCLUSION: According to orthopaedic sports medicine fellowship PDs, the fellowship interview is the most important factor in determining how an applicant will be ranked. Other factors, including LORs, the applicant's residency program, research production, and personal connections, were also considered to be important. This information provides orthopaedic sports medicine fellowship applicants with a better understanding of which areas to focus on when preparing for the fellowship interview and matching process.

Orthopaedic surgery is one of the most competitive fields in medicine, and only two-thirds of medical students who apply for orthopaedic residency matching are successful.[16,18] Orthopaedic residency directors take several factors into consideration when screening and ranking applicants, including research,[1] United States Medical Licensing Examination (USMLE) scores,[16,19] and impressions from the interview.[14,17] Most orthopaedic surgery residency programs are 5 years in duration; residents then have the option to complete 1 year of fellowship training after graduation.[5] With the advancements and variety of subspecialties within orthopaedic surgery, residency training may not be sufficient, thus many orthopaedic residency graduates currently pursue subspecialty fellowships.[5] In a survey conducted in 2012, Hariri et al[9] found that approximately 91% of orthopaedic surgery residency graduates were planning to enroll in a fellowship program, and there was an increasing trend for graduates to complete 2 fellowship programs.[5] Furthermore, 28% of residency graduates planned to pursue an orthopaedic sports medicine fellowship.[9]

As with residency applications, residents applying for fellowship training must submit applications and interview at programs before ranking these programs in order of preference. Similarly, each fellowship program ranks applicants whom they interview in order of preference, and prospective fellows and programs are matched.[6] Several studies have evaluated factors considered to be most important in ranking those applying to a variety of medical and surgical fellowships.[4,7,8,11-13,15] Although the most important factors tend to be similar across different fellowship programs,[7,10,13] the emphasis placed on some factors can vary depending on the specific fellowship program. In 2010, Muffly et al[12] surveyed directors of fellowships for female pelvic medicine and reconstructive surgery (FPM&RS) regarding factors considered to be important in their ranking process. The authors found that the applicant’s residency program, clinical research experience, and ability to work with others were considered most important. A study conducted by Poirier and Pruitt,[15] in which program directors (PDs) for pediatric emergency medicine were surveyed, identified recommendations from colleagues as being the most important factor and research potential as the second-most important factor. To our knowledge, no previous studies have evaluated which factors are considered to be most important by PDs in ranking orthopaedic sports medicine fellowship applicants. The purpose of this study was to determine the most important factors that are taken into account by orthopaedic sports medicine fellowship PDs when ranking applicants. We hypothesized that the most important factor when ranking orthopaedic sports medicine fellowship applicants is the interview.

Methods

A complete list of orthopaedic sports medicine fellowships was obtained from the American Orthopaedic Society for Sports Medicine (AOSSM) website. Of the 93 programs listed, 1 program was not accredited by the Accreditation Council for Graduate Medical Education (ACGME), and it was thus excluded from the study. A brief electronic survey was distributed (nonanonymously) to the remaining 92 fellowship PDs, asking them to rank, in order, the 5 most important factors from a total list of 13 (Table 1). To avoid bias, the order of the factors listed on each survey was randomized using the random-number generator in Microsoft Excel. An “other” option was also provided for PDs to list any additional factors not mentioned in the original list of 13. When scoring each survey, the authors gave 5 points to the factor that the respondent considered most important (ie, that he or she ranked as 1) and the authors gave 1 point to the factor that the respondent considered fifth-most important (ie, that he or she ranked as 5). Scores from all completed surveys were summed to determine the collective most important factors in ranking fellowship applicants.

TABLE 1

Survey Distributed to Orthopaedic Sports Medicine Fellowship Program Directors

When ranking the orthopaedic sports medicine fellowship applicants whom you interviewed, please rank the 5 most important factors (from the list below) in deciding your rank list, in order of most important (1) to fifth-most important (5). If there are any additional factors involved in your decision which are not listed below, please list them and rank them accordingly.

Rank from most important (1) to fifth-most important (5):

Geographical ties to the city-town of your fellowship program

History of being a competitive athlete in college or equivalent

Interest in a career in academics

OITE scores

The interview

Publications/research/posters

Letters of recommendation

USMLE scores

Personal connections made through the applicant

Volunteer experience

Extracurricular activities/hobbies

The residency program of the applicant

Research experience

Factors were listed in a random order on each survey to avoid bias. OITE, Orthopaedic In-Training Examination; USMLE, United States Medical Licensing Examination.

Results

Of the 92 fellowship PDs surveyed, 57 (62%) responded. Thirty-four PDs (37%) listed the interview as the most important factor in ranking fellowship applicants (overall score, 233). Letters of recommendation (LORs) were considered the second-most important factor (overall score, 196), followed by the applicant’s residency program (overall score, 133), publications/research/posters (overall score, 115), and personal connections made through the applicant (overall score, 90) (Figure 1). Of the 13 factors listed in the survey, the interview (60%) and LORs (33%) were most commonly ranked as 1 of the 5 most important by PDs (Figure 2). Six PDs (11%) mentioned other factors as important, including “future plans,” “surgical skills,” and “personal statement.”

Figure 1.

Overall score of the 13 factors taken into account when ranking orthopaedic sports medicine fellowship applicants. OITE, Orthopaedic In-Training Examination; USMLE, United States Medical Licensing Examination.

Figure 2.

Factors ranked as most important by orthopaedic sports medicine fellowship program directors.

Of the 13 factors listed in the survey, the 5 identified as being least important were volunteer experience (overall score, 33), geographical ties to the city/town of the fellowship program (overall score, 38), extracurricular activities/hobbies (overall score, 44), history of being a competitive athlete in college (overall score, 45), and Orthopaedic In-Training Examination (OITE) scores (overall score, 46) (Figure 1).

Discussion

The results of this study indicate that when applying for an orthopaedic sports medicine fellowship, the most important factors that the applicant should consider and prepare for are the interview, LORs, his or her residency program, publications/research/posters, and personal connections made through the applicant and fellowship PDs. Although the term “personal connections” was not clarified to PDs in this study, we included this option as a potential ranking factor because we have found from experience that when applicants have positive personal connections with faculty from a specific fellowship program (from previous research experiences, orthopaedic conferences, etc), this increases the opportunity for these applicants to match at a specific program. Furthermore, although not expanded upon in our survey responses, it is known that ranking lists in orthopaedic sports medicine are sometimes created after “negotiating” with the applicant, something that is not allowed by the ACGME in residency programs but that is not explicitly prohibited when applying to fellowship programs.

Of the 57 PDs who responded to the survey, 34 (60%) considered the interview to be the most important factor in ranking fellowship applicants. The interview process is an effective way for a PD to determine whether the candidate is a good fit for the fellowship program, as it allows for a more formal evaluation of the applicant’s personality, skills, and knowledge base. The interview has also been identified as one of the most important factors when ranking orthopaedic surgery residency applicants.[17] In 2016, Schenker et al[17] developed a systematic approach to screen and score medical students applying for orthopaedic surgery residency. A program was used to assess and score applicants to determine a final rank list.[17] The authors found that the interview was highly correlated with the final rank of the applicant. Furthermore, the interview is ranked highly among PDs for nonorthopaedic residencies and fellowships, including plastic surgery[20] and pediatric otolaryngology.[2] While our study found the interview to be the most important factor, Egro et al[7] found that PDs for aesthetic surgery fellowship programs rank the interview as the third-most important factor. In contrast, Miller et al[11] found that the interview is not taken into account when ranking applicants for surgical fellowship. This is further supported by Muffly et al,[12] who surveyed PDs for FPM&RS fellowships and found that the interview is not an important factor in ranking applicants. Although previous studies[2,7,17,20] support the value of the interview in ranking fellowship applicants, the emphasis placed on the interview itself varies depending on the type of fellowship.

The reputation of an orthopaedic surgery residency program and the degree of exposure to sports medicine cases may indicate the potential arthroscopic skill level that a sports medicine fellowship applicant may obtain by the time of residency graduation. Studies from other surgical fellowships have also reported frequent emphasis on an applicant’s residency program when applying for surgical fellowships.[11,12] In 2010, Miller et al[11] assessed the importance of the American Board of Surgery In-Training Examination (ABSITE) when applying for surgical fellowships. The authors surveyed PDs of surgical fellowships to determine the importance of ABSITE scores in ranking applicants and found that ABSITE scores were ranked as the third-most important factor behind LORs and the residency program of the applicant.[11] Similarly, Egro et al[7] found that the applicant’s residency program is ranked as the most important factor when surveying directors of FPM&RS fellowships. An applicant’s residency program may also influence whether he or she is awarded an interview.[13,15] Our study further supports the emphasis placed on an applicant’s residency program in the ranking process, as this was identified by PDs as the third-most important factor overall.

This study also demonstrated that sports medicine fellowship PDs consider research productivity during residency to be important. Previous studies from other medical and surgical specialties have identified research as either the most important or second-most important factor in selecting residents or fellows.[7,15] Recent medical school graduates are seen as more competitive and are likely to have a higher match rate when applying for orthopaedic surgery residency due to their research productivity.[18] In our study, although research was ranked as the sixth-most important factor, a few programs actually listed research as the most important factor in ranking fellowship applicants. In 2016, Cvetanovich et al[3] assessed the research productivity of faculty in all of the orthopaedic sports medicine fellowship programs using the Hirsh index (H-index, which is based on an author’s citation impact) to determine whether productivity varies depending on fellowship attributes and overall academic rank. The authors concluded that sports medicine fellowships with more fellows have a higher research productivity than those with fewer fellows.[3]

This study is the first to specifically determine the most important factors for orthopaedic sports medicine PDs in ranking the fellowship applicants whom they have interviewed. The limitations of this study should also be noted. First, since the survey was not distributed anonymously, some PDs may have been reluctant to submit a response. Second, not all of the PDs completed the survey (62% response rate), thus our results may not reflect the opinions of all current orthopaedic sports medicine fellowship PDs. Third, our survey listed only 13 factors and hence it was not exhaustive; however, PDs had the opportunity to select “other” and to list any additional factors they considered to be most important. Fourth, many of the factors (eg, personal connections) may have been ranked differently depending on how they were interpreted.

Conclusion

According to orthopaedic sports medicine fellowship PDs, the fellowship interview is the most important factor in determining how an applicant will be ranked. Other factors, including LORs, the applicant’s residency program, previous research, and personal connections, were also considered to be important. This information provides orthopaedic sports medicine fellowship applicants with a better understanding of which areas to focus on when preparing for the fellowship interview and match process.