Are There Gender-based Differences in Language in Letters of Recommendation to an Orthopaedic Surgery Residency Program?

BACKGROUND: Letters of recommendation are considered one of the most important factors for whether an applicant is selected for an interview for orthopaedic surgery residency programs. Language differences in letters describing men versus women candidates may create differential perceptions by gender. Given the gender imbalance in orthopaedic surgery, we sought to determine whether there are differences in the language of letters of recommendation by applicant gender. QUESTIONS/PURPOSES: (1) Are there differences in word count and word categories in letters of recommendation describing women and men applicants, regardless of author gender? (2) Is author gender associated with word category differences in letters of recommendation? (3) Do authors of different academic rank use different words to describe women versus men applicants?
METHODS: Using a linguistic analysis in a retrospective study, we analyzed all letters of recommendation (2834 letters) written for all 738 applicants with completed Electronic Residency Application Service applications submitted to the Johns Hopkins Orthopaedic Surgery Residency program during the 2018 to 2019 cycle to determine differences in word category use among applicants by gender, authors by gender, and authors by academic rank. Thirty nine validated word categories from the Linguistic Inquiry and Word Count dictionary along with seven additional word categories from previous publications were used in this analysis. The occurrence of words in each word category was divided by the number of words in the letter to obtain a word frequency for each letter. We calculated the mean word category frequency across all letters and analyzed means using non-parametric tests. For comparison of two groups, a p value threshold of 0.05 was used. For comparison of multiple groups, the Bonferroni correction was used to calculate an adjusted p value (p = 0.00058).
RESULTS: Letters of recommendation for women applicants were slightly longer compared with those for men applicants (366 ± 188 versus 339 ± 199 words; p = 0.003). When comparing word category differences by applicant gender, letters for women applicants had slightly more "achieve" words (0.036 ± 0.015 versus 0.035 ± 0.018; p < 0.0001). Letters for men had more use of their first name (0.016 ± 0.013 versus 0.014 ± 0.009; p < 0.0001), and more "young" words (0.001 ± 0.003 versus 0.000 ± 0.001; p < 0.0001) than letters for women applicants. These differences were very small as each 0.001 difference in mean word frequency was equivalent to one more additional word from the word category appearing when comparing three letters for women to three letters for men. For differences in letters by author gender, there were no word category differences between men and women authors. Finally, when looking at author academic rank, letters for men applicants written by professors had slightly more "research" terms (0.011 ± 0.010) than letters written by associate professors (0.010 ± 0.010) or faculty of other rank (0.009 ± 0.011; p < 0.0001), a finding not observed in letters written for women.
CONCLUSIONS: Although there were some minor differences favoring women, language in letters of recommendation to an academic orthopaedic surgery residency program were overall similar between men and women applicants. CLINICAL RELEVANCE: Given the similarity in language between men and women applicants, increasing women applicants may be a more important factor in addressing the gender gap in orthopaedics.