Evaluation of resident applicants by letters of recommendation: a comparison of traditional and behavior-based formats.

RATIONALE AND
OBJECTIVES: Traditional, narrative letters of recommendation solicited by medical students applying for radiology residency are widely used as a selection tool. Letters of recommendation are considered a source of reliable information about the attitudes and behaviors (non-cognitive variables) of the resident applicant. However, in many instances, this information is not present or is highly encoded and cannot be extracted. This study attempted to document the deficiencies of traditional letters of recommendation and determine the effectiveness of a structured letter of recommendation in obtaining information regarding noncognitive variables.
METHODS: One hundred thirteen randomly selected letters of recommendation were analyzed by two radiologists with experience in residency selection. Deficiencies in inclusion of information or the ability to extract information about noncognitive variables were documented. A standard behavioral assessment was sent to the writers of these letters of recommendation and these results tabulated.
RESULTS: The traditional letters of recommendation were frequently deficient in data regarding the noncognitive variables. In letters that contained such data, two experienced reviewers could not reliably extract the information. The structured form produced clearly identifiable information about the letter writer's assessment of noncognitive variables.
CONCLUSIONS: Traditional letters of information are frequently deficient in data regarding noncognitive variables. A standardized statement is effective in eliciting information on noncognitive variables related to applicant performance.