Paul Kukulski1,2, James Ahn1,3. 1. Both authors are with the University of Chicago Medical Center. 2. is Assistant Professor and Assistant Clerkship Director, Section of Emergency Medicine, Department of Medicine. 3. is Associate Professor and Program Director, Section of Emergency Medicine, Department of Medicine.
Abstract
BACKGROUND: The standardized letter of evaluation (SLOE) is the application component that program directors value most when evaluating candidates to interview and rank for emergency medicine (EM) residency. Given its successful implementation, other specialties, including otolaryngology, dermatology, and orthopedics, have adopted similar SLOEs of their own, and more specialties are considering creating one. Unfortunately, for such a significant assessment tool, no study to date has comprehensively examined the validity evidence for the EM SLOE. OBJECTIVE: We summarized the published evidence for validity for the EM SLOE using Messick's framework for validity evidence. METHODS: A scoping review of the validity evidence of the EM SLOE was performed in 2020. A scoping review was chosen to identify gaps and future directions, and because the heterogeneity of the literature makes a systematic review difficult. Included articles were assigned to an aspect of Messick's framework and determined to provide evidence for or against validity. RESULTS: There have been 22 articles published relating to validity evidence for the EM SLOE. There is evidence for content validity; however, there is a lack of evidence for internal structure, relation to other variables, and consequences. Additionally, the literature regarding response process demonstrates evidence against validity. CONCLUSIONS: Overall, there is little published evidence in support of validity for the EM SLOE. Stakeholders need to consider changing the ranking system, improving standardization of clerkships, and further studying relation to other variables to improve validity. This will be important across GME as more specialties adopt a standardized letter.
BACKGROUND: The standardized letter of evaluation (SLOE) is the application component that program directors value most when evaluating candidates to interview and rank for emergency medicine (EM) residency. Given its successful implementation, other specialties, including otolaryngology, dermatology, and orthopedics, have adopted similar SLOEs of their own, and more specialties are considering creating one. Unfortunately, for such a significant assessment tool, no study to date has comprehensively examined the validity evidence for the EM SLOE. OBJECTIVE: We summarized the published evidence for validity for the EM SLOE using Messick's framework for validity evidence. METHODS: A scoping review of the validity evidence of the EM SLOE was performed in 2020. A scoping review was chosen to identify gaps and future directions, and because the heterogeneity of the literature makes a systematic review difficult. Included articles were assigned to an aspect of Messick's framework and determined to provide evidence for or against validity. RESULTS: There have been 22 articles published relating to validity evidence for the EM SLOE. There is evidence for content validity; however, there is a lack of evidence for internal structure, relation to other variables, and consequences. Additionally, the literature regarding response process demonstrates evidence against validity. CONCLUSIONS: Overall, there is little published evidence in support of validity for the EM SLOE. Stakeholders need to consider changing the ranking system, improving standardization of clerkships, and further studying relation to other variables to improve validity. This will be important across GME as more specialties adopt a standardized letter.
Authors: D V Girzadas; R C Harwood; S N Delis; K Stevison; G Keng; N Cipparrone; A Carlson; G D Tsonis Journal: Acad Emerg Med Date: 2001-06 Impact factor: 3.451
Authors: Dowin Boatright; David Ross; Patrick O'Connor; Edward Moore; Marcella Nunez-Smith Journal: JAMA Intern Med Date: 2017-05-01 Impact factor: 21.873
Authors: Alexandra E Rojek; Raman Khanna; Joanne W L Yim; Rebekah Gardner; Sarah Lisker; Karen E Hauer; Catherine Lucey; Urmimala Sarkar Journal: J Gen Intern Med Date: 2019-05 Impact factor: 5.128
Authors: Julianna Jung; Douglas Franzen; Luan Lawson; David Manthey; Matthew Tews; Nicole Dubosh; Jonathan Fisher; Marianne Haughey; Joseph B House; Arleigh Trainor; David A Wald; Katherine Hiller Journal: West J Emerg Med Date: 2017-12-22