Laura R Hopson1, Linda Regan, Michael C Bond, Jeremy Branzetti, Elizabeth A Samuels, Bobby Naemi, Dana Dunleavy, Michael A Gisondi. 1. L.R. Hopson is emergency medicine residency program director and associate professor, Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, Michigan; ORCID: https://orcid.org/0000-0002-1183-4751. L. Regan is emergency medicine residency program director, vice chair for education, and associate professor, Department of Emergency Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland; ORCID: https://orcid.org/0000-0003-0390-4243. M.C. Bond is emergency medicine residency program director and associate professor, Department of Emergency Medicine, University of Maryland School of Medicine, Baltimore, Maryland; ORCID: https://orcid.org/0000-0001-5527-6758. J. Branzetti is emergency medicine residency program director, Department of Emergency Medicine, New York University School of Medicine, New York, New York; ORCID: http://orcid.org/0000-0002-2397-0566. E.A. Samuels is assistant professor of emergency medicine, Warren Alpert Medical School of Brown University, Providence, Rhode Island; ORCID: https://orcid.org/0000-0003-2414-110X. B. Naemi is manager, Admissions and Selection Research, Association of American Medical Colleges, Washington, DC. D. Dunleavy is director, Admissions and Selection Research, Association of American Medical Colleges, Washington, DC. M.A. Gisondi is associate professor and vice chair of education, Department of Emergency Medicine, Stanford University School of Medicine, Stanford, California; ORCID: https://orcid.org/0000-0002-6800-3932.
Abstract
PURPOSE: To compare the performance characteristics of the electronic Standardized Letter of Evaluation (eSLOE), a widely used structured assessment of emergency medicine (EM) residency applicants, and the Association of American Medical Colleges (AAMC) Standardized Video Interview (SVI), a new tool designed by the AAMC to assess interpersonal and communication skills and professionalism knowledge. METHOD: The authors matched EM residency applicants with valid SVI total scores and completed eSLOEs in the 2018 Match application cycle. They examined correlations and group differences for both tools, United States Medical Licensing Examination (USMLE) Step exam scores, and honor society memberships. RESULTS: The matched sample included 2,884 applicants. SVI score and eSLOE global assessment ratings demonstrated small positive correlations approaching r = 0.20. eSLOE ratings had higher correlations with measures of academic ability (USMLE scores, academic honor society membership) than did SVI scores. Group differences were minimal for the SVI, with scores slightly favoring women (d = -0.21) and US-MD applicants (d = 0.23-0.42). Group differences in eSLOE ratings were small, favoring women over men (approaching d = -0.20) and white applicants over black applicants (approaching d = 0.40). CONCLUSIONS: Small positive correlations between SVI score and eSLOE global assessment ratings, alongside varying correlations with academic ability indicators, suggest these are complementary tools. Findings suggest the eSLOE is subject to similar sources and degrees of bias as other common assessments; these group differences were not observed with the SVI. Further examination of both tools is necessary to understand their ability to predict clinical performance.
PURPOSE: To compare the performance characteristics of the electronic Standardized Letter of Evaluation (eSLOE), a widely used structured assessment of emergency medicine (EM) residency applicants, and the Association of American Medical Colleges (AAMC) Standardized Video Interview (SVI), a new tool designed by the AAMC to assess interpersonal and communication skills and professionalism knowledge. METHOD: The authors matched EM residency applicants with valid SVI total scores and completed eSLOEs in the 2018 Match application cycle. They examined correlations and group differences for both tools, United States Medical Licensing Examination (USMLE) Step exam scores, and honor society memberships. RESULTS: The matched sample included 2,884 applicants. SVI score and eSLOE global assessment ratings demonstrated small positive correlations approaching r = 0.20. eSLOE ratings had higher correlations with measures of academic ability (USMLE scores, academic honor society membership) than did SVI scores. Group differences were minimal for the SVI, with scores slightly favoring women (d = -0.21) and US-MD applicants (d = 0.23-0.42). Group differences in eSLOE ratings were small, favoring women over men (approaching d = -0.20) and white applicants over black applicants (approaching d = 0.40). CONCLUSIONS: Small positive correlations between SVI score and eSLOE global assessment ratings, alongside varying correlations with academic ability indicators, suggest these are complementary tools. Findings suggest the eSLOE is subject to similar sources and degrees of bias as other common assessments; these group differences were not observed with the SVI. Further examination of both tools is necessary to understand their ability to predict clinical performance.
Authors: Christopher Williams; Richard Alweis; Teresa Roth; Vera Luther; James Pile; Stephanie Elkins; Srikanth Nagalla; Elaine A Muchmore Journal: J Community Hosp Intern Med Perspect Date: 2021-03-23