Luan Lawson1, Julianna Jung2, Douglas Franzen3, Katherine Hiller4. 1. Department of Emergency Medicine, East Carolina University Brody School of Medicine, Greenville, North Carolina. 2. Department of Emergency Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland. 3. Division of Emergency Medicine, University of Washington School of Medicine, Seattle, Washington. 4. Department of Emergency Medicine, University of Arizona College of Medicine, Tucson, Arizona.
Abstract
BACKGROUND: Assessment practices in emergency medicine (EM) clerkships have not been previously described. Clinical assessment frequently relies on global ratings of clinical performance, or "shift cards," although these tools have not been standardized or studied. OBJECTIVE: We sought to characterize assessment practices in EM clerkships, with particular attention to shift cards. METHODS: A survey regarding assessment practices was administered to a national sample of EM clerkship directors (CDs). Descriptive statistics were compiled and regression analyses were performed. RESULTS: One hundred seventy-two CDs were contacted, and 100 (58%) agreed to participate. The most heavily weighted assessment methods in final grades were shift cards (66%) and written examinations (21-26%), but there was considerable variability in grading algorithms. EM faculty (100%) and senior residents (69%) were most commonly responsible for assessment, and assessors were often preassigned (71%). Forty-four percent of CDs reported immediate completion of shift cards, 27% within 1 to 2 days, and 20% within a week. Only 40% reported return rates >75%. Thirty percent of CDs do not permit students to review individual evaluations, and 54% of the remainder deidentify evaluations before student review. Eighty-six percent had never performed psychometric analysis on their assessment tools. Sixty-five percent of CDs were satisfied with their shift cards, but 90% supported the development of a national tool. CONCLUSION: There is substantial variability in assessment practices between EM clerkships, raising concern regarding the comparability of grades between institutions. CDs rely on shift cards in grading despite the lack of evidence of validity and inconsistent process variables. Standardization of assessment practices may improve the assessment of EM students. Copyright Â
BACKGROUND: Assessment practices in emergency medicine (EM) clerkships have not been previously described. Clinical assessment frequently relies on global ratings of clinical performance, or "shift cards," although these tools have not been standardized or studied. OBJECTIVE: We sought to characterize assessment practices in EM clerkships, with particular attention to shift cards. METHODS: A survey regarding assessment practices was administered to a national sample of EM clerkship directors (CDs). Descriptive statistics were compiled and regression analyses were performed. RESULTS: One hundred seventy-two CDs were contacted, and 100 (58%) agreed to participate. The most heavily weighted assessment methods in final grades were shift cards (66%) and written examinations (21-26%), but there was considerable variability in grading algorithms. EM faculty (100%) and senior residents (69%) were most commonly responsible for assessment, and assessors were often preassigned (71%). Forty-four percent of CDs reported immediate completion of shift cards, 27% within 1 to 2 days, and 20% within a week. Only 40% reported return rates >75%. Thirty percent of CDs do not permit students to review individual evaluations, and 54% of the remainder deidentify evaluations before student review. Eighty-six percent had never performed psychometric analysis on their assessment tools. Sixty-five percent of CDs were satisfied with their shift cards, but 90% supported the development of a national tool. CONCLUSION: There is substantial variability in assessment practices between EM clerkships, raising concern regarding the comparability of grades between institutions. CDs rely on shift cards in grading despite the lack of evidence of validity and inconsistent process variables. Standardization of assessment practices may improve the assessment of EM students. Copyright Â
Authors: Nicole M Dubosh; Jaime Jordan; Lalena M Yarris; Edward Ullman; Joshua Kornegay; Daniel Runde; Amy Miller Juve; Jonathan Fisher Journal: AEM Educ Train Date: 2018-12-14
Authors: Katherine M Hiller; Douglas Franzen; Luan Lawson; David Manthey; Jonathan Fisher; Marianne Haughey; Matthew Tews; Nicole Dubosh; Joseph House; Arleigh Trainor; David Wald; Julianna Jung Journal: West J Emerg Med Date: 2016-11-23
Authors: Jaewon Yoon; Jordan T Said; Leah L Thompson; Gabriel E Molina; Jeremy B Richards; Steven T Chen Journal: Int J Womens Dermatol Date: 2021-01-13
Authors: Julianna Jung; Douglas Franzen; Luan Lawson; David Manthey; Matthew Tews; Nicole Dubosh; Jonathan Fisher; Marianne Haughey; Joseph B House; Arleigh Trainor; David A Wald; Katherine Hiller Journal: West J Emerg Med Date: 2017-12-22