Rosemarie Fernandez1, Elizabeth D Rosenman2, Sarah Brolliar2, Anne K Chipman2, Colleen Kalynych3, Marie C Vrablik2, Joseph R Keebler4, Elizabeth H Lazzara4. 1. Department of Emergency Medicine and the Center for Experiential Learning and Simulation, College of Medicine University of Florida Gainesville FL. 2. Department of Emergency Medicine University of Washington Seattle WA. 3. Department of Emergency Medicine, Office of Educational Affairs University of Florida College of Medicine-Jacksonville Jacksonville FL. 4. Department of Human Factors and Behavioral Neurobiology, College of Arts and Sciences Embry-Riddle Aeronautical University Daytona Beach FL.
Abstract
BACKGROUND: Translational research in medical education requires the ability to rigorously measure learner performance in actual clinical settings; however, current measurement systems cannot accommodate the variability inherent in many patient care environments. This is especially problematic in emergency medicine, where patients represent a wide spectrum of severity for a single clinical presentation. Our objective is to describe and implement EBAM, an event-based approach to measurement that can be applied to actual emergency medicine clinical events. METHODS: We used a four-step event-based approach to create an emergency department trauma resuscitation patient care measure. We applied the measure to a database of 360 actual trauma resuscitations recorded in a Level I trauma center using trained raters. A subset (n = 50) of videos was independently rated in duplicate to determine inter-rater reliability. Descriptive analyses were performed to describe characteristics of resuscitation events and Cohen's kappa was used to calculate reliability. RESULTS: The methodology created a metric containing both universal items that are applied to all trauma resuscitation events and conditional items that only apply in certain situations. For clinical trauma events, injury severity scores ranged from 1 to 75 with a mean (±SD) of 21 (±15) and included both blunt (254/360; 74%) and penetrating (86/360; 25%) traumatic injuries, demonstrating the diverse nature of the clinical encounters. The mean (±SD) Cohen's kappa for patient care items was 0.7 (±0.3). CONCLUSION: We present an event-based approach to performance assessment that may address a major gap in translational education research. Our work centered on assessment of patient care behaviors during trauma resuscitation. More work is needed to evaluate this approach across a diverse array of clinical events.
BACKGROUND: Translational research in medical education requires the ability to rigorously measure learner performance in actual clinical settings; however, current measurement systems cannot accommodate the variability inherent in many patient care environments. This is especially problematic in emergency medicine, where patients represent a wide spectrum of severity for a single clinical presentation. Our objective is to describe and implement EBAM, an event-based approach to measurement that can be applied to actual emergency medicine clinical events. METHODS: We used a four-step event-based approach to create an emergency department trauma resuscitation patient care measure. We applied the measure to a database of 360 actual trauma resuscitations recorded in a Level I trauma center using trained raters. A subset (n = 50) of videos was independently rated in duplicate to determine inter-rater reliability. Descriptive analyses were performed to describe characteristics of resuscitation events and Cohen's kappa was used to calculate reliability. RESULTS: The methodology created a metric containing both universal items that are applied to all trauma resuscitation events and conditional items that only apply in certain situations. For clinical trauma events, injury severity scores ranged from 1 to 75 with a mean (±SD) of 21 (±15) and included both blunt (254/360; 74%) and penetrating (86/360; 25%) traumatic injuries, demonstrating the diverse nature of the clinical encounters. The mean (±SD) Cohen's kappa for patient care items was 0.7 (±0.3). CONCLUSION: We present an event-based approach to performance assessment that may address a major gap in translational education research. Our work centered on assessment of patient care behaviors during trauma resuscitation. More work is needed to evaluate this approach across a diverse array of clinical events.
Authors: James A Grand; Marina Pearce; Tara A Rench; Georgia T Chao; Rosemarie Fernandez; Steve W J Kozlowski Journal: BMJ Qual Saf Date: 2013-01-25 Impact factor: 7.035
Authors: Gerald P Whelan; John R Boulet; Danette W McKinley; John J Norcini; Marta van Zanten; Ronald K Hambleton; William P Burdick; Steven J Peitzman Journal: Med Teach Date: 2005-05 Impact factor: 3.650
Authors: William C McGaghie; Timothy J Draycott; William F Dunn; Connie M Lopez; Dimitrios Stefanidis Journal: Simul Healthc Date: 2011-08 Impact factor: 1.929
Authors: Rosemarie Fernandez; Marina Pearce; James A Grand; Tara A Rench; Kerin A Jones; Georgia T Chao; Steve W J Kozlowski Journal: Crit Care Med Date: 2013-11 Impact factor: 7.598
Authors: Elizabeth D Rosenman; Anthony Misisco; Jeffrey Olenick; Sarah M Brolliar; Anne K Chipman; Marie C Vrablik; Georgia T Chao; Steve W J Kozlowski; James A Grand; Rosemarie Fernandez Journal: J Am Coll Emerg Physicians Open Date: 2021-01-04