Tehnaz P Boyle1, Julianne N Dugas, James Liu, Stephanie N Stapleton, Ron Medzon, Barbara M Walsh, Pamela Corey, Leonard Shubitowski, John R Horne, Richard O'Connell, Graham Williams, Kerrie P Nelson, Vinay M Nadkarni, Carlos A Camargo, James A Feldman. 1. From the Department of Pediatrics (T.P.B., B.M.W.), Boston Medical Center, Boston University School of Medicine; Department of Emergency Medicine (J.N.D., J.L., S.N.S., R.M., J.A.F.), Boston Medical Center, Boston University School of Medicine; Solomont Center for Simulation (T.P.B., S.N.S., R.M., B.M.W., P.C.), Boston Medical Center; Boston Emergency Medical Services (L.S., J.R.H., R.O.C., G.W.); Department of Biostatistics (K.P.N.), Boston University, Boston, MA; Center for Simulation, Advanced Education and Innovation (V.M.N.); Department of Anesthesiology and Critical Care Medicine (V.M.N.); The Children's Hospital of Philadelphia (V.M.N.), University of Pennsylvania Perelman School of Medicine (V.M.N.), Philadelphia, PA; and Department of Emergency Medicine (C.A.C.), Massachusetts General Hospital, Harvard Medical School, Boston, MA.
Abstract
INTRODUCTION: Simulation tools to assess prehospital team performance and identify patient safety events are lacking. We adapted a simulation model and checklist tool of individual paramedic performance to assess prehospital team performance and tested interrater reliability. METHODS: We used a modified Delphi process to adapt 3 simulation cases (cardiopulmonary arrest, seizure, asthma) and checklist to add remote physician direction, target infants, and evaluate teams of 2 paramedics and 1 physician. Team performance was assessed with a checklist of steps scored as complete/incomplete by raters using direct observation or video review. The composite performance score was the percentage of completed steps. Interrater percent agreement was compared with the original tool. The tool was modified, and raters trained in iterative rounds until composite performance scoring agreement was 0.80 or greater (scale <0.20 = poor; 0.21-0.39 = fair, 0.40-0.59 = moderate; 0.60-0.79 = good; 0.80-1.00 = very good). RESULTS: We achieved very good interrater agreement for scoring composite performance in 2 rounds using 6 prehospital teams and 4 raters. The original 175 step tool was modified to 171 steps. Interrater percent agreement for the final modified tool approximated the original tool for the composite checklist (0.80 vs. 0.85), cardiopulmonary arrest (0.82 vs. 0.86), and asthma cases (0.80 vs. 0.77) but was lower for the seizure case (0.76 vs. 0.91). Most checklist items (137/171, 80%) had good-very good agreement. Among 34 items with fair-moderate agreement, 15 (44%) related to patient assessment, 9 (26%) equipment use, 6 (18%) medication delivery, and 4 (12%) cardiopulmonary resuscitation quality. CONCLUSIONS: The modified checklist has very good agreement for assessing composite prehospital team performance and can be used to test effects of patient safety interventions.
INTRODUCTION: Simulation tools to assess prehospital team performance and identify patient safety events are lacking. We adapted a simulation model and checklist tool of individual paramedic performance to assess prehospital team performance and tested interrater reliability. METHODS: We used a modified Delphi process to adapt 3 simulation cases (cardiopulmonary arrest, seizure, asthma) and checklist to add remote physician direction, target infants, and evaluate teams of 2 paramedics and 1 physician. Team performance was assessed with a checklist of steps scored as complete/incomplete by raters using direct observation or video review. The composite performance score was the percentage of completed steps. Interrater percent agreement was compared with the original tool. The tool was modified, and raters trained in iterative rounds until composite performance scoring agreement was 0.80 or greater (scale <0.20 = poor; 0.21-0.39 = fair, 0.40-0.59 = moderate; 0.60-0.79 = good; 0.80-1.00 = very good). RESULTS: We achieved very good interrater agreement for scoring composite performance in 2 rounds using 6 prehospital teams and 4 raters. The original 175 step tool was modified to 171 steps. Interrater percent agreement for the final modified tool approximated the original tool for the composite checklist (0.80 vs. 0.85), cardiopulmonary arrest (0.82 vs. 0.86), and asthma cases (0.80 vs. 0.77) but was lower for the seizure case (0.76 vs. 0.91). Most checklist items (137/171, 80%) had good-very good agreement. Among 34 items with fair-moderate agreement, 15 (44%) related to patient assessment, 9 (26%) equipment use, 6 (18%) medication delivery, and 4 (12%) cardiopulmonary resuscitation quality. CONCLUSIONS: The modified checklist has very good agreement for assessing composite prehospital team performance and can be used to test effects of patient safety interventions.
Authors: Paul A Harris; Robert Taylor; Robert Thielke; Jonathon Payne; Nathaniel Gonzalez; Jose G Conde Journal: J Biomed Inform Date: 2008-09-30 Impact factor: 6.317
Authors: Blair L Bigham; Jason E Buick; Steven C Brooks; Merideth Morrison; Kaveh G Shojania; Laurie J Morrison Journal: Prehosp Emerg Care Date: 2012 Jan-Mar Impact factor: 3.077
Authors: John D Hoyle; Remle P Crowe; Melissa A Bentley; Gerald Beltran; William Fales Journal: Prehosp Emerg Care Date: 2017-02-08 Impact factor: 3.077
Authors: Sylvain Boet; Cole Etherington; Sarah Larrigan; Li Yin; Hira Khan; Katrina Sullivan; James J Jung; Teodor P Grantcharov Journal: BMJ Qual Saf Date: 2018-10-11 Impact factor: 7.418