Anne-Laure Philippon1,2, Aurelien Baud3, Margaux Dumont2, Sidi Ahmed Remini4, Jeremy Leroy5, Jennifer Truchot6, Emmanuel Triby1, Yonathan Freund2,5. 1. Laboratoire Interuniversitaire de Sciences de l'Education (LISEC) Learning Sciences Department Université de Strasbourg Strasbourg France. 2. Emergency Department Hôpital Pitié-Salpêtrière Assistance Publique - Hôpitaux de Paris Paris France. 3. Emergency Department Hôpital Tenon Assistance Publique - Hôpitaux de Paris Paris France. 4. Centre de Simulation ILumens Paris Diderot Université de Paris Paris France. 5. Faculté de Médecine Sorbonne Université Paris France. 6. Emergency Department Hôpital Cochin Assistance Publique - Hôpitaux de Paris Paris France.
Abstract
OBJECTIVES: It is critical to assess competency of medical students and residents in emergency medicine (EM) during undergraduate and graduate medical education. However, very few valid tools exist to assess both technical and nontechnical skills in the specific context of EM. Three Acute Care Assessment Tools (ACAT 1, 2, and 3) have been previously developed for three acute care conditions: cardiac arrest (1), coma (2), and acute respiratory failure (3). This study aimed to evaluate the reproducibility of the tools. METHODS: The tool was tested using recorded videos of simulation sessions of fourth year medical students and first year residents in EM. Raters independently reviewed the videos two times in a 3-month interval, and interrater and intrarater reliability using intraclass correlation (ICC) was calculated. Secondary endpoints included the completeness rate and relevance of the ACAT. RESULTS: Sixty-two sessions were recorded and 48 videos analyzed (18 for CA and 15 for both respiratory failure and coma. The learners were residents in 32 (66%) of videos. Interrater reliability was excellent (ICC >0.9 for all three contexts) and so was the intrarater reliability (>0.88), both upon first review (month 0, M0) and at 3 months (M3). The usability of the ACAT was good, with a completeness of the items that ranged from 96% to 100%. Only one item of the ACAT 1 had a relevance of 27%, as it could not be completed in 13 scenarios out of 18. CONCLUSIONS: The results demonstrate educators can evaluate students similarly utilizing video recordings of simulated medical scenario. The excellent completeness of the rated items advocated for good usability. The three ACATs can be utilized to assess for completeness of predefined tasks in three acute care broad scenario in a competency-based medical education framework.
OBJECTIVES: It is critical to assess competency of medical students and residents in emergency medicine (EM) during undergraduate and graduate medical education. However, very few valid tools exist to assess both technical and nontechnical skills in the specific context of EM. Three Acute Care Assessment Tools (ACAT 1, 2, and 3) have been previously developed for three acute care conditions: cardiac arrest (1), coma (2), and acute respiratory failure (3). This study aimed to evaluate the reproducibility of the tools. METHODS: The tool was tested using recorded videos of simulation sessions of fourth year medical students and first year residents in EM. Raters independently reviewed the videos two times in a 3-month interval, and interrater and intrarater reliability using intraclass correlation (ICC) was calculated. Secondary endpoints included the completeness rate and relevance of the ACAT. RESULTS: Sixty-two sessions were recorded and 48 videos analyzed (18 for CA and 15 for both respiratory failure and coma. The learners were residents in 32 (66%) of videos. Interrater reliability was excellent (ICC >0.9 for all three contexts) and so was the intrarater reliability (>0.88), both upon first review (month 0, M0) and at 3 months (M3). The usability of the ACAT was good, with a completeness of the items that ranged from 96% to 100%. Only one item of the ACAT 1 had a relevance of 27%, as it could not be completed in 13 scenarios out of 18. CONCLUSIONS: The results demonstrate educators can evaluate students similarly utilizing video recordings of simulated medical scenario. The excellent completeness of the rated items advocated for good usability. The three ACATs can be utilized to assess for completeness of predefined tasks in three acute care broad scenario in a competency-based medical education framework.
Authors: Jason R Frank; Linda S Snell; Olle Ten Cate; Eric S Holmboe; Carol Carraccio; Susan R Swing; Peter Harris; Nicholas J Glasgow; Craig Campbell; Deepak Dath; Ronald M Harden; William Iobst; Donlin M Long; Rani Mungroo; Denyse L Richardson; Jonathan Sherbino; Ivan Silver; Sarah Taber; Martin Talbot; Kenneth A Harris Journal: Med Teach Date: 2010 Impact factor: 3.650
Authors: Robert Greif; Andrew S Lockey; Patricia Conaghan; Anne Lippert; Wiebe De Vries; Koenraad G Monsieurs Journal: Resuscitation Date: 2015-10-15 Impact factor: 5.262
Authors: Elizabeth D Rosenman; Aurora J Dixon; Jessica M Webb; Sarah Brolliar; Simon J Golden; Kerin A Jones; Sachita Shah; James A Grand; Steve W J Kozlowski; Georgia T Chao; Rosemarie Fernandez Journal: Acad Emerg Med Date: 2017-10-12 Impact factor: 3.451
Authors: David A Cook; Benjamin Zendejas; Stanley J Hamstra; Rose Hatala; Ryan Brydges Journal: Adv Health Sci Educ Theory Pract Date: 2013-05-02 Impact factor: 3.853
Authors: J Randall Curtis; Anthony L Back; Dee W Ford; Lois Downey; Sarah E Shannon; Ardith Z Doorenbos; Erin K Kross; Lynn F Reinke; Laura C Feemster; Barbara Edlund; Richard W Arnold; Kim O'Connor; Ruth A Engelberg Journal: JAMA Date: 2013-12-04 Impact factor: 56.272
Authors: Arna Banerjee; Jason M Slagle; Nathaniel D Mercaldo; Ray Booker; Anne Miller; Daniel J France; Lisa Rawn; Matthew B Weinger Journal: BMC Med Educ Date: 2016-11-16 Impact factor: 2.463