John B Zamjahn1, Raquel Baroni de Carvalho2, Megan H Bronson3, Deborah D Garbee4, John T Paige5. 1. Department of Cardiopulmonary Science, LSU Health New Orleans School of Allied Health Professions, 1900 Gravier Street, New Orleans, Louisiana 70112, USA. 2. Department of Social Medicine, Federal University of Espirito Santo, Vitoria, ES 29043-900, Brazil. 3. Epidemiology Data Center, LSU Health New Orleans School of Public Health, 2020 Gravier Street, New Orleans, Louisiana 70112, USA. 4. LSU Health New Orleans School of Nursing, 1900 Gravier Street, New Orleans, Louisiana 70112, USA. 5. Department of Surgery, LSU Health New Orleans School of Medicine, 1542 Tulane Ave, Rm 734, New Orleans, LA 70112, USA.
Abstract
Objective: The Objective Structured Assessment of Debriefing (OSAD) is an evidence-based, 8-item tool that uses a behaviorally anchored rating scale in paper-based form to evaluate the quality of debriefing in medical education. The objective of this project was twofold: 1) to create an easy-to-use electronic format of the OSAD (eOSAD) in order to streamline data entry; and 2) to pilot its use on videoed debriefings. Materials and Methods: The eOSAD was developed in collaboration with the LSU Health New Orleans Epidemiology Data Center using SurveyGizmo (Widgix Software, LLC, Boulder, CO, USA) software. The eOSAD was then piloted by 2 trained evaluators who rated 37 videos of faculty teams conducting pre-briefing and debriefing after a high-fidelity trauma simulation. Inter-rater reliability was assessed, and evaluators' qualitative feedback was obtained. Results: Inter-rater reliability was good [prebrief, intraclass correlation coefficient, ICC = 0.955 (95% CI, 0.912-0.977), P < .001; debrief, ICC = 0.853 (95% CI, 0.713-0.924), P < .001]. Qualitative feedback from evaluators found that the eOSAD was easy to complete, simple to read and add comments, and reliably stored data that were readily retrievable, enabling the smooth dissemination of information collected. Discussion: The eOSAD features a secure login, sharable internet access link for distant evaluators, and the immediate exporting of data into a secure database for future analysis. It provided convenience for end-users, produced reliable assessments among independent evaluators, and eliminated multiple sources of possible data corruption. Conclusion: The eOSAD tool format advances the post debriefing evaluation of videoed inter-professional team training in high-fidelity simulation.
Objective: The Objective Structured Assessment of Debriefing (OSAD) is an evidence-based, 8-item tool that uses a behaviorally anchored rating scale in paper-based form to evaluate the quality of debriefing in medical education. The objective of this project was twofold: 1) to create an easy-to-use electronic format of the OSAD (eOSAD) in order to streamline data entry; and 2) to pilot its use on videoed debriefings. Materials and Methods: The eOSAD was developed in collaboration with the LSU Health New Orleans Epidemiology Data Center using SurveyGizmo (Widgix Software, LLC, Boulder, CO, USA) software. The eOSAD was then piloted by 2 trained evaluators who rated 37 videos of faculty teams conducting pre-briefing and debriefing after a high-fidelity trauma simulation. Inter-rater reliability was assessed, and evaluators' qualitative feedback was obtained. Results: Inter-rater reliability was good [prebrief, intraclass correlation coefficient, ICC = 0.955 (95% CI, 0.912-0.977), P < .001; debrief, ICC = 0.853 (95% CI, 0.713-0.924), P < .001]. Qualitative feedback from evaluators found that the eOSAD was easy to complete, simple to read and add comments, and reliably stored data that were readily retrievable, enabling the smooth dissemination of information collected. Discussion: The eOSAD features a secure login, sharable internet access link for distant evaluators, and the immediate exporting of data into a secure database for future analysis. It provided convenience for end-users, produced reliable assessments among independent evaluators, and eliminated multiple sources of possible data corruption. Conclusion: The eOSAD tool format advances the post debriefing evaluation of videoed inter-professional team training in high-fidelity simulation.
Authors: Tara S Jankouskas; Kim Kopenhaver Haidet; Judith E Hupcey; Ann Kolanowski; W Bosseau Murray Journal: Simul Healthc Date: 2011-12 Impact factor: 1.929
Authors: Marisa Brett-Fleegler; Jenny Rudolph; Walter Eppich; Michael Monuteaux; Eric Fleegler; Adam Cheng; Robert Simon Journal: Simul Healthc Date: 2012-10 Impact factor: 1.929
Authors: Eduardo Salas; Cameron Klein; Heidi King; Mary Salisbury; Jeffey S Augenstein; David J Birnbach; Donald W Robinson; Christin Upshaw Journal: Jt Comm J Qual Patient Saf Date: 2008-09
Authors: John T Paige; Deborah D Garbee; Qingzhao Yu; John Zahmjahn; Raquel Baroni de Carvalho; Lin Zhu; Vadym Rusnak; Vladimir J Kiselov Journal: BMJ Simul Technol Enhanc Learn Date: 2021-01-27
Authors: Kaushik Baliga; Andrew Coggins; Sandra Warburton; Divya Mathias; Nicole K Yamada; Janene H Fuerch; Louis P Halamek Journal: BMC Med Educ Date: 2022-08-22 Impact factor: 3.263
Authors: Thomaz Bittencourt Couto; Francisco Maio Matos; Paula Dias de Toledo Rodovalho; Mary Fey; Robert Simon; Sacha Muller-Botti Journal: Adv Simul (Lond) Date: 2021-07-07