Louise Hull1, Stephanie Russ2, Maria Ahmed3, Nick Sevdalis1, David J Birnbach4. 1. Health Service and Population Research Department, Center for Implementation Science, King's College London, London, UK. 2. Institute for Education in Medical and Dental Sciences, University of Aberdeen, Aberdeen, UK. 3. Center for Primary Care, NIHR School for Primary Care Research, Manchester Academic Health Science Center, University of Manchester, Manchester, UK. 4. Miller School of Medicine, University of Miami, Miami, Florida, USA.
Abstract
Introduction: Debriefing is widely perceived to be the most important component of simulation-based training. This study aimed to explore the value of 360° evaluation of debriefing by examining expert debriefing evaluators, debriefers and learners' perceptions of the quality of interdisciplinary debriefings. Method: This was a cross-sectional observational study. 41 teams, consisting of 278 learners, underwent simulation-based team training. Immediately following the postsimulation debriefing session, debriefers and learners rated the quality of debriefing using the validated Objective Structured Assessment of Debriefing (OSAD) framework. All debriefing sessions were video-recorded and subsequently rated by evaluators trained to proficiency in assessing debriefing quality. Results: Expert debriefing evaluators and debriefers' perceptions of debriefing quality differed significantly; debriefers perceived the quality of debriefing they provided more favourably than expert debriefing evaluators (40.98% of OSAD ratings provided by debriefers were ≥+1 point greater than expert debriefing evaluators' ratings). Further, learner perceptions of the quality of debriefing differed from both expert evaluators and debriefers' perceptions: weak agreement between learner and expert evaluators' perceptions was found on 2 of 8 OSAD elements (learner engagement and reflection); similarly weak agreement between learner and debriefer perceptions was found on just 1 OSAD element (application). Conclusions: Debriefers and learners' perceptions of debriefing quality differ significantly. Both groups tend to perceive the quality of debriefing far more favourably than external evaluators. An overconfident debriefer may fail to identify elements of debriefing that require improvement. Feedback provided by learners to debriefers may be of limited value in facilitating improvements. We recommend periodic external evaluation of debriefing quality. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.
Introduction: Debriefing is widely perceived to be the most important component of simulation-based training. This study aimed to explore the value of 360° evaluation of debriefing by examining expert debriefing evaluators, debriefers and learners' perceptions of the quality of interdisciplinary debriefings. Method: This was a cross-sectional observational study. 41 teams, consisting of 278 learners, underwent simulation-based team training. Immediately following the postsimulation debriefing session, debriefers and learners rated the quality of debriefing using the validated Objective Structured Assessment of Debriefing (OSAD) framework. All debriefing sessions were video-recorded and subsequently rated by evaluators trained to proficiency in assessing debriefing quality. Results: Expert debriefing evaluators and debriefers' perceptions of debriefing quality differed significantly; debriefers perceived the quality of debriefing they provided more favourably than expert debriefing evaluators (40.98% of OSAD ratings provided by debriefers were ≥+1 point greater than expert debriefing evaluators' ratings). Further, learner perceptions of the quality of debriefing differed from both expert evaluators and debriefers' perceptions: weak agreement between learner and expert evaluators' perceptions was found on 2 of 8 OSAD elements (learner engagement and reflection); similarly weak agreement between learner and debriefer perceptions was found on just 1 OSAD element (application). Conclusions: Debriefers and learners' perceptions of debriefing quality differ significantly. Both groups tend to perceive the quality of debriefing far more favourably than external evaluators. An overconfident debriefer may fail to identify elements of debriefing that require improvement. Feedback provided by learners to debriefers may be of limited value in facilitating improvements. We recommend periodic external evaluation of debriefing quality. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.
Entities:
Keywords:
debriefing; feedback; interdisciplinary training; simulation-based training
Authors: Sonal Arora; Maria Ahmed; John Paige; Debra Nestel; Jane Runnacles; Louise Hull; Ara Darzi; Nick Sevdalis Journal: Ann Surg Date: 2012-12 Impact factor: 12.969
Authors: Georges L Savoldelli; Viren N Naik; Jason Park; Hwan S Joo; Roger Chow; Stanley J Hamstra Journal: Anesthesiology Date: 2006-08 Impact factor: 7.892
Authors: Adam Cheng; Vincent Grant; Peter Dieckmann; Sonal Arora; Traci Robinson; Walter Eppich Journal: Simul Healthc Date: 2015-08 Impact factor: 1.929
Authors: Marisa Brett-Fleegler; Jenny Rudolph; Walter Eppich; Michael Monuteaux; Eric Fleegler; Adam Cheng; Robert Simon Journal: Simul Healthc Date: 2012-10 Impact factor: 1.929
Authors: Adam Cheng; Walter Eppich; Vincent Grant; Jonathan Sherbino; Benjamin Zendejas; David A Cook Journal: Med Educ Date: 2014-07 Impact factor: 6.251
Authors: David A Davis; Paul E Mazmanian; Michael Fordis; R Van Harrison; Kevin E Thorpe; Laure Perrier Journal: JAMA Date: 2006-09-06 Impact factor: 56.272
Authors: Eduardo Salas; Cameron Klein; Heidi King; Mary Salisbury; Jeffey S Augenstein; David J Birnbach; Donald W Robinson; Christin Upshaw Journal: Jt Comm J Qual Patient Saf Date: 2008-09