Jane Runnacles1, Libby Thomas2, James Korndorffer3, Sonal Arora4, Nick Sevdalis5. 1. Department of Paediatrics, Royal Free London NHS Foundation Trust, London, UK. 2. Division of Medical Education, King's College, London, UK. 3. Department of Surgery, Tulane University, New Orleans, Louisiana, USA. 4. Department of Surgery and Cancer, Imperial College, London, UK. 5. Health Service & Population Research Department, Centre for Implementation Science, King's College, London, UK.
Abstract
Introduction: Debriefing is essential to maximise the simulation-based learning experience, but until recently, there was little guidance on an effective paediatric debriefing. A debriefing assessment tool, Objective Structured Assessment of Debriefing (OSAD), has been developed to measure the quality of feedback in paediatric simulation debriefings. This study gathers and evaluates the validity evidence of OSAD with reference to the contemporary hypothesis-driven approach to validity. Methods: Expert input on the paediatric OSAD tool from 10 paediatric simulation facilitators provided validity evidence based on content and feasibility (phase 1). Evidence for internal structure validity was sought by examining reliability of scores from video ratings of 35 postsimulation debriefings; and evidence for validity based on relationship to other variables was sought by comparing results with trainee ratings of the same debriefings (phase 2). Results: Simulation experts' scores were significantly positive regarding the content of OSAD and its instructions. OSAD's feasibility was demonstrated with positive comments regarding clarity and application. Inter-rater reliability was demonstrated with intraclass correlations above 0.45 for 6 of the 7 dimensions of OSAD. The internal consistency of OSAD (Cronbach α) was 0.78. Pearson correlation of trainee total score with OSAD total score was 0.82 (p<0.001) demonstrating validity evidence based on relationships to other variables. Conclusion: The paediatric OSAD tool provides a structured approach to debriefing, which is evidence-based, has multiple sources of validity evidence and is relevant to end-users. OSAD may be used to improve the quality of debriefing after paediatric simulations. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.
Introduction: Debriefing is essential to maximise the simulation-based learning experience, but until recently, there was little guidance on an effective paediatric debriefing. A debriefing assessment tool, Objective Structured Assessment of Debriefing (OSAD), has been developed to measure the quality of feedback in paediatric simulation debriefings. This study gathers and evaluates the validity evidence of OSAD with reference to the contemporary hypothesis-driven approach to validity. Methods: Expert input on the paediatric OSAD tool from 10 paediatric simulation facilitators provided validity evidence based on content and feasibility (phase 1). Evidence for internal structure validity was sought by examining reliability of scores from video ratings of 35 postsimulation debriefings; and evidence for validity based on relationship to other variables was sought by comparing results with trainee ratings of the same debriefings (phase 2). Results: Simulation experts' scores were significantly positive regarding the content of OSAD and its instructions. OSAD's feasibility was demonstrated with positive comments regarding clarity and application. Inter-rater reliability was demonstrated with intraclass correlations above 0.45 for 6 of the 7 dimensions of OSAD. The internal consistency of OSAD (Cronbach α) was 0.78. Pearson correlation of trainee total score with OSAD total score was 0.82 (p<0.001) demonstrating validity evidence based on relationships to other variables. Conclusion: The paediatric OSAD tool provides a structured approach to debriefing, which is evidence-based, has multiple sources of validity evidence and is relevant to end-users. OSAD may be used to improve the quality of debriefing after paediatric simulations. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.
Authors: Adam Cheng; Elizabeth A Hunt; Aaron Donoghue; Kristen Nelson; Judy Leflore; JoDee Anderson; Walter Eppich; Robert Simon; Jenny Rudolph; Vinay Nadkarni Journal: Simul Healthc Date: 2011-02 Impact factor: 1.929
Authors: Marisa Brett-Fleegler; Jenny Rudolph; Walter Eppich; Michael Monuteaux; Eric Fleegler; Adam Cheng; Robert Simon Journal: Simul Healthc Date: 2012-10 Impact factor: 1.929
Authors: Adam Cheng; Walter Eppich; Vincent Grant; Jonathan Sherbino; Benjamin Zendejas; David A Cook Journal: Med Educ Date: 2014-07 Impact factor: 6.251
Authors: Kaushik Baliga; Andrew Coggins; Sandra Warburton; Divya Mathias; Nicole K Yamada; Janene H Fuerch; Louis P Halamek Journal: BMC Med Educ Date: 2022-08-22 Impact factor: 3.263