Katherine A Barsness1, Deborah M Rooney2, Lauren M Davis3, Anthony C Chin4. 1. Division of Pediatric Surgery, Ann and Robert H Lurie Children's Hospital of Chicago, Northwestern University Feinberg School of Medicine, Chicago, IL, USA; Center for Simulation Technology and Immersive Learning, Northwestern University Feinberg School of Medicine, Chicago, IL, USA. Electronic address: kbarsness@luriechildrens.org. 2. Department of Medical Education, University of Michigan Medical School, Ann Arbor, MI, USA. 3. Center for Simulation Technology and Immersive Learning, Northwestern University Feinberg School of Medicine, Chicago, IL, USA. 4. Division of Pediatric Surgery, Ann and Robert H Lurie Children's Hospital of Chicago, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
Abstract
PURPOSE: A validated high fidelity simulation model would provide a safe environment to teach thoracoscopic EA/TEF repair to novices. The study purpose was to evaluate validity evidence for performance measures on an EA/TEF simulator. METHODS: IRB-exempt data were collected from 12 self-reported "novice" and 8 "experienced" pediatric surgeons. Participants evaluated the EA/TEF repair simulator using survey ratings that were analyzed for test content validity evidence. Additionally, deidentified operative performances were videotaped and independently rated by two surgeons using the Objective Structured Assessment for Technical Skills (OSATS) instrument. Novice and experienced OSATS were compared with p<.05 significant. RESULTS: Participants had high overall simulator ratings. Internal structure was supported by high interitem consistency (α=.95 and .96) and interrater agreement (ICC) [.52, .84] for OSATS ratings. Experienced surgeons performed at a significantly higher level than novices for all five primary and two supplemental OSATS items (p<.05). CONCLUSION: Favorable participant ratings indicate the simulator is relevant to clinical practice and valuable as a learning tool. Further, performance ratings can discriminate experienced and novice performances of EA/TEF repair. These findings support the use of the simulator for performance assessment, representing the first validated measures from a simulator intended for pediatric surgical training.
PURPOSE: A validated high fidelity simulation model would provide a safe environment to teach thoracoscopic EA/TEF repair to novices. The study purpose was to evaluate validity evidence for performance measures on an EA/TEF simulator. METHODS: IRB-exempt data were collected from 12 self-reported "novice" and 8 "experienced" pediatric surgeons. Participants evaluated the EA/TEF repair simulator using survey ratings that were analyzed for test content validity evidence. Additionally, deidentified operative performances were videotaped and independently rated by two surgeons using the Objective Structured Assessment for Technical Skills (OSATS) instrument. Novice and experienced OSATS were compared with p<.05 significant. RESULTS:Participants had high overall simulator ratings. Internal structure was supported by high interitem consistency (α=.95 and .96) and interrater agreement (ICC) [.52, .84] for OSATS ratings. Experienced surgeons performed at a significantly higher level than novices for all five primary and two supplemental OSATS items (p<.05). CONCLUSION: Favorable participant ratings indicate the simulator is relevant to clinical practice and valuable as a learning tool. Further, performance ratings can discriminate experienced and novice performances of EA/TEF repair. These findings support the use of the simulator for performance assessment, representing the first validated measures from a simulator intended for pediatric surgical training.
Authors: Jonathan J Neville; Carmen S Chacon; Reza Haghighi-Osgouei; Natasha Houghton; Fernando Bello; Simon A Clarke Journal: Pediatr Surg Int Date: 2021-09-02 Impact factor: 1.827