B Joseph Elmunzer1, Catharine M Walsh2, Gretchen Guiton3, Jose Serrano4, Amitabh Chak5, Steven Edmundowicz6, Richard S Kwon7, Daniel Mullady8, Georgios I Papachristou9, Grace Elta7, Todd H Baron10, Patrick Yachimski11, Evan L Fogel12, Peter V Draganov13, Jason R Taylor14, James Scheiman15, Vikesh K Singh16, Shyam Varadarajulu17, Field F Willingham18, Gregory A Cote1, Peter B Cotton1, Violette Simon6, Rebecca Spitzer1, Rajesh Keswani19, Sachin Wani6. 1. Division of Gastroenterology and Hepatology, Medical University of South Carolina, Charleston, South Carolina, USA. 2. Division of Gastroenterology, Hepatology, and Nutrition, Learning Institute and Research Institute, Hospital for Sick Children, Toronto, Ontario, Canada. 3. Department of Internal Medicine, University of Colorado School of Medicine, Aurora, Colorado, USA. 4. National Institute of Diabetes and Digestive and Kidney Diseases, National Institutes of Health, Bethesda, Maryland, USA. 5. Division of Gastroenterology and Liver Disease, Case Western Reserve University, Cleveland, Ohio, USA. 6. Division of Gastroenterology and Hepatology, University of Colorado Anschutz Medical Campus, Aurora, Colorado, USA. 7. Division of Gastroenterology, University of Michigan, Ann Arbor, Michigan, USA. 8. Division of Gastroenterology, Washington University School of Medicine, St Louis, Missouri, USA. 9. Division of Gastroenterology, Hepatology, and Nutrition, Ohio State University Wexner Medical Center, Columbus, Ohio, USA. 10. Division of Gastroenterology and Hepatology, University of North Carolina, Chapel Hill, North Carolina, USA. 11. Division of Gastroenterology, Vanderbilt University, Nashville, Tennessee, USA. 12. Division of Gastroenterology and Hepatology, Indiana University, Indianapolis, Indiana, USA. 13. Division of Gastroenterology, Hepatology, and Nutrition, University of Florida, Gainesville, Florida, USA. 14. Division of Gastroenterology and Hepatology, St Louis University, St Louis, Missouri, USA. 15. Division of Gastroenterology and Hepatology, University of Virginia, Charlottesville, Virginia, USA. 16. Division of Gastroenterology, Johns Hopkins Medical Institutions, Baltimore, Maryland, USA. 17. Center for Interventional Endoscopy, Advent Health, Orlando, Florida, USA. 18. Division of Digestive Diseases, Emory University, Atlanta, Georgia, USA. 19. Division of Gastroenterology, Northwestern University, Chicago, Illinois, USA.
Abstract
BACKGROUND AND AIMS: The accurate measurement of technical skill in ERCP is essential for endoscopic training, quality assurance, and coaching of this procedure. Hypothesizing that technical skill can be measured by analysis of ERCP videos, we aimed to develop and validate a video-based ERCP skill assessment tool. METHODS: Based on review of procedural videos, the task of ERCP was deconstructed into its basic components by an expert panel that developed an initial version of the Bethesda ERCP Skill Assessment Tool (BESAT). Subsequently, 2 modified Delphi panels and 3 validation exercises were conducted with the goal of iteratively refining the tool. Fully crossed generalizability studies investigated the contributions of assessors, ERCP performance, and technical elements to reliability. RESULTS: Twenty-nine technical elements were initially generated from task deconstruction. Ultimately, after iterative refinement, the tool comprised 6 technical elements and 11 subelements. The developmental process achieved consistent improvements in the performance characteristics of the tool with every iteration. For the most recent version of the tool, BESAT-v4, the generalizability coefficient (a reliability index) was .67. Most variance in BESAT scores (43.55%) was attributed to differences in endoscopists' skill, indicating that the tool can reliably differentiate between endoscopists based on video analysis. CONCLUSIONS: Video-based assessment of ERCP skill appears to be feasible with a novel instrument that demonstrates favorable validity evidence. Future steps include determining whether the tool can discriminate between endoscopists of varying experience levels and predict important outcomes in clinical practice.
BACKGROUND AND AIMS: The accurate measurement of technical skill in ERCP is essential for endoscopic training, quality assurance, and coaching of this procedure. Hypothesizing that technical skill can be measured by analysis of ERCP videos, we aimed to develop and validate a video-based ERCP skill assessment tool. METHODS: Based on review of procedural videos, the task of ERCP was deconstructed into its basic components by an expert panel that developed an initial version of the Bethesda ERCP Skill Assessment Tool (BESAT). Subsequently, 2 modified Delphi panels and 3 validation exercises were conducted with the goal of iteratively refining the tool. Fully crossed generalizability studies investigated the contributions of assessors, ERCP performance, and technical elements to reliability. RESULTS: Twenty-nine technical elements were initially generated from task deconstruction. Ultimately, after iterative refinement, the tool comprised 6 technical elements and 11 subelements. The developmental process achieved consistent improvements in the performance characteristics of the tool with every iteration. For the most recent version of the tool, BESAT-v4, the generalizability coefficient (a reliability index) was .67. Most variance in BESAT scores (43.55%) was attributed to differences in endoscopists' skill, indicating that the tool can reliably differentiate between endoscopists based on video analysis. CONCLUSIONS: Video-based assessment of ERCP skill appears to be feasible with a novel instrument that demonstrates favorable validity evidence. Future steps include determining whether the tool can discriminate between endoscopists of varying experience levels and predict important outcomes in clinical practice.
Authors: Sachin Gupta; John Anderson; Pradeep Bhandari; Brian McKaig; Pullan Rupert; Bjorn Rembacken; Stuart Riley; Matt Rutter; Roland Valori; Margaret Vance; Cees P M van der Vleuten; Brian P Saunders; Siwan Thomas-Gibson Journal: Gastrointest Endosc Date: 2011-06 Impact factor: 9.427
Authors: Sachin Wani; Rajesh N Keswani; Samuel Han; Eva M Aagaard; Matthew Hall; Violette Simon; Wasif M Abidi; Subhas Banerjee; Todd H Baron; Michael Bartel; Erik Bowman; Brian C Brauer; Jonathan M Buscaglia; Linda Carlin; Amitabh Chak; Hemant Chatrath; Abhishek Choudhary; Bradley Confer; Gregory A Coté; Koushik K Das; Christopher J DiMaio; Andrew M Dries; Steven A Edmundowicz; Abdul Hamid El Chafic; Ihab El Hajj; Swan Ellert; Jason Ferreira; Anthony Gamboa; Ian S Gan; Lisa M Gangarosa; Bhargava Gannavarapu; Stuart R Gordon; Nalini M Guda; Hazem T Hammad; Cynthia Harris; Sujai Jalaj; Paul S Jowell; Sana Kenshil; Jason Klapman; Michael L Kochman; Srinadh Komanduri; Gabriel Lang; Linda S Lee; David E Loren; Frank J Lukens; Daniel Mullady; V Raman Muthusamy; Andrew S Nett; Mojtaba S Olyaee; Kavous Pakseresht; Pranith Perera; Patrick Pfau; Cyrus Piraka; John M Poneros; Amit Rastogi; Anthony Razzak; Brian Riff; Shreyas Saligram; James M Scheiman; Isaiah Schuster; Raj J Shah; Rishi Sharma; Joshua P Spaete; Ajaypal Singh; Muhammad Sohail; Jayaprakash Sreenarasimhaiah; Tyler Stevens; James H Tabibian; Demetrios Tzimas; Dushant S Uppal; Shiro Urayama; Domenico Vitterbo; Andrew Y Wang; Wahid Wassef; Patrick Yachimski; Sergio Zepeda-Gomez; Tobias Zuchelli; Dayna Early Journal: Gastroenterology Date: 2018-07-26 Impact factor: 22.682
Authors: Jo Vandervoort; Roy M Soetikno; Tony C K Tham; Richard C K Wong; Angelo P Ferrari; Henry Montes; Alfred D Roston; Adam Slivka; David R Lichtenstein; Frederick W Ruymann; Jacques Van Dam; Mike Hughes; David L Carr-Locke Journal: Gastrointest Endosc Date: 2002-11 Impact factor: 9.427
Authors: Usman Khan; Rishad Khan; Eric Benchimol; Misbah Salim; Jennifer Telford; Robert Enns; Rachid Mohamed; Nauzer Forbes; Gurpal Sandha; Ali Kohansal; Jeffrey Mosko; Avijit Chatterjee; Gary May; Kevin Waschke; Alan Barkun; Paul D James Journal: Endosc Int Open Date: 2022-09-14