Catharine M Walsh1, Simon C Ling2, Nitin Khanna3, Samir C Grover4, Jeffrey J Yu5, Mary Anne Cooper6, Elaine Yong6, Geoffrey C Nguyen7, Gary May4, Thomas D Walters2, Richard Reznick8, Linda Rabeneck9, Heather Carnahan10. 1. Division of Gastroenterology, Hepatology and Nutrition, Hospital for Sick Children, Toronto, Ontario, Canada; Department of Paediatrics, University of Toronto, Toronto, Ontario, Canada; Wilson Centre, University of Toronto, Toronto, Ontario, Canada. 2. Division of Gastroenterology, Hepatology and Nutrition, Hospital for Sick Children, Toronto, Ontario, Canada; Department of Paediatrics, University of Toronto, Toronto, Ontario, Canada. 3. Division of Gastroenterology, St. Joseph's Health Centre, University of Western Ontario, London, Ontario, Canada. 4. Division of Gastroenterology, St. Michael's Hospital, Toronto, Ontario, Canada; Department of Medicine, University of Toronto, Toronto, Ontario, Canada. 5. Wilson Centre, University of Toronto, Toronto, Ontario, Canada. 6. Division of Gastroenterology, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada; Department of Medicine, University of Toronto, Toronto, Ontario, Canada. 7. Division of Gastroenterology, Mount Sinai Hospital, Toronto, Ontario, Canada; Department of Medicine, University of Toronto, Toronto, Ontario, Canada. 8. Faculty of Health Sciences, Queen's University Kingston, Ontario, Canada. 9. Division of Gastroenterology, Mount Sinai Hospital, Toronto, Ontario, Canada; Cancer Care Ontario, Toronto, Ontario, Canada; Department of Medicine, University of Toronto, Toronto, Ontario, Canada. 10. School of Human Kinetics and Recreation, Memorial University of Newfoundland, St. John's, Newfoundland, Canada.
Abstract
BACKGROUND: Rigorously developed and validated direct observational assessment tools are required to support competency-based colonoscopy training to facilitate skill acquisition, optimize learning, and ensure readiness for unsupervised practice. OBJECTIVE: To examine reliability and validity evidence of the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) for colonoscopy for use within the clinical setting. DESIGN: Prospective, observational, multicenter validation study. Sixty-one endoscopists performing 116 colonoscopies were assessed using the GiECAT, which consists of a 7-item global rating scale (GRS) and 19-item checklist (CL). A second rater assessed procedures to determine interrater reliability by using intraclass correlation coefficients (ICCs). Endoscopists' first and second procedure scores were compared to determine test-retest reliability by using ICCs. Discriminative validity was examined by comparing novice, intermediate, and experienced endoscopists' scores. Concurrent validity was measured by correlating scores with colonoscopy experience, cecal and terminal ileal intubation rates, and physician global assessment. SETTING: A total of 116 colonoscopies performed by 33 novice (<50 previous procedures), 18 intermediate (50-500 previous procedures), and 10 experienced (>1000 previous procedures) endoscopists from 6 Canadian hospitals. MAIN OUTCOME MEASUREMENTS: Interrater and test-retest reliability, discriminative, and concurrent validity. RESULTS: Interrater reliability was high (total: ICC=0.85; GRS: ICC=0.85; CL: ICC=0.81). Test-retest reliability was excellent (total: ICC=0.91; GRS: ICC=0.93; CL: ICC=0.80). Significant differences in GiECAT scores among novice, intermediate, and experienced endoscopists were noted (P<.001). There was a significant positive correlation (P<.001) between scores and number of previous colonoscopies (total: ρ=0.78, GRS: ρ=0.80, CL: Spearman's ρ=0.71); cecal intubation rate (total: ρ=0.81, GRS: Spearman's ρ=0.82, CL: Spearman's ρ=0.75); ileal intubation rate (total: Spearman's ρ=0.82, GRS: Spearman's ρ=0.82, CL: Spearman's ρ=0.77); and physician global assessment (total: Spearman's ρ=0.90, GRS: Spearman's ρ=0.94, CL: Spearman's ρ=0.77). LIMITATIONS: Nonblinded assessments. CONCLUSION: This study provides evidence supporting the reliability and validity of the GiECAT for use in assessing the performance of live colonoscopies in the clinical setting.
BACKGROUND: Rigorously developed and validated direct observational assessment tools are required to support competency-based colonoscopy training to facilitate skill acquisition, optimize learning, and ensure readiness for unsupervised practice. OBJECTIVE: To examine reliability and validity evidence of the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) for colonoscopy for use within the clinical setting. DESIGN: Prospective, observational, multicenter validation study. Sixty-one endoscopists performing 116 colonoscopies were assessed using the GiECAT, which consists of a 7-item global rating scale (GRS) and 19-item checklist (CL). A second rater assessed procedures to determine interrater reliability by using intraclass correlation coefficients (ICCs). Endoscopists' first and second procedure scores were compared to determine test-retest reliability by using ICCs. Discriminative validity was examined by comparing novice, intermediate, and experienced endoscopists' scores. Concurrent validity was measured by correlating scores with colonoscopy experience, cecal and terminal ileal intubation rates, and physician global assessment. SETTING: A total of 116 colonoscopies performed by 33 novice (<50 previous procedures), 18 intermediate (50-500 previous procedures), and 10 experienced (>1000 previous procedures) endoscopists from 6 Canadian hospitals. MAIN OUTCOME MEASUREMENTS: Interrater and test-retest reliability, discriminative, and concurrent validity. RESULTS: Interrater reliability was high (total: ICC=0.85; GRS: ICC=0.85; CL: ICC=0.81). Test-retest reliability was excellent (total: ICC=0.91; GRS: ICC=0.93; CL: ICC=0.80). Significant differences in GiECAT scores among novice, intermediate, and experienced endoscopists were noted (P<.001). There was a significant positive correlation (P<.001) between scores and number of previous colonoscopies (total: ρ=0.78, GRS: ρ=0.80, CL: Spearman's ρ=0.71); cecal intubation rate (total: ρ=0.81, GRS: Spearman's ρ=0.82, CL: Spearman's ρ=0.75); ileal intubation rate (total: Spearman's ρ=0.82, GRS: Spearman's ρ=0.82, CL: Spearman's ρ=0.77); and physician global assessment (total: Spearman's ρ=0.90, GRS: Spearman's ρ=0.94, CL: Spearman's ρ=0.77). LIMITATIONS: Nonblinded assessments. CONCLUSION: This study provides evidence supporting the reliability and validity of the GiECAT for use in assessing the performance of live colonoscopies in the clinical setting.
Authors: Meghan McCullough; Alex Campbell; Armando Siu; Libby Durnwald; Shubha Kumar; William P Magee; Jordan Swanson Journal: World J Surg Date: 2018-03 Impact factor: 3.352
Authors: Michael A Scaffidi; Catharine M Walsh; Rishad Khan; Colleen H Parker; Ahmed Al-Mazroui; Michael Abunassar; Alexander W Grindal; Peter Lin; Christopher Wang; Robert Bechara; Samir C Grover Journal: Endosc Int Open Date: 2019-05-03
Authors: Andreas Slot Vilmann; Christian Lachenmeier; Morten Bo Søndergaard Svendsen; Bo Søndergaard; Yoon Soo Park; Lars Bo Svendsen; Lars Konge Journal: Endosc Int Open Date: 2020-05-25
Authors: Rishad Khan; Parul Tandon; Michael A Scaffidi; Kirles Bishay; Katarzyna M Pawlak; Jan Kral; Sunil Amin; Mohammad Bilal; Rashid N Lui; Dalbir S Sandhu; Almoutaz Hashim; Steven Bollipo; Aline Charabaty; Enrique de-Madaria; Andrés F Rodríguez-Parra; Sergio A Sánchez-Luna; Michał Żorniak; Keith Siau; Catharine M Walsh; Samir C Grover Journal: J Can Assoc Gastroenterol Date: 2020-10-24