Keith Siau1,2, James Crossley3, Paul Dunckley1,4, Gavin Johnson1,5, Mark Feeney1,6, Marietta Iacucci7, John T Anderson1,8. 1. Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, UK. 2. Medical and Dental Sciences, University of Birmingham, Birmingham, UK. 3. Academic Unit of Medical Education, University of Sheffield, Sheffield, UK. 4. Department of Gastroenterology, Gloucestershire Hospitals NHSFT, Gloucester, UK. 5. Department of Gastroenterology, University College London Hospitals NHSFT, London, UK. 6. Department of Gastroenterology, Torbay and South Devon NHS Foundation Trust, Torquay, UK. 7. NIHR Biomedical Research Centre, University of Birmingham and University Hospitals NHS Foundation Trust, Birmingham, UK. 8. Endoscopy, Gloucestershire Hospitals NHSFT, Cheltenham, UK.
Abstract
INTRODUCTION: Formative colonoscopy direct observation of procedural skills (DOPS) assessments were updated in 2016 and incorporated into UK training but lack validity evidence. We aimed to appraise the validity of DOPS assessments, benchmark performance, and evaluate competency development during training in diagnostic colonoscopy. METHODS: This prospective national study identified colonoscopy DOPS submitted over an 18-month period to the UK training e-portfolio. Generalizability analyses were conducted to evaluate internal structure validity and reliability. Benchmarking was performed using receiver operator characteristic analyses. Learning curves for DOPS items and domains were studied, and multivariable analyses were performed to identify predictors of DOPS competency. RESULTS: Across 279 training units, 10,749 DOPS submitted for 1,199 trainees were analyzed. The acceptable reliability threshold (G > 0.70) was achieved with 3 assessors performing 2 DOPS each. DOPS competency rates correlated with the unassisted caecal intubation rate (rho 0.404, P < 0.001). Demonstrating competency in 90% of assessed items provided optimal sensitivity (90.2%) and specificity (87.2%) for benchmarking overall DOPS competence. This threshold was attained in the following order: "preprocedure" (50-99 procedures), "endoscopic nontechnical skills" and "postprocedure" (150-199), "management" (200-249), and "procedure" (250-299) domain. At item level, competency in "proactive problem solving" (rho 0.787) and "loop management" (rho 0.780) correlated strongest with the overall DOPS rating (P < 0.001) and was the last to develop. Lifetime procedure count, DOPS count, trainer specialty, easier case difficulty, and higher cecal intubation rate were significant multivariable predictors of DOPS competence. DISCUSSION: This study establishes milestones for competency acquisition during colonoscopy training and provides novel validity and reliability evidence to support colonoscopy DOPS as a competency assessment tool.
INTRODUCTION: Formative colonoscopy direct observation of procedural skills (DOPS) assessments were updated in 2016 and incorporated into UK training but lack validity evidence. We aimed to appraise the validity of DOPS assessments, benchmark performance, and evaluate competency development during training in diagnostic colonoscopy. METHODS: This prospective national study identified colonoscopy DOPS submitted over an 18-month period to the UK training e-portfolio. Generalizability analyses were conducted to evaluate internal structure validity and reliability. Benchmarking was performed using receiver operator characteristic analyses. Learning curves for DOPS items and domains were studied, and multivariable analyses were performed to identify predictors of DOPS competency. RESULTS: Across 279 training units, 10,749 DOPS submitted for 1,199 trainees were analyzed. The acceptable reliability threshold (G > 0.70) was achieved with 3 assessors performing 2 DOPS each. DOPS competency rates correlated with the unassisted caecal intubation rate (rho 0.404, P < 0.001). Demonstrating competency in 90% of assessed items provided optimal sensitivity (90.2%) and specificity (87.2%) for benchmarking overall DOPS competence. This threshold was attained in the following order: "preprocedure" (50-99 procedures), "endoscopic nontechnical skills" and "postprocedure" (150-199), "management" (200-249), and "procedure" (250-299) domain. At item level, competency in "proactive problem solving" (rho 0.787) and "loop management" (rho 0.780) correlated strongest with the overall DOPS rating (P < 0.001) and was the last to develop. Lifetime procedure count, DOPS count, trainer specialty, easier case difficulty, and higher cecal intubation rate were significant multivariable predictors of DOPS competence. DISCUSSION: This study establishes milestones for competency acquisition during colonoscopy training and provides novel validity and reliability evidence to support colonoscopy DOPS as a competency assessment tool.
Authors: Keith Siau; James Hodson; John T Anderson; Roland Valori; Geoff Smith; Paul Hagan; Marietta Iacucci; Paul Dunckley Journal: World J Gastroenterol Date: 2020-06-21 Impact factor: 5.742
Authors: Katarzyna M Pawlak; Jan Kral; Rishad Khan; Sunil Amin; Mohammad Bilal; Rashid N Lui; Dalbir S Sandhu; Almoutaz Hashim; Steven Bollipo; Aline Charabaty; Enrique de-Madaria; Andrés F Rodríguez-Parra; Sergio A Sánchez-Luna; Michał Żorniak; Catharine M Walsh; Samir C Grover; Keith Siau Journal: Gastrointest Endosc Date: 2020-06-11 Impact factor: 9.427
Authors: Rishad Khan; Parul Tandon; Michael A Scaffidi; Kirles Bishay; Katarzyna M Pawlak; Jan Kral; Sunil Amin; Mohammad Bilal; Rashid N Lui; Dalbir S Sandhu; Almoutaz Hashim; Steven Bollipo; Aline Charabaty; Enrique de-Madaria; Andrés F Rodríguez-Parra; Sergio A Sánchez-Luna; Michał Żorniak; Keith Siau; Catharine M Walsh; Samir C Grover Journal: J Can Assoc Gastroenterol Date: 2020-10-24