Anna H de Vries1, Arno M M Muijtjens2, Hilde G J van Genugten3, Ad J M Hendrikx3, Evert L Koldewijn3,4, Barbara M A Schout5,6, Cees P M van der Vleuten2, Cordula Wagner6,7, Irene M Tjiam8, Jeroen J G van Merriënboer4. 1. Department of Urology, Catharina Hospital, Michelangelolaan 2, 5623 EJ, Eindhoven, The Netherlands. a.h.de.vries@hotmail.com. 2. Department of Educational Development & Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, The Netherlands. 3. Department of Urology, Catharina Hospital, Michelangelolaan 2, 5623 EJ, Eindhoven, The Netherlands. 4. School of Health Professions Education, Maastricht University, Maastricht, The Netherlands. 5. Department of Urology, Alrijne Hospital, Leiden, The Netherlands. 6. Netherlands Institute for Health Services Research (NIVEL), Utrecht, The Netherlands. 7. Department of Public and Occupational Health, EMGO Institute for Health and Care Research, Amsterdam, The Netherlands. 8. Department of Urology, Canisius Wilhelmina Hospital, Nijmegen, The Netherlands.
Abstract
BACKGROUND: The current shift towards competency-based residency training has increased the need for objective assessment of skills. In this study, we developed and validated an assessment tool that measures technical and non-technical competency in transurethral resection of bladder tumour (TURBT). METHODS: The 'Test Objective Competency' (TOCO)-TURBT tool was designed by means of cognitive task analysis (CTA), which included expert consensus. The tool consists of 51 items, divided into 3 phases: preparatory (n = 15), procedural (n = 21), and completion (n = 15). For validation of the TOCO-TURBT tool, 2 TURBT procedures were performed and videotaped by 25 urologists and 51 residents in a simulated setting. The participants' degree of competence was assessed by a panel of eight independent expert urologists using the TOCO-TURBT tool. Each procedure was assessed by two raters. Feasibility, acceptability and content validity were evaluated by means of a quantitative cross-sectional survey. Regression analyses were performed to assess the strength of the relation between experience and test scores (construct validity). Reliability was analysed by generalizability theory. RESULTS: The majority of assessors and urologists indicated the TOCO-TURBT tool to be a valid assessment of competency and would support the implementation of the TOCO-TURBT assessment as a certification method for residents. Construct validity was clearly established for all outcome measures of the procedural phase (all r > 0.5, p < 0.01). Generalizability-theory analysis showed high reliability (coefficient Phi ≥ 0.8) when using the format of two assessors and two cases. CONCLUSIONS: This study provides first evidence that the TOCO-TURBT tool is a feasible, valid and reliable assessment tool for measuring competency in TURBT. The tool has the potential to be used for future certification of competencies for residents and urologists. The methodology of CTA might be valuable in the development of assessment tools in other areas of clinical practice.
BACKGROUND: The current shift towards competency-based residency training has increased the need for objective assessment of skills. In this study, we developed and validated an assessment tool that measures technical and non-technical competency in transurethral resection of bladder tumour (TURBT). METHODS: The 'Test Objective Competency' (TOCO)-TURBT tool was designed by means of cognitive task analysis (CTA), which included expert consensus. The tool consists of 51 items, divided into 3 phases: preparatory (n = 15), procedural (n = 21), and completion (n = 15). For validation of the TOCO-TURBT tool, 2 TURBT procedures were performed and videotaped by 25 urologists and 51 residents in a simulated setting. The participants' degree of competence was assessed by a panel of eight independent expert urologists using the TOCO-TURBT tool. Each procedure was assessed by two raters. Feasibility, acceptability and content validity were evaluated by means of a quantitative cross-sectional survey. Regression analyses were performed to assess the strength of the relation between experience and test scores (construct validity). Reliability was analysed by generalizability theory. RESULTS: The majority of assessors and urologists indicated the TOCO-TURBT tool to be a valid assessment of competency and would support the implementation of the TOCO-TURBT assessment as a certification method for residents. Construct validity was clearly established for all outcome measures of the procedural phase (all r > 0.5, p < 0.01). Generalizability-theory analysis showed high reliability (coefficient Phi ≥ 0.8) when using the format of two assessors and two cases. CONCLUSIONS: This study provides first evidence that the TOCO-TURBT tool is a feasible, valid and reliable assessment tool for measuring competency in TURBT. The tool has the potential to be used for future certification of competencies for residents and urologists. The methodology of CTA might be valuable in the development of assessment tools in other areas of clinical practice.
Authors: Anna H de Vries; Hilde G J van Genugten; Ad J M Hendrikx; Evert L Koldewijn; Barbara M A Schout; Irene M Tjiam; Jeroen J G van Merriënboer; Arno M M Muijtjens; Cordula Wagner Journal: J Endourol Date: 2016-01-22 Impact factor: 2.942
Authors: Maura E Sullivan; Adrian Ortega; Nir Wasserberg; Howard Kaufman; Julie Nyquist; Richard Clark Journal: Am J Surg Date: 2008-01 Impact factor: 2.565
Authors: Marko Babjuk; Maximilian Burger; Richard Zigeuner; Shahrokh F Shariat; Bas W G van Rhijn; Eva Compérat; Richard J Sylvester; Eero Kaasinen; Andreas Böhle; Joan Palou Redorta; Morgan Rouprêt Journal: Eur Urol Date: 2013-06-12 Impact factor: 20.096
Authors: Cora Griffin; Abdullatif Aydın; Oliver Brunckhorst; Nicholas Raison; Muhammad Shamim Khan; Prokar Dasgupta; Kamran Ahmed Journal: World J Urol Date: 2019-09-17 Impact factor: 4.226