Julia Dagnaes-Hansen1, Oria Mahmood2, Sarah Bube3, Flemming Bjerrum4, Yousif Subhi5, Malene Rohrsted6, Lars Konge2. 1. Copenhagen Academy for Medical Education and Simulation, Copenhagen, Capital Region, Denmark; Faculty of Health and Medical Sciences, Faculty of Medicine, University of Copenhagen, Capital Region, Denmark; Department of Urology, University Hospital of Copenhagen, Rigshospitalet, Copenhagen, Denmark. Electronic address: julia.dagnaes@gmail.com. 2. Copenhagen Academy for Medical Education and Simulation, Copenhagen, Capital Region, Denmark; Faculty of Health and Medical Sciences, Faculty of Medicine, University of Copenhagen, Capital Region, Denmark. 3. Copenhagen Academy for Medical Education and Simulation, Copenhagen, Capital Region, Denmark; Department of Urology, Zealand University Hospital, Roskilde, Denmark. 4. Department of Surgery, Herlev Gentofte Hospital, University Hospital of Copenhagen, Herlev, Denmark. 5. Faculty of Health and Medical Sciences, Faculty of Medicine, University of Copenhagen, Capital Region, Denmark; Department of Ophthalmology, Zealand University Hospital, Roskilde, Denmark. 6. Copenhagen Academy for Medical Education and Simulation, Copenhagen, Capital Region, Denmark; Department of Urology, University Hospital of Copenhagen, Rigshospitalet, Copenhagen, Denmark.
Abstract
OBJECTIVE: Direct observation in assessment of clinical skills is prone to bias, demands the observer to be present at a certain location at a specific time, and is time-consuming. Video-based assessment could remove the risk of bias, increase flexibility, and reduce the time spent on assessment. This study investigated if video-based assessment was a reliable tool for cystoscopy and if direct observers were prone to bias compared with video-raters. DESIGN: This study was a blinded observational trial. Twenty medical students and 9 urologists were recorded during 2 cystoscopies and rated by a direct observer and subsequently by 2 blinded video-raters on a global rating scale (GRS) for cystoscopy. Both intrarater and interrater reliability were explored. Furthermore, direct observer bias was explored by a paired samples t-test. RESULTS: Intrarater reliability calculated by Pearson's r was 0.86. Interrater reliability was 0.74 for single measure and 0.85 for average measures. A hawk-dove effect was seen between the 2 raters. Direct observer bias was detected when comparing direct observer scores to the assessment by an independent video-rater (p < 0.001). CONCLUSION: This study found that video-based assessment was a reliable tool for cystoscopy with 2 video-raters. There was a significant bias when comparing direct observation with blinded video-based assessment.
OBJECTIVE: Direct observation in assessment of clinical skills is prone to bias, demands the observer to be present at a certain location at a specific time, and is time-consuming. Video-based assessment could remove the risk of bias, increase flexibility, and reduce the time spent on assessment. This study investigated if video-based assessment was a reliable tool for cystoscopy and if direct observers were prone to bias compared with video-raters. DESIGN: This study was a blinded observational trial. Twenty medical students and 9 urologists were recorded during 2 cystoscopies and rated by a direct observer and subsequently by 2 blinded video-raters on a global rating scale (GRS) for cystoscopy. Both intrarater and interrater reliability were explored. Furthermore, direct observer bias was explored by a paired samples t-test. RESULTS: Intrarater reliability calculated by Pearson's r was 0.86. Interrater reliability was 0.74 for single measure and 0.85 for average measures. A hawk-dove effect was seen between the 2 raters. Direct observer bias was detected when comparing direct observer scores to the assessment by an independent video-rater (p < 0.001). CONCLUSION: This study found that video-based assessment was a reliable tool for cystoscopy with 2 video-raters. There was a significant bias when comparing direct observation with blinded video-based assessment.
Authors: Guglielmo Mantica; Federica Balzarini; Federico Dotta; Moises Rodriguez-Socarras; Silvia Proietti; Guido Giusti; Francesco Oneto; Marco Di Pierro; Paolo Traverso; Carlo Terrone Journal: Arab J Urol Date: 2019-04-24
Authors: Sarah H Bube; Pernille S Kingo; Mia G Madsen; Juan L Vásquez; Thomas Norus; Rikke G Olsen; Claus Dahl; Rikke B Hansen; Lars Konge; Nessn Azawi Journal: Eur Urol Open Sci Date: 2022-04-01