BACKGROUND: International guidelines suggest that trainees should perform at least 100 flexible bronchoscopies in a supervised setting, but this number is not evidence based. An objective assessment method could provide educational feedback to trainees and help supervisors decide when basic competency is established. No former assessment instrument has been able to distinguish between trainees and experts. OBJECTIVES: The aim of this study was to explore the validity and reliability of a new assessment procedure relating to testing operators across multiple tasks with increasing difficulty using a standardized scoring form. METHODS: The test was administered on a virtual reality bronchoscopy simulator to a total of 42 test subjects (14 senior consultants, 14 trainees and 14 medical students). The inter-rater reliability of the test procedure was explored according to examination of test results from 10 subjects using 3 raters (2 blinded). RESULTS: We found a high inter-rater reliability (Cronbach's α = 0.90). The assessment procedure sufficiently differentiated the performance of the 3 groups (p < 0.001). Assessment of 6 procedures was necessary to secure a generalizability coefficient >0.80. CONCLUSIONS: The new assessment procedure proved both valid and reliable.
BACKGROUND: International guidelines suggest that trainees should perform at least 100 flexible bronchoscopies in a supervised setting, but this number is not evidence based. An objective assessment method could provide educational feedback to trainees and help supervisors decide when basic competency is established. No former assessment instrument has been able to distinguish between trainees and experts. OBJECTIVES: The aim of this study was to explore the validity and reliability of a new assessment procedure relating to testing operators across multiple tasks with increasing difficulty using a standardized scoring form. METHODS: The test was administered on a virtual reality bronchoscopy simulator to a total of 42 test subjects (14 senior consultants, 14 trainees and 14 medical students). The inter-rater reliability of the test procedure was explored according to examination of test results from 10 subjects using 3 raters (2 blinded). RESULTS: We found a high inter-rater reliability (Cronbach's α = 0.90). The assessment procedure sufficiently differentiated the performance of the 3 groups (p < 0.001). Assessment of 6 procedures was necessary to secure a generalizability coefficient >0.80. CONCLUSIONS: The new assessment procedure proved both valid and reliable.
Authors: David Ryan Stather; Alex Chee; Paul Maceachern; Elaine Dumoulin; Christopher A Hergott; Jacob Gelberg; Sandra D Scott; Sylvia De Guzman; Alain Tremblay Journal: Can Respir J Date: 2013-05-28 Impact factor: 2.409
Authors: Morten la Cour; Ann Sofia Skou Thomsen; Mark Alberti; Lars Konge Journal: Graefes Arch Clin Exp Ophthalmol Date: 2019-01-15 Impact factor: 3.117
Authors: Lais Meirelles Nicoliello Vieira; Paulo Augusto Moreira Camargos; Cássio da Cunha Ibiapina Journal: J Bras Pneumol Date: 2022-04-29 Impact factor: 2.624