Martin Frendø1,2, Andreas Frithioff3,4, Lars Konge4, Søren Foghsgaard3, Peter Trier Mikkelsen5, Mads Sølvsten Sørensen3, Per Cayé-Thomasen3, Steven Arild Wuyts Andersen3,4. 1. Department of Otorhinolaryngology, Head & Neck Surgery and Audiology, Rigshospitalet, 9 Blegdamsvej, 2100, Copenhagen Ø, Denmark. martin.frendoe-soerensen.01@regionh.dk. 2. The Simulation Centre, Copenhagen Academy for Medical Education and Simulation (CAMES), The Capital Region of Denmark, Copenhagen, Denmark. martin.frendoe-soerensen.01@regionh.dk. 3. Department of Otorhinolaryngology, Head & Neck Surgery and Audiology, Rigshospitalet, 9 Blegdamsvej, 2100, Copenhagen Ø, Denmark. 4. The Simulation Centre, Copenhagen Academy for Medical Education and Simulation (CAMES), The Capital Region of Denmark, Copenhagen, Denmark. 5. Visual Computing Lab, The Alexandra Institute, Aarhus, Denmark.
Abstract
PURPOSE: To develop and gather validity evidence for a novel tool for assessment of cochlear implant (CI) surgery, including virtual reality CI surgery training. METHODS: Prospective study gathering validity evidence according to Messick's framework. Four experts developed the CI Surgery Assessment Tool (CISAT). A total of 35 true novices (medical students), trained novices (residents) and CI surgeons performed two CI-procedures each in the Visible Ear Simulator, which were rated by three blinded experts. Classical test theory and generalizability theory were used for reliability analysis. RESULTS: The CISAT significantly discriminated between the three groups (p < 0.001). The generalizability coefficient was 0.76 and most of the score variance (53.3%) was attributable to the participant and only 6.8% to the raters. When exploring a standard setting for CI surgery, the contrasting groups method suggested a pass/fail score of 36.0 points (out of 55), but since the trained novices performed above this, we propose using the mean CI surgeon performance score (45.3 points). CONCLUSION: Validity evidence for simulation-based assessment of CI performance supports the CISAT. Together with the standard setting, the CISAT might be used to monitor progress in competency-based training of CI surgery and to determine when the trainee can advance to further training.
PURPOSE: To develop and gather validity evidence for a novel tool for assessment of cochlear implant (CI) surgery, including virtual reality CI surgery training. METHODS: Prospective study gathering validity evidence according to Messick's framework. Four experts developed the CI Surgery Assessment Tool (CISAT). A total of 35 true novices (medical students), trained novices (residents) and CI surgeons performed two CI-procedures each in the Visible Ear Simulator, which were rated by three blinded experts. Classical test theory and generalizability theory were used for reliability analysis. RESULTS: The CISAT significantly discriminated between the three groups (p < 0.001). The generalizability coefficient was 0.76 and most of the score variance (53.3%) was attributable to the participant and only 6.8% to the raters. When exploring a standard setting for CI surgery, the contrasting groups method suggested a pass/fail score of 36.0 points (out of 55), but since the trained novices performed above this, we propose using the mean CI surgeon performance score (45.3 points). CONCLUSION: Validity evidence for simulation-based assessment of CI performance supports the CISAT. Together with the standard setting, the CISAT might be used to monitor progress in competency-based training of CI surgery and to determine when the trainee can advance to further training.
Authors: Sini Sipari; Matti Iso-Mustajärvi; Hanna Matikka; Jyrki Tervaniemi; Arto Koistinen; Antti Aarnisalo; Saku T Sinkkonen; Heikki Löppönen; Aarno Dietz Journal: Otol Neurotol Date: 2018-10 Impact factor: 2.311