PURPOSE: To develop and investigate an evidence-based performance test for assessment of vitreoretinal surgical skills on the EyeSi Surgical Simulator. METHODS: Ten junior residents without any surgical experience, eight senior residents with prior experience in cataract surgery and five vitreoretinal surgeons were included in the study. The test consisted of seven modules and was completed twice by all groups during a single session. Validity evidence was evaluated using Messick's validity framework. Senior residents completed four additional test sessions and were retested 3 months after to assess skill acquisition and retention. RESULTS: Content was aligned with vitreoretinal surgical skills as evaluated by expert surgeons. Response process was ensured through standardized instruction and data collection. The test showed satisfactory internal consistency with Cronbach's α = 0.76 (internal structure) and significant discriminative ability between the residents and the experienced surgeons (relation to other variables). A pass/fail level was determined at 596 using the contrasting groups' method. Consequences of applying this standard resulted in no false positive and no false negative. Senior residents significantly improved their simulator skills over time, reaching a plateau at the fifth iteration and equalling expert performance (p = 0.420). This level of competency was retained during the post-3-month retention testing (p = 0.062). CONCLUSION: We established a performance test with solid evidence for assessment of vitreoretinal surgical skills on the EyeSi Simulator and determined a benchmark criterion that may be used for future implementation of proficiency-based training for novices.
PURPOSE: To develop and investigate an evidence-based performance test for assessment of vitreoretinal surgical skills on the EyeSi Surgical Simulator. METHODS: Ten junior residents without any surgical experience, eight senior residents with prior experience in cataract surgery and five vitreoretinal surgeons were included in the study. The test consisted of seven modules and was completed twice by all groups during a single session. Validity evidence was evaluated using Messick's validity framework. Senior residents completed four additional test sessions and were retested 3 months after to assess skill acquisition and retention. RESULTS: Content was aligned with vitreoretinal surgical skills as evaluated by expert surgeons. Response process was ensured through standardized instruction and data collection. The test showed satisfactory internal consistency with Cronbach's α = 0.76 (internal structure) and significant discriminative ability between the residents and the experienced surgeons (relation to other variables). A pass/fail level was determined at 596 using the contrasting groups' method. Consequences of applying this standard resulted in no false positive and no false negative. Senior residents significantly improved their simulator skills over time, reaching a plateau at the fifth iteration and equalling expert performance (p = 0.420). This level of competency was retained during the post-3-month retention testing (p = 0.062). CONCLUSION: We established a performance test with solid evidence for assessment of vitreoretinal surgical skills on the EyeSi Simulator and determined a benchmark criterion that may be used for future implementation of proficiency-based training for novices.
Authors: Mina Iskander; Titilola Ogunsola; Rithambara Ramachandran; Richard McGowan; Lama A Al-Aswad Journal: Asia Pac J Ophthalmol (Phila) Date: 2021 May-Jun 01