Katrine Jensen1,2, Flemming Bjerrum3, Henrik Jessen Hansen4, René Horsleben Petersen4, Jesper Holst Pedersen4, Lars Konge5. 1. Department of Cardiothoracic Surgery, Sect. 2152, University Hospital of Copenhagen, Rigshospitalet, Copenhagen, Denmark. katrine.jensen@rh.regionh.dk. 2. Copenhagen Academy for Medical Education and Simulation (CAMES), Section 5404, University of Copenhagen and Capital Region, Rigshospitalet, Blegdamsvej 9, 2100 København Ø, Copenhagen, Denmark. katrine.jensen@rh.regionh.dk. 3. JMC Simulation Unit, The Juliane Marie Centre, Section 4704, University Hospital of Copenhagen, Rigshospitalet, Copenhagen, Denmark. 4. Department of Cardiothoracic Surgery, Sect. 2152, University Hospital of Copenhagen, Rigshospitalet, Copenhagen, Denmark. 5. Copenhagen Academy for Medical Education and Simulation (CAMES), Section 5404, University of Copenhagen and Capital Region, Rigshospitalet, Blegdamsvej 9, 2100 København Ø, Copenhagen, Denmark.
Abstract
BACKGROUND: The societies of thoracic surgery are working to incorporate simulation and competency-based assessment into specialty training. One challenge is the development of a simulation-based test, which can be used as an assessment tool. The study objective was to establish validity evidence for a virtual reality simulator test of a video-assisted thoracoscopic surgery (VATS) lobectomy of a right upper lobe. METHODS: Participants with varying experience in VATS lobectomy were included. They were familiarized with a virtual reality simulator (LapSim®) and introduced to the steps of the procedure for a VATS right upper lobe lobectomy. The participants performed two VATS lobectomies on the simulator with a 5-min break between attempts. Nineteen pre-defined simulator metrics were recorded. RESULTS: Fifty-three participants from nine different countries were included. High internal consistency was found for the metrics with Cronbach's alpha coefficient for standardized items of 0.91. Significant test-retest reliability was found for 15 of the metrics (p-values <0.05). Significant correlations between the metrics and the participants VATS lobectomy experience were identified for seven metrics (p-values <0.001), and 10 metrics showed significant differences between novices (0 VATS lobectomies performed) and experienced surgeons (>50 VATS lobectomies performed). A pass/fail level defined as approximately one standard deviation from the mean metric scores for experienced surgeons passed none of the novices (0 % false positives) and failed four of the experienced surgeons (29 % false negatives). CONCLUSION: This study is the first to establish validity evidence for a VATS right upper lobe lobectomy virtual reality simulator test. Several simulator metrics demonstrated significant differences between novices and experienced surgeons and pass/fail criteria for the test were set with acceptable consequences. This test can be used as a first step in assessing thoracic surgery trainees' VATS lobectomy competency.
BACKGROUND: The societies of thoracic surgery are working to incorporate simulation and competency-based assessment into specialty training. One challenge is the development of a simulation-based test, which can be used as an assessment tool. The study objective was to establish validity evidence for a virtual reality simulator test of a video-assisted thoracoscopic surgery (VATS) lobectomy of a right upper lobe. METHODS:Participants with varying experience in VATS lobectomy were included. They were familiarized with a virtual reality simulator (LapSim®) and introduced to the steps of the procedure for a VATS right upper lobe lobectomy. The participants performed two VATS lobectomies on the simulator with a 5-min break between attempts. Nineteen pre-defined simulator metrics were recorded. RESULTS: Fifty-three participants from nine different countries were included. High internal consistency was found for the metrics with Cronbach's alpha coefficient for standardized items of 0.91. Significant test-retest reliability was found for 15 of the metrics (p-values <0.05). Significant correlations between the metrics and the participants VATS lobectomy experience were identified for seven metrics (p-values <0.001), and 10 metrics showed significant differences between novices (0 VATS lobectomies performed) and experienced surgeons (>50 VATS lobectomies performed). A pass/fail level defined as approximately one standard deviation from the mean metric scores for experienced surgeons passed none of the novices (0 % false positives) and failed four of the experienced surgeons (29 % false negatives). CONCLUSION: This study is the first to establish validity evidence for a VATS right upper lobe lobectomy virtual reality simulator test. Several simulator metrics demonstrated significant differences between novices and experienced surgeons and pass/fail criteria for the test were set with acceptable consequences. This test can be used as a first step in assessing thoracic surgery trainees' VATS lobectomy competency.
Authors: M C Vassiliou; G A Ghitulescu; L S Feldman; D Stanbridge; K Leffondré; H H Sigman; G M Fried Journal: Surg Endosc Date: 2006-02-27 Impact factor: 4.584
Authors: Gaetano Rocco; Eveline Internullo; Stephen D Cassivi; Dirk Van Raemdonck; Mark K Ferguson Journal: Thorac Surg Clin Date: 2008-08 Impact factor: 1.750
Authors: Peter Vilmann; Paul Frost Clementsen; Sara Colella; Mette Siemsen; Paul De Leyn; Jean-Marc Dumonceau; Felix J Herth; Alberto Larghi; Enrique Vazquez-Sequeiros; Cesare Hassan; Laurence Crombag; Daniël A Korevaar; Lars Konge; Jouke T Annema Journal: Eur J Cardiothorac Surg Date: 2015-06-01 Impact factor: 4.191
Authors: Pia Iben Pietersen; Kristian Rørbæk Madsen; Ole Graumann; Lars Konge; Bjørn Ulrik Nielsen; Christian Borbjerg Laursen Journal: Crit Ultrasound J Date: 2018-09-03