Mona W Schmidt1, Karl-Friedrich Kowalewski1, Marc L Schmidt2, Erica Wennberg1, Carly R Garrow1, Sang Paik1, Laura Benner3, Marlies P Schijven4, Beat P Müller-Stich1, Felix Nickel5. 1. Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Im Neuenheimer Feld 110, 69120, Heidelberg, Germany. 2. , Karlsruhe, Germany. 3. Department of Medical Biometry and Informatics, University of Heidelberg, Im Neuenheimer Feld 130.3, 69120, Heidelberg, Germany. 4. Deparment of Surgery, Amsterdam Gastroenterology and Metabolism, Amsterdam UMC, University of Amsterdam, PO Box 22660, 1100 DD, Amsterdam, The Netherlands. 5. Department of General, Visceral, and Transplantation Surgery, Heidelberg University Hospital, Im Neuenheimer Feld 110, 69120, Heidelberg, Germany. felix.nickel@med.uni-heidelberg.de.
Abstract
INTRODUCTION: Virtual reality (VR-)trainers are well integrated in laparoscopic surgical training. However, objective feedback is often provided in the form of single parameters, e.g., time or number of movements, making comparisons and evaluation of trainees' overall performance difficult. Therefore, a new standard for reporting outcome data is highly needed. The aim of this study was to create a weighted, expert-based composite score, to offer simple and direct evaluation of laparoscopic performance on common VR-trainers. MATERIALS AND METHODS: An integrated analytic hierarchy process-Delphi survey was conducted with 14 international experts to achieve a consensus on the importance of different skill categories and parameters in evaluation of laparoscopic performance. A scoring algorithm was established to allow comparability between tasks and VR-trainers. A weighted composite score was calculated for basic skills tasks and peg transfer on the LapMentor™ II and III and validated for both VR-trainers. RESULTS: Five major skill categories (time, efficiency, safety, dexterity, and outcome) were identified and weighted in two Delphi rounds. Safety, with a weight of 67%, was determined the most important category, followed by efficiency with 17%. The LapMentor™-specific score was validated using 15 (14) novices and 9 experts; the score was able to differentiate between both groups for basic skills tasks and peg transfer (LapMentor™ II: Exp: 86.5 ± 12.7, Nov. 52.8 ± 18.3; p < 0.001; LapMentor™ III: Exp: 80.8 ± 7.1, Nov: 50.6 ± 16.9; p < 0.001). CONCLUSION: An effective and simple performance measurement was established to propose a new standard in analyzing and reporting VR outcome data-the Heidelberg virtual reality (VR) score. The scoring algorithm and the consensus results on the importance of different skill aspects in laparoscopic surgery are universally applicable and can be transferred to any simulator or task. By incorporating specific expert baseline data for the respective task, comparability between tasks, studies, and simulators can be achieved.
INTRODUCTION: Virtual reality (VR-)trainers are well integrated in laparoscopic surgical training. However, objective feedback is often provided in the form of single parameters, e.g., time or number of movements, making comparisons and evaluation of trainees' overall performance difficult. Therefore, a new standard for reporting outcome data is highly needed. The aim of this study was to create a weighted, expert-based composite score, to offer simple and direct evaluation of laparoscopic performance on common VR-trainers. MATERIALS AND METHODS: An integrated analytic hierarchy process-Delphi survey was conducted with 14 international experts to achieve a consensus on the importance of different skill categories and parameters in evaluation of laparoscopic performance. A scoring algorithm was established to allow comparability between tasks and VR-trainers. A weighted composite score was calculated for basic skills tasks and peg transfer on the LapMentor™ II and III and validated for both VR-trainers. RESULTS: Five major skill categories (time, efficiency, safety, dexterity, and outcome) were identified and weighted in two Delphi rounds. Safety, with a weight of 67%, was determined the most important category, followed by efficiency with 17%. The LapMentor™-specific score was validated using 15 (14) novices and 9 experts; the score was able to differentiate between both groups for basic skills tasks and peg transfer (LapMentor™ II: Exp: 86.5 ± 12.7, Nov. 52.8 ± 18.3; p < 0.001; LapMentor™ III: Exp: 80.8 ± 7.1, Nov: 50.6 ± 16.9; p < 0.001). CONCLUSION: An effective and simple performance measurement was established to propose a new standard in analyzing and reporting VR outcome data-the Heidelberg virtual reality (VR) score. The scoring algorithm and the consensus results on the importance of different skill aspects in laparoscopic surgery are universally applicable and can be transferred to any simulator or task. By incorporating specific expert baseline data for the respective task, comparability between tasks, studies, and simulators can be achieved.
Authors: Mona W Schmidt; Karl-Friedrich Kowalewski; Sarah M Trent; Laura Benner; Beat P Müller-Stich; Felix Nickel Journal: Surg Endosc Date: 2019-05-28 Impact factor: 4.584
Authors: Mona W Schmidt; Caelan M Haney; Karl-Friedrich Kowalewski; Vasile V Bintintan; Mohammed Abu Hilal; Alberto Arezzo; Marcus Bahra; Marc G Besselink; Matthias Biebl; Luigi Boni; Michele Diana; Jan H Egberts; Lars Fischer; Nader Francis; Daniel A Hashimoto; Daniel Perez; Marlies Schijven; Moritz Schmelzle; Marek Soltes; Lee Swanstrom; Thilo Welsch; Beat P Müller-Stich; Felix Nickel Journal: Surg Endosc Date: 2021-11-09 Impact factor: 3.453
Authors: M W Schmidt; K F Köppinger; C Fan; K-F Kowalewski; L P Schmidt; J Vey; T Proctor; P Probst; V V Bintintan; B-P Müller-Stich; F Nickel Journal: BJS Open Date: 2021-03-05
Authors: C Wild; F Lang; A S Gerhäuser; M W Schmidt; K F Kowalewski; J Petersen; H G Kenngott; B P Müller-Stich; F Nickel Journal: Surg Endosc Date: 2022-03-09 Impact factor: 3.453