BACKGROUND: Computer-based surgical simulators capture a multitude of metrics based on different aspects of performance, such as speed, accuracy, and movement efficiency. However, without rigorous assessment, it may be unclear whether all, some, or none of these metrics actually reflect technical skill, which can compromise educational efforts on these simulators. We assessed the construct validity of individual performance metrics on the LapVR simulator (Immersion Medical, San Jose, CA, USA) and used these data to create task-specific summary metrics. METHODS: Medical students with no prior laparoscopic experience (novices, N = 12), junior surgical residents with some laparoscopic experience (intermediates, N = 12), and experienced surgeons (experts, N = 11) all completed three repetitions of four LapVR simulator tasks. The tasks included three basic skills (peg transfer, cutting, clipping) and one procedural skill (adhesiolysis). RESULTS: We selected 36 individual metrics on the four tasks that assessed six different aspects of performance, including speed, motion path length, respect for tissue, accuracy, task-specific errors, and successful task completion. Four of seven individual metrics assessed for peg transfer, six of ten metrics for cutting, four of nine metrics for clipping, and three of ten metrics for adhesiolysis discriminated between experience levels. Time and motion path length were significant on all four tasks. We used the validated individual metrics to create summary equations for each task, which successfully distinguished between the different experience levels. CONCLUSION: Educators should maintain some skepticism when reviewing the plethora of metrics captured by computer-based simulators, as some but not all are valid. We showed the construct validity of a limited number of individual metrics and developed summary metrics for the LapVR. The summary metrics provide a succinct way of assessing skill with a single metric for each task, but require further validation.
BACKGROUND: Computer-based surgical simulators capture a multitude of metrics based on different aspects of performance, such as speed, accuracy, and movement efficiency. However, without rigorous assessment, it may be unclear whether all, some, or none of these metrics actually reflect technical skill, which can compromise educational efforts on these simulators. We assessed the construct validity of individual performance metrics on the LapVR simulator (Immersion Medical, San Jose, CA, USA) and used these data to create task-specific summary metrics. METHODS: Medical students with no prior laparoscopic experience (novices, N = 12), junior surgical residents with some laparoscopic experience (intermediates, N = 12), and experienced surgeons (experts, N = 11) all completed three repetitions of four LapVR simulator tasks. The tasks included three basic skills (peg transfer, cutting, clipping) and one procedural skill (adhesiolysis). RESULTS: We selected 36 individual metrics on the four tasks that assessed six different aspects of performance, including speed, motion path length, respect for tissue, accuracy, task-specific errors, and successful task completion. Four of seven individual metrics assessed for peg transfer, six of ten metrics for cutting, four of nine metrics for clipping, and three of ten metrics for adhesiolysis discriminated between experience levels. Time and motion path length were significant on all four tasks. We used the validated individual metrics to create summary equations for each task, which successfully distinguished between the different experience levels. CONCLUSION: Educators should maintain some skepticism when reviewing the plethora of metrics captured by computer-based simulators, as some but not all are valid. We showed the construct validity of a limited number of individual metrics and developed summary metrics for the LapVR. The summary metrics provide a succinct way of assessing skill with a single metric for each task, but require further validation.
Authors: F J Carter; M P Schijven; R Aggarwal; T Grantcharov; N K Francis; G B Hanna; J J Jakimowicz Journal: Surg Endosc Date: 2005-10-26 Impact factor: 4.584
Authors: Diana L Diesen; Loretta Erhunmwunsee; Kyla M Bennett; Kfir Ben-David; Basil Yurcisin; Eugene P Ceppa; Philip A Omotosho; Alexander Perez; Aurora Pryor Journal: J Surg Educ Date: 2011-05-04 Impact factor: 2.891
Authors: Jason Park; Helen MacRae; Laura J Musselman; Peter Rossos; Stanley J Hamstra; Stephen Wolman; Richard K Reznick Journal: Am J Surg Date: 2007-08 Impact factor: 2.565
Authors: Jaisa Olasky; Ganesh Sankaranarayanan; Neal E Seymour; J Harvey Magee; Andinet Enquobahrie; Ming C Lin; Rajesh Aggarwal; L Michael Brunt; Steven D Schwaitzberg; Caroline G L Cao; Suvranu De; Daniel B Jones Journal: Surg Innov Date: 2015-04-29 Impact factor: 2.058
Authors: Rickul Varshney; Saul Frenkiel; Lily H P Nguyen; Meredith Young; Rolando Del Maestro; Anthony Zeitouni; Elias Saad; W Robert J Funnell; Marc A Tewfik Journal: J Otolaryngol Head Neck Surg Date: 2014-10-24