Chaitanya S Kulkarni1, Shiyu Deng1, Tianzi Wang1, Jacob Hartman-Kenzler2, Laura E Barnes3, Sarah Henrickson Parker2, Shawn D Safford4, Nathan Lau5. 1. Grado Department of Industrial and Systems Engineering, Virginia Tech, 250 Durham Hall (0118), 1145 Perry Street, Blacksburg, VA, 24061, USA. 2. Virginia Tech Carilion Clinic School of Medicine, Roanoke, VA, USA. 3. Environmental and Systems Engineering, University of Virginia, Charlottesville, VA, USA. 4. Division of Pediatric General and Thoracic Surgery, UPMC Children's Hospital of Pittsburgh, Harrisburg, PA, USA. 5. Grado Department of Industrial and Systems Engineering, Virginia Tech, 250 Durham Hall (0118), 1145 Perry Street, Blacksburg, VA, 24061, USA. nathan.lau@vt.edu.
Abstract
INTRODUCTION: In laparoscopic surgery, looking in the target areas is an indicator of proficiency. However, gaze behaviors revealing feedforward control (i.e., looking ahead) and their importance have been under-investigated in surgery. This study aims to establish the sensitivity and relative importance of different scene-dependent gaze and motion metrics for estimating trainee proficiency levels in surgical skills. METHODS: Medical students performed the Fundamentals of Laparoscopic Surgery peg transfer task while recording their gaze on the monitor and tool activities inside the trainer box. Using computer vision and fixation algorithms, five scene-dependent gaze metrics and one tool speed metric were computed for 499 practice trials. Cluster analysis on the six metrics was used to group the trials into different clusters/proficiency levels, and ANOVAs were conducted to test differences between proficiency levels. A Random Forest model was trained to study metric importance at predicting proficiency levels. RESULTS: Three clusters were identified, corresponding to three proficiency levels. The correspondence between the clusters and proficiency levels was confirmed by differences between completion times (F2,488 = 38.94, p < .001). Further, ANOVAs revealed significant differences between the three levels for all six metrics. The Random Forest model predicted proficiency level with 99% out-of-bag accuracy and revealed that scene-dependent gaze metrics reflecting feedforward behaviors were more important for prediction than the ones reflecting feedback behaviors. CONCLUSION: Scene-dependent gaze metrics revealed skill levels of trainees more precisely than between experts and novices as suggested in the literature. Further, feedforward gaze metrics appeared to be more important than feedback ones at predicting proficiency.
INTRODUCTION: In laparoscopic surgery, looking in the target areas is an indicator of proficiency. However, gaze behaviors revealing feedforward control (i.e., looking ahead) and their importance have been under-investigated in surgery. This study aims to establish the sensitivity and relative importance of different scene-dependent gaze and motion metrics for estimating trainee proficiency levels in surgical skills. METHODS: Medical students performed the Fundamentals of Laparoscopic Surgery peg transfer task while recording their gaze on the monitor and tool activities inside the trainer box. Using computer vision and fixation algorithms, five scene-dependent gaze metrics and one tool speed metric were computed for 499 practice trials. Cluster analysis on the six metrics was used to group the trials into different clusters/proficiency levels, and ANOVAs were conducted to test differences between proficiency levels. A Random Forest model was trained to study metric importance at predicting proficiency levels. RESULTS: Three clusters were identified, corresponding to three proficiency levels. The correspondence between the clusters and proficiency levels was confirmed by differences between completion times (F2,488 = 38.94, p < .001). Further, ANOVAs revealed significant differences between the three levels for all six metrics. The Random Forest model predicted proficiency level with 99% out-of-bag accuracy and revealed that scene-dependent gaze metrics reflecting feedforward behaviors were more important for prediction than the ones reflecting feedback behaviors. CONCLUSION: Scene-dependent gaze metrics revealed skill levels of trainees more precisely than between experts and novices as suggested in the literature. Further, feedforward gaze metrics appeared to be more important than feedback ones at predicting proficiency.
Authors: J B Pagador; F M Sánchez-Margallo; L F Sánchez-Peralta; J A Sánchez-Margallo; J L Moyano-Cuevas; S Enciso-Sanz; J Usón-Gargallo; J Moreno Journal: Int J Comput Assist Radiol Surg Date: 2011-08-14 Impact factor: 2.924
Authors: Ignacio Oropesa; Patricia Sánchez-González; Magdalena K Chmarra; Pablo Lamata; Alvaro Fernández; Juan A Sánchez-Margallo; Frank Willem Jansen; Jenny Dankelman; Francisco M Sánchez-Margallo; Enrique J Gómez Journal: Surg Endosc Date: 2012-10-06 Impact factor: 4.584
Authors: Magdalena K Chmarra; Stefan Klein; Joost C F de Winter; Frank-Willem Jansen; Jenny Dankelman Journal: Surg Endosc Date: 2009-11-14 Impact factor: 4.584