Ernest D Gomez1,2, Rajesh Aggarwal3,4, William McMahan5, Karlin Bark5, Katherine J Kuchenbecker6. 1. Department of Otorhinolaryngology - Head and Neck Surgery, Hospital of the University of Pennsylvania, Philadelphia, PA, USA. ernest.gomez@uphs.upenn.edu. 2. Department of Mechanical Engineering and Applied Mechanics, University of Pennsylvania, 224 Towne Building, 220 South 33rd Street, Philadelphia, PA, 19104, USA. ernest.gomez@uphs.upenn.edu. 3. Department of Surgery, McGill University Health Centre, Philadelphia, PA, USA. 4. Arnold and Blema Steinberg Medical Simulation Centre, Philadelphia, PA, USA. 5. Department of Mechanical Engineering and Applied Mechanics, University of Pennsylvania, 224 Towne Building, 220 South 33rd Street, Philadelphia, PA, 19104, USA. 6. Department of Mechanical Engineering and Applied Mechanics, University of Pennsylvania, 224 Towne Building, 220 South 33rd Street, Philadelphia, PA, 19104, USA. kuchenbe@seas.upenn.edu.
Abstract
BACKGROUND: Surgical skill evaluation ordinarily requires tedious video review and survey completion, while new automatic approaches focus on evaluating the quality of the surgeon's movements in free space. Robotic surgical instrument vibrations are simple to measure and physically correspond to how roughly instruments are handled, but they have yet to be studied as a measure of technical surgical skill. METHODS: Thirteen surgeons used a robotic surgery system (da Vinci S by Intuitive Surgical) to perform four trials each of peg transfer (PT), needle pass (NP), and intracorporeal suturing (IS). Completion time, instrument vibrations, and applied forces were measured for each trial; root mean square (RMS) and total sum of squares (TSS) were calculated from both the vibration and force recordings. Four experienced surgeons blindly assessed the task videos using a Global Rating Scale (GRS), and skill metrics were compared between the eight novices and five experienced participants. Stepwise regression was performed to predict GRS score from objective skill metrics. The concurrent validity of each metric was evaluated using receiver operating characteristic (ROC) analysis. RESULTS: The GRS demonstrated excellent internal consistency (Cronbach's α = 0.91) and strong inter-rater reliability (ICC = 0.84). Compared to novices, experienced surgeons earned higher GRS scores and performed tasks with lower vibration magnitudes, lower forces, and shorter completion times in 15 of 18 task-metric combinations (p values ranging from 0.042 to <0.001). ROC analysis demonstrated that including vibration and force magnitudes along with completion time in skill prediction models improves the objective classification of subjects as novice or experienced for all tasks studied (PT: 90% sensitivity, 75% specificity; NP: 85% sensitivity, 84% specificity; suturing: 100% sensitivity, 100% specificity). CONCLUSIONS: RMS and TSS instrument vibrations are novel construct-valid measures of robotic surgical skill that enable the development of objective skill assessment models comparable to observer-based ratings.
BACKGROUND: Surgical skill evaluation ordinarily requires tedious video review and survey completion, while new automatic approaches focus on evaluating the quality of the surgeon's movements in free space. Robotic surgical instrument vibrations are simple to measure and physically correspond to how roughly instruments are handled, but they have yet to be studied as a measure of technical surgical skill. METHODS: Thirteen surgeons used a robotic surgery system (da Vinci S by Intuitive Surgical) to perform four trials each of peg transfer (PT), needle pass (NP), and intracorporeal suturing (IS). Completion time, instrument vibrations, and applied forces were measured for each trial; root mean square (RMS) and total sum of squares (TSS) were calculated from both the vibration and force recordings. Four experienced surgeons blindly assessed the task videos using a Global Rating Scale (GRS), and skill metrics were compared between the eight novices and five experienced participants. Stepwise regression was performed to predict GRS score from objective skill metrics. The concurrent validity of each metric was evaluated using receiver operating characteristic (ROC) analysis. RESULTS: The GRS demonstrated excellent internal consistency (Cronbach's α = 0.91) and strong inter-rater reliability (ICC = 0.84). Compared to novices, experienced surgeons earned higher GRS scores and performed tasks with lower vibration magnitudes, lower forces, and shorter completion times in 15 of 18 task-metric combinations (p values ranging from 0.042 to <0.001). ROC analysis demonstrated that including vibration and force magnitudes along with completion time in skill prediction models improves the objective classification of subjects as novice or experienced for all tasks studied (PT: 90% sensitivity, 75% specificity; NP: 85% sensitivity, 84% specificity; suturing: 100% sensitivity, 100% specificity). CONCLUSIONS: RMS and TSS instrument vibrations are novel construct-valid measures of robotic surgical skill that enable the development of objective skill assessment models comparable to observer-based ratings.
Authors: Ravikiran B Singapogu; Dane E Smith; Lindsay O Long; Timothy C Burg; Christopher C Pagano; Karen J L Burg Journal: J Surg Educ Date: 2012-09-05 Impact factor: 2.891
Authors: William McMahan; Ernest D Gomez; Liting Chen; Karlin Bark; John C Nappo; Eza I Koch; David I Lee; Kristoffel R Dumon; Noel N Williams; Katherine J Kuchenbecker Journal: J Robot Surg Date: 2013-04-05
Authors: W McMahan; J Gewirtz; D Standish; P Martin; J A Kunkel; M Lilavois; A Wedmid; D I Lee; K J Kuchenbecker Journal: IEEE Trans Haptics Date: 2011 May-Jun Impact factor: 2.487
Authors: Rodney L Dockter; Thomas S Lendvay; Robert M Sweet; Timothy M Kowalewski Journal: Int J Comput Assist Radiol Surg Date: 2017-05-18 Impact factor: 2.924
Authors: I-Hsuan Alan Chen; Ahmed Ghazi; Ashwin Sridhar; Danail Stoyanov; Mark Slack; John D Kelly; Justin W Collins Journal: World J Urol Date: 2020-11-06 Impact factor: 4.226