Sutuke Yibulayimu1, Yuneng Wang2, Yanzhen Liu1, Zhibin Sun1, Yu Wang3, Haiyue Jiang2, Facheng Li2. 1. School of Biomedical Science and Medical Engineering, Beihang University, Beijing, China. 2. Center of Body Contouring and Liposuction Center, Plastic Surgery Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China. 3. School of Biomedical Science and Medical Engineering, Beihang University, Beijing, China. wangyu@buaa.edu.cn.
Abstract
PURPOSE: Surgical skill assessment has received growing interest in surgery training and quality control due to its essential role in competency assessment and trainee feedback. However, the current assessment methods rarely provide corresponding feedback guidance while giving ability evaluation. We aim to validate an explainable surgical skill assessment method that automatically evaluates the trainee performance of liposuction surgery and provides visual postoperative and real-time feedback. METHODS: In this study, machine learning using a model-agnostic interpretable method based on stroke segmentation was introduced to objectively evaluate surgical skills. We evaluated the method on liposuction surgery datasets that consisted of motion and force data for classification tasks. RESULTS: Our classifier achieved optimistic accuracy in clinical and imitation liposuction surgery models, ranging from 89 to 94%. With the help of SHapley Additive exPlanations (SHAP), we deeply explore the potential rules of liposuction operation between surgeons with variant experiences and provide real-time feedback based on the ML model to surgeons with undesirable skills. CONCLUSION: Our results demonstrate the strong abilities of explainable machine learning methods in objective surgical skill assessment. We believe that the machine learning model based on interpretive methods proposed in this article can improve the evaluation and training of liposuction surgery and provide objective assessment and training guidance for other surgeries.
PURPOSE: Surgical skill assessment has received growing interest in surgery training and quality control due to its essential role in competency assessment and trainee feedback. However, the current assessment methods rarely provide corresponding feedback guidance while giving ability evaluation. We aim to validate an explainable surgical skill assessment method that automatically evaluates the trainee performance of liposuction surgery and provides visual postoperative and real-time feedback. METHODS: In this study, machine learning using a model-agnostic interpretable method based on stroke segmentation was introduced to objectively evaluate surgical skills. We evaluated the method on liposuction surgery datasets that consisted of motion and force data for classification tasks. RESULTS: Our classifier achieved optimistic accuracy in clinical and imitation liposuction surgery models, ranging from 89 to 94%. With the help of SHapley Additive exPlanations (SHAP), we deeply explore the potential rules of liposuction operation between surgeons with variant experiences and provide real-time feedback based on the ML model to surgeons with undesirable skills. CONCLUSION: Our results demonstrate the strong abilities of explainable machine learning methods in objective surgical skill assessment. We believe that the machine learning model based on interpretive methods proposed in this article can improve the evaluation and training of liposuction surgery and provide objective assessment and training guidance for other surgeries.
Authors: John D Birkmeyer; Jonathan F Finks; Amanda O'Reilly; Mary Oerline; Arthur M Carlin; Andre R Nunn; Justin Dimick; Mousumi Banerjee; Nancy J O Birkmeyer Journal: N Engl J Med Date: 2013-10-10 Impact factor: 91.245
Authors: Narges Ahmidi; Piyush Poddar; Jonathan D Jones; S Swaroop Vedula; Lisa Ishii; Gregory D Hager; Masaru Ishii Journal: Int J Comput Assist Radiol Surg Date: 2015-04-17 Impact factor: 2.924
Authors: Pablo Castillo-Segura; Carmen Fernández-Panadero; Carlos Alario-Hoyos; Pedro J Muñoz-Merino; Carlos Delgado Kloos Journal: Artif Intell Med Date: 2021-01-05 Impact factor: 5.326
Authors: Jeremy D Brown; Conor E O Brien; Sarah C Leung; Kristoffel R Dumon; David I Lee; Katherine J Kuchenbecker Journal: IEEE Trans Biomed Eng Date: 2016-12-02 Impact factor: 4.538