Jian Zhou1,2,3,4, Hu Liao1,2,3,4, Zhangyu Lin2, Mengyuan Lyu2,5, Weelic Chong6, Nan Chen1,2,3,4, Yang Hai1,2,6, Zihuai Wang1,2,3,4, Tianjian Lu1,2,3,4, Lunxu Liu7,8,9,10. 1. Department of Thoracic Surgery, West China Hospital, Sichuan University, No. 37, Guoxue Alley, Chengdu, Sichuan, 610041, China. 2. West China School of Medicine, Sichuan University, Chengdu, 610041, China. 3. Chest Oncology Institute, West China Hospital, Sichuan University, Chengdu, 610041, China. 4. Western China Collaborative Innovation Center for Early Diagnosis and Multidisciplinary Therapy of Lung Cancer, Sichuan University, Chengdu, 610041, China. 5. Department of Laboratory Medicine, West China Hospital, Sichuan University, Chengdu, 610041, China. 6. Sidney Kimmel School of Medicine, Thomas Jefferson University, Philadelphia, PA, 19107, USA. 7. Department of Thoracic Surgery, West China Hospital, Sichuan University, No. 37, Guoxue Alley, Chengdu, Sichuan, 610041, China. lunxu_liu@aliyun.com. 8. West China School of Medicine, Sichuan University, Chengdu, 610041, China. lunxu_liu@aliyun.com. 9. Chest Oncology Institute, West China Hospital, Sichuan University, Chengdu, 610041, China. lunxu_liu@aliyun.com. 10. Western China Collaborative Innovation Center for Early Diagnosis and Multidisciplinary Therapy of Lung Cancer, Sichuan University, Chengdu, 610041, China. lunxu_liu@aliyun.com.
Abstract
PURPOSE: We report a new thoracoscopic surgical skill training and assessment system with automatic scoring techniques, the Huaxi Intelligent Thoracoscopic Skill Training and Assessment (HITSTA) system. We also evaluated the discriminative ability of this system compared to our conventional scoring method at our institution. METHODS: We retrospectively collected training data of thoracic board-certified thoracic surgeons at West China Hospital, Sichuan University from January 1, 2018 to January 1, 2019. Surgeons were assessed by HITSTA system and human examiners simultaneously. Total scores were summed from 3 tasks (grasping with delivery, pattern cutting, and suture with knot). Bland-Altman analysis was used to test agreement of scores made by HITSTA system (automatic scoring) and human examiners (manual scoring). Differentiation ability was also compared between the two scoring methods. RESULTS: Thirty-nine surgeons were recruited. Scores made by HITSTA system and human examiners were not consistent. For suture with knot, automatic scoring method could detect the score differences between different training status (trained: 26.92 ± 12.04, untrained: 19.85 ± 11.12; p = 0.026) and training duration (< 10 h: 20.67 ± 15.23, ≥ 10 h: 31.92 ± 5.56; p = 0.003). For total scores, automatic scoring approach could discriminate between different training status (trained: 71.90 ± 12.63; untrained: 61.41 ± 13.87; p = 0.016) and training duration (< 10 h: 65.23 ± 15.31; ≥ 10 h 77.23 ± 6.94; p = 0.046). CONCLUSION: HITSTA system could discriminate the different levels of thoracoscopic surgical skills better than the traditional manual scoring method. Larger prospective studies are warranted to validate the differentiation ability of HITSTA system.
PURPOSE: We report a new thoracoscopic surgical skill training and assessment system with automatic scoring techniques, the Huaxi Intelligent Thoracoscopic Skill Training and Assessment (HITSTA) system. We also evaluated the discriminative ability of this system compared to our conventional scoring method at our institution. METHODS: We retrospectively collected training data of thoracic board-certified thoracic surgeons at West China Hospital, Sichuan University from January 1, 2018 to January 1, 2019. Surgeons were assessed by HITSTA system and human examiners simultaneously. Total scores were summed from 3 tasks (grasping with delivery, pattern cutting, and suture with knot). Bland-Altman analysis was used to test agreement of scores made by HITSTA system (automatic scoring) and human examiners (manual scoring). Differentiation ability was also compared between the two scoring methods. RESULTS: Thirty-nine surgeons were recruited. Scores made by HITSTA system and human examiners were not consistent. For suture with knot, automatic scoring method could detect the score differences between different training status (trained: 26.92 ± 12.04, untrained: 19.85 ± 11.12; p = 0.026) and training duration (< 10 h: 20.67 ± 15.23, ≥ 10 h: 31.92 ± 5.56; p = 0.003). For total scores, automatic scoring approach could discriminate between different training status (trained: 71.90 ± 12.63; untrained: 61.41 ± 13.87; p = 0.016) and training duration (< 10 h: 65.23 ± 15.31; ≥ 10 h 77.23 ± 6.94; p = 0.046). CONCLUSION: HITSTA system could discriminate the different levels of thoracoscopic surgical skills better than the traditional manual scoring method. Larger prospective studies are warranted to validate the differentiation ability of HITSTA system.
Authors: Rafael Alvarez; Aaron J Bonham; Colleen M Buda; Arthur M Carlin; Amir A Ghaferi; Oliver A Varban Journal: Ann Surg Date: 2019-12 Impact factor: 12.969