BACKGROUND: We have implemented Smart Endoscopic Surgery (SES), a surgical system that uses artificial intelligence (AI) to detect the anatomical landmarks that expert surgeons base on to perform certain surgical maneuvers. No report has verified the use of AI-based support systems for surgery in clinical practice, and no evaluation method has been established. To evaluate the detection performance of SES, we have developed and established a new evaluation method by conducting a clinical feasibility trial. METHODS: A single-center prospective clinical feasibility trial was conducted on 10 cases of LC performed at Oita University hospital. Subsequently, an external evaluation committee (EEC) evaluated the AI detection accuracy for each landmark using five-grade rubric evaluation and DICE coefficient. We defined LM-CBD as the expert surgeon's "judge" of the cystic bile duct in endoscopic images. RESULTS: The average detection accuracy on the rubric by the EEC was 4.2 ± 0.8 for the LM-CBD. The DICE coefficient between the AI detection area of the LM-CBD and the EEC members' evaluation was similar to the mean value of the DICE coefficient between the EEC members. The DICE coefficient was high score for the case that was highly evaluated by the EEC on a five-grade scale. CONCLUSION: This is the first feasible clinical trial of an AI system designed for intraoperative use and to evaluate the AI system using an EEC. In the future, this concept of evaluation for the AI system would contribute to the development of new AI navigation systems for surgery.
BACKGROUND: We have implemented Smart Endoscopic Surgery (SES), a surgical system that uses artificial intelligence (AI) to detect the anatomical landmarks that expert surgeons base on to perform certain surgical maneuvers. No report has verified the use of AI-based support systems for surgery in clinical practice, and no evaluation method has been established. To evaluate the detection performance of SES, we have developed and established a new evaluation method by conducting a clinical feasibility trial. METHODS: A single-center prospective clinical feasibility trial was conducted on 10 cases of LC performed at Oita University hospital. Subsequently, an external evaluation committee (EEC) evaluated the AI detection accuracy for each landmark using five-grade rubric evaluation and DICE coefficient. We defined LM-CBD as the expert surgeon's "judge" of the cystic bile duct in endoscopic images. RESULTS: The average detection accuracy on the rubric by the EEC was 4.2 ± 0.8 for the LM-CBD. The DICE coefficient between the AI detection area of the LM-CBD and the EEC members' evaluation was similar to the mean value of the DICE coefficient between the EEC members. The DICE coefficient was high score for the case that was highly evaluated by the EEC on a five-grade scale. CONCLUSION: This is the first feasible clinical trial of an AI system designed for intraoperative use and to evaluate the AI system using an EEC. In the future, this concept of evaluation for the AI system would contribute to the development of new AI navigation systems for surgery.
Authors: Pietro Mascagni; Armine Vardazaryan; Deepak Alapatt; Takeshi Urade; Taha Emre; Claudio Fiorillo; Patrick Pessaux; Didier Mutter; Jacques Marescaux; Guido Costamagna; Bernard Dallemagne; Nicolas Padoy Journal: Ann Surg Date: 2020-11-16 Impact factor: 13.787