Carly R Garrow1, Karl-Friedrich Kowalewski1,2, Linhong Li1, Martin Wagner1, Mona W Schmidt1, Sandy Engelhardt3, Daniel A Hashimoto4, Hannes G Kenngott1, Sebastian Bodenstedt5,6, Stefanie Speidel5,6, Beat P Müller-Stich1, Felix Nickel1. 1. Department of General, Visceral, and Transplantation Surgery, University Hospital of Heidelberg, Heidelberg, Germany. 2. Department of Urology, University Medical Center Mannheim, Heidelberg University, Mannheim, Germany. 3. Department of Computer Science, Mannheim University of Applied Sciences, Mannheim, Germany. 4. Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts. 5. Division of Translational Surgical Oncology, National Center for Tumor Diseases (NCT), Dresden, Germany. 6. Centre for Tactile Internet with Human-in-the-Loop (CeTI), TU Dresden, Dresden, Germany.
Abstract
OBJECTIVE: To provide an overview of ML models and data streams utilized for automated surgical phase recognition. BACKGROUND: Phase recognition identifies different steps and phases of an operation. ML is an evolving technology that allows analysis and interpretation of huge data sets. Automation of phase recognition based on data inputs is essential for optimization of workflow, surgical training, intraoperative assistance, patient safety, and efficiency. METHODS: A systematic review was performed according to the Cochrane recommendations and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. PubMed, Web of Science, IEEExplore, GoogleScholar, and CiteSeerX were searched. Literature describing phase recognition based on ML models and the capture of intraoperative signals during general surgery procedures was included. RESULTS: A total of 2254 titles/abstracts were screened, and 35 full-texts were included. Most commonly used ML models were Hidden Markov Models and Artificial Neural Networks with a trend towards higher complexity over time. Most frequently used data types were feature learning from surgical videos and manual annotation of instrument use. Laparoscopic cholecystectomy was used most commonly, often achieving accuracy rates over 90%, though there was no consistent standardization of defined phases. CONCLUSIONS: ML for surgical phase recognition can be performed with high accuracy, depending on the model, data type, and complexity of surgery. Different intraoperative data inputs such as video and instrument type can successfully be used. Most ML models still require significant amounts of manual expert annotations for training. The ML models may drive surgical workflow towards standardization, efficiency, and objectiveness to improve patient outcome in the future. REGISTRATION PROSPERO: CRD42018108907.
OBJECTIVE: To provide an overview of ML models and data streams utilized for automated surgical phase recognition. BACKGROUND: Phase recognition identifies different steps and phases of an operation. ML is an evolving technology that allows analysis and interpretation of huge data sets. Automation of phase recognition based on data inputs is essential for optimization of workflow, surgical training, intraoperative assistance, patient safety, and efficiency. METHODS: A systematic review was performed according to the Cochrane recommendations and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. PubMed, Web of Science, IEEExplore, GoogleScholar, and CiteSeerX were searched. Literature describing phase recognition based on ML models and the capture of intraoperative signals during general surgery procedures was included. RESULTS: A total of 2254 titles/abstracts were screened, and 35 full-texts were included. Most commonly used ML models were Hidden Markov Models and Artificial Neural Networks with a trend towards higher complexity over time. Most frequently used data types were feature learning from surgical videos and manual annotation of instrument use. Laparoscopic cholecystectomy was used most commonly, often achieving accuracy rates over 90%, though there was no consistent standardization of defined phases. CONCLUSIONS: ML for surgical phase recognition can be performed with high accuracy, depending on the model, data type, and complexity of surgery. Different intraoperative data inputs such as video and instrument type can successfully be used. Most ML models still require significant amounts of manual expert annotations for training. The ML models may drive surgical workflow towards standardization, efficiency, and objectiveness to improve patient outcome in the future. REGISTRATION PROSPERO: CRD42018108907.
Authors: Dolores Müller; Raphael Stier; Jennifer Straatman; Benjamin Babic; Lars Schiffmann; Jennifer Eckhoff; Thomas Schmidt; Christiane Bruns; Hans F Fuchs Journal: Chirurgie (Heidelb) Date: 2022-06-03
Authors: Martin Wagner; Johanna M Brandenburg; Sebastian Bodenstedt; André Schulze; Alexander C Jenke; Antonia Stern; Marie T J Daum; Lars Mündermann; Fiona R Kolbinger; Nithya Bhasker; Gerd Schneider; Grit Krause-Jüttler; Hisham Alwanni; Fleur Fritz-Kebede; Oliver Burgert; Dirk Wilhelm; Johannes Fallert; Felix Nickel; Lena Maier-Hein; Martin Dugas; Marius Distler; Jürgen Weitz; Beat-Peter Müller-Stich; Stefanie Speidel Journal: Surg Endosc Date: 2022-09-28 Impact factor: 3.453
Authors: Andrew A Gumbs; Vincent Grasso; Nicolas Bourdel; Roland Croner; Gaya Spolverato; Isabella Frigerio; Alfredo Illanes; Mohammad Abu Hilal; Adrian Park; Eyad Elyan Journal: Sensors (Basel) Date: 2022-06-29 Impact factor: 3.847
Authors: Kyle Lam; Junhong Chen; Zeyu Wang; Fahad M Iqbal; Ara Darzi; Benny Lo; Sanjay Purkayastha; James M Kinross Journal: NPJ Digit Med Date: 2022-03-03
Authors: E Willuth; S F Hardon; F Lang; C M Haney; E A Felinska; K F Kowalewski; B P Müller-Stich; T Horeman; F Nickel Journal: Surg Endosc Date: 2021-02-26 Impact factor: 4.584