Ozanan R Meireles1, Guy Rosman2,3, Maria S Altieri4, Lawrence Carin5, Gregory Hager6, Amin Madani7, Nicolas Padoy8,9, Carla M Pugh10, Patricia Sylla11, Thomas M Ward2, Daniel A Hashimoto12. 1. Department of Surgery, Massachusetts General Hospital, 15 Parkman Street, WAC460, Boston, MA, 02114, USA. ozmeireles@mgh.harvard.edu. 2. Department of Surgery, Massachusetts General Hospital, 15 Parkman Street, WAC460, Boston, MA, 02114, USA. 3. Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, USA. 4. Department of Surgery, East Carolina University, Greenville, USA. 5. Department of Electrical and Computer Engineering, Duke University, Durham, USA. 6. Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, USA. 7. Department of Surgery, University Health Network, Toronto, Canada. 8. ICube, University of Strasbourg, Strasbourg, France. 9. IHU Strasbourg, Strasbourg, France. 10. Department of Surgery, Stanford University, Stanford, USA. 11. Department of Surgery, Mount Sinai Medical Center, New York, USA. 12. Department of Surgery, Massachusetts General Hospital, 15 Parkman Street, WAC460, Boston, MA, 02114, USA. dahashimoto@mgh.harvard.edu.
Abstract
BACKGROUND: The growing interest in analysis of surgical video through machine learning has led to increased research efforts; however, common methods of annotating video data are lacking. There is a need to establish recommendations on the annotation of surgical video data to enable assessment of algorithms and multi-institutional collaboration. METHODS: Four working groups were formed from a pool of participants that included clinicians, engineers, and data scientists. The working groups were focused on four themes: (1) temporal models, (2) actions and tasks, (3) tissue characteristics and general anatomy, and (4) software and data structure. A modified Delphi process was utilized to create a consensus survey based on suggested recommendations from each of the working groups. RESULTS: After three Delphi rounds, consensus was reached on recommendations for annotation within each of these domains. A hierarchy for annotation of temporal events in surgery was established. CONCLUSIONS: While additional work remains to achieve accepted standards for video annotation in surgery, the consensus recommendations on a general framework for annotation presented here lay the foundation for standardization. This type of framework is critical to enabling diverse datasets, performance benchmarks, and collaboration.
BACKGROUND: The growing interest in analysis of surgical video through machine learning has led to increased research efforts; however, common methods of annotating video data are lacking. There is a need to establish recommendations on the annotation of surgical video data to enable assessment of algorithms and multi-institutional collaboration. METHODS: Four working groups were formed from a pool of participants that included clinicians, engineers, and data scientists. The working groups were focused on four themes: (1) temporal models, (2) actions and tasks, (3) tissue characteristics and general anatomy, and (4) software and data structure. A modified Delphi process was utilized to create a consensus survey based on suggested recommendations from each of the working groups. RESULTS: After three Delphi rounds, consensus was reached on recommendations for annotation within each of these domains. A hierarchy for annotation of temporal events in surgery was established. CONCLUSIONS: While additional work remains to achieve accepted standards for video annotation in surgery, the consensus recommendations on a general framework for annotation presented here lay the foundation for standardization. This type of framework is critical to enabling diverse datasets, performance benchmarks, and collaboration.
Authors: Elena P Padilla; Christopher C Stahl; Sarah A Jung; Alexandra A Rosser; Patrick B Schwartz; Taylor Aiken; Alexandra W Acher; Daniel E Abbott; Jacob A Greenberg; Rebecca M Minter Journal: Ann Surg Date: 2022-02-01 Impact factor: 13.787
Authors: Lena Maier-Hein; Matthias Eisenmann; Duygu Sarikaya; Keno März; Toby Collins; Anand Malpani; Johannes Fallert; Hubertus Feussner; Stamatia Giannarou; Pietro Mascagni; Hirenkumar Nakawala; Adrian Park; Carla Pugh; Danail Stoyanov; Swaroop S Vedula; Kevin Cleary; Gabor Fichtinger; Germain Forestier; Bernard Gibaud; Teodor Grantcharov; Makoto Hashizume; Doreen Heckmann-Nötzel; Hannes G Kenngott; Ron Kikinis; Lars Mündermann; Nassir Navab; Sinan Onogur; Tobias Roß; Raphael Sznitman; Russell H Taylor; Minu D Tizabi; Martin Wagner; Gregory D Hager; Thomas Neumuth; Nicolas Padoy; Justin Collins; Ines Gockel; Jan Goedeke; Daniel A Hashimoto; Luc Joyeux; Kyle Lam; Daniel R Leff; Amin Madani; Hani J Marcus; Ozanan Meireles; Alexander Seitel; Dogu Teber; Frank Ückert; Beat P Müller-Stich; Pierre Jannin; Stefanie Speidel Journal: Med Image Anal Date: 2021-11-18 Impact factor: 13.828
Authors: Kyle Lam; Michael D Abràmoff; José M Balibrea; Steven M Bishop; Richard R Brady; Rachael A Callcut; Manish Chand; Justin W Collins; Markus K Diener; Matthias Eisenmann; Kelly Fermont; Manoel Galvao Neto; Gregory D Hager; Robert J Hinchliffe; Alan Horgan; Pierre Jannin; Alexander Langerman; Kartik Logishetty; Amit Mahadik; Lena Maier-Hein; Esteban Martín Antona; Pietro Mascagni; Ryan K Mathew; Beat P Müller-Stich; Thomas Neumuth; Felix Nickel; Adrian Park; Gianluca Pellino; Frank Rudzicz; Sam Shah; Mark Slack; Myles J Smith; Naeem Soomro; Stefanie Speidel; Danail Stoyanov; Henry S Tilney; Martin Wagner; Ara Darzi; James M Kinross; Sanjay Purkayastha Journal: NPJ Digit Med Date: 2022-07-19