Anand Malpani1, Colin Lea2, Chi Chiung Grace Chen3, Gregory D Hager2. 1. Department of Computer Science, The Johns Hopkins University, 3400 N. Charles St., Malone Hall Room 340, Baltimore, MD, 21218, USA. amalpan1@jhu.edu. 2. Department of Computer Science, The Johns Hopkins University, 3400 N. Charles St., Malone Hall Room 340, Baltimore, MD, 21218, USA. 3. Department of Gynecology and Obstetrics, The Johns Hopkins University, Johns Hopkins Bayview Medical Center, 301 Mason Lord Drive, Suite 3200, Baltimore, MD, 21224, USA.
Abstract
PURPOSE: Surgical phase recognition using sensor data is challenging due to high variation in patient anatomy and surgeon-specific operating styles. Segmenting surgical procedures into constituent phases is of significant utility for resident training, education, self-review, and context-aware operating room technologies. Phase annotation is a highly labor-intensive task and would benefit greatly from automated solutions. METHODS: We propose a novel approach using system events-for example, activation of cautery tools-that are easily captured in most surgical procedures. Our method involves extracting event-based features over 90-s intervals and assigning a phase label to each interval. We explore three classification techniques: support vector machines, random forests, and temporal convolution neural networks. Each of these models independently predicts a label for each time interval. We also examine segmental inference using an approach based on the semi-Markov conditional random field, which jointly performs phase segmentation and classification. Our method is evaluated on a data set of 24 robot-assisted hysterectomy procedures. RESULTS: Our framework is able to detect surgical phases with an accuracy of 74 % using event-based features over a set of five different phases-ligation, dissection, colpotomy, cuff closure, and background. Precision and recall values for the cuff closure (Precision: 83 %, Recall: 98 %) and dissection (Precision: 75 %, Recall: 88 %) classes were higher than other classes. The normalized Levenshtein distance between predicted and ground truth phase sequence was 25 %. CONCLUSIONS: Our findings demonstrate that system events features are useful for automatically detecting surgical phase. Events contain phase information that cannot be obtained from motion data and that would require advanced computer vision algorithms to extract from a video. Many of these events are not specific to robotic surgery and can easily be recorded in non-robotic surgical modalities. In future work, we plan to combine information from system events, tool motion, and videos to automate phase detection in surgical procedures.
PURPOSE: Surgical phase recognition using sensor data is challenging due to high variation in patient anatomy and surgeon-specific operating styles. Segmenting surgical procedures into constituent phases is of significant utility for resident training, education, self-review, and context-aware operating room technologies. Phase annotation is a highly labor-intensive task and would benefit greatly from automated solutions. METHODS: We propose a novel approach using system events-for example, activation of cautery tools-that are easily captured in most surgical procedures. Our method involves extracting event-based features over 90-s intervals and assigning a phase label to each interval. We explore three classification techniques: support vector machines, random forests, and temporal convolution neural networks. Each of these models independently predicts a label for each time interval. We also examine segmental inference using an approach based on the semi-Markov conditional random field, which jointly performs phase segmentation and classification. Our method is evaluated on a data set of 24 robot-assisted hysterectomy procedures. RESULTS: Our framework is able to detect surgical phases with an accuracy of 74 % using event-based features over a set of five different phases-ligation, dissection, colpotomy, cuff closure, and background. Precision and recall values for the cuff closure (Precision: 83 %, Recall: 98 %) and dissection (Precision: 75 %, Recall: 88 %) classes were higher than other classes. The normalized Levenshtein distance between predicted and ground truth phase sequence was 25 %. CONCLUSIONS: Our findings demonstrate that system events features are useful for automatically detecting surgical phase. Events contain phase information that cannot be obtained from motion data and that would require advanced computer vision algorithms to extract from a video. Many of these events are not specific to robotic surgery and can easily be recorded in non-robotic surgical modalities. In future work, we plan to combine information from system events, tool motion, and videos to automate phase detection in surgical procedures.
Entities:
Keywords:
Robot-assisted surgery; Sensor data; Surgical phase detection; Surgical process modeling; Surgical task flow; Surgical workflow analysis; System events
Authors: Carol E Reiley; Henry C Lin; Balakrishnan Varadarajan; Balazs Vagvolgyi; Ssanjeev Khudanpur; David D Yuh; Gregory D Hager Journal: Stud Health Technol Inform Date: 2008
Authors: Andru Putra Twinanda; Jacques Marescaux; Michel de Mathelin; Nicolas Padoy Journal: Int J Comput Assist Radiol Surg Date: 2015-04-07 Impact factor: 2.924
Authors: Ccg Chen; E Tanner; A Malpani; S S Vedula; A N Fader; S A Scheib; I C Green; G D Hager Journal: J Minim Invasive Gynecol Date: 2015-10-15 Impact factor: 4.137
Authors: Jason D Kelly; Ashley Petersen; Thomas S Lendvay; Timothy M Kowalewski Journal: Int J Comput Assist Radiol Surg Date: 2020-09-30 Impact factor: 2.924