Nils Gessert1, Jens Beringhoff2, Christoph Otte2, Alexander Schlaefer2. 1. Hamburg University of Technology, Schwarzenbergstraße 95, 21073, Hamburg, Germany. nils.gessert@tuhh.de. 2. Hamburg University of Technology, Schwarzenbergstraße 95, 21073, Hamburg, Germany.
Abstract
PURPOSE: Estimating the interaction forces of instruments and tissue is of interest, particularly to provide haptic feedback during robot-assisted minimally invasive interventions. Different approaches based on external and integrated force sensors have been proposed. These are hampered by friction, sensor size, and sterilizability. We investigate a novel approach to estimate the force vector directly from optical coherence tomography image volumes. METHODS: We introduce a novel Siamese 3D CNN architecture. The network takes an undeformed reference volume and a deformed sample volume as an input and outputs the three components of the force vector. We employ a deep residual architecture with bottlenecks for increased efficiency. We compare the Siamese approach to methods using difference volumes and two-dimensional projections. Data were generated using a robotic setup to obtain ground-truth force vectors for silicon tissue phantoms as well as porcine tissue. RESULTS: Our method achieves a mean average error of [Formula: see text] when estimating the force vector. Our novel Siamese 3D CNN architecture outperforms single-path methods that achieve a mean average error of [Formula: see text]. Moreover, the use of volume data leads to significantly higher performance compared to processing only surface information which achieves a mean average error of [Formula: see text]. Based on the tissue dataset, our methods shows good generalization in between different subjects. CONCLUSIONS: We propose a novel image-based force estimation method using optical coherence tomography. We illustrate that capturing the deformation of subsurface structures substantially improves force estimation. Our approach can provide accurate force estimates in surgical setups when using intraoperative optical coherence tomography.
PURPOSE: Estimating the interaction forces of instruments and tissue is of interest, particularly to provide haptic feedback during robot-assisted minimally invasive interventions. Different approaches based on external and integrated force sensors have been proposed. These are hampered by friction, sensor size, and sterilizability. We investigate a novel approach to estimate the force vector directly from optical coherence tomography image volumes. METHODS: We introduce a novel Siamese 3D CNN architecture. The network takes an undeformed reference volume and a deformed sample volume as an input and outputs the three components of the force vector. We employ a deep residual architecture with bottlenecks for increased efficiency. We compare the Siamese approach to methods using difference volumes and two-dimensional projections. Data were generated using a robotic setup to obtain ground-truth force vectors for silicon tissue phantoms as well as porcine tissue. RESULTS: Our method achieves a mean average error of [Formula: see text] when estimating the force vector. Our novel Siamese 3D CNN architecture outperforms single-path methods that achieve a mean average error of [Formula: see text]. Moreover, the use of volume data leads to significantly higher performance compared to processing only surface information which achieves a mean average error of [Formula: see text]. Based on the tissue dataset, our methods shows good generalization in between different subjects. CONCLUSIONS: We propose a novel image-based force estimation method using optical coherence tomography. We illustrate that capturing the deformation of subsurface structures substantially improves force estimation. Our approach can provide accurate force estimates in surgical setups when using intraoperative optical coherence tomography.
Entities:
Keywords:
3D CNN; Force estimation; OCT; Siamese CNN
Authors: Benjamin Potsaid; Bernhard Baumann; David Huang; Scott Barry; Alex E Cable; Joel S Schuman; Jay S Duker; James G Fujimoto Journal: Opt Express Date: 2010-09-13 Impact factor: 3.894
Authors: Justis P Ehlers; Sunil K Srivastava; Daniel Feiler; Amanda I Noonan; Andrew M Rollins; Yuankai K Tao Journal: PLoS One Date: 2014-08-20 Impact factor: 3.240
Authors: O M Carrasco-Zevallos; B Keller; C Viehland; L Shen; G Waterman; B Todorich; C Shieh; P Hahn; S Farsiu; A N Kuo; C A Toth; J A Izatt Journal: Sci Rep Date: 2016-08-19 Impact factor: 4.379
Authors: Nils Gessert; Torben Priegnitz; Thore Saathoff; Sven-Thomas Antoni; David Meyer; Moritz Franz Hamann; Klaus-Peter Jünemann; Christoph Otte; Alexander Schlaefer Journal: Int J Comput Assist Radiol Surg Date: 2019-05-30 Impact factor: 2.924