Robert B Grupp1, Mathias Unberath2, Cong Gao2, Rachel A Hegeman3, Ryan J Murphy4, Clayton P Alexander5, Yoshito Otake6, Benjamin A McArthur7,8, Mehran Armand3,5,9, Russell H Taylor2. 1. Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA. grupp@jhu.edu. 2. Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA. 3. Research and Exploratory Development Department, Johns Hopkins University Applied Physics Laboratory, Laurel, MD, USA. 4. Auris Health, Inc., Redwood City, CA, USA. 5. Department of Orthopaedic Surgery, Johns Hopkins Medicine, Baltimore, MD, USA. 6. Graduate School of Information Science, Nara Institute of Science and Technology, Ikoma, Nara, Japan. 7. Department of Surgery and Perioperative Care, Dell Medical School, University of Texas, Austin, TX, USA. 8. Texas Orthopedics, Austin, TX, USA. 9. Department of Mechanical Engineering, Johns Hopkins University, Baltimore, MD, USA.
Abstract
PURPOSE: Fluoroscopy is the standard imaging modality used to guide hip surgery and is therefore a natural sensor for computer-assisted navigation. In order to efficiently solve the complex registration problems presented during navigation, human-assisted annotations of the intraoperative image are typically required. This manual initialization interferes with the surgical workflow and diminishes any advantages gained from navigation. In this paper, we propose a method for fully automatic registration using anatomical annotations produced by a neural network. METHODS: Neural networks are trained to simultaneously segment anatomy and identify landmarks in fluoroscopy. Training data are obtained using a computationally intensive, intraoperatively incompatible, 2D/3D registration of the pelvis and each femur. Ground truth 2D segmentation labels and anatomical landmark locations are established using projected 3D annotations. Intraoperative registration couples a traditional intensity-based strategy with annotations inferred by the network and requires no human assistance. RESULTS: Ground truth segmentation labels and anatomical landmarks were obtained in 366 fluoroscopic images across 6 cadaveric specimens. In a leave-one-subject-out experiment, networks trained on these data obtained mean dice coefficients for left and right hemipelves, left and right femurs of 0.86, 0.87, 0.90, and 0.84, respectively. The mean 2D landmark localization error was 5.0 mm. The pelvis was registered within [Formula: see text] for 86% of the images when using the proposed intraoperative approach with an average runtime of 7 s. In comparison, an intensity-only approach without manual initialization registered the pelvis to [Formula: see text] in 18% of images. CONCLUSIONS: We have created the first accurately annotated, non-synthetic, dataset of hip fluoroscopy. By using these annotations as training data for neural networks, state-of-the-art performance in fluoroscopic segmentation and landmark localization was achieved. Integrating these annotations allows for a robust, fully automatic, and efficient intraoperative registration during fluoroscopic navigation of the hip.
PURPOSE: Fluoroscopy is the standard imaging modality used to guide hip surgery and is therefore a natural sensor for computer-assisted navigation. In order to efficiently solve the complex registration problems presented during navigation, human-assisted annotations of the intraoperative image are typically required. This manual initialization interferes with the surgical workflow and diminishes any advantages gained from navigation. In this paper, we propose a method for fully automatic registration using anatomical annotations produced by a neural network. METHODS: Neural networks are trained to simultaneously segment anatomy and identify landmarks in fluoroscopy. Training data are obtained using a computationally intensive, intraoperatively incompatible, 2D/3D registration of the pelvis and each femur. Ground truth 2D segmentation labels and anatomical landmark locations are established using projected 3D annotations. Intraoperative registration couples a traditional intensity-based strategy with annotations inferred by the network and requires no human assistance. RESULTS: Ground truth segmentation labels and anatomical landmarks were obtained in 366 fluoroscopic images across 6 cadaveric specimens. In a leave-one-subject-out experiment, networks trained on these data obtained mean dice coefficients for left and right hemipelves, left and right femurs of 0.86, 0.87, 0.90, and 0.84, respectively. The mean 2D landmark localization error was 5.0 mm. The pelvis was registered within [Formula: see text] for 86% of the images when using the proposed intraoperative approach with an average runtime of 7 s. In comparison, an intensity-only approach without manual initialization registered the pelvis to [Formula: see text] in 18% of images. CONCLUSIONS: We have created the first accurately annotated, non-synthetic, dataset of hip fluoroscopy. By using these annotations as training data for neural networks, state-of-the-art performance in fluoroscopic segmentation and landmark localization was achieved. Integrating these annotations allows for a robust, fully automatic, and efficient intraoperative registration during fluoroscopic navigation of the hip.
Authors: Cornelius Johannes Frederik Reyneke; Marcel Luthi; Valerie Burdin; Tania S Douglas; Thomas Vetter; Tinashe E M Mutsvangwa Journal: IEEE Rev Biomed Eng Date: 2018-10-17
Authors: Michael Woerner; Ernst Sendtner; Robert Springorum; Benjamin Craiovan; Michael Worlicek; Tobias Renkawitz; Joachim Grifka; Markus Weber Journal: Acta Orthop Date: 2016-02-05 Impact factor: 3.717
Authors: Daniel F Malan; Stéfan J van der Walt; Renata G Raidou; Bas van den Berg; Berend C Stoel; Charl P Botha; Rob G H H Nelissen; Edward R Valstar Journal: Int J Comput Assist Radiol Surg Date: 2015-08-11 Impact factor: 2.924
Authors: Cong Gao; Amirhossein Farvardin; Robert B Grupp; Mahsan Bakhtiarinejad; Liuhong Ma; Mareike Thies; Mathias Unberath; Russell H Taylor; Mehran Armand Journal: IEEE Trans Med Robot Bionics Date: 2020-07-28
Authors: Lena Maier-Hein; Matthias Eisenmann; Duygu Sarikaya; Keno März; Toby Collins; Anand Malpani; Johannes Fallert; Hubertus Feussner; Stamatia Giannarou; Pietro Mascagni; Hirenkumar Nakawala; Adrian Park; Carla Pugh; Danail Stoyanov; Swaroop S Vedula; Kevin Cleary; Gabor Fichtinger; Germain Forestier; Bernard Gibaud; Teodor Grantcharov; Makoto Hashizume; Doreen Heckmann-Nötzel; Hannes G Kenngott; Ron Kikinis; Lars Mündermann; Nassir Navab; Sinan Onogur; Tobias Roß; Raphael Sznitman; Russell H Taylor; Minu D Tizabi; Martin Wagner; Gregory D Hager; Thomas Neumuth; Nicolas Padoy; Justin Collins; Ines Gockel; Jan Goedeke; Daniel A Hashimoto; Luc Joyeux; Kyle Lam; Daniel R Leff; Amin Madani; Hani J Marcus; Ozanan Meireles; Alexander Seitel; Dogu Teber; Frank Ückert; Beat P Müller-Stich; Pierre Jannin; Stefanie Speidel Journal: Med Image Anal Date: 2021-11-18 Impact factor: 13.828