Lasse Hansen1, Marlin Siebert2, Jasper Diesel3, Mattias P Heinrich2. 1. Institute of Medical Informatics, University of Lübeck, Ratzeburger Allee 160, 23562, Lübeck, Germany. hansen@imi.uni-luebeck.de. 2. Institute of Medical Informatics, University of Lübeck, Ratzeburger Allee 160, 23562, Lübeck, Germany. 3. Drägerwerk AG & Co. KGaA, Moislinger Allee 53-55, 23558, Lübeck, Germany.
Abstract
PURPOSE: For many years, deep convolutional neural networks have achieved state-of-the-art results on a wide variety of computer vision tasks. 3D human pose estimation makes no exception and results on public benchmarks are impressive. However, specialized domains, such as operating rooms, pose additional challenges. Clinical settings include severe occlusions, clutter and difficult lighting conditions. Privacy concerns of patients and staff make it necessary to use unidentifiable data. In this work, we aim to bring robust human pose estimation to the clinical domain. METHODS: We propose a 2D-3D information fusion framework that makes use of a network of multiple depth cameras and strong pose priors. In a first step, probabilities of 2D joints are predicted from single depth images. These information are fused in a shared voxel space yielding a rough estimate of the 3D pose. Final joint positions are obtained by regressing into the latent pose space of a pre-trained convolutional autoencoder. RESULTS: We evaluate our approach against several baselines on the challenging MVOR dataset. Best results are obtained when fusing 2D information from multiple views and constraining the predictions with learned pose priors. CONCLUSIONS: We present a robust 3D human pose estimation framework based on a multi-depth camera network in the operating room. Depth images as only input modalities make our approach especially interesting for clinical applications due to the given anonymity for patients and staff.
PURPOSE: For many years, deep convolutional neural networks have achieved state-of-the-art results on a wide variety of computer vision tasks. 3D human pose estimation makes no exception and results on public benchmarks are impressive. However, specialized domains, such as operating rooms, pose additional challenges. Clinical settings include severe occlusions, clutter and difficult lighting conditions. Privacy concerns of patients and staff make it necessary to use unidentifiable data. In this work, we aim to bring robust human pose estimation to the clinical domain. METHODS: We propose a 2D-3D information fusion framework that makes use of a network of multiple depth cameras and strong pose priors. In a first step, probabilities of 2D joints are predicted from single depth images. These information are fused in a shared voxel space yielding a rough estimate of the 3D pose. Final joint positions are obtained by regressing into the latent pose space of a pre-trained convolutional autoencoder. RESULTS: We evaluate our approach against several baselines on the challenging MVOR dataset. Best results are obtained when fusing 2D information from multiple views and constraining the predictions with learned pose priors. CONCLUSIONS: We present a robust 3D human pose estimation framework based on a multi-depth camera network in the operating room. Depth images as only input modalities make our approach especially interesting for clinical applications due to the given anonymity for patients and staff.
Entities:
Keywords:
2D–3D information fusion; Convolutional autoencoder; Deep learning; Human pose estimation; Operating room
Authors: Kenny Chen; Paolo Gabriel; Abdulwahab Alasfour; Chenghao Gong; Werner K Doyle; Orrin Devinsky; Daniel Friedman; Patricia Dugan; Lucia Melloni; Thomas Thesen; David Gonda; Shifteh Sattar; Sonya Wang; Vikash Gilja Journal: IEEE J Transl Eng Health Med Date: 2018-10-10 Impact factor: 3.316