Kilian Chandelon1,2, Adrien Bartoli3,4,5. 1. EnCoV, Institut Pascal, UMR6602 CNRS, UCA, Clermont-Ferrand University Hospital, 63000, Clermont-Ferrand, France. kilian.chandelon@gmail.com. 2. SurgAR - Surgical Augmented Reality, 63000, Clermont-Ferrand, France. kilian.chandelon@gmail.com. 3. EnCoV, Institut Pascal, UMR6602 CNRS, UCA, Clermont-Ferrand University Hospital, 63000, Clermont-Ferrand, France. 4. SurgAR - Surgical Augmented Reality, 63000, Clermont-Ferrand, France. 5. Department of Clinical Research and Innovation, Clermont-Ferrand University Hospital, Clermont-Ferrand, France.
Abstract
PURPOSE: We present a novel automatic system for markerless real-time augmented reality. Our system uses a dynamic keyframe database, which is required to track previously unseen or appearance-changing anatomical structures. Our main objective is to track the organ more accurately and over a longer time frame through the surgery. METHODS: Our system works with an offline stage which constructs the initial keyframe database and an online stage which dynamically updates the database with new keyframes automatically selected from the video stream. We propose five keyframe selection criteria ensuring tracking stability and a database management scheme ensuring real-time performance. RESULTS: Experimental results show that our automatic keyframe selection system based on a dynamic keyframe database outperforms the baseline system with a static keyframe database. An increase in number of tracked frames without requiring surgeon input is observed with an average improvement margin over the baseline of 11.9%. The frame rate is kept at the same values as the baseline, close to 50 FPS, and rendering remains smooth. CONCLUSION: Our software-based tracking system copes with new viewpoints and appearance changes during surgery. It improves surgical organ tracking performance. Its criterion-based architecture allows a high degree of flexibility in the implementation, hence compatibility with various use cases.
PURPOSE: We present a novel automatic system for markerless real-time augmented reality. Our system uses a dynamic keyframe database, which is required to track previously unseen or appearance-changing anatomical structures. Our main objective is to track the organ more accurately and over a longer time frame through the surgery. METHODS: Our system works with an offline stage which constructs the initial keyframe database and an online stage which dynamically updates the database with new keyframes automatically selected from the video stream. We propose five keyframe selection criteria ensuring tracking stability and a database management scheme ensuring real-time performance. RESULTS: Experimental results show that our automatic keyframe selection system based on a dynamic keyframe database outperforms the baseline system with a static keyframe database. An increase in number of tracked frames without requiring surgeon input is observed with an average improvement margin over the baseline of 11.9%. The frame rate is kept at the same values as the baseline, close to 50 FPS, and rendering remains smooth. CONCLUSION: Our software-based tracking system copes with new viewpoints and appearance changes during surgery. It improves surgical organ tracking performance. Its criterion-based architecture allows a high degree of flexibility in the implementation, hence compatibility with various use cases.
Authors: T Collins; D Pizarro; S Gasparini; N Bourdel; P Chauvet; M Canis; L Calvet; A Bartoli Journal: IEEE Trans Med Imaging Date: 2020-12-29 Impact factor: 10.048