Balazs P Vagvolgyi1, Ravikrishnan P Jayakumar2, Manu S Madhav3, James J Knierim4, Noah J Cowan5. 1. Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, USA. Electronic address: balazs@jhu.edu. 2. Mind/Brain Institute, Johns Hopkins University, Baltimore, USA; Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, USA; Mechanical Engineering Department, Johns Hopkins University, Baltimore, MD, USA. 3. Mind/Brain Institute, Johns Hopkins University, Baltimore, USA; Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, USA; Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, USA; School of Biomedical Engineering, Djawad Mowafaghian Centre for Brain Health, University of British Columbia, BC, Canada. 4. Mind/Brain Institute, Johns Hopkins University, Baltimore, USA; Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, USA. 5. Laboratory for Computational Sensing and Robotics, Johns Hopkins University, Baltimore, USA; Mechanical Engineering Department, Johns Hopkins University, Baltimore, MD, USA.
Abstract
BACKGROUND: Camera images can encode large amounts of visual information of an animal and its environment, enabling high fidelity 3D reconstruction of the animal and its environment using computer vision methods. Most systems, both markerless (e.g. deep learning based) and marker-based, require multiple cameras to track features across multiple points of view to enable such 3D reconstruction. However, such systems can be expensive and are challenging to set up in small animal research apparatuses. NEW METHODS: We present an open-source, marker-based system for tracking the head of a rodent for behavioral research that requires only a single camera with a potentially wide field of view. The system features a lightweight visual target and computer vision algorithms that together enable high-accuracy tracking of the six-degree-of-freedom position and orientation of the animal's head. The system, which only requires a single camera positioned above the behavioral arena, robustly reconstructs the pose over a wide range of head angles (360° in yaw, and approximately ± 120° in roll and pitch). RESULTS: Experiments with live animals demonstrate that the system can reliably identify rat head position and orientation. Evaluations using a commercial optical tracker device show that the system achieves accuracy that rivals commercial multi-camera systems. COMPARISON WITH EXISTING METHODS: Our solution significantly improves upon existing monocular marker-based tracking methods, both in accuracy and in allowable range of motion. CONCLUSIONS: The proposed system enables the study of complex behaviors by providing robust, fine-scale measurements of rodent head motions in a wide range of orientations.
BACKGROUND: Camera images can encode large amounts of visual information of an animal and its environment, enabling high fidelity 3D reconstruction of the animal and its environment using computer vision methods. Most systems, both markerless (e.g. deep learning based) and marker-based, require multiple cameras to track features across multiple points of view to enable such 3D reconstruction. However, such systems can be expensive and are challenging to set up in small animal research apparatuses. NEW METHODS: We present an open-source, marker-based system for tracking the head of a rodent for behavioral research that requires only a single camera with a potentially wide field of view. The system features a lightweight visual target and computer vision algorithms that together enable high-accuracy tracking of the six-degree-of-freedom position and orientation of the animal's head. The system, which only requires a single camera positioned above the behavioral arena, robustly reconstructs the pose over a wide range of head angles (360° in yaw, and approximately ± 120° in roll and pitch). RESULTS: Experiments with live animals demonstrate that the system can reliably identify rat head position and orientation. Evaluations using a commercial optical tracker device show that the system achieves accuracy that rivals commercial multi-camera systems. COMPARISON WITH EXISTING METHODS: Our solution significantly improves upon existing monocular marker-based tracking methods, both in accuracy and in allowable range of motion. CONCLUSIONS: The proposed system enables the study of complex behaviors by providing robust, fine-scale measurements of rodent head motions in a wide range of orientations.
Authors: Stanley M Tomkiewicz; Mark R Fuller; John G Kie; Kirk K Bates Journal: Philos Trans R Soc Lond B Biol Sci Date: 2010-07-27 Impact factor: 6.237
Authors: Manu S Madhav; Ravikrishnan P Jayakumar; Shahin G Lashkari; Francesco Savelli; Hugh T Blair; James J Knierim; Noah J Cowan Journal: J Neurosci Methods Date: 2021-08-26 Impact factor: 2.987
Authors: Ravikrishnan P Jayakumar; Manu S Madhav; Noah J Cowan; James J Knierim; Francesco Savelli; Hugh T Blair Journal: Nature Date: 2019-02-11 Impact factor: 49.962
Authors: Manu S Madhav; Ravikrishnan P Jayakumar; Shahin G Lashkari; Francesco Savelli; Hugh T Blair; James J Knierim; Noah J Cowan Journal: J Neurosci Methods Date: 2021-08-26 Impact factor: 2.987