Yuk-Hoi Yiu1, Moustafa Aboulatta2, Theresa Raiser3, Leoni Ophey1, Virginia L Flanagin1, Peter Zu Eulenburg4, Seyed-Ahmad Ahmadi5. 1. German Center for Vertigo and Balance Disorders (DSGZ), Feodor-Lynen-Str. 19, 81377 Munich, Germany. 2. Faculty of Informatics, Technical University of Munich, Boltzmannstr. 3, 85748 Garching, Germany. 3. German Center for Vertigo and Balance Disorders (DSGZ), Feodor-Lynen-Str. 19, 81377 Munich, Germany; Graduate School of Systemic Neurosciences, Ludwig-Maximilians Universität München, Großhadern Str. 2, 82152 Planegg, Germany. 4. German Center for Vertigo and Balance Disorders (DSGZ), Feodor-Lynen-Str. 19, 81377 Munich, Germany; Department of Neurology, Klinikum der Universität München, Marchioninistr. 15, 81377 Munich, Germany. 5. German Center for Vertigo and Balance Disorders (DSGZ), Feodor-Lynen-Str. 19, 81377 Munich, Germany. Electronic address: ahmad.ahmadi@med.uni-muenchen.de.
Abstract
BACKGROUND: A prerequisite for many eye tracking and video-oculography (VOG) methods is an accurate localization of the pupil. Several existing techniques face challenges in images with artifacts and under naturalistic low-light conditions, e.g. with highly dilated pupils. NEW METHOD: For the first time, we propose to use a fully convolutional neural network (FCNN) for segmentation of the whole pupil area, trained on 3946 VOG images hand-annotated at our institute. We integrate the FCNN into DeepVOG, along with an established method for gaze estimation from elliptical pupil contours, which we improve upon by considering our FCNN's segmentation confidence measure. RESULTS: The FCNN output simultaneously enables us to perform pupil center localization, elliptical contour estimation and blink detection, all with a single network and with an assigned confidence value, at framerates above 130 Hz on commercial workstations with GPU acceleration. Pupil centre coordinates can be estimated with a median accuracy of around 1.0 pixel, and gaze estimation is accurate to within 0.5 degrees. The FCNN is able to robustly segment the pupil in a wide array of datasets that were not used for training. COMPARISON WITH EXISTING METHODS: We validate our method against gold standard eye images that were artificially rendered, as well as hand-annotated VOG data from a gold-standard clinical system (EyeSeeCam) at our institute. CONCLUSIONS: Our proposed FCNN-based pupil segmentation framework is accurate, robust and generalizes well to new VOG datasets. We provide our code and pre-trained FCNN model open-source and for free under www.github.com/pydsgz/DeepVOG.
BACKGROUND: A prerequisite for many eye tracking and video-oculography (VOG) methods is an accurate localization of the pupil. Several existing techniques face challenges in images with artifacts and under naturalistic low-light conditions, e.g. with highly dilated pupils. NEW METHOD: For the first time, we propose to use a fully convolutional neural network (FCNN) for segmentation of the whole pupil area, trained on 3946 VOG images hand-annotated at our institute. We integrate the FCNN into DeepVOG, along with an established method for gaze estimation from elliptical pupil contours, which we improve upon by considering our FCNN's segmentation confidence measure. RESULTS: The FCNN output simultaneously enables us to perform pupil center localization, elliptical contour estimation and blink detection, all with a single network and with an assigned confidence value, at framerates above 130 Hz on commercial workstations with GPU acceleration. Pupil centre coordinates can be estimated with a median accuracy of around 1.0 pixel, and gaze estimation is accurate to within 0.5 degrees. The FCNN is able to robustly segment the pupil in a wide array of datasets that were not used for training. COMPARISON WITH EXISTING METHODS: We validate our method against gold standard eye images that were artificially rendered, as well as hand-annotated VOG data from a gold-standard clinical system (EyeSeeCam) at our institute. CONCLUSIONS: Our proposed FCNN-based pupil segmentation framework is accurate, robust and generalizes well to new VOG datasets. We provide our code and pre-trained FCNN model open-source and for free under www.github.com/pydsgz/DeepVOG.
Authors: Kun Qian; Tomoki Arichi; Anthony Price; Sofia Dall'Orso; Jonathan Eden; Yohan Noh; Kawal Rhode; Etienne Burdet; Mark Neil; A David Edwards; Joseph V Hajnal Journal: Sci Rep Date: 2021-08-11 Impact factor: 4.379