T W Rogers1, J Gonzalez-Bueno2, R Garcia Franco3, E Lopez Star3, D Méndez Marín3, J Vassallo4, V C Lansingh3,5, S Trikha6, N Jaccard2. 1. Unaffiliated, London, UK. thomas.rogers08@gmail.com. 2. Unaffiliated, London, UK. 3. Instituto Mexicano de Oftalmología (IMO), Instituto de la Retina del Bajío (INDEREB), Santiago de Querétaro, Mexico. 4. Mater Dei Hospital, Triq Dun Karm, Imsida, Malta. 5. HelpMeSee, Inc., 20 West 36th Street, Floor 4, New York, NY, 10018-8005, USA. 6. King's College Hospital NHS Foundation Trust, London, SE5 9RS, UK.
Abstract
OBJECTIVES: To evaluate the performance of an artificial intelligence (AI) system (Pegasus, Visulytix Ltd., UK*) at the detection of diabetic retinopathy (DR) from images captured by a handheld portable fundus camera. METHODS: A cohort of 6404 patients (~80% with diabetes mellitus) was screened for retinal diseases using a handheld portable fundus camera (Pictor Plus, Volk Optical Inc., USA) at the Mexican Advanced Imaging Laboratory for Ocular Research. The images were graded for DR by specialists according to the Scottish DR grading scheme. The performance of the AI system was evaluated, retrospectively, in assessing referable DR (RDR) and proliferative DR (PDR) and compared with the performance on a publicly available desktop camera benchmark dataset. RESULTS: For RDR detection, Pegasus performed with an 89.4% (95% CI: 88.0-90.7) area under the receiver operating characteristic (AUROC) curve for the MAILOR cohort, compared with an AUROC of 98.5% (95% CI: 97.8-99.2) on the benchmark dataset. This difference was statistically significant. Moreover, no statistically significant difference was found in performance for PDR detection with Pegasus achieving an AUROC of 94.3% (95% CI: 91.0-96.9) on the MAILOR cohort and 92.2% (95% CI: 89.4-94.8) on the benchmark dataset. CONCLUSIONS: Pegasus showed good transferability for the detection of PDR from a curated desktop fundus camera dataset to real-world clinical practice with a handheld portable fundus camera. However, there was a substantial, and statistically significant, decrease in the diagnostic performance for RDR when using the handheld device.
OBJECTIVES: To evaluate the performance of an artificial intelligence (AI) system (Pegasus, Visulytix Ltd., UK*) at the detection of diabetic retinopathy (DR) from images captured by a handheld portable fundus camera. METHODS: A cohort of 6404 patients (~80% with diabetes mellitus) was screened for retinal diseases using a handheld portable fundus camera (Pictor Plus, Volk Optical Inc., USA) at the Mexican Advanced Imaging Laboratory for Ocular Research. The images were graded for DR by specialists according to the Scottish DR grading scheme. The performance of the AI system was evaluated, retrospectively, in assessing referable DR (RDR) and proliferative DR (PDR) and compared with the performance on a publicly available desktop camera benchmark dataset. RESULTS: For RDR detection, Pegasus performed with an 89.4% (95% CI: 88.0-90.7) area under the receiver operating characteristic (AUROC) curve for the MAILOR cohort, compared with an AUROC of 98.5% (95% CI: 97.8-99.2) on the benchmark dataset. This difference was statistically significant. Moreover, no statistically significant difference was found in performance for PDR detection with Pegasus achieving an AUROC of 94.3% (95% CI: 91.0-96.9) on the MAILOR cohort and 92.2% (95% CI: 89.4-94.8) on the benchmark dataset. CONCLUSIONS: Pegasus showed good transferability for the detection of PDR from a curated desktop fundus camera dataset to real-world clinical practice with a handheld portable fundus camera. However, there was a substantial, and statistically significant, decrease in the diagnostic performance for RDR when using the handheld device.
Authors: Adam B Cohen; Lisa Stump; Harlan M Krumholz; Margaret Cartiera; Sanchita Jain; L Scott Sussman; Allen Hsiao; Walter Lindop; Anita Kuo Ying; Rebecca L Kaul; Thomas J Balcezak; Welela Tereffe; Matthew Comerford; Daniel Jacoby; Neema Navai Journal: NPJ Digit Med Date: 2022-06-02
Authors: Soledad Jimenez-Carmona; Pedro Alemany-Marquez; Pablo Alvarez-Ramos; Eduardo Mayoral; Manuel Aguilar-Diosdado Journal: J Clin Med Date: 2021-12-21 Impact factor: 4.241