Philippe Burlina1,2,3, William Paul1, T Y Alvin Liu2,3, Neil M Bressler3,4. 1. Applied Physics Laboratory, Johns Hopkins University, Baltimore, Maryland. 2. Department of Computer Science, Malone Center for Engineering in Healthcare, Johns Hopkins University, Baltimore, Maryland. 3. Retina Division, Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland. 4. Editor, JAMA Ophthalmology.
Abstract
IMPORTANCE: Anomaly detectors could be pursued for retinal diagnoses based on artificial intelligence systems that may not have access to training examples for all retinal diseases in all phenotypic presentations. Possible applications could include screening of population for any retinal disease rather than a specific disease such as diabetic retinopathy, detection of novel retinal diseases or novel presentations of common retinal diseases, and detection of rare diseases with little or no data available for training. OBJECTIVE: To study the application of anomaly detection to retinal diseases. DESIGN, SETTING, AND PARTICIPANTS: High-resolution retinal images from the publicly available EyePACS data set with fundus images with a corresponding label ranging from 0 to 4 for representing different severities of diabetic retinopathy. Sixteen variants of anomaly detectors were designed. For evaluation, a surrogate problem was constructed, using diabetic retinopathy images, in which only retinas with nonreferable diabetic retinopathy, ie, no diabetic macular edema, and no diabetic retinopathy or mild to moderate nonproliferative diabetic retinopathy were used for training an artificial intelligence system, but both nonreferable and referable diabetic retinopathy (including diabetic macular edema or proliferative diabetic retinopathy) were used to test the system for detecting retinal disease. MAIN OUTCOMES AND MEASURES: Anomaly detectors were evaluated by commonly accepted performance metrics, including area under the receiver operating characteristic curve, F1 score, and accuracy. RESULTS: A total of 88 692 high-resolution retinal images of 44 346 individuals with varying severity of diabetic retinopathy were analyzed. The best performing across all anomaly detectors had an area under the receiver operating characteristic of 0.808 (95% CI, 0.789-0.827) and was obtained using an embedding method that involved a self-supervised network. CONCLUSIONS AND RELEVANCE: This study suggests when abnormal (diseased) data, ie, referable diabetic retinopathy in this study, were not available for training of retinal diagnostic systems wherein only nonreferable diabetic retinopathy was used for training, anomaly detection techniques were useful in identifying images with and without referable diabetic retinopathy. This suggests that anomaly detectors may be used to detect retinal diseases in more generalized settings and potentially could play a role in screening of populations for retinal diseases or identifying novel diseases and phenotyping or detecting unusual presentations of common retinal diseases.
IMPORTANCE: Anomaly detectors could be pursued for retinal diagnoses based on artificial intelligence systems that may not have access to training examples for all retinal diseases in all phenotypic presentations. Possible applications could include screening of population for any retinal disease rather than a specific disease such as diabetic retinopathy, detection of novel retinal diseases or novel presentations of common retinal diseases, and detection of rare diseases with little or no data available for training. OBJECTIVE: To study the application of anomaly detection to retinal diseases. DESIGN, SETTING, AND PARTICIPANTS: High-resolution retinal images from the publicly available EyePACS data set with fundus images with a corresponding label ranging from 0 to 4 for representing different severities of diabetic retinopathy. Sixteen variants of anomaly detectors were designed. For evaluation, a surrogate problem was constructed, using diabetic retinopathy images, in which only retinas with nonreferable diabetic retinopathy, ie, no diabetic macular edema, and no diabetic retinopathy or mild to moderate nonproliferative diabetic retinopathy were used for training an artificial intelligence system, but both nonreferable and referable diabetic retinopathy (including diabetic macular edema or proliferative diabetic retinopathy) were used to test the system for detecting retinal disease. MAIN OUTCOMES AND MEASURES: Anomaly detectors were evaluated by commonly accepted performance metrics, including area under the receiver operating characteristic curve, F1 score, and accuracy. RESULTS: A total of 88 692 high-resolution retinal images of 44 346 individuals with varying severity of diabetic retinopathy were analyzed. The best performing across all anomaly detectors had an area under the receiver operating characteristic of 0.808 (95% CI, 0.789-0.827) and was obtained using an embedding method that involved a self-supervised network. CONCLUSIONS AND RELEVANCE: This study suggests when abnormal (diseased) data, ie, referable diabetic retinopathy in this study, were not available for training of retinal diagnostic systems wherein only nonreferable diabetic retinopathy was used for training, anomaly detection techniques were useful in identifying images with and without referable diabetic retinopathy. This suggests that anomaly detectors may be used to detect retinal diseases in more generalized settings and potentially could play a role in screening of populations for retinal diseases or identifying novel diseases and phenotyping or detecting unusual presentations of common retinal diseases.
Authors: Philippe M Burlina; Neil Joshi; Katia D Pacheco; T Y Alvin Liu; Neil M Bressler Journal: JAMA Ophthalmol Date: 2019-03-01 Impact factor: 7.389
Authors: Philipp Seebock; Jose Ignacio Orlando; Thomas Schlegl; Sebastian M Waldstein; Hrvoje Bogunovic; Sophie Klimscha; Georg Langs; Ursula Schmidt-Erfurth Journal: IEEE Trans Med Imaging Date: 2019-05-31 Impact factor: 10.048
Authors: Phillippe Burlina; Neil Joshi; Katia D Pacheco; David E Freund; Jun Kong; Neil M Bressler Journal: JAMA Ophthalmol Date: 2018-11-01 Impact factor: 7.389
Authors: Philippe Burlina; William Paul; Philip Mathew; Neil Joshi; Katia D Pacheco; Neil M Bressler Journal: JAMA Ophthalmol Date: 2020-10-01 Impact factor: 7.389