Aaron S Coyner1, Ryan Swan1, J Peter Campbell2, Susan Ostmo2, James M Brown3, Jayashree Kalpathy-Cramer4, Sang Jin Kim5, Karyn E Jonas6, R V Paul Chan6, Michael F Chiang7. 1. Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon. 2. Department of Ophthalmology, Casey Eye Institute, Oregon Health and Science University, Portland, Oregon. 3. Department of Radiology, Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, Massachusetts. 4. Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon; Department of Radiology, Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Harvard Medical School, Charlestown, Massachusetts; Massachusetts General Hospital and Brigham and Women's Hospital Center for Clinical Data Science, Boston, Massachusetts. 5. Department of Ophthalmology, Casey Eye Institute, Oregon Health and Science University, Portland, Oregon; Department of Ophthalmology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea. 6. Department of Ophthalmology, University of Illinois at Chicago, Chicago, Illinois. 7. Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon; Department of Ophthalmology, Casey Eye Institute, Oregon Health and Science University, Portland, Oregon. Electronic address: chiangm@ohsu.edu.
Abstract
PURPOSE: Accurate image-based ophthalmic diagnosis relies on fundus image clarity. This has important implications for the quality of ophthalmic diagnoses and for emerging methods such as telemedicine and computer-based image analysis. The purpose of this study was to implement a deep convolutional neural network (CNN) for automated assessment of fundus image quality in retinopathy of prematurity (ROP). DESIGN: Experimental study. PARTICIPANTS: Retinal fundus images were collected from preterm infants during routine ROP screenings. METHODS: Six thousand one hundred thirty-nine retinal fundus images were collected from 9 academic institutions. Each image was graded for quality (acceptable quality [AQ], possibly acceptable quality [PAQ], or not acceptable quality [NAQ]) by 3 independent experts. Quality was defined as the ability to assess an image confidently for the presence of ROP. Of the 6139 images, NAQ, PAQ, and AQ images represented 5.6%, 43.6%, and 50.8% of the image set, respectively. Because of low representation of NAQ images in the data set, images labeled NAQ were grouped into the PAQ category, and a binary CNN classifier was trained using 5-fold cross-validation on 4000 images. A test set of 2109 images was held out for final model evaluation. Additionally, 30 images were ranked from worst to best quality by 6 experts via pairwise comparisons, and the CNN's ability to rank quality, regardless of quality classification, was assessed. MAIN OUTCOME MEASURES: The CNN performance was evaluated using area under the receiver operating characteristic curve (AUC). A Spearman's rank correlation was calculated to evaluate the overall ability of the CNN to rank images from worst to best quality as compared with experts. RESULTS: The mean AUC for 5-fold cross-validation was 0.958 (standard deviation, 0.005) for the diagnosis of AQ versus PAQ images. The AUC was 0.965 for the test set. The Spearman's rank correlation coefficient on the set of 30 images was 0.90 as compared with the overall expert consensus ranking. CONCLUSIONS: This model accurately assessed retinal fundus image quality in a comparable manner with that of experts. This fully automated model has potential for application in clinical settings, telemedicine, and computer-based image analysis in ROP and for generalizability to other ophthalmic diseases.
PURPOSE: Accurate image-based ophthalmic diagnosis relies on fundus image clarity. This has important implications for the quality of ophthalmic diagnoses and for emerging methods such as telemedicine and computer-based image analysis. The purpose of this study was to implement a deep convolutional neural network (CNN) for automated assessment of fundus image quality in retinopathy of prematurity (ROP). DESIGN: Experimental study. PARTICIPANTS: Retinal fundus images were collected from preterm infants during routine ROP screenings. METHODS: Six thousand one hundred thirty-nine retinal fundus images were collected from 9 academic institutions. Each image was graded for quality (acceptable quality [AQ], possibly acceptable quality [PAQ], or not acceptable quality [NAQ]) by 3 independent experts. Quality was defined as the ability to assess an image confidently for the presence of ROP. Of the 6139 images, NAQ, PAQ, and AQ images represented 5.6%, 43.6%, and 50.8% of the image set, respectively. Because of low representation of NAQ images in the data set, images labeled NAQ were grouped into the PAQ category, and a binary CNN classifier was trained using 5-fold cross-validation on 4000 images. A test set of 2109 images was held out for final model evaluation. Additionally, 30 images were ranked from worst to best quality by 6 experts via pairwise comparisons, and the CNN's ability to rank quality, regardless of quality classification, was assessed. MAIN OUTCOME MEASURES: The CNN performance was evaluated using area under the receiver operating characteristic curve (AUC). A Spearman's rank correlation was calculated to evaluate the overall ability of the CNN to rank images from worst to best quality as compared with experts. RESULTS: The mean AUC for 5-fold cross-validation was 0.958 (standard deviation, 0.005) for the diagnosis of AQ versus PAQ images. The AUC was 0.965 for the test set. The Spearman's rank correlation coefficient on the set of 30 images was 0.90 as compared with the overall expert consensus ranking. CONCLUSIONS: This model accurately assessed retinal fundus image quality in a comparable manner with that of experts. This fully automated model has potential for application in clinical settings, telemedicine, and computer-based image analysis in ROP and for generalizability to other ophthalmic diseases.
Authors: Tobias J Dietrich; Erika J Ulbrich; Marco Zanetti; Sandro F Fucentese; Christian W A Pfirrmann Journal: AJR Am J Roentgenol Date: 2011-12 Impact factor: 3.959
Authors: Aaron S Coyner; Ryan Swan; James M Brown; Jayashree Kalpathy-Cramer; Sang Jin Kim; J Peter Campbell; Karyn E Jonas; Susan Ostmo; R V Paul Chan; Michael F Chiang Journal: AMIA Annu Symp Proc Date: 2018-12-05
Authors: F X Castellanos; J N Giedd; W L Marsh; S D Hamburger; A C Vaituzis; D P Dickstein; S E Sarfatti; Y C Vauss; J W Snell; N Lange; D Kaysen; A L Krain; G F Ritchie; J C Rajapakse; J L Rapoport Journal: Arch Gen Psychiatry Date: 1996-07
Authors: Miles F Greenwald; Ian D Danford; Malika Shahrawat; Susan Ostmo; James Brown; Jayashree Kalpathy-Cramer; Kacy Bradshaw; Robert Schelonka; Howard S Cohen; R V Paul Chan; Michael F Chiang; J Peter Campbell Journal: J AAPOS Date: 2020-04-11 Impact factor: 1.220
Authors: Rene Y Choi; Aaron S Coyner; Jayashree Kalpathy-Cramer; Michael F Chiang; J Peter Campbell Journal: Transl Vis Sci Technol Date: 2020-02-27 Impact factor: 3.283
Authors: Nita G Valikodath; Emily Cole; Daniel S W Ting; J Peter Campbell; Louis R Pasquale; Michael F Chiang; R V Paul Chan Journal: Transl Vis Sci Technol Date: 2021-06-01 Impact factor: 3.283
Authors: J Peter Campbell; Sang Jin Kim; James M Brown; Susan Ostmo; R V Paul Chan; Jayashree Kalpathy-Cramer; Michael F Chiang Journal: Ophthalmology Date: 2020-10-27 Impact factor: 14.277