Rene Y Choi1, James M Brown2, Jayashree Kalpathy-Cramer3, R V Paul Chan4, Susan Ostmo1, Michael F Chiang5, J Peter Campbell6. 1. Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon. 2. Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, Massachusetts; Department of Computer Science, University of Lincoln, Lincoln, United Kingdom. 3. Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, Massachusetts. 4. Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois. 5. Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon; Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon. 6. Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon. Electronic address: campbelp@ohsu.edu.
Abstract
PURPOSE: Retinopathy of prematurity is a leading cause of childhood blindness worldwide, but clinical diagnosis is subjective, which leads to treatment differences. Our goal was to determine objective differences in the diagnosis of plus disease between clinicians using an automated retinopathy of prematurity (ROP) vascular severity score. DESIGN: This retrospective cohort study used data from the Imaging and Informatics in ROP Consortium, which comprises 8 tertiary care centers in North America. Fundus photographs of all infants undergoing ROP screening examinations between July 1, 2011, and December 31, 2016, were obtained. PARTICIPANTS: Infants meeting ROP screening criteria who were diagnosed with plus disease and treatment initiated by an examining physician based on ophthalmoscopic examination results. METHODS: An ROP severity score (1-9) was generated for each image using a deep learning (DL) algorithm. MAIN OUTCOME MEASURES: The mean, median, and range of ROP vascular severity scores overall and for each examiner when the diagnosis of plus disease was made. RESULTS: A total of 5255 clinical examinations in 871 babies were analyzed. Of these, 168 eyes were diagnosed with plus disease by 11 different examiners and were included in the study. The mean ± standard deviation vascular severity score for patients diagnosed with plus disease was 7.4 ± 1.9, median was 8.5 (interquartile range, 5.8-8.9), and range was 1.1 to 9.0. Within some examiners, variability in the level of vascular severity diagnosed as plus disease was present, and 1 examiner routinely diagnosed plus disease in patients with less severe disease than the other examiners (P < 0.01). CONCLUSIONS: We observed variability both between and within examiners in the diagnosis of plus disease using DL. Prospective evaluation of clinical trial data using an objective measurement of vascular severity may help to define better the minimum necessary level of vascular severity for the diagnosis of plus disease or how other clinical features such as zone, stage, and extent of peripheral disease ought to be incorporated in treatment decisions.
PURPOSE:Retinopathy of prematurity is a leading cause of childhood blindness worldwide, but clinical diagnosis is subjective, which leads to treatment differences. Our goal was to determine objective differences in the diagnosis of plus disease between clinicians using an automated retinopathy of prematurity (ROP) vascular severity score. DESIGN: This retrospective cohort study used data from the Imaging and Informatics in ROP Consortium, which comprises 8 tertiary care centers in North America. Fundus photographs of all infants undergoing ROP screening examinations between July 1, 2011, and December 31, 2016, were obtained. PARTICIPANTS: Infants meeting ROP screening criteria who were diagnosed with plus disease and treatment initiated by an examining physician based on ophthalmoscopic examination results. METHODS: An ROP severity score (1-9) was generated for each image using a deep learning (DL) algorithm. MAIN OUTCOME MEASURES: The mean, median, and range of ROP vascular severity scores overall and for each examiner when the diagnosis of plus disease was made. RESULTS: A total of 5255 clinical examinations in 871 babies were analyzed. Of these, 168 eyes were diagnosed with plus disease by 11 different examiners and were included in the study. The mean ± standard deviation vascular severity score for patients diagnosed with plus disease was 7.4 ± 1.9, median was 8.5 (interquartile range, 5.8-8.9), and range was 1.1 to 9.0. Within some examiners, variability in the level of vascular severity diagnosed as plus disease was present, and 1 examiner routinely diagnosed plus disease in patients with less severe disease than the other examiners (P < 0.01). CONCLUSIONS: We observed variability both between and within examiners in the diagnosis of plus disease using DL. Prospective evaluation of clinical trial data using an objective measurement of vascular severity may help to define better the minimum necessary level of vascular severity for the diagnosis of plus disease or how other clinical features such as zone, stage, and extent of peripheral disease ought to be incorporated in treatment decisions.
Authors: J Peter Campbell; Esra Ataer-Cansizoglu; Veronica Bolon-Canedo; Alican Bozkurt; Deniz Erdogmus; Jayashree Kalpathy-Cramer; Samir N Patel; James D Reynolds; Jason Horowitz; Kelly Hutcheson; Michael Shapiro; Michael X Repka; Phillip Ferrone; Kimberly Drenser; Maria Ana Martinez-Castellanos; Susan Ostmo; Karyn Jonas; R V Paul Chan; Michael F Chiang Journal: JAMA Ophthalmol Date: 2016-06-01 Impact factor: 7.389
Authors: J Peter Campbell; Michael C Ryan; Emily Lore; Peng Tian; Susan Ostmo; Karyn Jonas; R V Paul Chan; Michael F Chiang Journal: Ophthalmology Date: 2016-05-27 Impact factor: 12.079
Authors: Travis K Redd; John Peter Campbell; James M Brown; Sang Jin Kim; Susan Ostmo; Robison Vernon Paul Chan; Jennifer Dy; Deniz Erdogmus; Stratis Ioannidis; Jayashree Kalpathy-Cramer; Michael F Chiang Journal: Br J Ophthalmol Date: 2018-11-23 Impact factor: 4.638
Authors: J Peter Campbell; Jayashree Kalpathy-Cramer; Deniz Erdogmus; Peng Tian; Dharanish Kedarisetti; Chace Moleta; James D Reynolds; Kelly Hutcheson; Michael J Shapiro; Michael X Repka; Philip Ferrone; Kimberly Drenser; Jason Horowitz; Kemal Sonmez; Ryan Swan; Susan Ostmo; Karyn E Jonas; R V Paul Chan; Michael F Chiang Journal: Ophthalmology Date: 2016-08-31 Impact factor: 12.079
Authors: James D Reynolds; Velma Dobson; Graham E Quinn; Alistair R Fielder; Earl A Palmer; Richard A Saunders; Robert J Hardy; Dale L Phelps; John D Baker; Michael T Trese; David Schaffer; Betty Tung Journal: Arch Ophthalmol Date: 2002-11
Authors: Stanford Taylor; James M Brown; Kishan Gupta; J Peter Campbell; Susan Ostmo; R V Paul Chan; Jennifer Dy; Deniz Erdogmus; Stratis Ioannidis; Sang J Kim; Jayashree Kalpathy-Cramer; Michael F Chiang Journal: JAMA Ophthalmol Date: 2019-07-03 Impact factor: 7.389
Authors: Adam Hanif; Charles Lu; Ken Chang; Praveer Singh; Aaron S Coyner; James M Brown; Susan Ostmo; Robison V Paul Chan; Daniel Rubin; Michael F Chiang; Jayashree Kalpathy-Cramer; John Peter Campbell Journal: Ophthalmol Retina Date: 2022-03-16
Authors: J Peter Campbell; Michael F Chiang; Jimmy S Chen; Darius M Moshfeghi; Eric Nudleman; Paisan Ruambivoonsuk; Hunter Cherwek; Carol Y Cheung; Praveer Singh; Jayashree Kalpathy-Cramer; Susan Ostmo; Malvina Eydelman; R V Paul Chan; Antonio Capone Journal: Ophthalmology Date: 2022-02-12 Impact factor: 14.277
Authors: J Peter Campbell; Praveer Singh; Travis K Redd; James M Brown; Parag K Shah; Prema Subramanian; Renu Rajan; Nita Valikodath; Emily Cole; Susan Ostmo; R V Paul Chan; Narendran Venkatapathy; Michael F Chiang; Jayashree Kalpathy-Cramer Journal: Pediatrics Date: 2021-03 Impact factor: 7.124