J Peter Campbell1, Michael C Ryan1, Emily Lore2, Peng Tian3, Susan Ostmo1, Karyn Jonas4, R V Paul Chan4, Michael F Chiang5. 1. Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon. 2. Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon. 3. Cognitive Systems Laboratory, Northeastern University, Boston, Massachusetts. 4. Department of Ophthalmology and Visual Sciences, Illinois Eye and Ear Infirmary, University of Illinois at Chicago, Chicago, Illinois. 5. Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon; Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon. Electronic address: chiangm@ohsu.edu.
Abstract
PURPOSE: To identify the most common areas for discrepancy in retinopathy of prematurity (ROP) classification between experts. DESIGN: Prospective cohort study. PARTICIPANTS: A total of 281 infants were identified as part of a multicenter, prospective, ROP cohort study from 7 participating centers. Each site had participating ophthalmologists who provided the clinical classification after routine examination using binocular indirect ophthalmoscopy (BIO) and obtained wide-angle retinal images, which were independently classified by 2 study experts. METHODS: Wide-angle retinal images (RetCam; Clarity Medical Systems, Pleasanton, CA) were obtained from study subjects, and 2 experts evaluated each image using a secure web-based module. Image-based classifications for zone, stage, plus disease, and overall disease category (no ROP, mild ROP, type II or pre-plus, and type I) were compared between the 2 experts and with the clinical classification obtained by BIO. MAIN OUTCOME MEASURES: Inter-expert image-based agreement and image-based versus ophthalmoscopic diagnostic agreement using absolute agreement and weighted kappa statistic. RESULTS: A total of 1553 study eye examinations from 281 infants were included in the study. Experts disagreed on the stage classification in 620 of 1553 comparisons (40%), plus disease classification (including pre-plus) in 287 of 1553 comparisons (18%), zone in 117 of 1553 comparisons (8%), and overall ROP category in 618 of 1553 comparisons (40%). However, agreement for presence versus absence of type 1 disease was >95%. There were no differences between image-based and clinical classification except for zone III disease. CONCLUSIONS: The most common area of discrepancy in ROP classification is stage, although inter-expert agreement for clinically significant disease, such as presence versus absence of type 1 and type 2 disease, is high. There were no differences between image-based grading and clinical examination in the ability to detect clinically significant disease. This study provides additional evidence that image-based classification of ROP reliably detects clinically significant levels of ROP with high accuracy compared with the clinical examination.
PURPOSE: To identify the most common areas for discrepancy in retinopathy of prematurity (ROP) classification between experts. DESIGN: Prospective cohort study. PARTICIPANTS: A total of 281 infants were identified as part of a multicenter, prospective, ROP cohort study from 7 participating centers. Each site had participating ophthalmologists who provided the clinical classification after routine examination using binocular indirect ophthalmoscopy (BIO) and obtained wide-angle retinal images, which were independently classified by 2 study experts. METHODS: Wide-angle retinal images (RetCam; Clarity Medical Systems, Pleasanton, CA) were obtained from study subjects, and 2 experts evaluated each image using a secure web-based module. Image-based classifications for zone, stage, plus disease, and overall disease category (no ROP, mild ROP, type II or pre-plus, and type I) were compared between the 2 experts and with the clinical classification obtained by BIO. MAIN OUTCOME MEASURES: Inter-expert image-based agreement and image-based versus ophthalmoscopic diagnostic agreement using absolute agreement and weighted kappa statistic. RESULTS: A total of 1553 study eye examinations from 281 infants were included in the study. Experts disagreed on the stage classification in 620 of 1553 comparisons (40%), plus disease classification (including pre-plus) in 287 of 1553 comparisons (18%), zone in 117 of 1553 comparisons (8%), and overall ROP category in 618 of 1553 comparisons (40%). However, agreement for presence versus absence of type 1 disease was >95%. There were no differences between image-based and clinical classification except for zone III disease. CONCLUSIONS: The most common area of discrepancy in ROP classification is stage, although inter-expert agreement for clinically significant disease, such as presence versus absence of type 1 and type 2 disease, is high. There were no differences between image-based grading and clinical examination in the ability to detect clinically significant disease. This study provides additional evidence that image-based classification of ROP reliably detects clinically significant levels of ROP with high accuracy compared with the clinical examination.
Authors: Susan K Gelman; Rony Gelman; Alison B Callahan; M Elena Martinez-Perez; Daniel S Casper; John T Flynn; Michael F Chiang Journal: Arch Ophthalmol Date: 2010-09
Authors: J Peter Campbell; Esra Ataer-Cansizoglu; Veronica Bolon-Canedo; Alican Bozkurt; Deniz Erdogmus; Jayashree Kalpathy-Cramer; Samir N Patel; James D Reynolds; Jason Horowitz; Kelly Hutcheson; Michael Shapiro; Michael X Repka; Phillip Ferrone; Kimberly Drenser; Maria Ana Martinez-Castellanos; Susan Ostmo; Karyn Jonas; R V Paul Chan; Michael F Chiang Journal: JAMA Ophthalmol Date: 2016-06-01 Impact factor: 7.389
Authors: Jane S Myung; Robison Vernon Paul Chan; Michael J Espiritu; Steven L Williams; David B Granet; Thomas C Lee; David J Weissgold; Michael F Chiang Journal: J AAPOS Date: 2011-12 Impact factor: 1.220
Authors: James D Reynolds; Velma Dobson; Graham E Quinn; Alistair R Fielder; Earl A Palmer; Richard A Saunders; Robert J Hardy; Dale L Phelps; John D Baker; Michael T Trese; David Schaffer; Betty Tung Journal: Arch Ophthalmol Date: 2002-11
Authors: Michael A Klufas; Samir N Patel; Michael C Ryan; Mrinali Patel Gupta; Karyn E Jonas; Susan Ostmo; Maria Ana Martinez-Castellanos; Audina M Berrocal; Michael F Chiang; R V Paul Chan Journal: Ophthalmology Date: 2015-05-28 Impact factor: 12.079
Authors: R V Paul Chan; Steven L Williams; Yoshihiro Yonekawa; David J Weissgold; Thomas C Lee; Michael F Chiang Journal: Retina Date: 2010-06 Impact factor: 4.256
Authors: Esra Ataer-Cansizoglu; Veronica Bolon-Canedo; J Peter Campbell; Alican Bozkurt; Deniz Erdogmus; Jayashree Kalpathy-Cramer; Samir Patel; Karyn Jonas; R V Paul Chan; Susan Ostmo; Michael F Chiang Journal: Transl Vis Sci Technol Date: 2015-11-30 Impact factor: 3.283
Authors: Chace Moleta; J Peter Campbell; Jayashree Kalpathy-Cramer; R V Paul Chan; Susan Ostmo; Karyn Jonas; Michael F Chiang Journal: Am J Ophthalmol Date: 2017-01-11 Impact factor: 5.258
Authors: Hilal Biten; Travis K Redd; Chace Moleta; J Peter Campbell; Susan Ostmo; Karyn Jonas; R V Paul Chan; Michael F Chiang Journal: JAMA Ophthalmol Date: 2018-05-01 Impact factor: 7.389
Authors: Layla Ghergherehchi; Sang Jin Kim; J Peter Campbell; Susan Ostmo; R V Paul Chan; Michael F Chiang Journal: Asia Pac J Ophthalmol (Phila) Date: 2018-05-24
Authors: Rene Y Choi; James M Brown; Jayashree Kalpathy-Cramer; R V Paul Chan; Susan Ostmo; Michael F Chiang; J Peter Campbell Journal: Ophthalmol Retina Date: 2020-05-04
Authors: J Peter Campbell; Jayashree Kalpathy-Cramer; Deniz Erdogmus; Peng Tian; Dharanish Kedarisetti; Chace Moleta; James D Reynolds; Kelly Hutcheson; Michael J Shapiro; Michael X Repka; Philip Ferrone; Kimberly Drenser; Jason Horowitz; Kemal Sonmez; Ryan Swan; Susan Ostmo; Karyn E Jonas; R V Paul Chan; Michael F Chiang Journal: Ophthalmology Date: 2016-08-31 Impact factor: 12.079
Authors: Sang Jin Kim; Alexander D Port; Ryan Swan; J Peter Campbell; R V Paul Chan; Michael F Chiang Journal: Surv Ophthalmol Date: 2018-04-19 Impact factor: 6.048
Authors: Shelbi L Olson; Tsengelmaa Chuluunbat; Emily D Cole; Karyn E Jonas; Munkhuu Bayalag; Chimgee Chuluunkhuu; Nita G Valikodath; D Hunter Cherwek; Nathan Congdon; Leslie D MacKeen; Joelle Hallak; Vivien Yap; Susan Ostmo; Wei Chi Wu; J Peter Campbell; Michael F Chiang; R V Paul Chan Journal: J Pediatr Ophthalmol Strabismus Date: 2020-09-01 Impact factor: 1.402
Authors: Kellyn N Bellsmith; James Brown; Sang Jin Kim; Isaac H Goldstein; Aaron Coyner; Susan Ostmo; Kishan Gupta; R V Paul Chan; Jayashree Kalpathy-Cramer; Michael F Chiang; J Peter Campbell Journal: Ophthalmology Date: 2020-02-07 Impact factor: 12.079