Chace Moleta1, J Peter Campbell1, Jayashree Kalpathy-Cramer2, R V Paul Chan3, Susan Ostmo1, Karyn Jonas3, Michael F Chiang4. 1. Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon. 2. Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology, Massachusetts General Hospital, Charlestown, Massachusetts. 3. Department of Ophthalmology, University of Illinois-Chicago, Chicago, Illinois. 4. Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon; Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon. Electronic address: chiangm@ohsu.edu.
Abstract
PURPOSE: To identify any temporal trends in the diagnosis of plus disease in retinopathy of prematurity (ROP) by experts. DESIGN: Reliability analysis. METHODS: ROP experts were recruited in 2007 and 2016 to classify 34 wide-field fundus images of ROP as plus, pre-plus, or normal, coded as "3," "2," and "1," respectively, in the database. The main outcome was the average calculated score for each image in each cohort. Secondary outcomes included correlation on the relative ordering of the images in 2016 vs 2007, interexpert agreement, and intraexpert agreement. RESULTS: The average score for each image was higher for 30 of 34 (88%) images in 2016 compared with 2007, influenced by fewer images classified as normal (P < .01), a similar number of pre-plus (P = .52), and more classified as plus (P < .01). The mean weighted kappa values in 2006 were 0.36 (range 0.21-0.60), compared with 0.22 (range 0-0.40) in 2016. There was good correlation between rankings of disease severity between the 2 cohorts (Spearman rank correlation ρ = 0.94), indicating near-perfect agreement on relative disease severity. CONCLUSIONS: Despite good agreement between cohorts on relative disease severity ranking, the higher average score and classifications for each image demonstrate that experts are diagnosing pre-plus and plus disease at earlier stages of disease severity in 2016, compared with 2007. This has implications for patient care, research, and teaching, and additional studies are needed to better understand this temporal trend in image-based plus disease diagnosis.
PURPOSE: To identify any temporal trends in the diagnosis of plus disease in retinopathy of prematurity (ROP) by experts. DESIGN: Reliability analysis. METHODS: ROP experts were recruited in 2007 and 2016 to classify 34 wide-field fundus images of ROP as plus, pre-plus, or normal, coded as "3," "2," and "1," respectively, in the database. The main outcome was the average calculated score for each image in each cohort. Secondary outcomes included correlation on the relative ordering of the images in 2016 vs 2007, interexpert agreement, and intraexpert agreement. RESULTS: The average score for each image was higher for 30 of 34 (88%) images in 2016 compared with 2007, influenced by fewer images classified as normal (P < .01), a similar number of pre-plus (P = .52), and more classified as plus (P < .01). The mean weighted kappa values in 2006 were 0.36 (range 0.21-0.60), compared with 0.22 (range 0-0.40) in 2016. There was good correlation between rankings of disease severity between the 2 cohorts (Spearman rank correlation ρ = 0.94), indicating near-perfect agreement on relative disease severity. CONCLUSIONS: Despite good agreement between cohorts on relative disease severity ranking, the higher average score and classifications for each image demonstrate that experts are diagnosing pre-plus and plus disease at earlier stages of disease severity in 2016, compared with 2007. This has implications for patient care, research, and teaching, and additional studies are needed to better understand this temporal trend in image-based plus disease diagnosis.
Authors: J Peter Campbell; Esra Ataer-Cansizoglu; Veronica Bolon-Canedo; Alican Bozkurt; Deniz Erdogmus; Jayashree Kalpathy-Cramer; Samir N Patel; James D Reynolds; Jason Horowitz; Kelly Hutcheson; Michael Shapiro; Michael X Repka; Phillip Ferrone; Kimberly Drenser; Maria Ana Martinez-Castellanos; Susan Ostmo; Karyn Jonas; R V Paul Chan; Michael F Chiang Journal: JAMA Ophthalmol Date: 2016-06-01 Impact factor: 7.389
Authors: J Peter Campbell; Michael C Ryan; Emily Lore; Peng Tian; Susan Ostmo; Karyn Jonas; R V Paul Chan; Michael F Chiang Journal: Ophthalmology Date: 2016-05-27 Impact factor: 12.079
Authors: J Peter Campbell; Jayashree Kalpathy-Cramer; Deniz Erdogmus; Peng Tian; Dharanish Kedarisetti; Chace Moleta; James D Reynolds; Kelly Hutcheson; Michael J Shapiro; Michael X Repka; Philip Ferrone; Kimberly Drenser; Jason Horowitz; Kemal Sonmez; Ryan Swan; Susan Ostmo; Karyn E Jonas; R V Paul Chan; Michael F Chiang Journal: Ophthalmology Date: 2016-08-31 Impact factor: 12.079
Authors: R V Paul Chan; Steven L Williams; Yoshihiro Yonekawa; David J Weissgold; Thomas C Lee; Michael F Chiang Journal: Retina Date: 2010-06 Impact factor: 4.256
Authors: Steven L Williams; Lu Wang; Steven A Kane; Thomas C Lee; David J Weissgold; Audina M Berrocal; Daniel Rabinowitz; Justin Starren; John T Flynn; Michael F Chiang Journal: Br J Ophthalmol Date: 2009-12-02 Impact factor: 4.638
Authors: Aaron Nagiel; Michael J Espiritu; Ryan K Wong; Thomas C Lee; Andreas K Lauer; Michael F Chiang; R V Paul Chan Journal: Ophthalmology Date: 2012-12 Impact factor: 12.079
Authors: Mrinali Patel Gupta; R V Paul Chan; Rachelle Anzures; Susan Ostmo; Karyn Jonas; Michael F Chiang Journal: Am J Ophthalmol Date: 2015-12-15 Impact factor: 5.258
Authors: Layla Ghergherehchi; Sang Jin Kim; J Peter Campbell; Susan Ostmo; R V Paul Chan; Michael F Chiang Journal: Asia Pac J Ophthalmol (Phila) Date: 2018-05-24
Authors: Rene Y Choi; James M Brown; Jayashree Kalpathy-Cramer; R V Paul Chan; Susan Ostmo; Michael F Chiang; J Peter Campbell Journal: Ophthalmol Retina Date: 2020-05-04
Authors: Adam Hanif; Charles Lu; Ken Chang; Praveer Singh; Aaron S Coyner; James M Brown; Susan Ostmo; Robison V Paul Chan; Daniel Rubin; Michael F Chiang; Jayashree Kalpathy-Cramer; John Peter Campbell Journal: Ophthalmol Retina Date: 2022-03-16
Authors: J Peter Campbell; Michael F Chiang; Jimmy S Chen; Darius M Moshfeghi; Eric Nudleman; Paisan Ruambivoonsuk; Hunter Cherwek; Carol Y Cheung; Praveer Singh; Jayashree Kalpathy-Cramer; Susan Ostmo; Malvina Eydelman; R V Paul Chan; Antonio Capone Journal: Ophthalmology Date: 2022-02-12 Impact factor: 14.277
Authors: Nita G Valikodath; Emily Cole; Daniel S W Ting; J Peter Campbell; Louis R Pasquale; Michael F Chiang; R V Paul Chan Journal: Transl Vis Sci Technol Date: 2021-06-01 Impact factor: 3.283
Authors: Ryan Swan; Sang Jin Kim; J Peter Campbell; R V Paul Chan; Kemal Sonmez; Kent D Taylor; Xiaohui Li; Yii-Der Ida Chen; Jerome I Rotter; Charles Simmons; Michael F Chiang Journal: Ophthalmol Retina Date: 2018-03-08
Authors: Jeanie L Y Cheong; Joy E Olsen; Li Huang; Kim M Dalziel; Rosemarie A Boland; Alice C Burnett; Anjali Haikerwal; Alicia J Spittle; Gillian Opie; Alice E Stewart; Leah M Hickey; Peter J Anderson; Lex W Doyle Journal: BMJ Open Date: 2020-09-10 Impact factor: 2.692