Shruti Chandra1,2, Rajna Rasheed1, Piyali Sen1,2, Deepthy Menon1, Sobha Sivaprasad3,4. 1. National Institute of Health Research Moorfields Biomedical Research Centre, Moorfields Eye Hospital NHS Foundation Trust, London, UK. 2. University College London, Institute of Ophthalmology, London, UK. 3. National Institute of Health Research Moorfields Biomedical Research Centre, Moorfields Eye Hospital NHS Foundation Trust, London, UK. sobha.sivaprasad@nhs.net. 4. University College London, Institute of Ophthalmology, London, UK. sobha.sivaprasad@nhs.net.
Abstract
PURPOSE: To evaluate the inter-rater reliability for identification of complete retinal pigment epithelium and outer retinal atrophy (cRORA) on SD-OCT images as defined by the Classification of Atrophy Meetings (CAM) group. METHODS: Fifty images of anonymized SD-OCT line scans of eyes with cRORA due to AMD were selected. Each .tiff image was saved in both black-on-white (BW) and white-on-black (WB) format. Five retina-trained clinicians graded both sets of images twice for the diagnosis of cRORA based on the CAM group definition. Fleiss kappa statistic was calculated for inter-rater reliability and Cohen's kappa statistic for intra-grader and inter-grader reliability between any two graders. RESULTS: The inter-grader reliability varied from as low as 0.28 to 0.92 for WB images and 0.34 to 0.86 for BW images. However, the inter-grader and intra-grader agreement was ĸ WB 0.92; ĸ BW 0.86 and ĸ 0.92 respectively, for graders accustomed to the CAM criteria. Fleiss kappa was ĸ 0.49 (p value < 0.0001) for WB images and ĸ 0.34 (p value < 0.0001 for BW images. Overall, the agreement was better using WB images for all parameters except RPE attenuation/loss. CONCLUSION: There is significant variability in diagnosis of cRORA on SD-OCT by retina-trained ophthalmologists in the real world. The study highlights the need for training to recognise the different features of cRORA prior to its implementation in clinical practice.
PURPOSE: To evaluate the inter-rater reliability for identification of complete retinal pigment epithelium and outer retinal atrophy (cRORA) on SD-OCT images as defined by the Classification of Atrophy Meetings (CAM) group. METHODS: Fifty images of anonymized SD-OCT line scans of eyes with cRORA due to AMD were selected. Each .tiff image was saved in both black-on-white (BW) and white-on-black (WB) format. Five retina-trained clinicians graded both sets of images twice for the diagnosis of cRORA based on the CAM group definition. Fleiss kappa statistic was calculated for inter-rater reliability and Cohen's kappa statistic for intra-grader and inter-grader reliability between any two graders. RESULTS: The inter-grader reliability varied from as low as 0.28 to 0.92 for WB images and 0.34 to 0.86 for BW images. However, the inter-grader and intra-grader agreement was ĸ WB 0.92; ĸ BW 0.86 and ĸ 0.92 respectively, for graders accustomed to the CAM criteria. Fleiss kappa was ĸ 0.49 (p value < 0.0001) for WB images and ĸ 0.34 (p value < 0.0001 for BW images. Overall, the agreement was better using WB images for all parameters except RPE attenuation/loss. CONCLUSION: There is significant variability in diagnosis of cRORA on SD-OCT by retina-trained ophthalmologists in the real world. The study highlights the need for training to recognise the different features of cRORA prior to its implementation in clinical practice.