Qualitative and quantitative assessment of posterior segment optical coherence tomography images using standard photos: the Liwan Eye Study.

BACKGROUND/AIMS: To develop a standardised grading scheme, using standard photos, for spectral-domain ocular coherence tomography (SD-OCT) images of the posterior eye and evaluate the interobserver agreement among trained ophthalmologists in identifying pathological changes.
METHODS: Subjects were recruited from Liwan District, Guangzhou, with SD-OCT data collection from June 2013 to November 2013 as part of 10-year follow-up visits from the Liwan Eye Study. All subjects underwent SD-OCT imaging of the macula with scanning lines analysed by two ophthalmologists to assess for the presence of 12 different posterior segment lesions. Per cent agreement for each lesion between the graders and quantitative measures of dome-shaped macula (DSM) height and choroidal thickness were calculated.
RESULTS: A total of 679 SD-OCT images from 679 subjects were independently evaluated by the two graders. Each of the 12 lesions was successfully graded as present or absent in over 96% of images. For all lesions, per cent agreement between observers was over 90%, ranging from 90.7% for epiretinal membranes and retinal pigment epithelium thickenings to 99.7% for full thickness macular holes and retinal detachments. Quantitative measurements of DSM height and choroidal thickness at three locations of the eye all exhibited intraclass correlation scores between the two graders of greater than 0.9.
CONCLUSION: Our study demonstrates high concordance between graders in characterising posterior segment lesions using SD-OCT images, validating the continued use of this imaging modality in the diagnosis of posterior eye disease. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

The study proposes and validates a set of standardised criteria for grading posterior segment spectral-domain ocular coherence tomography images in accordance with International Nomenclature for Optical Coherence Tomography consensus anatomic landmarks.

The study assessed for 12 different pathological changes of the vitreoretinal interface, neurosensory retina and retinal pigment epithelium/Bruch’s complex and presents standard photos for each.

The study reports interobserver variability between trained ophthalmologists in grading posterior segment lesions, analogous to how these images are interpreted in most medical settings.

Ocular coherence tomography images were obtained from a population-based study, resulting in low prevalence for several of the types of lesions.

Introduction

First introduced in 1991, optical coherence tomography (OCT) is a non-invasive imaging modality based on interferometry, able to generate real-time cross-sectional maps of the retina.1 Analogous to B-scan ultrasound, OCT divides light from a broadband light source into sample and reference beams and measures the interference pattern following backscatter from the back of the eye,2 resulting in a reflectivity versus depth profile of the retina useful in diagnostic decisions. Recently, widespread adoption of spectral-domain OCT (SD-OCT) technology has effectively replaced previous slower time-domain OCT devices, allowing for image acquisition rates of up to 29 000 axial scans per second with a 6 µm resolution or 14 500 scans per second with 3.5 µm resolution.3 With the advent of swept-source OCT using longer bandwidth light sources and higher resolution photodetectors, these numbers will only continue to improve.

Natural variation in reflectivity of normal retinal tissue underlies the basis of OCT to resolve the multiple layers comprising the vitreoretinal interface, neurosensory retina, retinal pigment epithelium (RPE)/Bruch’s complex and inner choroid. While nerve fibres and the RPE demonstrate high reflectivity, plexiform and nuclear layers show medium reflectivity, and photoreceptors display low reflectivity.4 5 In contrast to many subfields of medicine in which a tissue in question can readily be biopsied, ophthalmology and particularly the study of the retina are disproportionately dependent on imaging. Consequently, in clinical practice, OCT has become a routine and invaluable test to identify an extensive array of lesions ranging from epiretinal membranes (ERMs) and macular holes to pathological changes in the RPE.

OCT reading guidelines to date have been proposed for a number of conditions including glaucoma, diabetic maculopathy and age-related macular degeneration,6–8 but a comprehensive OCT classification scheme for the posterior segment changes has not yet been developed. In 2015, Heng et al 9 demonstrated a high percentage concordance between two retinal specialists in evaluating features of diabetic macular oedema. Similarly, in 2016, Kim et al 7 quantified the intergrader agreement among glaucoma specialists of using OCT images in the structural diagnosis of glaucoma. However, for many ocular lesions detectable on OCT, the reliability in grading among the specialists who diagnose them has not yet been characterised.

The purpose of this study was to develop a standardised grading scheme, using standard photos, for SD-OCT images of the posterior eye and to evaluate the interobserver agreement among trained ophthalmologists in identifying pathological changes. Although the applications of OCT in managing ocular disease have been widely celebrated, only recently has consensus terminology for normal posterior segment SD-OCT imaging been proposed,10 and little has been reported on the interobserver variability in assessing posterior segment disease by SD-OCT. Here, we define and investigate the grading of 12 different posterior eye lesions by SD-OCT to determine areas of agreement among the images collected in the Liwan Eye Study, a population-based study.

Methods

Subjects

All subjects were enrolled from Liwan District, Guangzhou, with OCT data collection from June 2013 to November 2013 as part of 10-year follow-up visits from the Liwan Eye Study. The methodology of Liwan Eye Study has been described in detail elsewhere.11 In short, all subjects aged 50 years and older residing in the selected study clusters in Liwan District of Guangzhou for more than 6 months were enrolled in 2003. Written informed consent was obtained from all subjects after explaining the purpose of the study and the risks and benefits of the examination. From these subjects, those who underwent SD-OCT imaging between June 2013 and November 2013 were then included in the present analysis. This research adhered to the Declaration of Helsinki.

Image acquisition

SD-OCT imaging of the macula was performed by a trained technician using high-definition OCT (Model Ivue100, Optovue, Fremont, California, USA) with a retina map scanning protocol. Pupils were not dilated for OCT scannings. Images with a quality score less than 30 as self-evaluated by the OCT were retaken. If low quality of the image persisted despite multiple retakes, the image was excluded from grading. For each graded eye, all scanning lines of the macular were analysed to assess for the presence of lesions. Images of the left eye were taken only if the examination of the right eye was not possible. Database entries were managed with EpiData V.3.0 or greater (EpiData Association, Odense, Denmark).

Classification of lesions

The layers of the macula in physiological conditions were identified on the basis of the classification system proposed by the International Nomenclature for Optical Coherence Tomography (IN•OCT) Panel, which that subdivides the retina into 18 layers progressing from the vitreoretinal interface towards the choroid–scleral interface.10 For each OCT scan, the vitreoretinal interface, neurosensory retina (layers 1–13), RPE/Bruch’s membrane complex (layer 14) and choroid (layers 15–18) were examined. A list of pathological lesions on SD-OCT proposed in this study are presented in table 1. Additional grading details and standard photographs for each lesion are presented in online supplementary table 1 and figures 1–3.

Table 1

Definitions of assessed SD-OCT lesions

Lesion	Definition
Vitreoretinal interface
Vitreomacular adhesion	Elevation of the vitreous body above the macular area with remaining attachment of the vitreous to the fovea21 (figure 1A).
Vitreomacular traction	Vitreofoveal adhesion, distortion of the foveal surface and elevation of the foveal floor21 (figure 1B).
Epiretinal membrane	Highly reflective layer on or above the inner surface of the retina22 23 (figure 1C).
Neurosensory retina (layers 1–13)
Full thickness macular hole	Defect of the neurosensory retina affecting all layers from the internal limiting membrane to the ellipsoid zone24 (figure 2A).
Lamellar macular hole	Irregular fovea with partial loss of the inner layers of retina but with intact photoreceptor layer and RPE/Bruch’s complex25 (figure 2B).
Macular retinoschisis	Single or multiple separations within neurosensory retina, either between different layers or within one neurosensory layer26 (figure 2C).
Retinal detachment	Separation of neurosensory retina from the RPE27 28 (figure 2D).
Macular oedema	(1) Increased macular thickness throughout all layers of the retina, (2) focal thickening with or without intraretinal cyst formation or (3) thickness exceeding >300 µm on any sector of the macular tomographic map29 30 (figure 2E,F).
Intraretinal hyper-reflective lesion	Hyper-reflective area in the neurosensory retina with or without shadowing effect on the underlying outer retina and choroid31 (figure 2G).
RPE/Bruch’s complex (layer 14)
RPE thickening	Regions of increased thickness in the RPE that tend to be localised and irregular32 (figure 3A).
Pigment epithelium detachment	Separation of the RPE from Bruch’s membrane, with accumulation of sub-RPE fluid, blood, fibrovascular tissue or lipoprotein-derived debris (figure 3B).
Dome-shaped macula	Inward bulge of the RPE layer resulting in a ‘dome-like’ protrusion >50 µm from peak to base above a presumed line tangent to the outer surface of the RPE33 34 (figure 3C).

RPE, retinal pigment epithelium; SD-OCT, spectral-domain optical coherence tomography.

Optical coherence tomography photographs from the Liwan Eye Study showing examples of (A) vitreomacular adhesion, (B) vitreomacular traction and (C) epiretinal membrane.

Optical coherence tomography photographs from the Liwan Eye Study showing examples of (A) full thickness macular hole, (B) lamellar macular hole, (C) macular retinoschisis, (D) retinal detachment, (E) macular oedema and (G) intraretinal hyper-reflective lesion. Figure part F depicts the macular tomographic map for the image displayed in figure part E featuring regions with retinal thickness exceeding 300 µm.

Optical coherence tomography photographs from the Liwan Eye Study showing examples of (A) retinal pigment epithelium thickening, (B) pigment epithelium detachment and (C) dome-shaped macula.

Grading of lesions

Grading criteria and a set of standard photos for all OCT lesions assessed in this study are detailed in online supplementary materials. Briefly, lesions on each image were provided without other clinical information and were independently graded by trained observers into three mutually exclusive categories based on presence or absence of the lesion: ‘None’ (greater than 50% certainty that a lesion is absent), ‘Yes’ (greater than 50% certainty that a lesion if present) or ‘Cannot grade’ (poor image quality or image obscured by mixture of lesions present). Images from each subject were also graded on the ability to measure dome-shaped macula (DSM) height if such a lesion was present and choroidal thickness at three locations along the horizontal scan.

Two ophthalmologists served as observers to grade the OCT images. For qualitative OCT evaluations, interobserver agreement (percent of graded cases agreed between two assessments by different graders) was determined as a measure of inter-rater reliability. For quantitative measures of DSM height and choroidal thickness, descriptive statistics using the Student’s t-test and intraclass correlation (ICC) scores were also calculated. ICC was determined with Stata V.12.0 using a two-way random effects model. For statistical analyses, a P value less than 0.05 was deemed significant.

Results

A total of 679 OCT images from 679 subjects (379 women and 300 men; mean age 70.8±8.1 years) were independently graded by two trained observers. On average, grading required approximately 5 min per image. Each lesion from table 1 was successfully scored as present or absent in over 96% of images with successful grading of all images for retinal detachments (RDs), intraretinal hyper-reflective lesions, pigment epithelium detachments (PEDs) and DSMs.

Table 2 presents the number of cases graded as present or absent by both observers for each lesion, as well as the interobserver agreement for each type of lesion assessed. For all 12 lesions in this study, per cent agreement between observers was over 90%, ranging from 90.7% for ERMs and RPE thickenings to 99.7% for full thickness macular holes (FTMHs) and RDs.

Table 2

Interobserver grading agreement on qualitative optical coherence tomography evaluations

Variable	Sample size	Graded (%)	Agreement
			Present	Absent	% Agreement
Vitreoretinal interface
Vitreomacular adhesion	679	673 (99.1)	66	571	94.7
Vitreomacular traction	679	677 (99.7)	0	666	98.4
Epiretinal membrane	679	677 (99.7)	66	548	90.7
Neurosensory retina
Full thickness macular holes	679	678 (99.9)	1	676	99.7
Lamellar macular hole	679	677 (99.7)	5	666	99.1
Macular retinoschisis	679	676 (99.6)	7	651	97.3
Retinal detachment	679	679 (100)	1	676	99.7
Macular oedema	679	654 (96.3)	5	621	95.7
Intraretinal hyper-reflective lesion	677	677 (100)	8	654	97.8
RPE/Bruch’s complex
RPE thickening	679	677 (99.7)	29	585	90.7
Pigment epithelium detachment	678	678 (100)	3	662	98.1
Dome-shaped macula	679	678 (100)	6	656	97.6

RPE, retinal pigment epithelium.

Quantitative evaluations of DSM lesion height and choroidal thickness at three locations of the eye are shown in table 3. For all quantitative evaluations, the ICC score between the two graders was greater than 0.9, indicating high reliability between graders for these measurements.

Table 3

Interobserver grading agreement on quantitative OCT evaluations

Variable	Statistic	Grader 1	Grader 2	Paired difference	P value
DSM height (μm)	N			6
	Mean (SD)	123.8 (71.1)	123.7 (71.3)	0.17 (0.41)	0.363
	Min, median, max	57, 101.5, 215	56, 101.5, 215	0, 0, 1
	ICC (95% CI)			1.00 (1.00 to 1.00)
Subfoveal choroidal thickness (μm)	N			548
	Mean (SD)	174.4 (69.9)	168.6 (68.3)	5.74 (25.89)	<0.001
	Min, median, max	42, 172.5, 371	42, 166, 400	−198, 0, 201
	ICC (95% CI)			0.93 (0.91 to 0.94)
Choroidal thickness at nasal 1 mm (μm)	N			548
	Mean (SD)	156.2 (67.8)	151.9 (65.6)	4.29 (20.88)	<0.001
	Min, median, max	21, 154, 371	21, 150, 347	−178, 0, 204
	ICC (95% CI)			0.95 (0.94 to 0.96)
Choroidal thickness at temporal 1 mm (μm)	N			548
	Mean (SD)	174.7 (65.8)	167.3 (64.9)	7.43 (27.85)
	Min, median, max	44, 175, 362	44, 166.5, 418	−210, 0, 187	<0.001
	ICC (95% CI)			0.90 (0.89 to 0.92)

DSM, dome-shaped macula; ICC, intraclass correlation; OCT, optical coherence tomography.

Discussion

Based on recent definitions proposed for normal anatomic landmarks by the IN•OCT Panel,10 we defined and compared the scoring of 12 types of posterior eye lesions collected from a population of 679 subjects to evaluate the reliability of SD-OCT grading between clinicians. The highest per cent agreement scores were observed in the grading of FTMHs, lamellar macular holes and RDs, all of which are routinely diagnosed and monitored with the assistance of OCT in practice.12–14 Quantitative OCT evaluations also demonstrated a small but statistically significant difference in measurements between graders for DSM height and choroid thickness at nasal, temporal and subfoveal locations. The ability to accurately and non-invasively measure these parameters with OCT is important for both detecting new lesions and monitoring existing ones. In particular, SD-OCT may be of value to track conditions with associated choroidal thickness changes such as pachychoroid neovasculopathy and neovascular age-related macular degeneration.15 16 However, the potential for small discrepancies between independent graders measuring choroidal thickness with this modality should be noted.

From our analyses, the lowest per cent agreement scores were seen with assessments of macular oedema (MO), vitreomacular adhesions, ERMs and RPE thickening. The diversity of disease states leading to MO make it a challenging condition to diagnose, with prior attempts at characterising MO features by SD-OCT exhibiting good but imperfect agreement between independent image readers.9 17 Likewise, several forms of RPE thickening have been reported in the literature, yet the reliability among clinicians in using SD-OCT to identify this type of lesion has not previously been examined. For RPE thickening in particular, our grading criteria were based on examples from the literature of RPE proliferation, hypertrophy, pigmentation and migration changes, all of which notably lack a quantitative definition.18 19 The more subjective definition of RPE thickening on OCT may have made grading this lesion more difficult and may help explain its lower per cent agreement scores relative to other lesions in our study population. Further research should attempt to more precisely define criteria for these lesions to determine the reliability of these diagnoses from SD-OCT with greater confidence. Nonetheless, none of the lesions examined in this study showed per cent agreement between the graders of less than 90%, underscoring the acceptable clinical utility of OCT for evaluating these changes in the eye.

Limitations of this study include the low prevalence in the enrolled population of FTMH, RD, PED and vitreomacular traction (VMT). VMT was not graded as present in any of the 679 subjects, dampening any conclusions that we can draw about this lesion. Limitations regarding the low prevalence of these lesions are to be expected as subjects were drawn from a population-based study. Given the growing adoption of SD-OCT in ophthalmic clinics and its increasing role in patient care, it is important that future studies verify the reliability of OCT image interpretation in daily clinical practice to avoid potential missed or erroneous diagnoses. Fortunately, in practice, the identification of these lesions is complemented by a range of imaging and diagnostic tools including fluorescein angiography, fundus autofluorescence and B-scan ultrasound.20 The collective findings from these tests would inform the results of SD-OCT, improving diagnostic accuracy and the possibility to intervene with appropriate treatment.

In summary, we proposed a standardised grading scheme for 12 different ocular pathological changes of the vitreoretinal interface, neurosensory retina and RPE/Bruch’s complex by SD-OCT using recent IN•OCT consensus anatomic landmarks and assessed the agreement between trained clinicians in identifying these lesions. Our study demonstrates high concordance between graders in characterising SD-OCT images of these regions, providing validation for the continued use of SD-OCT as an ancillary test in the diagnosis of posterior eye pathology.