Agreement Between Spectral-Domain and Swept-Source Optical Coherence Tomography Retinal Thickness Measurements in Macular and Retinal Disease.

INTRODUCTION: To assess inter-device agreement in optical coherence tomography-derived retinal thickness measurements in patients with known macular conditions between spectral-domain and swept-source optical coherence tomography (OCT).
METHODS: Two hundred seventy-two subjects were included in the study. They consisted of 91 male (33.5%) and 181 female (66.5%) subjects, and 132 left (48.5%) and 140 right (51.5%) eyes. Each subject underwent spectral-domain OCT (SD-OCT, Spectralis, Heidelberg Engineering; RTVue XR Avanti XR HD, Optovue) and swept-source OCT (SS-OCT; DRI-OCT-1, Atlantis, Topcon) in a single imaging session performed by the same clinical trial-certified technician. The comparison of retinal thickness reproducibility between devices was performed using Bland-Altman analyses and across the entire data set using the intraclass correlation coefficient (ICC).
RESULTS: The ICC of the retinal thickness measurements (95% confidence interval) made using all three OCT instruments was 0.81 (0.77-0.84). The mean difference in mean retinal thickness between Spectralis SD-OCT and Topcon SS-OCT was 59.1 μm (95% limit of agreement [LoA] -21.7 to 139.8 μm). The mean difference in mean retinal thickness between Optovue SD-OCT and Topcon SS-OCT was 21.8 μm (95% LoA -34.7  to 78.3 μm).
CONCLUSIONS: Retinal layer thickness measurements vary between SS-OCT and SD-OCT devices. We describe inter-device agreement in retinal thickness between SS-OCT and SD-OCT in patients with macular conditions. Clinicians should be aware of the differences in retinal thickness values when imaging patients using different OCT devices and should consider using the same OCT device model in order to monitor clinical change. TRIAL REGISTRATION: ClinicalTrials.gov Identifier (NCT02828215).

Summary Slide

Introduction

Advances in optical coherence tomography (OCT) technology have led to improvements in both image acquisition and analysis [1]. The development of spectral-domain OCT (SD-OCT) has allowed faster imaging, enabling more macular line scans (denser sampling) and re-scan at the same retinal locus, thus improving the signal-to-noise ratio [2]. Swept-source OCT (SS-OCT) uses a tuneable narrow-band light source that emits changing light frequencies measured by a photodetector that can respond to fast temporal variation [3]. Faster image acquisition in SD-OCT and SS-OCT enables line scan averaging and so delivers enhanced visualization of the retina through an improvement in the signal-to-noise ratio of OCT imaging. On-board segmentation algorithms generate automated evaluation of retinal thickness by detection of the inner and outer retinal boundaries [4].

Several diseases affecting the retina, vitreous and choroid are associated with significant sight loss. In England and Wales, age-related macular degeneration (AMD) was responsible for 4788 cases of severe sight impairment registration in 2007–2008 (58.6% of total cases of severe sight impairment registration) [5]. It has been estimated that AMD represented 7% of worldwide blindness, behind both cataract (33%) and uncorrected refractive error (21%), in 2010 [6]. Indeed, cataract, AMD and uncorrected refractive error appeared to be the most prevalent cause of visual loss in a study of high-income countries in 2015 [7]. Analysis in 2013 described the prevalence of early AMD as 8.01% worldwide and projected 288 million cases by 2040 [8]. It is also becoming increasingly evident that alterations in the retina and choroid may be present in forms of neurodegenerative diseases, such as Alzheimer’s disease and multiple sclerosis [9, 10]. Alterations in OCT-derived retinal thickness measurements are used as part of anti-VEGF (anti-vascular endothelial growth factor) intravitreal injection retreatment criteria for a range of macular diseases. It is important to understand possible differences in OCT-derived retinal thickness measurements made with various OCT systems in order to prevent suboptimal treatment decisions based on these OCT metrics. Given the number of commercially available OCT devices on the market, it is important to understand the magnitude of any difference in retinal thickness measurement. This variation is particularly important when patients are imaged on different OCT systems, where differences in retinal thickness measurement could arise from measurement-related factors (relating to the OCT system and segmentation algorithm) rather than true clinical change.

Previous studies have compared retinal thickness measurements made with SD-OCT and time-domain OCT, confirming that although the repeatability within each device was similar, the measurement of retinal thickness differed significantly [11]. Similarly, studies with multiple SD-OCT commercial instruments comparing time-domain OCT confirm that there is a significant difference in absolute retinal thickness measurements across these devices [12]. Furthermore, a previous study that compared SS-OCT and SD-OCT in healthy eyes found a mean difference of 15.6–37 μm in central retinal thickness measurements [13]. However, it has been suggested that the difference between SS-OCT and SD-OCT can be greater in eyes with retinal conditions. Indeed, a comparison of SD-OCT and SS-OCT retinal thickness measurements in neovascular age-related macular degeneration (nAMD) confirmed that this variation was larger, at 97 μm, in the central macular subfield [14]. In this large prospective study, we compare retinal thickness measurements in a range of retinal and macular conditions (including AMD, diabetic retinopathy, retinal dystrophy, posterior uveitis) using SD-OCT (Spectralis, Heidelberg Engineering; Optovue HD-OCT) and SS-OCT (Atlantis DRI-OCT-1, Topcon). All measurements were performed by a clinical trial-certified technician. As OCT usage varies, particularly as forms of tele-ophthalmology and digital imaging services develop, it is essential for clinicians to be aware of the clinical variation associated with the use of different forms of technology.

The aim of this study was to determine retinal thickness in patients with macular disease using both SD-OCT and SS-OCT and to assess the degree of their agreement.

Materials and Methods

This study formed part of the Adaptive Optical Coherence Tomography and Adaptive Optics Imaging in retinal disease (ACAD) Study (NCT02828215). This was a cross-sectional study assessing retinal imaging with retinal conditions. All subjects gave informed written consent prior to study participation. This study was approved by the local ethics committee (NRES South East Coast–Surrey) and adhered to the tenets set forth in the Declaration of Helsinki.

Study Population

All subjects with retinal and macular pathological conditions were identified from medical retinal clinics at Moorfields Eye Hospital (MEH). Subjects were eligible if they were at least 18 years old. All participants were informed of the nature and purpose of the study and provided the required informed consent.

Inclusion criteria consisted of the presence of disease in the vitreous, retina and choroid (involving but not limited to vitreomacular traction, vitreous opacities, epiretinal membrane, nerve fibre loss in glaucoma, proliferative diabetic retinopathy, age-related macular degeneration, inherited retinal dystrophy, chronic central serous chorioretinopathy, intermediate and posterior uveitis, and choroidal tumours). The study eye needed to have evidence of vitreous, retinal or choroidal disease requiring treatment or monitoring at MEH. If both eyes were eligible, the eye to be studied was that with higher visual acuity (VA) or was chosen at random in those with equal VA. Exclusion criteria included media opacity sufficient to prevent adequate fundoscopy and OCT imaging. Other exclusion criteria included a history of previous significant ocular trauma or any condition which, in the investigators’ opinion, would conflict with or otherwise prevent the subject from complying with the required procedures, schedule or other study content.

OCT Imaging Protocol

After informed consent was obtained, VA and medical history were noted from the medical record. Each subject underwent SD-OCT (Spectralis, Heidelberg Engineering; RTVue XR Avanti XR HD, Optovue OCT) and SS-OCT (DRI-OCT-1 Atlantis, Topcon) in a single imaging session completed by the same clinical trial-certified technician. All subjects had adequate pupillary dilation prior to imaging using 2.5% phenylephrine and 1% tropicamide eye drops.

Each Spectralis SD-OCT imaging process consisted of a volume scan of 25 horizontal line scans (512 A-scans per B-scan, 242 μm interscan distance) that covered 20° × 20° centred on the fovea using the automated retinal thickness: high-speed (ART: HS) mode. Each Optovue SD-OCT acquisition consisted of 12 × 9 mm macular cube scan. Each DRI-OCT-1 SS-OCT acquisition consisted of 12 × 9 mm raster scan centred on the fovea.

Analysis of Macular Thickness

The automated retinal thickness measurements are calculated by the on-board segmentation algorithm that defines the inner retinal boundary as the inner limiting membrane and the outer retinal boundary as the outer aspect of the retinal pigment epithelium (RPE)/Bruch’s membrane complex. This algorithm calculates the retinal thickness of each of nine ETDRS (Early Treatment in Diabetic Retinopathy Study) macular subfields; the thickness of the central macular subfield (centred on the fovea with a diameter of 1 mm) was recorded.

Statistical Analysis

The mean and standard deviation (SD) of automated central macular thickness measures were calculated. Method agreement was determined using techniques described by Bland and Altman [15, 16]. Bland–Altman analysis including limits of agreement (LoA) was completed. The intraclass correlation coefficient (ICC) was assessed for retinal thickness measurements made. All statistical analyses were performed using Excel, Microsoft Excel for Mac 2011, GraphPad Prism version 7 and SPSS version 3.2 software.

Results

Subject Characteristics

Two hundred seventy-two eyes from 272 patients were included in the study. The sample consisted of 91 male (33.5%) and 181 female (66.5%) subjects, and included 132 left (48.5%) and 140 right (51.5%) eyes. Mean ± SD age of the study participants was 66.6 ± 17.4 years. Subjects included a range of macular diagnoses (including choroidal neovascular membrane [CNV] and macular dystrophy) and retinal diagnoses (including posterior uveitis and diabetic retinopathy). The most prevalent diagnoses were age-related macular degeneration and retinal dystrophy. Demographics and range of retinal and macular diagnoses are shown in Table 1.

Table 1

Demographics of study subjects in the Advanced OCT and Adaptive Optics Imaging in Retinal Disease (the ACAD Study)

	Study characteristics
Mean ± SD age (years)	66.6 ± 17.4 years
Laterality (n, %)	132 left (48.5%) and 140 right (51.5%)
Gender (n, %)	91 male (33.5%) and 181 female (66.5%)
Ethnicity (n, %)	White—British (199, 73.2) Not recorded (25, 9.2) Black or Black British—Caribbean (5, 1.8) White—Irish (12, 4.4) Other mixed (1, 0.4) Asian or Asian British—Indian (6, 2.2) White—Other (11, 3.7) Any other ethnic group (1, 0.4)
Macular diagnosis (n, %)	Choroidal neovascular membrane (134, 49.3) Macular oedema (29, 10.7) Central serous retinopathy (19, 7.0) Macular dystrophy (3, 1.1) Dry age-related macular degeneration (41, 15.1) Serous pigment epithelial detachment (6, 2.2) Epiretinal membrane (4. 1.5)
Retinal diagnosis (n, %)	Posterior uveitis (15, 5.5) Scleritis (9, 3.3) Retinal dystrophy (5, 1.8) Diabetic retinopathy (15, 5.5) Retinal vein occlusion (5, 1.8) Retinal artery occlusion (2, 0.74) Choroidal naevus (2, 0.74)

Macular and retinal diagnoses are shown for both study and fellow eyes

Retinal Thickness Measurements

The largest mean retinal thickness (± SD) measurements were found on the Spectralis SD-OCT (298.3 μm ± 70.6 μm). Mean ± SD retinal thickness measurements made on the Optovue OCT and DRI-OCT-1 were 258.8 μm ± 57.9 μm and 239.2 μm ± 56.1 μm, respectively.

The ICC of the retinal thickness measurements (95% confidence interval) made using all three OCT s was 0.81 (0.77–0.84). There was no correlation between mean retinal thickness and absolute retinal thickness measurements made in this cohort. Pearson’s correlation coefficient for mean retinal thickness and standard deviation for Optovue SD-OCT, Spectralis SD-OCT and Topcon SS-OCT were 0.44, 0.61 and 0.53, respectively (all p > 0.05).

Bland–Altman Analyses

Bland–Altman analysis was performed (Fig. 1) comparing retinal thickness measurements between OCT devices (Optovue vs Spectralis, Optovue vs Topcon, Spectralis vs Topcon). The mean difference in mean retinal thickness between Spectralis SD-OCT and Topcon SS-OCT was 59.1 μm (95% LoA −21.7 to 139.8 μm). The mean difference in mean retinal thickness between Optovue SD-OCT and Spectralis SD-OCT was −37.5 μm (95% LoA −121.2  to 46.3 μm). The mean difference in mean retinal thickness between Optovue SD-OCT and Topcon SS-OCT was 21.8 μm (95% LoA −34.7 to 78.3 μm).

Fig. 1

Bland–Altman analyses comparing retinal thickness measurements between optical coherence tomography devices: A Optovue vs Spectralis; B Optovue vs Topcon; C Topcon vs Spectralis. Mean difference (dashed line) and 95% limits of agreement (dotted lines) are shown

Subgroup Analysis by Diagnosis

Mean retinal thickness and agreement analysis was completed for subgroups (CNV, dry AMD and macular oedema), where n > 20. Mean ± SD retinal thickness measurements with Optovue SD-OCT, Spectralis SD-OCT and Topcon SS-OCT were 241.1 μm ± 46.3 μm, 284.1 μm ± 60.7 μm, and 225.7 μm ± 48.2 μm, respectively, for CNV; 258.7 μm ± 43.1 μm, 310.1 μm ± 75.0 μm, and 244.5 μm ± 52.5 μm for dry AMD; and 319.2 μm ± 152.5 μm, 332.8 μm ± 94.5 μm, and 357.1 μm ± 101.2 μm for macular oedema.

Mean difference between retinal thickness measurements and their associated 95% LoA are shown in Table 2. The largest mean difference in retinal thickness difference (95% LoA) was between Topcon SS-OCT and Spectralis SD-OCT, at 65.4 μm (−20.5 to 153.1 μm), for patients with dry age-related macular degeneration.

Table 2

Mean difference in optical coherence tomography-derived retinal thickness measurements made In diagnosis subgroups

	Mean difference in retinal thickness measurements (95% limits of agreement) (μm)
	Optovue SD-OCT—Spectralis SD-OC	Optovue SD-OCT—Topcon SS-OCT	Topcon SS-OCT—Spectralis SD-OCT
Choroidal neovascular membrane	−40.9 (−114.3 to 32.5)	18.6 (−24.2 to 61.4)	59.4 (−14.8 to 133.6)
Dry age-related macular degeneration	−51.2 (−157.5 to 55.1)	15.7 (−54.5 to 85.9)	65.4 (−20.5 to 153.1)
Macular oedema	−10.5 (−265.1 to 244.2)	−44.7 (−645.3 to 556.3)	−23.6 (−514.6 to 467.3)

Discussion

The ACAD Study investigated the inter-device agreement in OCT-derived retinal thickness measurements in patients with macular and retinal disease. The ICC for retinal thickness measurements made with Optovue SD-OCT, Topcon SS-OCT and Spectralis SD-OCT was 0.81 (95% CI 0.77–0.84), suggesting good agreement in retinal thickness measurements in our cohort. The smallest mean retinal thickness difference was between Optovue SD-OCT and Topcon SS-OCT, at 21.8 μm (95% LoA −34.7 to 78.3 μm), suggesting the highest degree of agreement. Our results indicate that the largest difference in mean retinal thickness was between Spectralis SD-OCT and Topcon SS-OCT, at 59.1 μm (95% LoA −21.7  to 139.8 μm). Spectralis SD-OCT appears to systematically overestimate retinal thickness compared to both Topcon and Optovue (Fig. 1A, C) across the diverse range of conditions of our cohort.

These findings are important, as changes in retinal thickness values form part of disease progression assessment for macular disease, particularly nAMD and diabetic macular oedema, when treated with anti-angiogenic injections. This assessment of measurement variability is essential to help guide the use of different types of OCT instruments in clinical trials and could possibly define new clinical trial endpoints. As OCT-based indices of disease activity become increasingly important in clinical decision-making, understanding differences in retinal thickness measurements taken on different OCT devices is imperative (see Fig. 2).

Fig. 2

Optical coherence tomography images of example subject in the Adaptive Optical Coherence Tomography and Adaptive Optics Imaging in retinal disease (ACAD Study) acquired with Optovue (A), Spectralis (B) and Topcon (C)

Differences in the retinal thickness measurements may be explained, in part, by differences in the methods of scan acquisition and automated segmentation of retinal layers. Differences in the segmentation algorithms, particularly the location of the inner and outer retinal boundary, may generate differences in retinal thickness measurements [17, 18]. Spectralis, DRI-OCT-1 and Optovue OCT utilize different imaging wavelengths (840 nm, 1050 nm and 840 nm, respectively) and at different spatial resolution (7 μm × 14 μm, 8 μm × 20 μm, and 5 μm × 15 μm, respectively). A previous study indicated that retinal thickness measurements made in healthy subjects are similar between Spectralis SD-OCT and Cirrus HD-OCT [12]. Tan et al. reported that the mean difference in central retinal thickness ranged between 15.6 and 37 μm when comparing central retinal thickness measurements between Topcon DRI-OCT-1, Spectralis SD-OCT and Cirrus HD-OCT [13]. In our study, the mean difference between Optovue and Topcon OCT appeared to be small, but the Spectralis SD-OCT and Topcon SS-OCT difference was only slightly larger, at 59.1 μm. However, this cohort included a mixture of mostly healthy and highly myopic eyes. In eyes with retinal disease, alteration in normal retinal layers can lead to a larger difference in absolute retinal thickness measurements due to alteration in segmentation and image acquisition. We previously reported that in eyes with nAMD, SD-OCT and SS-OCT retinal thickness measurements were surprisingly similar, with SD-OCT measurements just 3% smaller than the SS-OCT measurements [14].

The strengths of this study include the relatively large sample size, with a diverse range of retinal and macular diagnoses using both OCT technologies commercially available across two forms of OCT technology and three different devices (Optovue, Spectralis and Topcon) in a prospective design, with imaging performed by a clinical trial-certified technician. Limitations include the study protocol involving a single scan being completed per OCT device and unavailability of clinical information on a proportion of the study subjects. Segmentation error has been described as an important cause of inaccuracy in automated OCT-derived retinal thickness measurements. The agreement found in this study would perhaps have been stronger with manual retinal thickness measurements. Nevertheless, as automated measurements are an important tool in clinical practice, understanding the agreement between automated retinal thickness measurements is critical to informing real-world clinical practice [19, 20]. Future areas of study include an assessment of factors affecting repeatability including the incidence of automated segmentation error and assessment of other clinically relevant indices including the retinal nerve fibre layer in patients with glaucoma. Differences in axial resolution and placement of segmentation lines will contribute to the differences in retinal thickness measurements we see between devices. We did not compensate for errors, artefacts or resolution differences between devices, in order to make our results relevant to real-world settings where post hoc changes to OCT images do not form part of routine clinical care. Though this could be viewed as a limitation, it can also be seen as a strength in making the results more relevant for patient care and is comparable to other studies which use a similar approach to analysis [11, 13, 18, 21].

Conclusions

In summary, we report the mean retinal thickness measurements in a cohort of patients with macular and retinal conditions. We describe the inter-device agreement between different forms of OCT technology; the largest mean difference and LoA appeared to be between Spectralis SD-OCT and Topcon SS-OCT. Clinicians should be aware of the variability in OCT measurements derived from different forms of OCT acquisition technology and, importantly, their associated segmentation algorithms. Clinicians should consider using the same form of OCT technology in order to accurately monitor patients, or perhaps analyse clinical indices in the context of these considerations. As OCT technology continues to evolve, such studies should be further broadened to consider other parameters such as the speed of operation, dependence of results on tracking procedures, lateral or axial resolution, and segmentation methods, and with trends towards automation, in evaluating the variability between the use of manual and automatic collection of data.

Why carry out this study?

This study assessed inter-device agreement in optical coherence tomography-derived retinal thickness measurements in patients with known macular and retinal disease.

A total of 272 subjects were included and underwent both spectral-domain and swept-source optical coherence tomography imaging.

What was learned from this study?

The intraclass correlation coefficient of retinal thickness measurements made across the modes of optical coherence tomography was 0.81.

The mean difference in retinal thickness measurements was largest between Topcon swept-source and Spectralis spectral-domain optical coherence tomography imaging.

The mean difference in retinal thickness measurements was greatest in patients diagnosed with age-related macular degeneration and choroidal neovascular membrane.

Clinicians should be aware of the differences in retinal thickness values when imaging patients using different OCT devices and should consider using the same OCT device model in order to monitor clinical change.