Repeatability of retinal thickness and volume metrics in neovascular age-related macular degeneration using the Topcon 3DOCT-1000.

INTRODUCTION: Optical coherence tomography (OCT) is a commonly used imaging modality that provides detailed cross-sectional retinal images. This has revolutionised management of neovascular age-related macular degeneration. The need for repeated anti-vascular endothelial growth factor injections has led to therapy being delivered using OCT-guided retreatment strategies with both qualitative OCT features of disease activity (e.g. macular fluid) and changes in retinal thickness as triggers for retreatment The purpose of this study is to determine the intra-session repeatability of retinal thickness and volume measurements using the Topcon 3DOCT-1000 spectral-domain optical coherence tomography (SDOCT) device in patients with neovascular age-related macular degeneration (nAMD). This is the largest study to date looking specifically at the Topcon 3DOCT-1000.
MATERIALS AND METHODS: Two SDOCT raster scans were performed by the same blinded observer in the same sitting in consecutive patients attending for nAMD treatment as part of standard validation of a new device. Retrospective analysis was undertaken, with retinal thickness and volume measurements automatically calculated by the onboard software for each Early Treatment of Diabetic Retinopathy Study subfield for each scan. Bland-Altman methods of analysis were used to assess repeatability.
RESULTS: Data from the 73 patients were analyzed with a mean age of 78 years (standard deviation 8). The 95% coefficient of repeatability (CR) was 64 μm and 0.050 mm 3 for retinal thickness and volume respectively in the central 1 mm macular subfield. The CR did not exceed 85 μm (0.30 mm 3 ) in any subfield. The revised CR for retinal thickness and volume for the subgroup of 37 patients with no segmentation error in the central 1 mm subfield was 53 μm and 0.050 mm 3 respectively. Discussion : We report relatively modest intra-sessional repeatability of SDOCT retinal thickness and volume metrics in patients with nAMD in a clinical setting. Though useful in detecting clinical change from measurement variability in clinical practice, these results suggest the precision of macular thickness measurement does not approach the theoretical resolution of SDOCT.

Optical coherence tomography (OCT) imaging has revolutionized the assessment of the patient with macular disease.[12] This non-invasive and rapid imaging modality provides detailed cross-sectional information about retinal morphology. Recent advances have seen the development of spectral-domain optical coherence tomography (SDOCT) offering more rapid imaging and improved resolution over older time-domain OCT technology.[3]

The rapid development of this imaging technology has occurred at a time of unprecedented advancement in the treatment of neovascular age-related macular degeneration (nAMD).

It is therefore important to estimate the repeatability of SDOCT based retinal thickness and volume metrics in patients with nAMD in order to distinguish true clinical change from measurement variability. A better understanding of the repeatability of SDOCT based retinal thickness and volume measurements would potentially improve the quality of retreatment decisions in clinical trials and in clinical practice. Previous work has provided repeatability estimates of time-domain OCT measures in nAMD[4] and of the Cirrus HD-OCT (Carl Zeiss Meditec, Dublin, CA) in nAMD.[5] Recent studies have repeatability in various OCT machines, but not the Topcon.[67] One previous study has looked at the Topcon in OCT, but in only 12 eyes with nAMD. This is the largest report of the repeatability of the Topcon 3DOCT-1000 SDOCT in a large cohort of patients with nAMD.

Materials and Methods

Data from patients with nAMD undergoing treatment with ranibizumab in the Medical Retina Service were included in the study. All patients in this study had active subfoveal CNV due to AMD in the study eye and either had or were about to undergo treatment. For each patient, only images from the eye undergoing treatment were used in the analysis.

For this study, similar methods were used to collect the data as in a previous study by the same group carried out at Moorfields Eye Hospital.[4] All scans were obtained between November 26, 2008 and February 17, 2009 on a single Topcon 3DOCT-1000 machine by a single observer (JA). All patients had imaging performed after pupil dilation with one drop of 2.5% phenylephrine hydrochloride and 1% tropicamide. Patients had consented to imaging as part of their clinical care and the research followed the tenets of the Declaration of Helsinki. In addition, approval for this research had been obtained from the Research Governance Committee of the Eye Hospital. As the scans were from consecutive patients at different stages of treatment, they represent a wide range of disease activity and retinal thicknesses with some patients well treated, quiescent lesions and relatively normal thicknesses to other patients with active CNV and gross retinal thickening.

All SDOCT imaging was performed using the commercially available Topcon 3DOCT-1000 machine (Topcon Inc., Paramus NJ) with software version 3.2. This is an SDOCT system machine and provides an axial resolution of approximately 6 μm. The OCT machine is serviced regularly in line with manufacturer recommendations by authorized technicians evaluated by authorized technicians and personnel from Topcon Inc. to ensure that the machine is calibrated and operating correctly.

In each imaging session, a single experienced technician (JA) acquired the OCT scans. The raster scan protocol (512 A scans × 128 B scans) was used to scan the macula (6 × 6 mm area) in eyes undergoing treatment for nAMD. This provides a scan density of 47 μm per B scan. For each patient, 2 consecutive macular raster scan sets were used for analysis. Consecutive scan sets were acquired in a single imaging session. The patient was aligned correctly with the OCT device and was asked to look at an internal fixation target. If no target was seen, the patient was asked to look straight ahead by use of an external fixation light to ensure the scans were taken through the fovea. The technician was experienced in identifying common artefacts in OCT images and scan sets were reacquired as needed to optimize scan quality. Patients with low-quality scans with poor signal strength (defined by the authors as a Q-factor less than 30) were not rescanned and were therefore excluded in the repeatability analysis. Repeated scans were taken in a clinical setting as part of the normal validation of a new device introduced into a clinical setting. The data were then retrospectively analysed and the OCT technician did not know that the measurements would be used for a repeatability analysis. This was done in an attempt to minimise patient fatigue and additional fixation losses in the second scan. Once image quality had been optimised, the technician acquired 2 raster scans in the same session with the patient sitting back from the machine between the 2 scans but with the second scan acquired without delay. Each scan was analyzed using the onboard Topcon 3DOCT software (version 3.2) with segmentation of the retinal layers and quantitative measurement of retinal thickness and volume across circular Early Treatment of Diabetic Retinopathy Study (ETDRS) sectors with 9 sectoral thickness values for circles with diameters of 1, 3 and 6 mm as previously described.[10] The Topcon 3D-1000 onboard FastMap software defines the inner and outer retinal boundaries as the internal limiting membrane and the inner boundary of the retinal pigment epithelium (RPE), respectively. The automated measurements (in μm) of retinal thickness in each of the 9 ETDRS subfields was recorded from the macular thickness map analysis as were the corresponding volume metrics in mm3.

As retinal boundary placement (segmentation) error may significantly affect the variability of the automated measures of retinal thickness, a note was made of the scans with segmentation error in the central 1 mm subfield, by one observer (VT) by analysing the first of the pair of scans taken in the same sitting. A retinal boundary placement error was defined by the authors as a visible difference between computer algorithm determined and observer determined inner or outer retinal boundaries. This permitted a recalculation of coefficient of repeatability excluding any pairs of scan sets with segmentation error for the central 1 mm subfield.

Macular subfields for FMTM protocol analysis. The central A1 field has a diameter of 1 mm, fields A2 to A5 are zones of a circle 3 mm in diameter, and fields A6 to A9 are zones of a circle 6 mm in diameter.

Summary statistics (mean and SD) for demographic and SDOCT retinal thickness and volume data were calculated (SPSS version 16.0). The difference between measures across the two images was analysed using a Wilcoxon signed-rank test, as Kolmogorov-Smirnov statistics showed that the retinal thickness and volume values were not normally distributed (P < 0.05) at the 2 visits. A P < 0.05 was considered statistically significant.

Assessment of repeatability of SDOCT parameters (retinal thickness and volume in each ETDRS subfield) was performed in line with methods outlined by Bland and Altman.[8] For each parameter, within subject differences were plotted against within subject means [Figs. 1, 2 and supplemental Figs). We confirmed that the differences between retinal thickness measurements were normally distributed. The mean intra-subject standard deviation (sw) was used to calculate the coefficient of repeatability (CR) defined by Bland and Altman[8] as 1.96× √(2 s2w) or 2.77sw. The difference between 2 measurements for the same subject is expected to be less than the coefficient of repeatability for 95% of pairs of observations. The term s2w is the within subject residual mean square in the one-way ANOVA table. In addition, 95% confidence intervals of each estimated CR were also calculated. This confidence interval provides an upper and lower limit for each estimate of repeatability and depends on the sample size and the number of repeated measurements taken per subject. For a sample size of 73 and 2 measurements, the 95% CI of the estimated CR is CR +/- CRX 0.162. A revised CR for the central 1 mm A1 subfield data free of segmentation error in the central 1 mm zone was also calculated.

Figure 1

Bland Altman plot of difference in thickness (μm) against mean retinal thickness for A1 macular subfield (μm)

Figure 2

Bland Altman plot of difference in volume (μm3) against mean retinal volume for A1 macular subfield (μm3)

Suppl. Figure

Macular subfields for FMTM protocol analysis

Results

Data from 73 eyes of 73 patients were included in the analysis from 80 eyes of 80 patients in total, similar to our work from previous published studies.[4] Exclusion was attributed to poor quality OCT images (Q-factor less than 30 in 5 patients) from the first OCT and where co-pathological signs unaccounted for previously were found on OCT (2 eyes with epiretinal membrane). There were 47 female and 26 males with a median age of 78 (range, 59 to 91) years. There were 35 right and 38 left eyes. Sixty-nine patients were Caucasian. 36 (49%) of the initial scans had segmentation error within the central 1 mm zone. This leaves 37 patients with the initial scan free of segmentation error for sensitivity analysis. The median and interquartile range for retinal thickness and volume measurements are shown in Tables 1 and 2 respectively. The Wilcoxon signed rank test (5% significance level) showed no statistical difference between the intra-session measurements. The Bland Altman plots (for all patients) of retinal thickness and volume for each subfield showed no obvious relationship between difference and magnitude (Figs. 1 and 2 show data for the central 1 mm A1 subfield. The other plots looked similar, see Figs. 3 and 4). The coefficient of repeatability (with 95% confidence limits) for the central macular A1 subfield was 64 μm (53 to 74 μm) for retinal thickness and 0.050 mm3 (0.040 to 0.063 mm3) for retinal volume.

Table 1

Median retinal thickness, interquartile range and Wilcoxon signed rank test P value for each of the ETDRS macular subfields

Table 2

Median retinal volume, interquartile range and Wilcoxon signed rank test P value for each of the ETDRS macular subfields

Figure 3

Supplemental Bland Altman plots of differences in retinal thickness (μm) against mean retinal thickness (μm) for macular subfields A2-8

Figure 4

Supplemental Bland Altman plots of differences in retinal volume (μm3) against mean retinal thickness (μm3) for macular subfields A2-8

In patients with segmentation error the coefficient of reliability (CR) was 69μm (50 to 87μm) and 0.08μm (0.064 to 0.096μm) for retinal volume.

In patients with segmentation error the CR was 69 μm (50 to 87 μm) and 008 μm (0.064 to 096 μm) for retinal volume. The higher CR in the total patient group and higher still in the segmentation error group was seen as a general trend in all subfields. The coefficients of repeatability for other subfields are presented in Tables 3–8.

Table 3

Mean of the difference between retinal thickness measurements 1 and 2 with coefficients of repeatability (CR) for each of the ETDRS subfields for the 37 patients with no segmentation error

Table 8

Mean of the difference between retinal volume measurements 1 and 2 with coefficients of repeatability (CR) for each of the ETDRS subfields for the 36 patients only with segmentation error

There were 37 patients with scans sets free of segmentation error in the central 1 mm zone of the first scan. The revised CR (with 95% confidence limits) was 53 μm (41 to 65 μm) for the central A1 subfield and 0.05 mm3 (0.04 to 0.06 mm3) for the volume measure in this subfield. The 95% CI for the estimated CR for a sample size of 37 and 2 measurements is CR +/to CR × 0.228.

Discussion

In this study, we report the repeatability of SDOCT measures of retinal thickness and volume using the Topcon 3DOCT-1000 in a cohort of patients with nAMD. The results provide an estimate of repeatability and suggest that a retinal thickness or volume change of greater than 64 μm or 0.050 mm3 in the central 1 mm A1 subfield is needed to distinguish clinical change from measurement variability. It is important to report repeatability estimates for SDOCT devices as OCT thickness change is used both in clinical practice and in clinical trials to guide retreatment with anti-VEGF agents in the treatment of nAMD.[91011] Understanding the repeatability of retinal thickness measurements allows the identification of true clinical change from measurement variability therefore permitting a more informed assessment of disease progression or improvement.

Retinal thickness measurements using OCT imaging in nAMD may be less accurate and less repeatable than in normal subjects. Pathological features such as choroidal neovascularisation, subretinal fluid and haemorrhages make differentiation of retinal boundaries by OCT more challenging. Previous work using time-domain and SDOCT report a high incidence of retinal boundary detection error (OCT segmentation algorithm failure) in patients with nAMD.[1213] This reduces the accuracy of automated retinal thickness measurements and if the segmentation algorithm is unreliable, this may lead to variability in identifying inner and retina boundaries on consecutive scans reducing the repeatability of measurements. Another source of variability is eye movement between scans and this may also be encountered in patients with nAMD who may have poor fixation. With the improved resolution and increased speed of SDOCT imaging, it is thought that this will lead to improved accuracy and precision of retinal thickness measurement. Although SDOCT is much faster than time-domain OCT imaging, this is off-set by the increased number of line scans sampled in a raster scan protocol (typically 128 B scans as in this study) leading to a scan time of approximately 3 seconds. This is therefore longer than the 1.92 seconds taken to acquire a low-resolution (128 A scans per line scan) fast macular thickness scan with the time-domain Stratus OCT.

Most previous reports have estimated the repeatability of SDOCT retinal thickness measurement in normal subjects or in conditions other than nAMD. Relatively little work has been done to assess repeatability of OCT metrics in nAMD patients with previous reports[413] from our group of repeatability of 67 μm (for the central macular subfield) and another report (intra-class correlation coefficient of 0.72) using the Stratus (time-domain) OCT.[13] With the arrival of SDOCT technology there is now a need to report repeatability estimates of OCT measures with this technology in patients with nAMD. One group[5] reported repeatability of 42.4 μm with the Cirrus HD-OCT in 49 patients with nAMD with scans performed by an operator certified by image reading centres for OCT imaging in clinical trials. The estimate of repeatability of 64 μm for the central macular thickness subfield in this manuscript is comparable to the value obtained in the work by this group[5] as we applied a different formula to calculate the coefficient of repeatability (2.77Sw in this study rather than 1.96Sw).[5] It is also important to consider that unlike in that report,[5] the images in the present study were not captured by a reading centre technician with accreditation for reading centre work. Although, the mean central macular thickness (341 μm) in the cohort sampled[5] was higher than in this current study (272 μm), the repeatability estimates are similar further supporting the finding that repeatability is independent of the magnitude of macular thickness over this range. Another group[14] reported a coefficient of variation of 3.6% with the Topcon 3D-OCT 1000 in 12 patients with nAMD but they excluded patients with unstable fixation or a signal strength of <40. It is therefore difficult to apply this estimate of repeatability in clinical practice as patients with nAMD undergoing treatment often have unstable fixation which limits the repeatability of OCT derived retinal thickness measurements.

Retinal boundary detection algorithms have a high rate of failure in patients with nAMD, leading to the potential for inaccurate and imprecise retinal thickness measurement.[13] The rate of segmentation error in this cohort of patients (49%) is higher than in previously published work at 25%[14] and 12.4%.[11] This may reflect the different settings for the different studies with a greater rate of segmentation error encountered in a busy clinical practice setting as described in this current work meaning inevitable faster rates of patient turnover and scan acquisition. It may also reflect use of a different machine judging boundaries. The outer retinal barrier of the Zeiss Cirrus for example uses the RPE line to determine outer retinal boundary where the Topcon uses the IS/OS line. A more recent study,[6] has shown that the Cirrus was more free of segmenation error in 72% compared to 42% in the Spectralis. One group,[7] although not looking at nAMD specifically noted variability in different machines for retinal pathology. The results from our study, the largest know looking specifically at the Topcon 3D-1000 SDOCT, highlights the apparent variability between different machines occurs and demonstrates the need for further study to evaluate this machine compared to others in nAMD.

Despite the high rate of segmentation error in this cohort, in a sensitivity analysis of the 37 patients (51%) with the first scan free of retinal boundary detection error in the central 1 mm macular subfield, there was only a modest improvement in repeatability (from 64 to 53 μm). This suggests that factors other than segmentation error contribute to limit the repeatability of central macular thickness measurements in this cohort of patients with nAMD undergoing treatment. One such factor could be unstable fixation which could lead to non-correspondence of retinal loci between scans after rescanning. If there is great variation of retinal thickness across the raster scan (as is likely in patients with nAMD with choroidal neovascularisation and macular fluid) then fixation change between scans will lead to variability in retinal thickness measurement. Although the Topcon 3D-1000 allows repositioning of the shadowgram generated by the raster OCT over the fundus image to compensate for eccentric fixation, the operator did not utilise this function. Although this could be deemed a weakness of the study, it reflects what occurs in a busy clinical environment. A further limitation of the study is the lack of visual acuity data or lesion data which prevented an analysis of the correlation between these factors and the repeatability of SDOCT retinal thickness metrics.

In summary, we report the repeatability of SDOCT retinal thickness and volume metrics for the Topcon 3DOCT-1000 in consecutive patients receiving treatment for nAMD. These estimates may be used to determine change criteria to more accurately identify disease progression. These differences may be used to determine change criteria to more accurately identify disease progression. The relatively modest intra-sessional repeatability of SDOCT retinal thickness and volume metrics in patients with nAMD report in a clinical setting suggest that the precision of macular thickness measurement does not approach the theoretical resolution of SDOCT. Though useful in detecting clinical change from measurement variability in clinical practice these results suggest further improvements in software and hardware may be needed (e.g. better segmentation of retinal boundaries and eye tracking hardware capability) to improve repeatability of such SDOCT devices.

Table 4

Mean of the difference between retinal thickness measurements 1 and 2 with coefficients of repeatability (CR) for each of the ETDRS subfields for all 73 patients

Table 5

Mean of the difference between retinal thickness measurements 1 and 2 with coefficients of repeatability (CR) for each of the ETDRS subfields for the 36 patients only with segmentation error

Table 6

Mean of the difference between retinal volume measurements 1 and 2 with coefficients of repeatability (CR) for each of the ETDRS subfields for the 37 patients free of segmentation error

Table 7

Mean of the difference between retinal volume measurements 1 and 2 with coefficients of repeatability (CR) for each of the ETDRS subfields for all 73 patients