Repeatability of Optical Coherence Tomography Angiography Measurements in Patients with Retinal Vein Occlusion.

PURPOSE: To analyze the repeatability of vessel density (VD) measurements and manual foveal avascular zone (FAZ) measurements using optical coherence tomography angiography (OCTA) in patients with retinal vein occlusion (RVO) without macular edema.
METHODS: The study population consisted of patients with RVO and central macular thickness <300 μm. For each subject, measurements were performed twice with a 5-minute interval. The intraclass correlation coefficient (ICC) and coefficient of variation (CV) were calculated to analyze the repeatability of results obtained with the OCTA device. To identify factors related to repeatability, we performed Pearson correlation analyses based on the CV of potential factors.
RESULTS: A total of 48 eyes were included in the study. The ICCs of the VDs in superficial capillary plexus (SCP) and the deep capillary plexus (DCP) were 0.748 and 0.665, respectively, and the CVs of the VDs in SCP and DCP were 9.1% and 12.6%, respectively. The ICCs associated with the FAZ of the superficial layer (SFAZ) and that of the deep layer (DFAZ) were 0.965 and 0.956, respectively, and the CV of the SFAZ and DFAZ were 8.8% and 9.7%, respectively. From Pearson correlation analyses, OCTA quality was significantly correlated with the CV of the VDs of SCP and DCP. However, there were no variables that were significantly correlated with the CV of SFAZ and DFAZ, including OCTA quality.
CONCLUSIONS: VD measurements in the SCP layer using OCTA exhibited good repeatability, and VD measurements in the DCP layer exhibited relatively low repeatability compared to that of SCP layer measurements in patients with RVO without macular edema after treatment with bevacizumab. Manual measurement of the FAZ area in both SCP and DCP layers resulted in good repeatability. In addition, the repeatability of VD measurements in SCP and DCP layers was correlated with OCTA image quality.

Retinal vein occlusions (RVOs) comprise a heterogeneous group of disorders that manifest as impaired venous return from the retinal circulation [1]. RVO is the second-most common retinal vascular disorder after diabetic retinopathy and is considered an important cause of visual loss [2]. The pathogenic mechanism of RVO is thought to involve a combination of compression of veins at arteriovenous crossings, degenerative changes within venous walls, and hypercoagulability [3,4]. The application of optical coherence tomography angiography (OCTA) in RVO allows detection of foveal avascular zone (FAZ) enlargement, capillary nonperfusion, microvascular abnormalities, and signs of vascular congestion in both the superficial and deep capillary networks [5]. Therefore, it is important to observe the retinal vascular structure using OCTA in patients with RVO.

The recent development of OCTA has made it easier to observe the vascular structure accurately and quickly, and several studies using OCTA in RVO patients have been reported.

Kang et al. [6] reported that OCTA allowed the detection of FAZ enlargement, increased parafoveal capillary nonperfusion, and decreased parafoveal vessel density (VD) in eyes with RVO. Kashani et al. [7] also reported that OCTA was able to reveal almost all of the clinically relevant findings in subjects with RVO. These observations are consistent with clinical, anatomical, and fluorescein angiographic findings, including areas of impaired vascular perfusion, retinal atrophy, vascular dilation, shunt vessels, and some forms of intraretinal edema. As above, the observation of microvasculature using OCTA in RVO can be useful, but high repeatability is required for reliability of the measurement. Recently, Kim et al. [8] reported that the repeatability of VD measurements in RVO eyes with or without macular edema (ME) was favorable. However, there have been few studies examining OCTA variables altogether such as the superficial capillary plexus (SCP), deep capillary plexus (DCP), and FAZ area in analyses of the repeatability of OCTA parameter measurements in patients with RVO. Additionally, it is found that two measurements of VD could be different each other in RVO patients without ME, which is relatively unlikely to cause segmentation error clinically, even though the test was performed on the same day in our retina clinic.

The present study was performed to analyze the repeatability of VD and manual FAZ area measurements using OCTA in RVO patients and to identify the factors related with such repeatability.

Materials and Methods

Subjects

This study adhered to the tenets of the Declaration of Helsinki and was approved by the institutional review board of Konyang University Hospital (2020-10-008). The requirement for informed consent was waived due to the retrospective nature of the study. The study population consisted of patients who visited our retina clinic between May 2017 and May 2020. Patients were diagnosed with RVO through comprehensive examinations, including history taking, fundus photography, optical coherence tomography (OCT), and fluorescence angiography. We excluded cases with ME, defined as central macular thickness ≥300 μm, because ME makes it difficult to distinguish between the superficial layer and the deep layer, causing segmentation error.

OCTA measurements

OCTA was performed using a Spectralis OCT2 device (Heidelberg Engineering, Heidelberg, Germany). For each subject, measurements were performed twice with a 5-minute interval. The Spectralis OCT2 instrument is capable of 70,000 A-scan/sec using a light source centered at 870 nm, with axial and transverse resolutions of 3.9 and 6 μm in tissue, respectively. En face images of the SCP, defined as the layer originating from the internal limiting membrane to the inner plexiform layer, and the DCP, defined as the layer starting from the outer border of the outer plexiform layer, were visualized automatically by segmenting two separate slabs defined by arbitrary segmentation lines created by the device software. VD was calculated using ImageJ (National Institutes of Health, Bethesda, MD, USA). In particular, 8-bit images were split into the red, green, and blue channels, with the red channel used as the reference. The adjust threshold tool was applied with the default settings, and the dark background option was selected. This tool automatically set the lower and upper threshold values (arbitrarily chosen as 110 and 255, respectively, for every image), and segmented grayscale images into features of interest and the background. Two graders (YML and MWL) manually outlined the FAZ, defined as the avascular area in the center of fovea, and measured the area using a built-in program that provides measurement of outlined areas. The mean values of two measurements were used in the analysis (Fig. 1A–1F). Subjects were excluded if OCTA could not reveal the FAZ area. Images with loss of fixation, segmentation errors, and motion artifacts were also excluded. OCTA quality is literally the quality of OCTA that the device automatically measured in consideration of the image quality and clarity of the OCTA device. The OCTA quality score ranges from 0 (no signal) to 40 (excellent quality). If the score is 15 or less, the quality bar turns red indicating a poor scan quality. If the score is between 15 and 25 the scan quality is considered marginal. If the score is 25 or above the scan quality is considered good. In this study, we excluded OCTA images with a OCTA quality <20.

Fig. 1

Original optical coherence tomography angiography (OCTA) images, images converted using ImageJ, and foveal avascular zone area measurements. (A) Representative original OCTA image of the superficial capillary plexus layer. (B) Image of the superficial capillary plexus layer converted with ImageJ. (C) Representative original OCTA image of the deep capillary plexus layer. (D) Image of the deep capillary plexus layer converted with ImageJ. (E,F) Two graders manually outlined the foveal avascular zone, defined as the avascular area at the center of the fovea, and measured the area using a built-in program that provides measurements of outlined areas.

Statistical analysis

The intraclass correlation coefficient (ICC) and coefficient of variation (CV) were calculated to analyze the repeatability of VD and FAZ area measurements in the SCP and DCP. The ICC is the ratio of subject variance to total variance, and a value close to 1 for the same examination indicates less variability (<0.40, poor; 0.40–0.59, fair; 0.60–0.74, good; 0.75–1.00, excellent). The CV (%) was calculated as 100% × standard deviation/overall mean, and a value <10% indicated good repeatability of the measurements. The agreement between two measurements was evaluated using Bland-Altman plots. To identify factors related to repeatability, we performed Pearson correlation analyses based on the CV of potential factors.

Results

A total of 48 eyes were included in the study: 33 eyes with branch RVO, 15 eyes with central RVO. The patients had a mean age of 62.27 ± 9.87 years, best-corrected visual acuity of 0.16 ± 0.23, spherical equivalent of −0.02 ± 1.16 diopters, intraocular pressure of 13.08 ± 2.74 mmHg, axial length of 23.78 ± 0.70 mm, and central macular thickness of 264.73 ± 23.82 μm. The average number of intravitreal Avastin injections was 7.17 ± 6.36 (Table 1).

Table 1

Demographic and baseline characteristics of the patient

Characteristics	Value
No. of patients	48
Age (yr)	62.27 ± 9.87
Sex (male)	20 (42)
Laterality (right)	21 (44)
BCVA (logMAR)	0.16 ± 0.23
Spherical equivalent (D)	−0.02 ± 1.16
IOP (mmHg)	13.08 ± 2.74
Axial length (mm)	23.78 ± 0.70
CMT (μm)	264.73 ± 23.82

Values are presented as mean ± standard deviation or number (%).

BCVA = best-corrected visual acuity; logMAR = logarithm of the minimum angle of resolution; IOP = intraocular pressure; CMT = central macular thickness.

In all 48 eyes with RVO, the mean VD in the SCP at the first and second examinations were 60.89 ± 15.63 and 62.36 ± 14.12, respectively (p = 0.748), and those in the DCP were 51.21 ± 13.73 and 53.22 ± 13.53, respectively (p = 0.665) (Table 2). In analyses of interexam repeatability of FAZ measurements, the FAZ areas measured by the two graders exhibited excellent intergrader repeatability that the ICC was 0.96 and CV was 3.1% in superficial layer and the ICC was 0.97 and CV was 3.8% in deep layer. The mean areas of the FAZ of the superficial layer (SFAZ) at the first and second examinations (interexam repeatability) were 0.54 ± 0.31 and 0.53 ± 0.32 mm2, respectively (p = 0.965), and those of the FAZ of the deep layer (DFAZ) were 0.38 ± 0.25 and 0.38 ± 0.23 mm2, respectively (p = 0.961). The mean OCTA quality at the first and second examinations were 32.21 ± 2.68 and 32.58 ± 2.87 dB, respectively and there was no difference between the two values (p = 0.394).

Table 2

Mean measurement values at the first and second examinations and ICCs and CV associated with VD and mean FAZ area measurements in patients with retinal vein occlusion

	VD				FAZ area
	First	Second	ICC	CV	First	Second	ICC	CV
SCP	60.89 ± 15.63	62.36 ± 14.12	0.748	9.1	0.54 ± 0.31	0.53 ± 0.32	0.965	8.8
DCP	51.21 ± 13.73	53.22 ± 13.53	0.665	12.6	0.38 ± 0.25	0.38 ± 0.23	0.956	9.7

Values are presented as mean ± standard deviation.

ICC = intraclass correlation coefficient; CV = coefficients of variation; VD = vessel density; FAZ = foveal avascular zone; SCP = superficial capillary plexus; DCP = deep capillary plexus.

The ICCs of the VDs in the SCP and DCP were 0.748 and 0.665, respectively, and the corresponding CVs were 9.1% and 12.6%, respectively. The ICCs of the SFAZ and DFAZ were 0.965 and 0.956, respectively, and the corresponding CVs were 8.8% and 9.7%, respectively. In Bland-Altman plots, the differences were close to 0 for the VD, which indicated no systematic differences in measurement values between the two scans (Fig. 2A–2D).

Fig. 2

The agreement between two measurements was evaluated using Bland-Altman plots. The differences were close to 0 for measurements of vessel density and foveal avascular zone (FAZ) area, indicating no systematic differences in measurement values between the two scans. (A) Vessel density of the superficial capillary plexus layer in the retinal vein occlusion (RVO) patients, (B) vessel density of the deep capillary plexus layer in the RVO patients, (C) the FAZ area measurement of the superficial capillary plexus layer in the RVO patients, and (D) the FAZ area measurement of the deep capillary plexus layer in the RVO patients.

From Pearson correlation analyses, OCTA quality was significantly correlated with the CV of the VDs in the SCP (p = 0.001) and DCP (p = 0.003) (Table 3 and Fig. 3A, 3B). However, no variables including OCTA quality were significantly correlated with the CV of the SFAZ (p = 0.169) and DFAZ (p = 0.280) (Table 4 and Fig. 4A–4D). We compared parameters that affecting the CV of the SCP and DCP layer by dividing them by 10%. In the SCP layer, there were 19 patients with the CV greater than 10% and 29 patients with less than 10%. We compared parameters that affecting the repeatability such as age (p = 0.360), sex (p = 0.661), best-corrected visual acuity (p = 0.126), spherical equivalent (p = 0.661), intraocular pressure (p = 0.281), axial length (p = 0.523), central macular thickness (p = 0.567), the number of intravitreal bevacizumab injection (p = 0.057) and OCTA quality (p = 0.019). In the DCP layer, there were 25 patients with the CV greater than 10% and 23 patients with less than 10%. Similarly, in the DCP layer, a comparison was conducted for the same category (age, p = 0.190; sex, p = 0.411; best-corrected visual acuity, p = 0.801; spherical equivalent, p = 0.812; intraocular pressure, p = 0.917; axial length, p = 0.815; central macular thickness, p = 0.926; the number of intravitreal bevacizumab injection, p = 0.867; OCTA quality, p = 0.000). We also conducted a subgroup analysis which compared all parameters according to the foveal involvement of RVO, but the difference between the two groups was only best-corrected visual acuity and the FAZ area and there was no difference in CV-related factors (CV of the VD in the SCP layer, p = 0.080; CV in the VD of the DCP layer, p = 0.438; CV of the FAZ area measurement of the SCP layer, p = 0.142; CV of the FAZ area measurement of the DCP layer, p = 0.496). As a result, it was revealed that the fovea involvement of the lesion was not related to repeatability.

Table 3

Results of Pearson correlation analyses to identify factors correlated with the CV of vessel density measurements

	SCP		DCP
	CV	p-value	CV	p-value
Age	0.186	0.206	0.146	0.321
Sex	0.262	0.072	0.236	0.107
BCVA	0.110	0.456	−0.033	0.825
IOP	−0.198	0.176	−0.039	0.790
Axial length	−0.271	0.063	−0.120	0.417
SE	−0.084	0.568	−0.065	0.659
CMT	−0.188	0.202	−0.146	0.321
OCTA quality	−0.450	0.001	−0.414	0.003

CV = coefficients of variation; SCP = superficial capillary plexus; DCP = deep capillary plexus; BCVA = best-corrected visual acuity; IOP = intraocular pressure; SE = spherical equivalent; CMT = central macular thickness; OCTA = optical coherence tomography angiography.

Fig. 3

Optical coherence tomography angiography (OCTA) quality was significantly correlated with the coefficients of variation of the vessel density in (A) the superficial capillary plexus (p = 0.001) and (B) deep capillary plexus (p = 0.003).

Table 4

Results of Pearson correlation analyses to identify factors correlated with the CV of foveal avascular zone area measurements

	SFAZ		DFAZ
	CV	p-value	CV	p-value
Age	−0.101	0.496	0.063	0.669
Sex	0.019	0.898	0.159	0.281
BCVA	0.012	0.933	0.067	0.653
IOP	−0.200	0.134	0.335	0.120
Axial length	0.018	0.902	−0.170	0.247
SE	−0.200	0.174	0.037	0.804
CMT	−0.028	0.850	0.256	0.079
OCTA quality	0.202	0.169	−0.159	0.280

CV = coefficients of variation; SFAZ = superficial foveal avascular zone; DFAZ = deep foveal avascular zone; BCVA = best-corrected visual acuity; IOP = intraocular pressure; SE = spherical equivalent; CMT = central macular thickness; OCTA = optical coherence tomography angiography.

Fig. 4

The representative optical coherence tomography angiography (OCTA) images in a same patient. (A) Representative OCTA image of the superficial capillary plexus (SCP) layer when the image quality was 28 dB. The vessel density (VD) was 38.30 and the foveal avascular zone (FAZ) area measurement was 0.29 mm2. (B) Representative OCTA image of the SCP layer when the image quality was 36 dB. The VD was 73.91 and the FAZ area measurement was 0.28 mm2. In SCP layer, the coefficient of variation (CV) of VD was 44.8% and the CV of FAZ area measurement was 5.6%. (C) Representative OCTA image of the deep capillary plexus (DCP) layer when the image quality was 28 dB. The VD was 40.137 and the FAZ area measurement was 0.13 mm2. (D) Representative OCTA image of the DCP layer when the image quality was 36 dB. The VD was 67.98 and the FAZ area measurement was 0.12 mm2. In DCP layer, the CV of VD was 36.4% and the CV of FAZ area measurement was 2.4%.

Discussion

Fluorescein angiography, which was the standard method for diagnosing RVO, is invasive and affected by a number of variables, including contrast, timing of the dye injection, obscuration from leakage, and degree of pigmentation. By contrast, OCTA is a noninvasive, dye-free OCT-based imaging technique that offers the best visualization of retinal vasculature with depth-resolved imaging. Therefore, it has the potential to become a mainstay in the assessment of retinal ischemia [9,10]. OCTA also provides quantitative information, such as that on FAZ area and VD. However, such quantitative information requires high repeatability. Repeatability of any diagnostic device is crucial in terms of reliability of examination. Therefore, there have been a number of studies of the repeatability and reproducibility of OCTA in normal eyes, and the results showed high repeatability [11,12]. There have also been studies of the repeatability of OCTA in diseased eyes. Lee et al. [13] reported that measurements of superficial VD using OCTA exhibited relatively good repeatability in various retinal diseases, including diabetic ME, RVO with ME, epiretinal membrane, and age-related macular degeneration. However, there have been few studies on measuring VD in superficial and deep layers as well as the FAZ area in RVO patients, as in the present study. In addition, the present study was performed in patients without ME for accurate examination of each SCP and DCP. In our study, VD measurements in the SCP layer exhibited good repeatability, and VD measurements in the DCP layer exhibited relatively low repeatability compared to the SCP layer. In addition, manual measurements of the FAZ area in both SCP and DCP layers exhibited good repeatability.

In the present study, the ICC of the VDs in the SCP was 0.748 and the CV was 9.1%, which was a favorable result. Lee et al. [13] reported that VD measurements in the SCP using OCTA exhibited relatively good repeatability in various retinal diseases, including RVO. They reported that the CV of the VD of the SCP was 10.65% in RVO. Kim et al. [8] also reported favorable repeatability of VD measurements in eyes with RVO. They reported that the CV of the VD was 10.61% and the ICC was 0.800. Although there could be differences between the devices used, quantitative measurements of the VD in the SCP using OCTA exhibited good repeatability in patients with RVO, indicating that it may be a reliable diagnostic tool.

In the present study, the ICC of the VD in the DCP was 0.665 and the CV was 12.6%, which was an unfavorable result. Although we excluded patients with ME to reduce the possibility of segmentation error, repeatability was low compared to measurements of VD in the SCP. Measurements of VD in the DCP were reported to be inferior to those in the SCP with regard to repeatability and accuracy [14,15]. Durbin et al. [14] reported that VD measurements in the DCP using OCTA had lower utility than those in the SCP because of artifacts, so VD measurements in the SCP had better diagnostic efficacy. Spaide et al. [15] suggested that OCT projection artifacts occur due to superficial retinal vessels, which can appear in deeper retinal layers. In addition, the OCTA projection artifacts are nearly always present and appear in any structure that is located below the vasculature. Therefore, physicians should take this into consideration when analyzing quantitative measurements of VD in the DCP taken using OCTA in RVO patients.

In the present study, the ICC of the SFAZ area was 0.965 and the CV was 8.8%, and the ICC of the DFAZ area was 0.956 and the CV was 9.7%, which indicated good repeatability of measurements taken with the OCTA device. Carpineto et al. [16] reported that FAZ area measurements by OCTA exhibited excellent reproducibility and repeatability in healthy eyes. They also reported that the ICCs associated with FAZ area measurements obtained with two devices were 0.997 and 0.998, and the corresponding CV were 1.83% and 1.86%, indicating excellent repeatability. Fang et al. [17] reported that FAZ area measurements using OCTA exhibited excellent repeatability and correlation in healthy eyes and that the ICC associated with FAZ area measurements was 0.961. Lee et al. [18] reported that the repeatability of manual FAZ area measurements was good in high myopia patients. The ICC associated with the SFAZ area measurements was 0.891 and the CV was 8.8%, and those associated with the DFAZ were 0.788 and 11.2%, respectively. Czako et al. [19] also reported excellent repeatability of FAZ area measurements in diabetic patients. The mean VDs of the FAZ in the first and second examinations were 0.30 ± 0.11 and 0.31 ± 0.11/mm2, respectively (p > 0.05). The ICC associated with the FAZ area measurements was 0.970 and the CV was 7.79%. Therefore, in various diseases, FAZ measurement is considered to be reliable for assessing both the SCP and DCP layers.

Lee et al. [12] reported that differences in signal strengths among measurements affected the repeatability of VD measurements using OCTA. Lim et al. [20] examined the effects of signal strength on OCTA measurements in healthy young subjects and found that peripapillary VD was significantly correlated with signal strength. Lee et al. [21] also reported that signal strength was associated with the repeatability of OCTA measurements and recommended taking signal strength into consideration in the analysis of retinal VD and FAZ area in normal eyes. However, previous studies analyzed only the SCP layer, and we showed that both the DCP layer and the SCP layer are affected by OCTA quality. Additionally, in subgroup analysis dividing subjects according to the CV of the SCP and DCP layer by 10%, the difference between the two groups was statistically significant only in the OCTA quality. OCTA quality is very important for the repeatability of OCTA measurements not only in normal eyes but also in diseased eyes. If OCTA quality is low, the reliability of quantitative measurements of VD using OCTA could be reduced.

On the other hand, measurements of the FAZ area were not influenced by OCTA quality. Czako et al. [22] reported that the signal strength index affected the repeatability of VD measurements, and FAZ area measurements were also affected by the signal strength index. However, their study differed from the present study in that we measured the FAZ area manually, whereas Czako et al. [22] measured the FAZ area automatically. In addition, Kim et al. [8] reported that they could not analyze the repeatability of automatic FAZ area measurements using built-in program in OCTA in patients with RVO because there were many cases in which the line of the FAZ area was not drawn properly or the device could not detect the FAZ area. Therefore, although there could be differences for each OCTA device, the recommendation is that the FAZ area should be measured manually in patients with RVO, as manual measurements exhibited good repeatability in both SCP and DCP layers and were affected less by OCTA quality in the present study. The lower the OCTA quality, the more blurry the borderline of FAZ, which cannot be detected precisely by the device because the device cannot detect such blurry line below a certain threshold. However, examiner may distinguish the blurry borderline due to low OCTA quality, which results in more accurate measurement with high reliability than auto measurement by the device.

This study had several limitations. First, the data were obtained from a single-center population, which may have limited the generalizability of our results. However, we assumed that given the number of eyes included in this study, the conclusions derived from our results are reliable. Second, this study was carried out with a single OCT device. Because there are several OCT devices currently in use, there could be subtle differences among the different OCT devices currently available. Third, this study demonstrated the short-term repeatability of OCTA in RVO patients, and it will be necessary to study the long-term repeatability of OCTA in future studies. Fourth, accurate segmentation is difficult due to retinal damage, and it is difficult to rule out the possibility of a slight difference in segmentation between two examinations of the SCP and DCP layers. However, we checked if the auto-analyzed sections of the device were the same level on b-scan images by two examiners, and images showing prominent segmentation errors were excluded.

In conclusion, VD measurements in the SCP layer using OCTA exhibited good repeatability, whereas those in the DCP layer exhibited relatively low repeatability compared to the SCP layer in patients with RVO without ME after treatment with bevacizumab. Manual measurements of the FAZ area in both SCP and DCP layers exhibited good repeatability. In addition, the repeatability of VD measurements in SCP and DCP layers was correlated with OCTA image quality, whereas that of manual measurements of the FAZ area was not. Physicians should take this into consideration when analyzing measurements of VD and the FAZ area in patients with RVO.