Repeatability and Reproducibility of Peripapillary Choroidal Thickness Using a Medical Image-Processing Software.

Although choroid has been shown to have a vital role in the pathophysiology of many ocular diseases, its role in the pathogenesis of several other conditions remains uncertain. The authors propose a novel methodology to establish a more accurate Peripapillary Choroidal Thickness (PPCT), using an image-processing software. This study sought to evaluate the reproducibility and repeatability of PPCT evaluation with ImageJ software in healthy volunteers. Forty-eight eyes of 24 volunteers were subjected to PPCT area determination, after imaging acquisition and recording with Spectral-domain Optical Coherence Tomography (SD-OCT) Heidelberg Spectralis®, during two different sessions, by two blinded assessors. The age of the subjects was 29.2 ± 4.5 years (mean ± SD). The Coefficient of Repeatability (CR) average measurements of PPCT area was 17.06 mm2 and 9.48 mm2 correspondingly, for the first and second examiners. Intra-class Correlation Coefficient (ICC) was 0.994 (95% CI 0.989 to 0.997) and 0.998 (95% CI 0.997 to 0.999). Inter-observer Concordance Correlation Coefficient (CCC) was 0.998 (95% CI 0.996 to 0.999) for both examiners. Intra-observer CCC ranged from 0.997 (95% CI 0.996 to 0.999) to 0.998 (95% CI 0.997 to 0.999), correspondingly, for the first and second examiners. The PPCT quantification by means of the proposed methodology showed good inter- and intra-observer agreement for both operators, indicating feasibility and good reproducibility of the proposed methodology. This approach might be used in different clinical settings and potentially contributes to elucidation of the choroid role in ocular pathology.

INTRODUCTION

Apart from the retina, the choroid provides oxygen and nourishment to the prelaminar portion of the optic nerve [1]. In the recent years, there has been growing recognition of the role played by choroid in various chorioretinal diseases and dilated choroidal vessels (pachy vessels) with areas of choroidal thickening visualized on SD-OCT, being associated with several disorders, coining the term “pachychoroid clinical spectrum” [2-4]. Non-Arteritic Anterior Ischemic Optic Neuropathy (NAION) is an idiopathic, ischemic, acute, monocular, painless, and insult of the optic nerve head, ensuing permanent visual impairment. Furthermore, NAION is the most common acute optic neuropathy in older adults [5]. The etiology is believed to be multifactorial, nevertheless, the exact etiopathogeny remains unproven.

Recently, several studies of the pathogenesis of NAION, using Spectral-Domain Optical Coherence Tomography (SD-OCT) imaging technology, have focused on Peripapillary Choroidal Thickness (PPCT) with contradictory results [6-8]. It could be suggested that the ambiguity among previous findings can be attributed, at least in part, to differences in the used methodology. As observed from Fig 1, imaging the choroid on SD-OCT results in an irregular layer, consequently, the measured thickness will depend on the studied spot. Previous studies have performed various measurements, usually by quadrants, and taking this problem into account, however, it is still not fully compensated.

Figure 1

Peripapillary Choroidal Thickness (PPCT) was Calculated as an Area (μm²), manually defined by the Yellow Line, using a Medical Image-Processing Software (ImageJ v1.51).

The ImageJ software is an open source tool for the analysis of scientific images. Furthermore, SD-OCT image processing with ImageJ has been extensively published in the literature and applied by the authors in different clinical settings [9, 10].

By minimizing measuring errors, the authors trust that this new methodology can contribute to the establishment of the true role of PPCT in NAION, and eventually other ocular pathologies. This study sought to validate this methodology by evaluating the reproducibility and repeatability of PPCT evaluation with ImageJ software in healthy volunteers.

MATERIALS AND METHODS

Study Design and Population

The current cross-sectional study, performed at a tertiary single-center (Department of Ophthalmology, Egas Moniz Hospital, Lisbon, Portugal), included volunteers from hospital staff. No compensation, reward, or incentive was offered in exchange for participation in the study. The study was conducted in compliance with the tenets of the Declaration of Helsinki and informed consent was obtained before clinical exams. Each subject was evaluated in a comprehensive ophthalmic examination, including Corrected Distance Visual Acuity (CDVA) using a Snellen chart, slit-lamp biomicroscopy, intraocular pressure using a Goldmann applanation tonometer, and funduscopic examination, using a 90 Diopters (D) lens.

Inclusion/Exclusion Criteria

Following Ethics Committee approval, the study was advertised to hospital staff. Inclusion criteria included (1) age > 18 years; (2) CDVA ≥ 0.045 logarithm of the minimum angle of resolution (logMAR); (3) spherical refraction within ± 3.0 D; and (4) cylinder correction within ± 2.0 D. Subjects were excluded if (1) they had previous history of ocular disease; (2) surgical or laser treatments; (3) systemic disease with ocular involvement; (4) any condition that precluded good quality SD-OCT; and (5) inability to consent.

Choroid Imaging

Choroidal imaging was performed using the SD-OCT Heidelberg Spectralis® (Heidelberg Engineering, Heidelberg, Germany). ImageJ v. 1.51 (U. S. National Institutes of Health, Bethesda, Maryland, USA, https://imagej.nih.gov/ij/, 1997-2016) allowed planimetric measurements of areas in SD-OCT volume stacks and the PPCT was measured (mm²) from the external border of the hyper-reflective line, corresponding to the RPE to the internal scleral border (Fig 1). A video of the imaging protocol is available online, as supplementary data. All SD-OCT images were obtained, stored, and numbered, by two well-trained technicians blinded to the purpose of the study. After image collection, two different PPCT measurement sessions were performed in random order for each examiner at baseline (S1) and after ten days (S2). Randomization was achieved using the Research Randomizer (free software, available online at https://www.randomizer.org/).

At S1, three measurements were performed using the ImageJ software, each by two trained examiners. At S2, measurements were repeated in the same fashion. The observers (A and B, P. Silva and M. Cordeiro, correspondingly) were blinded with respect to subject clinical information and measurement results.

Statistical Analysis and Sample Size

The Stata software (StataCorp. 2017. Stata Statistical Software: Release 15. College Station, TX: StataCorp LLC) and MedCalc Statistical Software, version 15.8 (MedCalc Software bvba, Ostend, Belgium; https://www.medcalc.org; 2015), were used to analyze the data. Participant demographics as well as PPCT, were summarized with descriptive statistics including number, mean, and standard deviation (SD), median, minimum, and maximum.

Reproducibility was evaluated by means of Lin's Concordance Correlation Coefficient (CCC), within 95% Confidence Interval (CI) [11].

Intra-session and intersession reproducibility were assessed by the CCC. Concordance correlation coefficient measures agreement by evaluating the degree to which pairs of measurements fall on the 45° line through the origin [11]. Based on the criteria listed by previous studies, a good correlation will be considered when the ICC is between 0.95 and 0.99 [12, 13].

Agreement of the method was assessed with the Bland-Altman plot by comparing repeated observations for each single assessor. The Coefficient of Repeatability (CR) was determined as 1.96 times the SD of the differences between the evaluations [14]. The coefficient of repeatability is a degree of the 95% Limits of Agreement (LoA) and gives an interval, within which 95% of test-retest measurement differences fit.

The desirable sample size to estimate the width of the 95% CI within 15%, for an assumed within-subject Standard Deviation (SD) of 15% and three examinations per patient by two examiners, was 43 eyes [15].

RESULTS

A total of 60 eyes of 30 subjects underwent PPCT area measurements with previously described methods. Although all studied subjects offered appropriate SD-OCT exams, 12 eyes from six subjects were excluded from the study (three due to refractive error, two because of ocular hypertension/glaucoma suspect, and one as a result of previous surgical treatment).

Sixteen subjects were female and eight were male. The mean ± SD of age of the subjects was 29.2 ± 4.5 years. Additional descriptive and clinical data for eyes of the subjects enrolled in the study are summarized in Table 1. The mean ± SD PPCT area measurements by session and observer are shown in Table 2.

Table 1

Descriptive and Clinical Data of the Subjects

Subject	Characteristics
Number of eyes (subjects)	48 (24)
Age (years)	29.2 (±4.5); Min:24; Max:41
Sex (men/women)	8/16
Refractive error subjects (myopic/ hyperopic) *	8/7
IOP (mmHg)	15.6 (±2.2); Min:10; Max:20
CDVA (Snellen)	0.97 (±0.05); Min:0.9; Max:1.0
PPCT Area (mm²)	491.5 (±80.2) Min: 263.2; Max: 699.7

Abbreviations: D: Diopters; CDVA: Corrected Distance Visual Acuity; IOP: Intraocular Pressure; PPCT: Peripapillary Choroidal Thickness; Min: Minimum; Max: Maximum, mmHg: Millimeter of Mercury, mm2: Square Millimeters.

Subjects were Respectively Considered Myopic or Hyperopic when Spherical equivalent was > -1D or >+1D.

Table 2

Measurements by each Observer (A or B) at a given Session (1, 2 or 3)

PPCT Observer/Session Measurement	Mean	SD	Min.	Max.
A1	491.08	80.42	266.3	689.7
A2	490.36	80.02	264.8	690.7
A3	491.53	80.10	263.8	693.7
B1	492.39	81.37	264.4	698.3
B2	491.43	81.32	263.2	691.1
B3	492.18	81.20	266.1	699.7

Measurements by Observer A at the First Session (A1 and A2) and at Second Session (A3). Measurements by Observer B at the First Session (B1 and B2) and at Second Session (B3). SD: Standard Deviation; Min: Minimum; Max: Maximum; PPCT: Peripapillary Choroidal Thickness

The indicators of repeatability, ICC, and CR, are shown in Table 3. The ICC values were 0.994 (95% CI 0.989 to 0.997) and 0.998 (95% CI 0.997 to 0.999) for the first and second examiners, correspondingly. The CR average measurements of PPCT area were 17.06 mm2 and 9.48 mm2 for the first and second examiners, correspondingly. Bland-Altman plots did not find systematic bias or important outliers (Fig 2). The indicator of reproducibility, CCC, is presented in Table 4.

Table 3

Intra-session Repeatability

Intra-session repeatability	CR	ICC	95% CI
Observer A	17.056	0.994	(0.989 - 0.997)
Observer B	9.483	0.998	(0.997 - 0.999)

CR: Coefficient of Repeatability; ICC: Intra-class Correlation Coefficient; CI: Confidence Interval.

Figure 2

Bland-Altman Plots with the Representation of the Mean Difference and Limits of Agreement

Table 4

Intra-session and Intersession Reproducibility

Inter-observer intra-session reproducibility	CCC	95% CI
A1 vs B1	0.998	(0.996 - 0.999)
A3 vs B3	0.998	(0.996 - 0.999)
Intra-observer intersession reproducibility
A1 vs A3	0.997	(0.996 - 0.999)
B1 vs B3	0.998	(0.997 - 0.999)

Measurements by Observer (A and B) at First Session (1) and at the Second Session, 10 Days Later (3). CCC: Concordance Correlation Coefficient

Left: Measurements by Observer A at the same, First Session (A1 and A2). Right: Measurements by observer B at the same, First Session (B1 and B2).

Inter-observer intra-session CCCs for the PPCT average were 0.998 (95% CI: 0.996 to 0.999) for the first session and 0.998 (95% CI: 0.996 to 0.999) for the second session. Intra-observer intersession CCCs for the PPCT average were 0.997 (95% CI: 0.996 to 0.999) for the first session and 0.998 (95% CI: 0.997 to 0.999) for the second session.

DISCUSSION

In the recent years, choroid dysfunction has been implicated in the pathogenesis of different ocular diseases [2]. However, recent studies evaluating PPCT, tend to report contradictory findings, regarding the same pathology, as is the case of NAION [3, 6, 7, 16]. The authors of this study hypothesized that the ambiguity among previous studies can be attributed, at least in part, to differences in their methodology, and propose, in this paper, a new approach to quantify the PPCT area with minimization of measurement errors. Currently, there is no gold standard for the evaluation of PPCT. The different methods reported in the literature, not only differ in their quantification approach yet are also dependent on the SD-OCT manufacturer and equipment. As ImageJ is an image-processing software, it can be used, in the same way, on exported exams from different image acquisition equipment. Therefore, the current study could be important to validate the proposed methodology and to construct a reference standard in healthy volunteers before its application in relevant pathologic conditions, namely NAION. Precision incorporates both faces of reproducibility and repeatability [15]. Repeatability denotes the variability in repeated evaluations, in which the main aspects that may contribute to the variability found in repeated evaluations are: (1) observer, (2) device used, (3) device calibration, (4) environment, and (5) time interval between evaluations, are considered constant [17]. Differently, reproducibility denotes variability when one or more of the mentioned five factors vary. In this study, examiner change was performed for reproducibility assessment [18]. This study showed both good inter- and intra-observer agreement for both operators, feasibility, and good reproducibility of the proposed methodology.

The only limitation of the current study was including healthy young subjects.

Although the SD-OCT exam quality can be affected by ocular pathology or patient-associated factors (movement disorders, poor cooperation, poor visual acuity or nystagmus), modern high-resolution imaging systems with eye-tracking technology allow quality imaging in a myriad of clinical applications with declining limitations [19-21]. Another potential pitfall is the inclusion of both eyes from the same subject. A statistical matter in precision analysis is whether one or two eyes of the same patient should be included. In this study, the authors opted to include both eyes from the same subject. This approach has been conducted before in several studies, and the observers are not responsible for image acquisition, and are masked to any information about the subjects and the images are evaluated independently in a random order [22].

CONCLUSION

The findings revealed good reproducibility and repeatability of PPCT evaluation in normal eyes, from healthy volunteers, using a novel methodology with a medical image-processing software. The authors believe that this study opens new research avenues and their findings may help elucidate the role of choroid in ocular pathology. The proposed methodology could potentially be merged on SD-OCT imaging software, simplifying the choroidal assessment at the clinical setting in the real world.