Evaluation of a Qualitative Approach for Detecting Glaucomatous Progression Using Wide-Field Optical Coherence Tomography Scans.

PURPOSE: To determine the effectiveness of detecting glaucomatous progression by a qualitative evaluation of wide-field (12 × 9 mm) scans on optical coherence tomography imaging. This method was compared to a conventional quantitative analysis of the global circumpapillary retinal nerve fiber layer (cpRNFL) thickness.
METHODS: A total of 409 eyes with a clinical diagnosis of glaucoma or suspected glaucoma for which two wide-field scans were obtained at least 1 year apart (n = 125) and within one session (n = 284) were included to determine the sensitivity of detecting progression at 95% specificity. Qualitative OCT evaluation was performed in a similar manner to flicker chronoscopy by superimposing the two scans, and the progression probability was graded. A quantitative event-based analysis of the global cpRNFL thickness also was performed.
RESULTS: Thirty-three and 25 eyes were deemed to have progressed based on qualitative and quantitative approaches, respectively (P = 0.152). A post hoc review of cases where the two methods disagreed revealed that all eyes missed by the quantitative analysis had established glaucomatous damage that appeared to show characteristic patterns of progression. All eyes missed by the qualitative evaluation appeared to be free of such established damage, and instead showed a generalized reduction in cpRNFL thickness.
CONCLUSIONS: Qualitative evaluation of OCT imaging information more frequently detected change consistent with known patterns of glaucomatous progression than global cpRNFL thickness, warranting further studies to evaluate its value. TRANSLATIONAL RELEVANCE: A framework for qualitatively evaluating progressive glaucomatous changes on OCT imaging clinically shows promise.

Introduction

Detecting disease progression is an important, yet challenging, task in the clinical management of glaucoma.[1] In recent years, optical coherence tomography (OCT) has been used increasingly for this purpose,[2-6] since progressive neuroretinal changes can be visualized and quantified with this high-resolution imaging modality. Nonetheless, optimal methods for detecting such changes remain to be established.

Current methods for detecting glaucomatous progression with OCT imaging typically involve trend-based (e.g., using linear regression analysis) or event-based (i.e., identifying change from baseline exceeding a threshold) analyses of neuroretinal thickness measurements.[7-14] These analyses can be performed using global or local measurements, such as with the trend-based analysis of global circumpapillary retinal nerve fiber layer (cpRNFL) thickness or local event-based analysis of the retinal nerve fiber layer (RNFL) thickness maps. Progression then is simply defined after a certain prespecified set of criteria has been met with these quantitative methods.

Instead, we hypothesized that a qualitative evaluation of the OCT information, which takes into consideration the full wealth of information available from OCT imaging and known patterns of glaucomatous progression, could improve the accuracy of detecting progressive damage.[15] Such an approach would be similar to assessing progressive optic disc changes on stereophotographs, where changes in the extent and appearance of the neuroretinal tissue are considered in a manner that would not be captured by quantitative measurements of the cup-to-disc ratio. As such, we compared the performance of a qualitative evaluation for detecting progressive glaucomatous damage to an event-based analysis of the global cpRNFL thickness measurements.

Methods

This longitudinal observational study was approved by the institutional review boards of Columbia University and the New York Eye and Ear Infirmary of Mount Sinai, and adhered to the tenets of the Declaration of Helsinki and Health Insurance Portability and Accountability Act. Written informed consent was obtained from all participants involved in this study.

Participants

This study included participants with a clinical diagnosis of glaucoma or suspected glaucoma based on a comprehensive examination by a glaucoma specialist (RR). Eligibility criteria for this study included eyes being free from retinal pathology that could affect the inner retina (e.g., epiretinal membranes). Eyes also were required to undergo a reliable visual field test using the Swedish Interactive Threshold Algorithm (SITA) standard 24-2 testing strategy on a Humphrey Field Analyzer II-i (Carl Zeiss Meditec, Inc, Dublin, CA). Visual field tests were considered unreliable if they had >33% fixation losses, >15% false-positive errors or >33% false-negative errors (except when mean deviation [MD] was <−12 dB for the latter).

Healthy eyes were included to provide estimates of normal age-related changes in the wide-field RNFL thickness measurements. Healthy participants were originally included in a reference database study by the OCT device manufacturers (data provided by Topcon, Inc., Tokyo, Japan). One eye was chosen randomly as the study eye for each participant, and they were required to have a best-corrected visual acuity of 20/40 or better and intraocular pressure of ≤21 mm Hg. Eyes with narrow angles or any ocular pathology or glaucomatous visual field abnormalities on a visual field test performed using the SITA Standard 24-2 strategy were not included. Participants also were required to be free from a significant medical history that could affect the test results.

OCT Imaging

Wide-field volume scans consisting of 512 × 256 A-scans over a 12 × 9 mm region encompassing the optic disc and macula were acquired for all eyes using a swept-source OCT device (DRI OCT-1 and DRI OCT Triton for the glaucoma and healthy eyes, respectively; Topcon, Inc.). Any scan with significant eye movements or blink artifacts affecting the central 10 × 7 mm region (excluding 1-mm from each edge) were excluded. Glaucomatous eyes were required to have two wide-field scans obtained either at least 1 year apart to examine longitudinal change in RNFL thickness measurements (“signal”), or two scans obtained within the same session to determine estimates of measurement variability (“noise”). Eyes that met these criteria formed the “longitudinal” and “variability” groups, respectively, and eyes that met both criteria (i.e., eligible to be included in either group) were automatically allocated to the longitudinal group.

Methods Used to Identify Progression

Two methods were evaluated for detecting progressive glaucomatous damage using the wide-field OCT scans, including: (1) a quantitative analysis to detect age-adjusted global cpRNFL thickness changes that exceed estimates of measurement variability, and (2) a qualitative evaluation of OCT imaging to assess the probability of progressive glaucomatous damage being present. For both methods, a customized MATLAB program (MathWorks, Natick, MA) was written to manually coregister the pairs of wide-field OCT scans from the same eye. This was performed for eyes included in the longitudinal and variability groups, and the coregistration was performed using retinal features (including the optic disc and retinal blood vessels) visible on en face projection images.

For the first method, the global cpRNFL thickness measurements were obtained from a derived circle scan from the wide-field scans. This derived circle scan had a diameter of 3.4 mm and was averaged over an annulus of 100 μm in width. The normal age-related change in global cpRNFL thickness was estimated by calculating the slope of a linear regression model fitted between these values and age for all eyes in the normative group. Its measurement variability was then estimated by calculating the standard deviation (SD) of the test–retest differences of its values for all eyes in the variability group (see the Statistical Analysis section below for further details). The threshold for progression was then defined as the 5% lower limit of the test–retest differences, determined by multiplying the SD of the differences by −1.645. Eyes in the longitudinal group were then deemed to have progressed if the change in their global cpRNFL thickness over time, subtracting the expected age-related changes (determined by multiplying the normal rate of age-related change by the follow-up duration), exceeded this threshold.

For the second method involving a qualitative evaluation of the wide-field OCT scans, customized one-page reports were generated for the eyes in the longitudinal and variability groups (Fig. 1). This report was modeled after a customized one-page wide-field OCT scan report that we described previously.[15,16] However, it was modified to ensure that all its components were spatially coregistered between the two reports from an eye; thus, allowing subtle changes in each component to be visualized when both reports were overlaid and alternated, in a similar manner to flicker chronoscopy. These report pairs were de-identified and presented in random order using a custom-written program that allowed alternations of these wide-field OCT scan reports, and were graded by one experienced examiner. The examiner was required to provide a grading of the probability of progression occurring using a continuous probability scale (between 0% and 100%; with higher values indicating a higher perceived likelihood of progression). This grading was entered in the customized program, which also automatically recorded the time spent performing the grading. Similar to the first method, a threshold for progression was determined by calculating the upper fifth percentile of the probability grading of the eyes in the variability group. Thus, eyes in the longitudinal group that had a probability grading higher than this threshold were deemed to have progressed. This continuous probability scale is advantageous, since it allowed the sensitivity to be calculated at a fixed specificity for any examiner, even if different examiners used the scale in a different manner.

Figure 1

Example of the one-page report used for qualitative evaluation of the probability of progression occurring, consisting of a derived circumpapillary circle scan and its corresponding RNFL thickness profile (top left), the corresponding quadrant and clock-hour circumpapillary RNFL thickness values shown using pie charts (bottom middle), an en face slab image of the inner retina and RNFL thickness map (bottom left), RNFL thickness probability map presented in field view, with 24-2 visual field (VF) test locations overlaid (top right), and the macular ganglion cell plus inner plexiform layer (GCL+) thickness map and GCL+ thickness probability map presented in field view, with 10-2 VF test locations overlaid (bottom right). Note: information presented in field view are inverted relative to information presented in retina view.

Statistical Analysis

The SD of the test–retest differences of the global cpRNFL thickness in the variability group was determined using a random intercept model (a type of linear mixed model) to account for the hierarchical nature of the data (i.e., to account for two eyes originating from the same participant). The proportion of eyes deemed to have progressed by the two methods evaluated in this study was compared using McNemar's test. All statistical analyses were performed using MATLAB and Stata (StataCorp LP, College Station, TX).

Results

Participant Characteristics

A total of 125 eyes of 82 participants diagnosed clinically with glaucoma or suspected glaucoma were included in the longitudinal group, in which two wide-field scans were obtained at least 1 year apart. Median (interquartile range [IQR]) age and follow-up duration were 61 (IQR, 49–68) and 1.6 (IQR, 1.1–2.0) years, respectively. Median visual field MD and pattern standard deviation (PSD) were −2.22 (−4.69 to −0.49) and 1.96 (1.58–6.24) dB, respectively. Another nonoverlapping cohort of 284 eyes of 183 participants, also with a clinical diagnosis of glaucoma or suspected glaucoma, was included in the variability group, in which two wide-field scans were obtained within the same session. Median age was 62 years (IQR, 48–69), and median visual field MD and PSD were −2.43 (−5.84 to −0.87) and 2.23 (1.55–6.53) dB, respectively. Finally, 418 eyes of 418 healthy participants were included to provide a cross-sectional estimate of age-related changes; median age of these participants was 53 ± 16 years (range, 23–87).

Qualitative and Quantitative Results and Parameters

The median graded probability of progression for eyes in the longitudinal and variability group was 5% (IQR, 1%–60%) and 5% (IQR, 1%–15%), respectively (although note that the mean was 25% and 12%, respectively; similar values were not unexpected given that only a relatively small proportion of eyes in the longitudinal group were expected to progress). The median change in global cpRNFL thickness between the two visits for the eyes in the longitudinal group was −0.8 μm/y (IQR, −1.7 to 0.3 μm/y), and the SD of the test–retest difference in the variability group was 2.1 μm). Mean age-related change estimate of cpRNFL thickness from the normative group was −0.07 μm/y.

Comparison of Methods to Detect Progression

At a specificity of 95%, 33 (26%) and 25 (20%) eyes were identified as having progressed according to the qualitative and quantitative methods, respectively. However, this difference was not significant (P = 0.152). A total of 17 eyes were identified using both methods (κ = 0.46 ± 0.09), and these findings are summarized in Figure 2.

Figure 2

Proportional Venn diagram illustrating the number of eyes deemed as having progressed with qualitative evaluation of the OCT imaging information (left) and quantitative analysis of the cpRNFL thickness measurements (right) over time.

One example where progression was identified using both methods is shown in Figure 3. In this case, an inferior-temporal arcuate defect appeared to deepen and widen based on changes on the cpRNFL thickness profile, wide-field RNFL thickness, and macular ganglion cell plus inner plexiform layer (GCL+) thickness maps. This example along with two further examples, are shown in Supplementary Video S1 using the alternation flicker method.

Figure 3

An example of a case where qualitative evaluation of the OCT imaging information and quantitative analysis of the cpRNFL thickness measurements over time considered the eye as having progressed.

Review of Cases Missed by Each Method

All cases showing disagreement between the methods were reviewed to better understand the factors contributing to their disagreement. All eight eyes where progression was detected by the quantitative analysis of global cpRNFL thickness, but not the qualitative evaluation, were observed to be free of established glaucomatous damage; only one eye appeared suspicious of having glaucomatous damage. All of these eyes also appeared to exhibit a generalized decline in the neuroretinal thicknesses, without any changes consistent with glaucomatous patterns of progression. An example of one of these eyes is shown in Figure 4. This example along with three others (including the eye suspicious of having glaucomatous damage on OCT), are shown in Supplementary Video S2 using the alternation flicker method. These findings suggested that the quantitative analysis of the global cpRNFL thickness values may have overestimated the proportion of eyes that were truly progressing.

Figure 4

An example of a case where the quantitative analysis of the cpRNFL thickness measurements considered progression to have occurred, but qualitative evaluation of the OCT imaging information did not on the basis that no characteristic pattern of glaucomatous progression was present.

All 16 eyes where progression was detected by qualitative analysis of the OCT information, but not quantitative analysis of the global cpRNFL thickness, showed characteristic deepening and widening of regions of established glaucomatous damage. These changes often were subtle, and, indeed, the median rate of global cpRNFL thickness change for these eyes was lower than that of the remaining 17 eyes detected by both methods (−1.2 and −5.3 μm/y, respectively; P < 0.001). An example of one of these eyes also is shown in Figure 5, and this along with three further examples also are shown in Supplementary Video S3 using the alternation flicker method.

Figure 5

An example of a case where the qualitative evaluation of the OCT imaging information detected progression missed by quantitative analysis of the cpRNFL thickness measurements.

Discussion

This study demonstrated that the proportion of glaucomatous eyes deemed to have progressed by qualitative evaluation of OCT imaging information and quantitative event-based analysis of global cpRNFL thickness measurements was not statistically significantly different, but there was only a moderate level of agreement between the two methods. A post hoc evaluation revealed that all eyes missed by qualitative evaluation did not show any characteristic patterns of established glaucomatous damage or its progression, but rather a generalized decline in neuroretinal thickness measurements. All eyes missed by quantitative analysis instead demonstrated characteristic patterns of glaucomatous progression, such as deepening and/or widening of existing defects. These findings, thus, highlight the potential value of the qualitative approach for detecting glaucomatous progression, and warrant further investigation of this technique.

Qualitative evaluation of OCT imaging information for detecting glaucomatous progression was based on a similar concept to flicker chronoscopy of optic disc photographs. In particular, to allow detection of subtle changes, images were superimposed and viewed by alternation flicker. This approach has been used for decades to detect changes in optic disc appearance, and has been reported to perform better than conventional methods, including side-by-side or manual evaluations.[17-20] However, this approach has been rarely used in clinical practice due to technological limitations associated with an accurate, automatic coregistration of the optic disc photographs and the burden of manual image alignment. On the other hand, rapid advancements with OCT imaging technology now allow precise scan acquisition at the same retinal locations over time.[21-23] As such, a qualitative evaluation of longitudinal OCT results with alternation flicker or serial presentations could be implemented more readily. To our knowledge, this is the first study to implement this approach for OCT imaging results, and to compare it to a conventional quantitative analysis of global cpRNFL thickness measurements.

Our finding of only a moderate agreement between the eyes deemed to have progressed by the qualitative and quantitative approaches on OCT imaging is comparable with similar observations with longitudinal visual field results, where previous studies reported only a fair to moderate level of agreement between these two approaches.[24-26] However, those previous studies did not attempt to scrutinize the cases of disagreement to understand potential contributing factors. Undertaking such a post hoc review instead revealed important reasons underlying the disagreements observed.

Firstly, we observed that all eyes considered to have progressed by qualitative evaluation, but not by quantitative analysis, exhibited characteristic deepening and widening of established glaucomatous defects.[27] These progressive changes were missed by quantitative analysis, since they occurred locally and were too subtle to be detected by the global cpRNFL thickness metric relative to measurement variability. Furthermore, qualitative evaluation of the entire wide-field OCT scan also allowed neuroretinal changes outside the circumpapillary circle scan to be considered, which provided greater precision in the assessment of progressive changes visible on the cpRNFL thickness profiles.

Secondly, the post hoc review of the cases detected by quantitative, but not by qualitative, evaluation revealed that none had regions of established glaucomatous damage or showed characteristic patterns of glaucomatous progression. Instead, they showed a generalized reduction in the cpRNFL thickness. It is possible that such changes may represent true diffuse glaucomatous progression, and future studies are required to determine their long-term clinical significance, such as on visual field endpoints.[28-30] However, given that these eyes did not appear to have glaucomatous damage, it is more likely that these changes occurred simply as a result of measurement variability. Note that these are eight (or 6%) of the 125 eyes evaluated that showed change outside the 5% lower limit of test–retest variability. Thus, we hypothesized that the qualitative approach actually detected a greater proportion of eyes exhibiting true glaucomatous progression, but further longitudinal studies are required to determine this as described above.

While these findings highlight the potential advantages of a qualitative evaluation of OCT imaging for detecting glaucomatous progression using an alternation flicker technique, this approach could be improved further when multiple OCT scans are serially evaluated, since progressive changes might be better distinguished from imaging variability when multiple scans are available. Such an approach has been used with various imaging modalities in macular diseases,[31-34] often allowing subtle progressive changes to be detected. Inclusion of quantitative analyses of change over time (e.g., topographic event- and trend-based analyses[9,10]) during the qualitative evaluation also could improve the detection of progressive glaucomatous changes, by making progressive changes more apparent.

Some limitations should be considered when interpreting our findings. First, the within-session variability estimates were used to determine the thresholds of change at a fixed specificity, which often are slightly less variable than short-term between-session estimates.[35-37] Despite this, the between-method comparison for detecting progression still remains robust because the estimates of specificity were obtained from the same cohort for both methods. Second, progression was evaluated using only information from two visits over a relatively short duration. However, increasing the number of tests and follow-up duration would likely strengthen this study without altering its conclusions, given how the post hoc review of the cases of disagreement revealed that there were consistent factors that contributed to the observed differences. However, we recognize that the post hoc review also can be subject to confirmation bias, and, thus, presented numerous cases for public review. Ultimately, future longitudinal studies are needed to determine the implications of the qualitative approach to determine whether it truly is advantageous, but this study provides an initial investigation into this promising novel approach. Third, this study only included a single examiner, so the generalizability and intraexaminer repeatability of this method remain to be established. Finally, this study only evaluated the commonly-used quantitative parameter of global cpRNFL thickness change, although future studies could further exploit the information from the whole scan for detecting progressive neuroretinal changes, such as through using topographic event- and trend-based analyses.[9,10]

In summary, our study observed that qualitative evaluation of OCT imaging information identified a higher proportion of glaucomatous eyes as having progressed compared to quantitative analysis of global cpRNFL thickness measurements at matched specificities, although this difference was not statistically significant. However, a post hoc review of the cases where the two methods disagreed revealed how the qualitative approach more often detected changes consistent with known patterns of glaucomatous damage, thus, highlighting the potential value of this technique.

Click here for additional data file.

Click here for additional data file.

Click here for additional data file.