The effect of software upgrade on optical coherence tomography measurement of the retinal nerve fiber layer thickness.

PURPOSE: To determine the effect of software upgrades on retinal nerve fiber layer (RNFL) thickness measurements taken by spectral domain optical coherence tomography (SD-OCT).
METHODS: Eighty normal eyes (40 patients) were scanned for RNFL thickness measurements using Spectralis (Heidelberg Engineering, Heidelberg, Germany) SD-OCT. Scan analysis was performed using version 4.0 software and then reanalyzed with version 5.1.3. Student paired t testing and Pearson's correlation coefficient were used for statistical analysis.
RESULTS: Average and quadrant RNFL thicknesses generated using version 4.0 and 5.1.3 software on Spectralis demonstrated high correlation (r = 0.955-0.998). Average RNFL thickness using version 4.0 was 0.08 μm thinner than version 5.1.3 (p = 0.409). Quadrant RNFL differences ranged from -0.26 to +0.97 μm (p = 0.146-0.915). Segmentation errors were reduced 33% after the upgrade.
CONCLUSION: Minor RNFL thickness changes may occur after software upgrades in Spectralis OCT. The differences did not reach statistical significance but segmentation errors were improved.

INTRODUCTION

Since the commercial introduction of optical coherence tomography (OCT), clinicians have used this laser diagnostic modality for the identification and monitoring of various ophthalmic diseases. OCT has been particularly useful in assessing the retinal nerve fiber layer (RNFL) thickness in patients being followed for glaucoma or those diagnosed as glaucoma suspects.12 These measurements not only serve as a possible method for diagnosing preperimetric glaucoma, but also may objectively measure disease progression after the initial diagnosis has taken place.3–5 Three major iterations of the Stratus time-domain OCT (TD–OCT) have been released over the past two decades. Now, these machines are being replaced by spectral-domain OCT (SD-OCT) instruments. The progression of the technology has brought about significant improvements in speed, resolution, and reproducibility. Image acquisition speed has increased from 512 A-scans/s to 29,000–55,000 A-scans/s depending on the machine used.267 Image resolution of commercially available SD-OCT models range between 5 and 7 μm compared to 10-15 μm with TD-OCT.27–9 Three-dimensional (3D) volume data sets can now be generated as well.2

With each step in the evolution of OCT, measurements must be compared to the prior generations to assure a seamless transition in clinical practice. This is especially true when monitoring glaucomatous RNFL loss where changes may span over multiple years.10 In the previous work, we have reported that RNFL measurements taken in normal subjects on different SD-OCT instruments are significantly different than prior TD-OCT machines.11 Several other groups have analyzed these relationships in diseased eyes and have shown similar findings.12–14 While the change in hardware from TD-OCT to SD-OCT imaging is readily recognizable to the treating clinician, other software upgrades within the same OCT machines may not be as obvious. These changes in software may alter image quality, speed of acquisition, and/or segmentation algorithms. In this study, we analyze the difference in RNFL thickness measurements taken with Spectralis (Heidelberg Engineering, Heidelberg, Germany) SD-OCT before and after manufacturer released software upgrades and explore changes that may or may not be clinically significant when caring for glaucoma patients.

MATERIALS AND METHODS

The study protocol was written in accordance with the Declaration of Helsinki and approval from the Colorado Institutional Review Board was obtained prior to recruitment and testing of subjects. All subjects signed an informed consent and separate Health Insurance Portability and Accountability Act authorization form. A total of 40 normal subjects were recruited at the University of Colorado Hospital Eye Center in Aurora Colorado. Each participant underwent baseline ophthalmologic examination to ensure proper eligibility for the study. Screening examination-included assessment of snellen visual acuity, refractive error by auto-refraction, full slit lamp exam, intraocular pressure (IOP) by goldmann applanation, and gonioscopy. A Swedish Interactive Threshold Algorithm 24-2 full-threshold Humphrey Visual Field (Carl Zeiss Meditec, Dublin, CA, USA) was also performed. All examinations were performed by one of two ophthalmologists (LKS and MYK). A single experienced ophthalmic technician administered the visual field examinations and the results were interpreted by a glaucoma specialist (MYK).

Inclusion criteria were set before recruitment and consisted of a best-corrected visual acuity of ≥20/40 or better, spherical equivalent between +3.00 and -6.00 Diopters, normal IOP (10-21 mmHg), normal gonioscopic examination with full view to at least the scleral spur, reliable visual field examination with normal glaucoma hemifield testing, optic nerves without hemorrhage or abnormalities of the neuroretinal rim or cup to disc ratio, and absence of cup to disk asymmetry greater than 0.2. Exclusion criteria consisted of a history of glaucoma or ocular hypertension, suspicion of glaucoma on exam, or presence of any ocular disease other than mild age-related cataracts. Both eyes of every subject were scanned on a single Spectralis OCT instrument by a single experience ophthalmic technician. Scans were performed in random order over the course of 1 month depending on subject convenience. If an unreliable scan was performed secondary to patient movement or blinking, the scan was repeated at the discretion of the technician. All scans were reviewed to make certain a signal strength of >15 dB was achieved. High-resolution mode was used for all scans in addition to utilization of the eye tracking ART (automatic real-time) feature.

At study completion, RNFL reports were generated and measurements were collected using the then-current Spectralis software version 4.0. Next, a software upgrade to version 5.1.3 was performed on the site by a Heidelberg Engineering service representative. All study scan data were then retrieved from the identical study date as before and analyzed in similar fashion for quality. If a scan report possessed an error in segmentation using only version 4.0 or 5.1.3, the entire eye was discarded from the study.

Statistical analysis

All statistical analyses were performed using STATA software (Stata Corp., College Station, TX, USA). Both the average and each of the four quadrants RNFL thickness measurements from software version 4.0 reports were compared with those of version 5.1.3 by use of Student paired t-tests. Standard errors were also calculated for each mean value. Scatter plots were generated for version 4.0 RNFL thickness against version 5.1.3 RNFL thickness. Agreement was evaluated using Bland–Altman plots and calculated Pearson correlation coefficients.

RESULTS

Out of the 80 eyes of 40 subjects enrolled in the study, only one did not qualify for inclusion in the study due to prior trauma. Sixteen (40%) of the 40 subjects were male and 24 (60%) were female. The mean (± standard deviation) age of subjects was 37.1 ± 11.0 years (range 21–61 years). Mean best-corrected Snellen visual acuity was 20/20, with spherical equivalent of -0.8 ± 1.7 Diopters. The mean IOP was 15.6 ± 2.3 mmHg, and mean cup-to-disk ratio was 0.3 ± 0.1. The mean deviation from visual field testing was 0.18 ± 1.25 dB while mean pattern standard deviation was 1.62 ± 0.47 dB. All patients had normal glaucoma hemifield test results.

From the 79 qualifying eyes, 6 (7.6%) were removed from analysis due to inaccurate segmentation of the RNFL boundaries using version 4.0. Complete failure of segmentation occurred in two of these, while the remaining four contained errors of <50% of the scan circle length. After software upgrade to 5.1.3, two of the six (33%) scans were correctly segmented. One example of complete segmentation algorithm failure and subsequent resolution after version upgrade is shown in Figure 1.

Figure 1

(a) Peripapillary RNFL scan with an infrared image and RNFL map using Spectralis OCT software version 4.0 (b) Exactly same scan reprocessed after software upgrade to version 5.1.3

Mean RNFL thickness measurements are listed in Table 1. Using version 4.0 software, the average RNFL thickness was found to be 106.74 ± 13.31 μm. After the upgrade to version 5.1.3, the average thickness measured was 106.65 ± 13.14 μm. All quadrant and average thickness measurements decreased slightly after the upgrade except for the superior quadrant. The greatest change in the thickness was noted in the superior quadrant (+0.97 μm), while the inferior quadrant showed the smallest change (-0.04 μm). None of these differences were statistically significant. Pearson correlation coefficients were extremely high for all measurements (0.986–0.998). The scatter plot in Figure 2 displays the high correlation of measurements from version 4.0 to 5.1.3. Bland–Altman plots were constructed to demonstrate the relationship of mean and quadrant RNFL thickness measurements using version 4.0 and 5.1.3. Figure 3 demonstrates the fairly consistent trend of differences between the two software versions across all thicknesses. Similar plots for RNFL measurements in each quadrant were also produced and demonstrated a similar consistent difference between the software versions despite variations in thickness (not shown due to space constraints).

Table 1

Comparison of retinal nerve fiber layer thickness measurements from Spectralis optical coherence tomography using software version 4.0 and 5.1.3

Figure 2

Scatter plot of average RNFL thickness using software version 4.0 against that of version 5.1.3 along with linear regression line

Figure 3

Bland–Altman plot demonstrating agreement between mean RNFL thickness measurements (in μm) between software versions 4.0 and 5.1.3. The linear regression line demonstrates any change in agreement across different RNFL thicknesses

DISCUSSION

One of two types of software revisions may be supplied to the clinician according to the Heidelberg Engineering website (http://www.heidelbergengineering.com/technical-support-heidelberg-engineering/software-policy/). Minor software revisions called “updates” are provided to consumers at no additional cost and are signified by a change in the version number after the decimal point (e.g., 1.0 to 1.1). These updates are released in order to fix minor software inconsistencies, maintain the product, or insert basic new functions. Major software revisions termed “upgrades” may require additional cost to the consumer and can be identified by a change in the version number before the decimal point (e.g., 1.0 to 2.0). Upgrades may consist of new analysis methods, applications, significant new function upgrades, and/or the expansion of clinical normative data.

There is a continuously growing body of literature detailing the many factors that may significantly alter RNFL thickness measurements by OCT. We have previously demonstrated a significant difference in RNFL measurements when moving from TD-OCT to one of three SD-OCT instruments.11 Others have shown similar findings that remain evident in diseased eyes as well.12–14 Work by Balasubramanian et al. showed the profound effect of image quality on the RNFL thickness. A +2.0 diopter defocus resulted in a 10 μm decrease in thickness for Cirrus OCT and a 20–40 μm increase in thickness for Spectralis.15 Similarly, Mwanza et al. discovered a 9.3% increase in average RNFL thickness after signal quality improvement from cataract extraction.16 Recent work by Mylonas et al. specifically looked at segmentation errors for the total retinal thickness in macular OCT scans. Stratus TD-OCT demonstrated moderate-to-severe errors in 38% of scans while newer SD-OCT instrument errors were as low as 4%. In their study, the error frequency for Spectralis was 27% for moderate and severe errors. The rate of segmentation errors for identification of RNFL boundaries in our study was much lower at 7.6%.17 However, their study contained a population of eyes with age-related macular degeneration which may have influenced alignment of the scan circle due to fixation problems.

In this study, we analyze the significance of a major software upgrade for the Spectralis SD-OCT as it relates to RNFL measurements and scan quality. While average and quadrant RNFL measurements did infact change (-0.26 to +0.97 μm) after the change in software, the differences failed to reach the level of statistical significance. Recent work by Medeiros et al. analyzed Stratus TD-OCT's ability to detect progression of disease as evidenced by ONH stereo-photography change and/or HVF decline. In their cohort of 253 patients, 31 (13%) demonstrated progression over the mean follow-up time of 4.1 years. OCT successfully differentiated progressors from nonprogressors with significantly different rates of RNFL changes between the two groups. Progressors’ mean RNFL loss occurred at a rate of -0.72 μm/year while nonprogressors changed at 0.14 μm/year.10 Given this relatively slow rate of RNFL decline in progressors and the small difference compared to nonprogressors, it is reasonable to have some concerns regarding the minor changes in the RNFL thickness that we observed by simply changing the system software. A difference of 0.97 μm (superior quadrant change from V4.0 to V5.1.3) could alter the detection of progression by OCT monitoring alone by more than a year. More research is needed to elucidate the rate of progression of RNFL thinning in glaucoma “progressors” using SD-OCT and whether or not these values would be in the range noted by Medeiros and colleagues with TD-OCT.

The most obvious explanation for the minor changes in the RNFL thickness after software upgrade is an improved segmentation algorithm. Without histological gold standards, it is impossible to determine if such an alteration in software algorithms is more or less accurate. However, a noticeable improvement was noted in the number of usable scans. The number of segmentation errors/failures was decreased by 33% with the new software version. This is an obvious benefit to the clinician. Failures such as these have even motivated the development of external software to process OCT scan data.18

While we believe that our findings of only slight changes in RNFL measurement thickness after software upgrade are of clinical value, several limitations should be kept in mind. First, not all software revisions are equal. Our data only account for one specific upgrade on a single OCT instrument. Each upgrade may contain different features or improvements to analysis and would therefore result in differing degrees of data alteration. Second, our study group consisted of normal eyes only and the patients consisted were younger than the typical glaucoma population.

CONCLUSIONS

In conclusion, major software upgrades should be taken into account when following an RNFL thickness with SD-OCT. Clinicians should be aware of this fact and remain alert to the potential impact on segmentation and perceived changes in RNFL data. While these changes are measurable, they do not reach statistical significance in this particular instance of upgrade and patient population. Improved algorithms can however resolve errors in segmentation increasing reliability of thickness measurements in addition to the number of readable scans. Further studies are needed to determine the effect of software upgrades on scan analysis by other SD-OCT devices that are commercially available.