Signal Normalization Reduces Image Appearance Disparity Among Multiple Optical Coherence Tomography Devices.

PURPOSE: To assess the effect of the previously reported optical coherence tomography (OCT) signal normalization method on reducing the discrepancies in image appearance among spectral-domain OCT (SD-OCT) devices.
METHODS: Healthy eyes and eyes with various retinal pathologies were scanned at the macular region using similar volumetric scan patterns with at least two out of three SD-OCT devices at the same visit (Cirrus HD-OCT, Zeiss, Dublin, CA; RTVue, Optovue, Fremont, CA; and Spectralis, Heidelberg Engineering, Heidelberg, Germany). All the images were processed with the signal normalization. A set of images formed a questionnaire with 24 pairs of cross-sectional images from each eye with any combination of the three SD-OCT devices either both pre- or postsignal normalization. Observers were asked to evaluate the similarity of the two displayed images based on the image appearance. The effects on reducing the differences in image appearance before and after processing were analyzed.
RESULTS: Twenty-nine researchers familiar with OCT images participated in the survey. Image similarity was significantly improved after signal normalization for all three combinations (P ≤ 0.009) as Cirrus and RTVue combination became the most similar pair, followed by Cirrus and Spectralis, and RTVue and Spectralis.
CONCLUSIONS: The signal normalization successfully minimized the disparities in the image appearance among multiple SD-OCT devices, allowing clinical interpretation and comparison of OCT images regardless of the device differences. TRANSLATIONAL RELEVANCE: The signal normalization would enable direct OCT images comparisons without concerning about device differences and broaden OCT usage by enabling long-term follow-ups and data sharing.

Introduction

The invention of optical coherence tomography (OCT) has led the medical diagnostic imaging technology into a new era.[1,2] With the ability to generate in vivo cross-sectional images of tissue structures with microscopic resolution (as high as 1 μm) in a noninvasive and noncontact fashion, OCT has become an indispensable standard in daily clinical care in ophthalmology; moreover, it improved not only the understanding of disease pathogenesis,[3-5] but also clinical diagnosis and management of various diseases.[3-9]

OCT has been proven to be very helpful in diagnosing retinal pathologies, elucidating pathological processes, determining whether surgical intervention is required, or monitoring surgical outcome.[5,10-13] However, the diversity of OCT images produced by the variety of OCT manufacturers results in OCT image data incompatibility, as well as generates substantial differences in both qualitative and quantitative OCT measurements and interpretations among the different devices.[14-17] The variability in OCT image intensity, ocular tissues contrast, sampling density in the axial direction, image quality, and OCT signal noise level may influence the interpretation of OCT images and present a substantial clinical challenge when images acquired with different OCT devices are compared. Previously we reported that a signal normalization method successfully reduced the variability among OCT signals from various OCT devices.[16,17] The signal normalization method also proved its ability to minimize the systematic retinal layer thickness measurement differences among three commercially available spectral-domain (SD-) OCT devices.[16-18] Furthermore, differences were minimized even between images where frame averaging is used during image acquisition and images without averaging.

In this study, we investigated the ability of our signal normalization method to reduce the discrepancies in OCT image appearance among different OCT devices. Our hypothesis was that our signal normalization technique would improve the similarity in image appearance among different OCT devices. The purpose of this study was to assess the effect of the signal normalization on reducing the discrepancies of image appearance among SD-OCT devices by subjectively evaluating the image similarity before and after signal normalization.

Methods

Subjects

Subjects providing OCT cross-sectional images for the questionnaire were recruited at the University of Pittsburgh Medical Center Eye Center, including a variety of retinal pathologies to test the effect of the signal normalization method on image appearance with various pathologies. We enrolled eyes with glaucoma, age-related macular degeneration (AMD), diabetic retinopathy, macular hole, and cystoid macular edema (CME). The University of Pittsburgh Review Board and ethics committee approvals were obtained for the study, and informed consent was obtained from all subjects who contributed the OCT images.

This study followed the tenets of the Declaration of Helsinki, and was conducted in compliance with the Health Insurance Portability and Accountability Act.

Image Acquisition

The macular region from all eyes were scanned using at least two out of three SD-OCT devices (listed below) at the same visit: Cirrus HD-OCT (Zeiss, Dublin, CA; software version 6.1), RTVue OCT (Optovue, Fremont, CA; software version 6.1), and Spectralis (Heidelberg Engineering, Heidelberg, Germany; software version 1.5).

Cirrus HD-OCT

Macular Cube 200 × 200 scan pattern was used to obtain volumetric cube data centered on the foveola. The scanning protocol collected 200 × 200 A-scans over a 6 × 6 mm2 area centered on the foveola with 1024 sampling points along 2.0-mm axial direction for each point.

RTVue OCT

Raster cube scan patterns (200 × 200 and 513 × 101) centered on the foveola were used to acquire RTVue image data at the macular region. Isotropic 200 × 200 raster cube scan pattern collected 200 × 200 A-scans over a 6 × 6 mm2 area centered on the foveola and 640 sampling points along 1.96-mm axial direction for each point. Anisotropic 513 × 101 raster cube scan pattern had the same specifications as the 200 × 200 raster cube scan, except for the uneven sampling density in x- and y-directions.

Spectralis

High definition macular raster volume scan pattern centered at the foveola covering a 20° × 20° region (193 sections with nine-frame averaged for each section) was used to acquire the image data for the macular region. This anisotropic raster cube scan pattern collected 1024 × 193 A-scans with 496 sampling points along 1.9-mm axial direction.

Images with an image quality below the manufacturer's recommended cutoff (signal strength [SS] below six for Cirrus, signal strength index [SSI] below 40 for RTVue, and image quality below 15 for Spectralis), or images with apparent eye movement during scanning were considered poor quality images and discarded. Eye movement was subjectively defined as image artifacts on OCT en face images showing a horizontal frame shift larger than one average sized retinal blood vessel diameter or a major distortion of the fovea region. All of the OCT raw data were exported to a standalone computer for signal normalization and further analysis.

Signal Normalization Processing

Previously reported signal normalization methods were applied to all the OCT images.[16,17] In brief, the signal normalization contained four steps: z-scaling and sampling density normalization, speckle noise reduction, amplitude normalization, and image quality compensation. Z-scaling and sampling density normalization resolved the scaling and sampling density variation in the axial direction; speckle noise reduction removed the speckle noise; amplitude normalization optimized and equalized the dynamic range of the retinal signals; and the image quality compensation (high dynamic range and histogram matching techniques) fixed the image quality discrepancy.[19,20] RTVue and Spectralis image data were converted to Cirrus equivalent data format. For the Cirrus and RTVue data, the previously described virtual averaging technique was applied to minimize the differences in image appearance between nonframe averaged (Cirrus and RTVue) and frame-averaged OCT images (Spectralis).[18] All the processing steps were fully automated with predefined coefficients and parameters. No manual adjustment was needed during signal normalization processing.

Subjective Evaluation

To subjectively evaluate the effect of our signal normalization on reducing the variation in image appearance among OCT devices, a questionnaire was created. The questionnaire contained 24 cases each including a pair of cross-sectional OCT images scanned at the corresponding location on the same eye at the same visit but with two different OCT devices (any combination among Cirrus, RTVue, and Spectralis; Fig. 1). Image pairs were displayed in a random and masked fashion. Observers were asked to evaluate the similarity of the image appearance for the two displayed images based on (1) the contrast between the retinal signal and the background noise (Fig. 2A), (2) the contrast between adjacent retinal layers (Fig. 2B), and (3) the textures or patterns of retinal nerve fiber layer (RNFL) and retinal pigment epithelium (RPE; the major high intensity retinal layers) according to their visual perception (Fig. 2C). The similarity was recorded in a Likert-type five-point scale ranging from one point to five points,[21] using the following scale: (1) Significantly different (0%–20% similarity), (2) Somewhat different (20%–40% similarity), (3) Cannot decide if it is similar (40%–60% similarity), (4) Looks similar (60%–80% similarity), and (5) Nearly identical (as if taken by the same device) (80%–100% similarity). When judging the similarity, the differences in retinal axial location in the scanning window, retinal orientation, and pathological contexts were not considered. The first image pair in Figure 1 represents an example of a “significantly different” image pair (point 1), while the second image pair represents an example of the “nearly identical” image pair (point 5).

Figure 1

Sample pairs of the questionnaire. For each question, two images are displayed and observers are asked to judge the similarity based on the image appearance (contrast between retinal signal and background noise, contrast between adjacent retinal layers, and textures in the retina). Image pairs before and after signal normalization are displayed in a random and masked manner. A wide variety of retinal pathology was included in the questionnaire.

Figure 2

Image appearance similarity comparison criteria: (A) the contrast between the retinal signal and the background noise, (B) the contrast between adjacent retinal layers, the red and blue arrows point to the RNFL and the ganglion cell layer (GCL) as an example, and (C) the textures of RNFL and RPE (the major high intensity retinal layers).

Eye care specialists of various levels including residents in ophthalmology, attendings in glaucoma and retina specialties, medical students, medical imaging device technicians, and researchers having experience with OCT were invited to participate in the study. An electronic invitation was sent out first, and if the recipient agreed to participate, a face-to-face meeting was scheduled to complete the questionnaire.

A verbal explanation about the nature of this survey was given followed by a training set of five pairs of images presented as examples before the actual survey started. The expected decisions were demonstrated to the graders (without disclosing whether this image pair is before or after signal normalization) in order to establish a common similar judging standard.

Questionnaire Repeatability

A subset of observers repeated the questionnaire within a 2-year interval to evaluate the reliability and repeatability of the questionnaire. Intraclass correlation coefficient (ICC) two-way mixed model with its 95% confidence interval (CI) on absolute agreement was used to determine the test–rest reliability.[22] The values of the ICC can range from 0 to 1, with a higher value indicating better reliability. ICC less than 0.40 was considered as poor, values between 0.4 and 0.6 as fair, values between 0.61 and 0.8 as good, and values higher than 0.8 as excellent.[23]

Statistical Analysis

Odds ratios were used to analyze the effect of the signal normalization on improving the similarity between OCT images. Cumulative link mixed model was used to estimate the parameters for the ordinal mixed effects models in order to assess the effects of signal normalization technique. The R Environment and Language for Statistics (version 3.1.0) was used to conduct the statistical analysis.[24] IBM SPSS Statistics for Windows (IBM, Armonk, NY; version 23.0) was used for computing ICC.

Results

Two glaucoma, two AMD, three macular hole, two diabetic retinopathy, one CME with nonproliferative diabetic retinopathy (NPDR), and two CME eyes were enrolled and imaged to create the subjective evaluation questionnaire. A total of 29 observers participated in the survey. They were six ophthalmologists (four glaucoma and two retina specialists), 14 researchers, four residents in ophthalmology, and five OCT technicians. All observers have ample experience with OCT imaging either in clinical practice or research.

Figure 3 presents the distribution of the similarity scores of all observers before and after signal normalization. The histograms show that the similarity responses shifted from dissimilar to mostly similar, suggesting that after signal normalization, the similarity between OCT images was improved.

Figure 3

Histogram of subjective evaluation of similarity. A substantial improvement in similarity is noticeable after the signal normalization.

Figure 4 also presents the descriptive summaries, broken down to comparison between devices: Cirrus versus RTVue, Cirrus versus Spectralis, and RTVue versus Spectralis. The same trend was observed in the individual groups: the responses shifted in a positive direction (from dissimilar to similar), further indicating that the signal normalization was able to increase the similarity between OCT images regardless of comparison groups. Another observation was that the image similarity increased after signal normalization to varying degrees among groups where group Cirrus versus RTVue became the most similar pair, followed by group Cirrus versus Spectralis, and then group RTVue versus Spectralis.

Figure 4

Histogram of subjective evaluation of similarity for each comparison group. A substantial improvement in similarity is noticeable after the signal normalization in all comparison groups.

Table 1 summarizes the odds ratio of the comparison between various devices before and after signal normalization. Statistically significant improvement in image similarity was detected for the individual comparison groups after signal normalization. Varying degrees of the effects depending on the comparison combination were observed as Cirrus versus RTVue became the most similar pair (odds ratio in log scale = 6.1, P < 0.0001), followed by Cirrus versus Spectralis (2.9, P < 0.0001), and then by RTVue versus Spectralis (1.2, P = 0.009).

Table 1

Statistical Analysis Results of the Subjective Evaluation

In addition, Figure 5 further shows the detailed information of how the similarity of each image pair was affected by the signal normalization processing in the form of heat maps. In the heat map, the diagonal between the top-left and bottom-right corners indicates the similarity scores remained unchanged, the upper-right triangle means the similarity scores were improved, while the lower-left triangle indicates the similarity scores were degraded by signal normalization processing. As shown in Figure 5, the similarity scores of most image pairs were considered dissimilar (1 and 2) before signal normalization and improved after signal normalization as more red cells were observed in the upper-right triangle of each heat map.

Figure 5

Similarity score of image pairs affected by signal normalization for: (A) overall comparison (a total of 384 comparisons), (B) between Cirrus and RTVue, (C) between Cirrus and Spectralis, and (D) between RTVue and Spectralis (for B, C, and D there are 116 comparisons).

Six observers participated the test–retest reliability analysis. The ICC for test–retest reliability of the questionnaire was 0.87 (95% CI, 0.81–0.91), demonstrating excellent reliability and repeatability of the similarity analysis method.

Discussion

The signal normalization method statistically significantly improved the image similarity, when considering contrasts between retinal layers, textures in ocular tissues, and overall image appearance regardless of pathology. All the steps in the signal normalization method, including normalizing the sampling density and amplitude, reducing the difference in the noise level, and optimizing the dynamic range to the meaningful retinal signal, worked together to standardize the retinal image across OCT technology, and therefore successfully improved the image appearance similarity among OCT devices.

Another important factor was the virtual averaging. By mimicking the image acquisition of frame-averaged images, the virtual averaging reduces the differences between nonframe-averaged and frame-averaged OCT images. The process not only decreased the quantitative measurement differences,[18] but also improved the qualitative comparison between nonframe-averaged and frame-averaged OCT image data. Figure 6 shows an example of an image pair from Cirrus and Spectralis devices of an eye diagnosed with NPDR and CME. Before processing (top row, Figs. 6A, 6B), the textures or patterns in the retinal layers and ocular tissues looked more granular in nonframe-averaged image as can be clearly appreciated in the RNFL (red arrow) and inner nuclear and outer plexiform layers (IN-OPLs; blue arrow). After virtual averaging (bottom row; Figs. 6C, 6D), the retinal layers became smoother, clearer, and showed more detailed structural information, especially for inner retinal layers. Therefore, the signal normalization with virtual averaging was able to reduce the discrepancies in the texture and patterns of retinal layers and enable direct comparisons between nonframe-averaged and frame-averaged OCT image data.

Figure 6

An example of image pair of Cirrus (right) and Spectralis (left) from an eye with NPDR and CME. (A) and (B) show the image pair before signal normalization with virtual averaging, and bottom row, (C) and (D), show the image pair after processing. After virtual averaging, the retinal layers became smoother, clearer, and showed more detailed retinal structural information clearly noticeable in the RNFL (red arrow) and IN-OPLs (blue arrow). Therefore, the signal normalization with virtual averaging was able to reduce the discrepancies in the texture and patterns of retinal layers.

Signal normalization method has demonstrated its ability to reduce the systematic measurement differences among multiple SD-OCT devices and make the RNFL thickness measurements acquired from various SD-OCT devices directly comparable. In this study, we showed that the similarity of OCT cross-sections from different OCT devices improved after signal normalization, allowing direct comparison and evaluation of OCT images across various OCT devices. Subjective qualitative assessment of OCT cross-sectional images is a major part of clinical activity in ophthalmology, especially for evaluating retinal pathologies. However, the variability of OCT signal dynamic range, noise level, and scanning depth difference among different OCT devices results in the inconsistent OCT image appearance that may lead to inconsistency in clinical reading of the images, especially when patients are referred from other practices where different OCT devices are in use. This inconsistency can influence the interpretation of OCT images and cause a serious clinical challenge when clinicians compare images from different OCT devices for diagnosis and decision-making. With the signal normalization method and standardized OCT image appearance, retinal specialists can ignore the specific device that was used to acquire the image and can directly have two images displayed side-by-side even when previous images were acquired by different OCT instruments. This would broaden the use of OCT by enabling a long-term follow-up and allowing data sharing between clinics and organizations.

Image similarity clearly increased after signal normalization for each individual group, but with different degrees of effects. The combination of Cirrus and RTVue became the most similar, followed by the combination of Cirrus and Spectralis, and finally the combination of RTVue and Spectralis. It was interesting that the signal normalization method successfully made the image appearance comparable between Cirrus and RTVue, and substantially reduced the discrepancies in image appearance between Cirrus and Spectralis, but the combination of RTVue and Spectralis showed the least similarity. On the other hand, the combination of Cirrus and RTVue showed the most improvement in similarity (odds ratio in log scale increased from 0.0 to 6.1), followed by the combination of RTVue and Spectralis (from −4.0 to 1.2), and then by the combination of Cirrus and Spectralis (from −0.1 to 2.9). The differences in attained similarity among comparison groups may indicate the limited ability of our signal normalization method, or there are other factors resulting in OCT signal characteristics variability, such as the various device built-in image processing methods applied prior to raw data exportation. In addition to the various device built-in image processing methods, the questionnaire was composed of 24 cases with six pathologies scanned with two out of three OCT devices. This unbalanced disease and device combination may cause the ocular pathology to induce confounding effect in the similarity improvement analysis. Even though observers were instructed to ignore the pathological context in the cross-sectional images similarity evaluation to minimize the effect of the signal variation caused by diseases, we cannot completely rule out this possibility. Further investigation with balanced combination or fixed disease type with variation device combination is warranted.

Concerns may arise that, since the purpose of this study—to evaluate the performance of the signal normalization—was clearly explained to our graders, their decision in judging the image similarity may be biased. In order to avoid this situation and to acquire survey results that reflect the actual truth, the information of whether this image pair was from original devices or after signal normalization was strictly masked as well as the order of the images as appearing on the screen. When establishing the similarity judging standard using the training image set, the graders were guided but not forced to agree with the demonstrated results. From our results that the image similarity was significantly improved after signal normalization but showing various degrees of effects with different device combination, it indicated that our graders were not biased for postnormalization pairs of images during the surveys.

A potential limitation of our study design is that it is not testing the effect of the individual aspects (the contrast between retinal signal and background noise, the contrast between adjacent retinal layers, and the textures in the retinal layer) separately to allow us a better understanding of what aspects in our signal normalization has more power. Our speculation was that the impact of each individual component of the whole signal normalization process is small and may be difficult to detect, but only the combination of all the steps makes noticeable changes in the final outcome. Further investigation is warranted.

In conclusion, the qualitative validation showed that the proposed signal normalization method is able to improve the image similarity among SD-OCT devices. By improving the similarity in image appearance among SD-OCT devices, signal normalization allows direct comparison of OCT images among various instruments, which would broaden the use of OCT technology in both clinical and research applications.