Algorithms for the Automated Analysis of Age-Related Macular Degeneration Biomarkers on Optical Coherence Tomography: A Systematic Review.

PURPOSE: To assess the quality of optical coherence tomography (OCT) grading algorithms for retinal biomarkers of age-related macular degeneration (AMD).
METHODS: Following a systematic review of the literature data on detection and quantification of AMD retinal biomarkers by available algorithms were extracted and descriptively synthesized. Algorithm quality was assessed using a modified version of the Quality Assessment of Diagnostic Accuracy Studies 2 checklist with a focus on accuracy against established reference standards and risk of bias.
RESULTS: Thirty five studies reporting computer-aided diagnosis (CAD) tools for qualitative analysis or algorithms for quantitative analysis were identified. Compared with manual assessment in reference standards correlation coefficients ranged from 0.54 to 0.97 for drusen, 0.80 to 0.98 for geographic atrophy (GA), and 0.30 to 0.98 for intra- or subretinal fluid and pigment epithelial detachment (PED) detection by automated algorithms. CAD tools achieved area under the curve (AUC) values of 0.94 to 0.99, sensitivity of 0.90 to 1.00, and specificity of 0.89 to 0.92.
CONCLUSIONS: Automated analysis of AMD biomarkers on OCT is promising. However, most of the algorithm validation was performed in preselected patients, exhibiting the targeted biomarker only. In addition, type and quality of reported algorithm validation varied substantially. TRANSLATIONAL RELEVANCE: The development of algorithms for combined, simultaneous analysis of multiple AMD biomarkers including AMD staging and the agreement on standardized validation procedures would be of considerable translational value for the clinician and the clinical researcher.

Introduction

To increase our understanding of risk factors for the onset and progression of age-related macular degeneration (AMD), the leading cause of irreversible severe vision loss among Caucasians in all developed countries, we need large, prospective epidemiological studies.[1] In order to accurately stage AMD and assess progression, we increasingly rely on multimodal retinal imaging in a clinical context. This has not yet been translated into epidemiological studies, as the manual grading of the increasing data volumes generated in multimodal imaging is unfeasible. Thus, semi- or fully automated algorithms are necessary to grade image data generated in epidemiological studies.

Optical coherence tomography (OCT) is an integral part of multimodal imaging and of increasing importance in standard clinical care. The image data volume generated by OCT is particularly high (up to hundreds of B-Scans per examination), its manual analysis can consume a tremendous amount of time and, to date, we lack established and validated means of semi- or fully automated grading for AMD biomarkers on OCT. The first, very general approach in automated OCT analysis was automated retinal layer segmentation with subsequent thickness calculations, which can be helpful in a variety of retinal diseases as an unspecific diagnostic marker.[2] However, automated segmentation implemented in proprietary OCT software is of varying quality and frequently encounters segmentation errors in the presence of retinal pathology.[3-5] Recently, substantial efforts have been made to develop automated image analysis for detection of AMD-specific biomarkers, like drusen, geographic atrophy (GA), and sub- and intraretinal fluid, enabling a more detailed quantification of these biomarkers.

Thus, in this article we present an overview of the currently available algorithms for semi- and fully automated OCT image analysis of retinal AMD biomarkers.

Methods

Eligibility Criteria for Considering Studies for This Review

Search Methods for Identifying Studies

Our search strategy, selection of publications, and reporting of results were conducted in accordance with the Cochrane recommendations for systematic reviews. Literature was searched in MEDLINE, MEDLINE In-Process, Science Citation Index Expanded, Conference Proceedings Citation Index - Science, Book Citation Index - Science, Emerging Sources Citation Index, Korean Citation Index, and Scientific Electronic Library Online Citation Index for published studies up to March 2016. Detailed information about the search terms and formulas can be found in Supplementary Table S1. The initial search yielded 926 articles. After screening of all abstracts for eligibility, 65 references were included in the full-text review (Fig. 1). Study authors were contacted to provide additional data if required. Reference lists of manuscripts reviewed in full were hand searched for additional relevant articles. All included studies reported having obtained ethics approval.

Figure 1

Flowchart depicting the literature search on algorithms for analysis of AMD biomarkers on OCT.

Study Selection

Articles reporting algorithms quantifying or qualitatively analyzing retinal biomarkers of AMD on OCT images were included. Studies using OCT angiography or polarization-sensitive OCT were excluded, as were nonhuman studies. A huge body of literature is available on the general topic of automated OCT image analysis and much of it may, with some modifications, be applicable to AMD. However, in order to limit the scope of our already quite extensive survey, we decided to exclude works that did not report qualitative or quantitative AMD biomarker analysis. For example, studies improving visualization only or reporting retinal layer segmentation or retinal thickness measurements only were deemed irrelevant for this review. In case of any uncertainty a senior investigator (RPF) was consulted. Conference abstracts were excluded in case they were later published as a journal article, which was included in the review.

Data Collection and Risk of Bias Assessment

Data on OCT devices and image acquisition protocols, number of included eyes, used reference standards, algorithm functionality, validity, reliability, limitations, relations to other algorithms, key conclusions of the authors, and other relevant information were extracted. Two independent assessors (MWMW and JB) used a modified version of the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) checklist (see Supplementary Table S2 for details), which was adapted to the requirements for assessment of automated OCT analysis by including the following: algorithm training/development and testing in separate patient collectives, objective comparison to reference standard with index test, manual double grading for the reference standard, and blinding of graders and intragrader repeatability. Risk of bias was assessed regarding patient selection, conduct, and interpretation of index and reference tests and flow and timing. Agreement on study quality was high between both assessors with a kappa score of 0.85. Any ambiguities were adjudicated by a senior investigator (RPF). Risk of bias was defined as high if it was increased in any of the aforementioned fields. A summarizing statement regarding increased risk of bias is included in Tables 1 through 5. Detailed assessments of risk of bias are provided in Supplementary Table S2.

Table 1

Algorithms for Drusen Quantification

Table 5

Algorithms for Qualitative Analysis

Data Synthesis and Analysis

Algorithms were categorized based on qualitative or quantitative analysis, and the latter were further categorized dependent on the analyzed biomarkers. Quality of the algorithms was assessed by comparison of their reported accuracy with the reference standard employed in each respective study as well as by assessing the appropriateness of chosen reference standards and validation samples.

Results

We identified a total of 35 algorithms for automated or semiautomated analysis of retinal AMD biomarkers on OCT (Fig. 1). These include 27 for quantitative and eight for qualitative assessment. Algorithms for the quantitative analysis include the following biomarkers: drusen, GA, pigment epithelial detachment (PED), and intra-/subretinal fluid (Fig. 2). Algorithms for qualitative analysis will be referred to as computer-aided diagnosis (CAD) tools. The algorithm characteristics are summarized in Tables 1 through 5. Quantitative algorithms will be presented by detected biomarker. For each category, one exemplary algorithm is presented in brief and interesting aspects of additional algorithms are highlighted.

Figure 2

Overview of the entity of included algorithms.

In brief, algorithmic image analysis is based on intensity values, intensity gradients, and pixel position within the image, which was used to generate a segmentation of retinal layers or pathological structures. Based on the two-dimensional analysis, a three-dimensional interpolation can be generated for further analysis. En face images can be generated by axial projection, referred to as summed voxel projection (SVP). This can also be calculated for only a part of the axial intensity, resulting in a partial SVP (e.g., only for a specific part of the image, such as a segmented layer on OCT). The different image processing approaches used by the reviewed algorithms are listed in Supplementary Table S3.

Drusen

Seven algorithms were published on drusen detection, mainly focusing on area covered by drusen and total drusen volume, but also on drusen number and maximum diameter (see Table 1). Where available, their coefficient of correlation (CC) ranged from 0.54 to 0.97 when validated against manual grading on OCT or color fundus photography (CFP). All except one algorithm[6] relied on calculation of the difference between the actual retinal pigment epithelium (RPE) segmentation and a calculated ideal RPE or Bruch's membrane. Two algorithms[6,7] used active contours for RPE segmentation in which an object is delineated by an energy-minimizing contour guided by the surrounding image (e.g., based on intensity gradients and internal forces dependent on the contour itself such as continuity and smoothness).[8] In the algorithm by Chen et al.,[9] the RPE was detected using an intensity threshold and interpolated to achieve a smooth line. First, an ideal RPE free of any deformations and then the difference between the ideal and real RPE layer were calculated for drusen identification. Finally, using an en face projection, possible false-positive drusen were removed if they are only present in one B-scan or based on their intensity or shape information. De Sisternes et al.[10] published an approach in which 11 drusen features, including information about drusen geometry, reflectivity, texture, number, area, and volume were used for the calculation of likelihood of progression from early and intermediate to exudative AMD. Piecewise linear regression with Lasso regularization was used and prediction of progression was estimated. A frequent limitation of algorithms for drusen detection was underestimation of overall drusen burden.[9,11,12] The authors attributed this to a “blind angle” of the algorithms for very small drusen with only minimum RPE elevation because of necessary preprocessing steps for noise reduction and absolute thresholds for RPE deviations detected as drusen.

Geographic Atrophy

Six algorithms on GA detection and area calculation were published (Table 2). Where available, their CC and overlap ratio (OR) ranged from 0.80 to 0.98 and 0.59 to 0.82, respectively, when validated against manual grading in partial OCT SVP, fundus-autofluorescence (FAF), and red-free photography (RFP). The common approach for GA detection on OCT was a partial SVP of the choroid based on the increase in reflectance intensity underneath Bruch's membrane in the area of GA. Chen et al.[23] published an exemplary algorithm using this approach. Their algorithm first segmented the RPE with an adopted version of the RPE-detection method used for their drusen detection algorithm.[9] Then, a partial SVP beneath the RPE was generated and the average axial intensity within this slab was used to generate an en face image. In the en face projection, an active contour model identified GA. In order to speed up the algorithm, a global binarization method was used to narrow down the image regions analyzed for GA. In those cases where the binarization results were far away from the real GA boundaries, a region around the GA was specified manually for further GA analysis. The algorithm was also compared with the algorithm described by Yehoshua et al.,[24] currently implemented in the Cirrus Software (Carl Zeiss Meditec, Jena, Germany). A limitation of the algorithm by Chen et al.[23] was a generally underestimated GA area.

Table 2

Algorithms for GA and GA + Drusen Quantification

Geographic Atrophy and Drusen

One algorithm combined GA and drusen detection and was published in a fully automated and a semiautomated version (Table 2). Chiu et al.[25] used abnormal thinning and thickening of a multiple-layer complex called RPE-drusen complex (RPEDC), defined by the inner aspect of the RPE plus drusen material and the outer aspect of Bruch's membrane, in order to identify GA and drusen, respectively. Subretinal drusenoid deposits were also included in the RPEDC. After image downsampling, binarization was used to separate the hyperreflective inner layers (retinal nerve fiber layer and the two plexiform layers) from the outer hyperreflective RPEDC. Subsequently, the internal limiting membrane, the inner border of the RPEDC and Bruch's membrane were segmented by iteratively finding shortest paths in a derived graph. In this approach, each pixel corresponds to a node, edge weights were based on intensity gradients, and shortest paths were found between the left and right image boundaries. The algorithm was less accurate on OCT scans containing both GA and drusen versus solely drusen. Beside the validation against manual segmentation on OCT, no additional statistical analysis was reported.

Pigment Epithelial Detachment

Out of four algorithms quantifying PED volume (Table 3), two[35,36] were based on the same principle of ideal to actual RPE comparison presented above, and two[37,38] used the graph-based surface segmentation approach by Li et al.[39] for fluid detection within PEDs. The first two calculated PED area as well. Three of the algorithms were validated against manual segmentation on OCT; however, different statistical methods were used. One of these was the drusen quantification algorithm developed by Gregori et al.[15] (see Table 1), which was licensed to Carl Zeiss and used for PED quantification.[38] Segmentation with this algorithm performed less well in cases with GA.[36]

Table 3

Algorithms for PED Quantification

Intra-/Subretinal Fluid and Pigment Epithelial Detachment

Eight algorithms on fluid-associated alterations were found, four for intra- and subretinal fluid detection only, and four that also include PEDs (see Table 4). All were validated against manual segmentation on OCT. However, the variety of statistical methods used makes a direct comparison difficult. The four algorithms detecting intra- and subretinal fluid only were based on a variety of image analysis methods such as gray level– or gradient-based segmentation, active contours, and convolutional neural networks. The latter is a state of the art machine learning technique inspired by biological neural networks within the visual cortex. The algorithms detecting intra- and subretinal fluid as well as PEDs were all graph-based and used classifiers for fluid detection, except one.[40] In brief, classifiers assign an object to a class, for example, based on the class of the k most similar objects (k nearest-neighbor classifier) or a “forest” of randomly generated decision-trees during a learning process (random forest classifier). The algorithm by Zheng et al.[41] used intensity gradient-based edge maps for segmentation of fluid-filled regions. The true positive delineated regions were then manually selected. In validation against manual OCT segmentation the dice coefficient was used, which is a commonly used measure for comparison of similarity in image analysis.

Table 4

Algorithms for Intra- or Subretinal Fluid and PED Quantification

Some of the presented algorithms were implemented in the Cirrus software and are commercially available. This is the case for the algorithm developed by Gregori et al.,[15,36] which was used for drusen and PED[36] quantification and for the algorithm on GA quantification described by Yehoshua et al.[24] The implemented algorithms for drusen and GA detection got Food and Drug Administration approval as a part of the Cirrus HD-OCT 6.0 software.

Computer-Aided Diagnosis Tools

For algorithms with qualitative analysis a classification process is needed, for which machine learning algorithms are often used. In order to build a model for the classification process, the algorithm analyzes a training dataset, in which the diagnosis of all subjects is known. Then, a testing dataset is used to evaluate the model's performance. A frequently used method in machine learning are support-vector machines (SVMs), in which an optimal separating linear boundary is generated for classification,[62] possibly after implicit nonlinear mapping to a higher-dimensional space. For mapping of the data, any OCT-derived information can be used. Eight CAD tools were identified, seven for AMD and one for PED classification. They had a variety of readouts and used different approaches (Table 5). Where available, their sensitivity and specificity ranged from 0.58 to 1.0 and 0.64 to 1.0, respectively, against the reference standard of manual classification on OCT. As an example, the algorithm by Srinivasan et al.,[63] distinguished between healthy retina, dry AMD, and diabetic macular edema (DME). A relatively small area of the B-scan, consisting of the 150 central A-scans and only the 40 pixels above and 5 pixels below the mean RPE level was cropped for analysis. For classification, histograms of oriented gradients (HOG) descriptors were used. To compute these, the image was divided into small spatial cells. For each cell, a contrast-normalized one-dimensional histogram of the directions of the spatial gradients, weighted by their magnitudes, was calculated. This process was performed four times with different properties. Three SVMs were trained to correctly classify the B-scans based on the extracted HOG features: normal versus AMD, normal versus DME, and AMD versus DME.

Discussion

Most of the algorithms identified perform well when compared with manual grading. However, the samples used for assessment of algorithm quality were small and preselected for the presence of a particular biomarker and the absence of additional pathology in most cases. In addition, availability of high-quality images suitable for algorithm development and assessment was a prerequisite for inclusion into studies. Against this background, available algorithms for the automated detection and quantification of AMD biomarkers on OCT image data are promising; however, further quality assessment as well as assessment of their performance in samples that may contain multiple pathologies, and are thus more representative of a wider utilization are warranted.

Of all AMD biomarkers, GA was detected most robustly, while the greatest variability of performance was observed in algorithms for intra-/subretinal fluid and PED detection. This might be due to a greater diversity of how the biomarker fluid can present on OCT compared with drusen and GA. While the technical approaches for the different algorithms for drusen and GA segmentation are very similar for the respective biomarker, there is a greater variability of image analysis techniques for intra-/subretinal fluid, PED, and CAD tools. This might be due to the availability of very general and easily accessible image analysis approaches for drusen and GA as well as their comparatively uniform appearance on OCT. However, although comparison of the different algorithms reported is difficult due to a lack of standardization in their quality assessment, the best overall performance seems to have been achieved by CAD tools. This is likely the case as the sole step of detecting a pathology might be less prone to errors than the combination of detecting and quantifying it.

The most impeding barrier for the comparison of identified algorithms is the inconsistency in their quality assessment. A substantial number of articles reported repeatability or a numerical comparison of measurements with the reference standard only. For most algorithms, it is unclear whether the development and evaluation of the algorithm was made in separate patient samples, which is necessary in order to prevent risk of bias and achieve reliable validation. A highly relevant question is whether an algorithm can be used in a routine, not preselected patient sample. However, in most of the reviewed studies the patient pool for testing was preselected for the respective pathology (e.g., drusen). While this is reasonable for the first stages of algorithm development, it is not representative of typical AMD patients, in whom multiple AMD biomarkers are present simultaneously. Few algorithms combining analysis of different AMD biomarkers were described so far, like the very general approaches of Chiu et al.[25] on drusen and atrophy and of Chen et al.[46] on “symptomatic exudate-associated derangements” including PEDs, intra- and subretinal fluid. Both algorithms actually measured only one feature, which comprised the respective biomarkers (RPEDC thickness in the first and fluid regions in the second).

Algorithms for quantitative analysis of retinal AMD biomarkers are not only available for OCT images but also for other imaging modalities. A significant number of algorithms for the analysis of CFP were extensively reviewed elsewhere.[74] Overall, high accuracy was achieved. These algorithms functioned with image filtering, texture-, threshold-, clustering-, and edge-based detection, and recently also included machine learning technology. In our opinion, the combination of algorithms for different imaging modalities like CFP and OCT would be a promising approach for enhancing algorithm accuracy and allowing comprehensive disease classification.

Most of the drusen quantification algorithms for OCT calculate total drusen area and volume and do not include individual drusen parameters such as size, which are necessary for AMD classification (see Table 1). Interesting exceptions are two algorithms, in which individual drusen size was assessed.[11,12] Diniz et al.[12] differentiated small (<63 μm), intermediate (63–125 μm), and large (>125 μm) drusen, however individual drusen size was just roughly estimated under the assumption of all drusen being perfect circles (area = πr2 → diameter = 2 r). Another study classified AMD on OCT according to the Age-related Eye Disease Study Grading System (AGS) using maximum drusen size and percentage of drusen area within the AGS grid.[11] These two algorithms are promising examples of how automated OCT analysis can be implemented in AMD classification of early and intermediate AMD. Classification is also available for other AMD stages: CAD tools are capable of distinguishing between healthy and early AMD[33] and healthy and exudative AMD (Schlegl T, et al. IOVS 2015;56: ARVO E-abstract 5920).[64,67] A classifier able to differentiate early from advanced AMD is the next step. There is already one algorithm, which partially fulfills this and can discriminate between DME and early AMD.[63] As ME can also be due to exudative AMD, this algorithm might also be capable to differentiate early from wet AMD. Other algorithms for OCT analysis for relatively unspecific retinal biomarkers like intraretinal fluid were developed for unrelated pathology, but might function similarly well in AMD (e.g., for cystoid ME in vitreoretinal disease[75] and for microcystic ME in multiple sclerosis[76]).

So far, no automated quantification of other retinal AMD biomarkers like pigmentary abnormalities, reticular pseudodrusen (RPD), changes preceding GA, and choroidal neovascularization (CNV) is available. However, pigmentary changes can be identified on OCT with a sensitivity of 66.5% and specificity of 78.7% using hyperreflectivity.[77] The algorithm for combined drusen and GA detection mentioned above[25] includes possible RDP within the analyzed RPEDC, yet differentiation from conventional drusen is not possible. One study quantified drusen-associated photoreceptor layer thinning semiautomatically.[78]

The strengths of this first review on automated OCT image analysis of AMD retinal biomarkers are its systematic approach and the standardized quality assessment of included algorithms. The main limitation of this study is the absence of a uniform quality assessment due to an inconsistent assessment of algorithm quality and performance across studies.

In conclusion, automated analysis of AMD biomarkers on OCT is promising; however, type and quality of reported algorithm validation vary substantially and most validation has been performed in preselected patients only. The development of algorithms for combined, simultaneous analysis of multiple AMD biomarkers including AMD staging and the agreement on standardized validation procedures would be of considerable translational value for the clinician and the clinical researcher.

Click here for additional data file.

Table 4

Extended