Quantification of Nonperfusion Area in Montaged Widefield OCT Angiography Using Deep Learning in Diabetic Retinopathy.

Purpose: To examine the efficacy of a deep learning-based algorithm to quantify the nonperfusion area (NPA) on montaged widefield OCT angiography (OCTA) for assessment of diabetic retinopathy (DR) severity. Design: Cross-sectional study. Participants: One hundred thirty-seven participants with a full range of DR severity and 26 healthy participants.
Methods: A deep learning-based algorithm was developed for detecting and quantifying NPA in the superficial vascular complex on widefield OCTA comprising 3 horizontally montaged 6 × 6-mm OCTA scans from the nasal, macular, and temporal regions. We trained the algorithm on 978 volumetric OCTA scans from all participants using 5-fold cross-validation. The algorithm can distinguish NPA from shadow artifacts. The F1 score evaluated segmentation accuracy. The area under the receiver operating characteristic curve and sensitivity with specificity fixed at 95% quantified network performance to distinguish patients with diabetes from healthy control participants, referable DR from nonreferable DR (nonproliferative DR [NPDR] less than moderate severity), and severe DR (severe NPDR, proliferative DR, or DR with edema) from nonsevere DR (mild to moderate NPDR). Main Outcome Measures: Widefield OCTA NPA, visual acuity (VA), and DR severities.
Results: Automatically segmented NPA showed high agreement with the manually delineated ground truth, with a mean ± standard deviation F1 score of 0.78 ± 0.05 in nasal, 0.82 ± 0.07 in macular, and 0.78 ± 0.05 in temporal scans. The extrafoveal avascular area (EAA) in the macular scan showed the best sensitivity at 54% for differentiating those with diabetes from healthy control participants, whereas montaged widefield OCTA scan showed significantly higher sensitivity than macular scans (P < 0.0001, McNemar's test) for detecting eyes with DR at 66%, referable DR at 63%, and severe DR at 62%. Montaged widefield OCTA showed the highest correlation (Spearman ρ = 0.74; P < 0.0001) between EAA and DR severity. The macular scan showed the strongest negative correlation (Pearson ρ = -0.42; P < 0.0001) between EAA and best-corrected VA. Conclusions: A deep learning-based algorithm for montaged widefield OCTA can detect NPA accurately and can improve the detection of clinically important DR.

Numerous studies have reported on the association between the OCT angiography (OCTA) nonperfusion metric and diabetic retinopathy (DR) clinical severity. Because the metrics are quantitative and objective, in contrast to the qualitative clinical grading, they offer the possibility of improving the management of diabetic retinopathy through reliable automated evaluation of eyes at risk. However, the reported approaches have significant limitations that prevent wide application of OCTA for evaluation of DR. First is the small 3 × 3-mm central field that most studies evaluated. The first generation of OCTA devices provided capillary-level resolution in 3 × 3-mm fields only. Increasing evidence suggests that a subset of DR eyes that have predominantly peripheral lesions and nonperfusion may be present outside the central area.1, 2, 3 A widefield evaluation of nonperfusion would increase the sensitivity of detecting capillary damage secondary to diabetes.

However, even with 3 × 3-mm scans, only approximately 60% of OCTA images may be usable for quantitative analysis. Widefield OCTA presents additional technical challenges. It is more prone to the artifacts related to defocus, vignetting, and vitreous opacities. Normative interpapillary space varies dramatically from the optic disc to peripheral region. This makes the quantification of nonperfusion in a wider OCTA field of view even more challenging.

Currently, few reports of nonperfusion quantification using widefield OCTA exist in the literature. Alibhai et al described quantification of nonperfusion in widefield OCTA with a 12 × 12-mm single scan. This study required manual exclusion of low-signal artifacts, which would be impractical in clinical settings. Other studies quantifying nonperfusion with widefield OCTA have used manual delineation of nonperfusion.5, 6, 7, 8 To the best of our knowledge, no study has demonstrated automated quantification of nonperfusion in widefield OCTA.

In this study, we examined automated quantification of nonperfusion area (NPA) on montaged widefield OCTA with a 17 × 6-mm field of view on diabetic eyes. Although we previously reported on a deep learning algorithm that can discriminate true nonperfusion versus low-signal artifact on 6 × 6-mm central macular scans,, one cannot assume the same network will be valid for wider peripheral views because the nature and extent of low-signal artifacts change with widening field of view and montaging. This study introduced a novel idea by using a deep learning method to quantitate nonperfusion automatically, to exclude widefield specific artifacts, and to validate the technique against clinically meaningful end points.

Methods

Data Acquisition

The Oregon Health and Science University institutional review board approved the study, which adhered to the tenets of the Declaration of Helsinki. Patients with diabetes and healthy participants were recruited at a tertiary academic ophthalmology department, and informed consent was obtained. All participants underwent 6 × 6-mm volumetric scans centered at the macula and the immediate areas nasal and temporal to the macular scan using a 70-kHz OCT AngioVue system (RTVue-XR; Optovue, Inc) with 2 repeated B-scans obtained at each of 400 raster positions and each B-scan containing 400 A-lines. The split-spectrum amplitude-decorrelation angiography algorithm computed the OCTA data. Widefield OCTA was achieved by montaging the 3 scans. We also obtained fundus photographs of the patients with diabetes, which were graded by a trained, masked grader (L.G.) based on the standardized 7-field color fundus images according to the Early Treatment of Diabetic Retinopathy Study scale. An Early Treatment Diabetic Retinopathy Study protocol visual acuity (VA) was obtained for each participant.

A total of 1092 volumetric OCT scans from 182 participants (30 healthy control participants, 36 participants with diabetes without retinopathy, 72 participants with mild or moderate nonproliferative DR [NPDR], and 44 participants with severe NPDR or proliferative DR [PDR]; mean age ± standard deviation, 57 ± 15 years) were collected from a DR study. We excluded 114 invalid scans from 19 participants (4 healthy control participants, 6 participants with diabetes without retinopathy, 6 participants with mild or moderate NPDR, and 3 participants with severe NPDR or PDR) because of scanning failure when the OCT system could not acquire valid signals because of strong eye movements or an inability to complete a scan because of the volunteers being unable to fixate well. All the scanning was conducted with participants’ eyes in mydriasis.

Dataset Preparation

In this study, we segmented NPA in the superficial vascular complex (SVC) from en face angiograms. To create an SVC en face angiogram, a guided bidirectional graph search algorithm segmented the retinal layer boundaries. A maximum projection method was used to project OCTA data within the slab. Vitreous floaters, pupil vignetting, or eyelash shadows can cause localized shadow artifacts. A simple NPA detection algorithm using apparent flow signal loss can mistake these artifacts for NPA. However, an algorithm that is aware that shadow artifacts reduce both the OCTA and structural OCT signal simultaneously, whereas true NPA has a loss of flow signal and preservation of structural OCT signal, potentially can distinguish these artifacts from NPAs. Although this can help to differentiate some artifacts from true NPA, in practice, a substantial variation in OCT reflectance between imaging subjects, between imaging sessions, and even between different regions of the eye make this approach to artifact detection imprecise. In particular, near the central macula where the retina is thin, structural OCT often exhibits low reflectance. This means that a conventional rule-based algorithm cannot distinguish NPA from low-signal artifacts reliably.

In contrast, deep learning-based methods could distinguish these artifacts accurately despite these variations, as long as an accurate ground truth maps are available to supervise the learning process. In our work, 3 certified graders (L.G., Q.Y., B.W.) manually delineated NPA and shadow artifacts independently. The final ground truth maps were generated from these 3 manual delineation results using a majority voting method. We excluded the manually delineated areas smaller than 20 pixels (0.0045 mm2) from ground truth maps to remove errors caused by noise.

The input data set consisted of the SVC angiogram, the OCT reflectance image of the inner retina, the inner retinal thickness map, and the corresponding manually delineated ground truth containing NPA and shadow area. In the healthy participants, the foveal avascular zone (FAZ) also was considered NPA, because it has similar patterns from the perspective of pattern recognition theory. Thus, each participant had some region in the images that the network can use for training purposes. The size of all en face images was 400 × 400 pixels. A convolutional neural network, which is a state-of-the-art deep learning-based segmentation method, needs a large dataset to train. To increase the number of training samples, we applied several data augmentation operations, including vertical flips, horizontal flips, transpositions, and rotations.

Development of the Deep Learning System

We trained a deep learning algorithm to segment NPA at 3 different locations, including nasal, macular, and temporal scans. We adopted the residual module from ResNet, which can improve the training phase with faster convergence and higher accuracy using identity short-cut connections. We also used a U-Net-like network architecture as the backbone of our convolutional neural network. U-Nets are used widely in medical image processing tasks because they have shown good adaptability to small datasets, in part because of skip connections between the encoder and decoder arms. The network architecture used in this study also contained several adaptions to aid NPA detection in widefield images (Fig 1). To reduce computational complexity, we combined the inner retinal thickness (Fig 1A) and OCT reflectance images of the inner retina (Fig 1B) and fed them to a subnet (Fig 1D1) to segment shadow artifact affected areas. The SVC en face images (Fig 1C) were fed to a subnet (Fig 1D2) to extract retinal capillary features. Then, the output of subnets for shadow and vessel detection were fed to 3 parallel subnets from which we expected to learn features from each of the 3 regions used in this work (nasal, macular, and temporal; Fig 1D3–D5). The results from these 3 parallel subnets then were concatenated and used to predict the true NPA area and the area affected by shadow artifacts (Fig 1E).

Figure 1

Network architecture. A, B, D, The thickness map (blue is thin and red is thick) (A) and structural OCT image of the inner retina (B) were fed to 1 branch of the network (D1) to extract features from shadow-affected areas. C, D, The en face angiogram of the superficial vascular complex was input into another branch of the network (D2) to extract the nonperfusion area (NPA). D, E, Then, the network combined these 2 types of features and fed them to 3 parallel subnets (D3–D5) to output the NPA (blue in panel E) and shadow artifacts (yellow in panel E).

Evaluation and Statistical Analysis

We evaluated the performance of our algorithm using 5-fold cross-validation on our dataset. In this study, the entire dataset was split into 5 individual groups. Care was taken to ensure that different severities comprise the same proportions in all groups. In each training and evaluation step, 4 groups were chosen as the training dataset, whereas the fifth group was reserved to evaluate performance. We repeated this process 5 times, with each of the groups used once as the test dataset, to obtain the average performance of our method. The same eye can only be used for training or testing. This approach allowed us to evaluate the performance of the proposed method on the entire dataset. The 3 retinal regions were evaluated separately and then montaged (Fig 2). The montage process was performed by an invariant features-based algorithm.

Figure 2

Automated nonperfusion area (NPA) segmentation results on montaged widefield OCT angiography imaging, shown by (A) a representative healthy control, (B) an eye with diabetes, but without retinopathy, (C) an eye with mild to moderate nonproliferative diabetic retinopathy, and (D) an eye with severe diabetic retinopathy. In each case, the first panel is the superficial vascular complex angiogram, the second panel is the ground truth manually delineated NPA (green) and shadow area (yellow), and the third panel is the automated segmentation result showing NPA (blue) and shadow artifacts area (yellow) that closely matches the ground truth.

To evaluate our method comprehensively, we analyzed the impact of various factors on the performance of the algorithm, including DR severity and scan quality measured by signal strength index. We quantified the agreement between the output of the network and manual delineation using the F1 score (Equation 1):where TP is the true-positive finding (the number of pixels that were predicted correctly to be NPA), TN is the true-negative finding (the number of pixels that were predicted correctly to be non-NPA), FP is the false-positive finding (the number of pixels that were predicted wrongly as NPA but in fact were non-NPA), and FN is the false-negative finding (the number of pixels that were predicted wrongly as non-NPA but in fact were NPA). To eliminate the impact of normal variation in the size of the FAZ, we used the extrafoveal avascular area (EAA), defined as the avascular area outside of the 1-mm-diameter circle centered on the fovea, to quantify diagnostic performance. We also used the extent of NPA measured by the algorithm in each location to stage DR according to several classification levels, including healthy versus patients with diabetes mellitus (which includes any stage of DR, but also diabetic eyes without retinopathy), eyes with DR versus eyes without retinopathy (which includes diabetic eyes in which retinopathy has not developed), referable (defined as moderate to severe NPDR, PDR, or DR with diabetic macular edema) versus nonreferable DR, and severe (severe NPDR, PDR, or DR with diabetic macular edema) versus nonsevere DR. The sensitivity and the area under the receiver operating characteristic curve (AUC) were used to evaluate the diagnostic accuracy between healthy and DR groups, and sensitivity with specificity fixed at 95% also were quantified to assess the diagnostic accuracy of using NPA to stage DR. We compared the AUC of NPA on montaged widefield OCTA and conventional macular OCTA using the Delong method and compared their sensitivities using McNemar’s test. Spearman correlation coefficient ρ values were used to quantify the correlation between NPA and DR severity, and Pearson correlation coefficient ρ values were used to quantify the correlation between NPA and VA. We also compared the correlations between NPA and DR severity or VA using widefield versus conventional macular scans with a bootstrap difference test.

Results

Performance Evaluation

Good agreement, indicated by F1 scores, was found between the network’s output and the manually delineated ground truth (Table 1). The average F1 scores in the macular scans were slightly higher than those of the nasal or temporal side. When analyzed by DR severity, the agreement was similar and not statistically different between DR severities. For the montaged widefield scans, the agreement showed slight deterioration. The average intergrader agreements (measured by intersection over union) in the macular scans were higher than those of the nasal or temporal side.

Table 1

Agreement Quantification on Widefield OCT Angiography with Different Diabetic Retinopathy Severities

Variable	Nasal	Temporal	Macula	Montaged Widefield OCT Angiography
Agreement (F1 score) between automatically segmented NPA and ground truth map
Diabetes without retinopathy	0.80 ± 0.05	0.75 ± 0.04	0.88 ± 0.07	0.84 ± 0.08
Mild to moderate NPDR	0.78 ± 0.06	0.78 ± 0.04	0.80 ± 0.06	0.79 ± 0.06
Severe NPDR and PDR	0.78 ± 0.05	0.79 ± 0.05	0.77 ± 0.06	0.78 ± 0.05
Average	0.78 ± 0.05	0.78 ± 0.05	0.82 ± 0.07	0.80 ± 0.07
Intergrader agreement (IoU) for 3 experts
Expert 1 vs. expert 2	0.66 ± 0.12	0.72 ± 0.07	0.76 ± 0.09	0.72 ± 0.10
Expert 1 vs. expert 3	0.66 ± 0.18	0.63 ± 0.11	0.70 ± 0.13	0.67 ± 0.14
Expert 2 vs. expert 3	0.69 ± 0.09	0.65 ± 0.10	0.78 ± 0.18	0.71 ± 0.14
Average	0.67 ± 0.14	0.66 ± 0.10	0.75 ± 0.14	0.70 ± 0.13

DR = diabetic retinopathy; NPA = nonperfusion area; NPDR= nonproliferative diabetic retinopathy; PDR = proliferative diabetic retinopathy; IoU = intersection over union.

Data are presented as mean ± standard deviation.

Robustness to Scan Quality

We used healthy eyes to quantify the correlation between scan quality (measured by signal strength index) and NPA segmentation accuracy (measured by the following: segmentation specificity – network output perfusion area / ground truth perfusion area) in widefield OCTA imaging (Fig 3). The segmentation accuracy was independent from scan quality in the disc (r = 0.04; P = 0.775), macula (r = 0.08; P = 0.595), and temporal (r = 0.08; P = 0.584) scans.

Figure 3

Graphs showing the correlation between signal strength index (SSI) with specificity of automated nonperfusion area segmentation on widefield OCT angiography imaging from healthy control participants (red dots; n = 52) showing that the algorithm’s output was not compromised by signal strength variation.

Performance in Diabetic Retinopathy Diagnosis

With specificity fixed at 95%, EAA in the central macula showed the best sensitivity for detecting patients with diabetes from healthy control individuals, and the montaged widefield OCTA imaging showed the highest sensitivity for detecting patients with different severities of DR (Table 2). Similarly, EAA in macular scans showed the highest AUC for detecting healthy control participants from patients with diabetes, and the montaged widefield OCTA scans achieved the best AUC for distinguishing different DR severities. Compared with the traditional macular OCTA imaging, montaged widefield OCTA imaging significantly improved the diagnostic sensitivity (P < 0.001) for different severities of DR.

Table 2

Nonperfusion Area Diagnostic Accuracy

Diabetic Retinopathy Group	Nasal	Temporal	Macular	Montaged Widefield OCT Angiography	P Value
Sensitivity to detect retinopathy, % (95% CI)
Healthy vs. DM	38 (31–45)	38 (32–45)	54 (48–61)	39 (34–44)	0.0001∗
DR vs. non-DR	33 (25–41)	37 (29–45)	34 (27–42)	66 (60–71)	<0.0001∗
Referable DR vs. nonreferable DR	29 (21–38)	42 (33–51)	35 (27–44)	63 (57–69)	<0.0001∗
Severe DR vs. nonsevere DR and non-DR	27 (19–37)	35 (27–45)	24 (17–33)	62 (56–68)	<0.0001∗
AUC (95% CI)
Healthy vs. DM	0.73 (0.68– 0.79)	0.81 (0.76–0.86)	0.88 (0.84–0.92)	0.82 (0.78–0.85)	0.0425†
DR vs. non-DR	0.74 (0.68–0.79)	0.84 (0.79–0.88)	0.86 (0.81–0.90)	0.92 (0.89–0.94)	0.0182†
Referable DR vs. nonreferable DR	0.78 (0.73–0.83)	0.83 (0.78–0.87)	0.86 (0.81–0.90)	0.91 (0.88–0.93)	0.0429†
Severe DR vs. nonsevere DR and non-DR	0.76 (0.70–0.81)	0.81 (0.76–0.86)	0.84 (0.79–0.88)	0.89 (0.86–0.92)	0.0513†

AUC = area under the receiver operating characteristic curve; CI = confidence interval; DM = diabetic mellitus; DR = diabetic retinopathy; non-DR = healthy and diabetic mellitus; nonreferable DR = healthy, diabetic mellitus, and mild DR; nonsevere DR = mild and moderate diabetic retinopathy without edema; referable DR = moderate diabetic retinopathy, severe nonproliferative diabetic retinopathy, and proliferative diabetic retinopathy; severe DR = severe nonproliferative diabetic retinopathy, proliferative diabetic retinopathy, and eyes with edema.

Best performance among all scan areas appears in boldface.

McNemar test was used to compare the sensitivities of nonperfusion area between widefield OCT angiography and conventional macular OCT angiography.

DeLong test was used to compare the AUCs of nonperfusion area between widefield OCT angiography and conventional macular OCT angiography.

Correlation of Extrafoveal Avascular Area with Diabetic Retinopathy Severity and Visual Acuity

The montaged widefield OCTA showed the highest correlation between EAA and DR severity. The best-corrected VA showed a negative univariate correlation with EAA, and the macular scan showed the strongest negative univariate correlation between EAA and best-corrected VA (Table 3).

Table 3

Correlation Coefficients (ρ Values) of Nonperfusion Area with Diabetic Retinopathy Severity and Best-Corrected Visual Acuity in Different Regions

Variable	Nasal	Temporal	Macular	Montaged Widefield OCT Angiography	Correlation Differences∗
Correlation with DR severity, Spearman’s ρ (95% CI)	0.50 (0.40–0.59), P <0.0001	0.55 (0.46–0.63), P <0.0001	0.69 (0.58–0.77), P <0.0001	0.74 (0.63–0.82), P <0.0001	0.052 ± 0.067, P = 0.436†
Correlation with best-corrected VA, Pearson’s ρ (95% CI)	–0.17 (–0.30 to –0.05), P = 0.0082	–0.31 (–0.51 to –0.30), P <0.0001	–0.42 (–0.55 to –0.26), P <0.0001	–0.40 (–0.57 to –0.20), P <0.0001	0.018 ± 0.12, P = 0.888†

CI = confidence interval; DR = diabetic retinopathy; VA= visual acuity.

Best performance among all scan areas appears in boldface.

Bootstrapped difference test between macular-centered and montaged widefield OCT angiography image based on 9999 bootstrapping. Data appear as mean ± standard deviation.

Two-side P value of 9999 bootstrapped difference test.

Discussion

In this study, we developed and validated a deep learning neural network capable of quantifying the NPA by distinguishing it from low-signal artifacts in the superficial vascular complex in widefield OCTA images constructed from a montage of nasal, macular, and temporal scans. The algorithm showed good agreement with manual delineation (average F1 score, >0.78) for NPA segmentation on widefield OCTA images. We performed cross-validation, indicating that these results were generalizable. We also demonstrated that NPA on widefield OCTA imaging correlates with DR severity and VA, and its diagnostic accuracy in distinguishing DR, referable DR, and severe DR was superior to NPA quantified from macular OCTA imaging.

Although it is well recognized that quantification of nonperfusion using a wider field of view likely would improve its sensitivity in assessment of DR, most studies to date have been based on 3 × 3-mm scans. Recent efforts in quantification using wider-field OCTA have required either manual exclusion of low-signal artifacts or manual delineation of nonperfusion areas. To avoid tedious manual intervention, several conventional (i.e., nonlearning-based) algorithms to detect capillary dropout regions in en face images automatically have been developed.23, 24, 25, 26 These techniques can detect NPA in en face images automatically, but this approach suffers on low-quality scans and scans with high noise levels. This innovative algorithm proposed in this study demonstrated a fully automated quantification of nonperfusion area in a widefield OCTA image using a deep learning-based method. Because the deep learning-based method has shown great success in OCTA images, our approach did not exclude scans because of quality or noise levels.

The proposed algorithm can detect NPA accurately in 3 different retinal regions, largely immune to shadow artifacts (Fig 2) or low scan quality (Fig 3). Recent research has demonstrated that machine learning methods can detect shadow artifacts. In our previous work, we demonstrated a deep learning algorithm that can distinguish NPA from shadow artifacts by adding the reflectance OCT image and inner retinal thickness map as inputs. Herein, we adopted a similar strategy to distinguish NPA from shadow artifacts on montaged widefield images by using the above-mentioned 3 channels as input. The reflectance OCT image could help to identify the low-reflectance areas, including shadow areas and FAZ. The retinal thickness map is useful for distinguishing shadow from NPA near the FAZ. By using all 3 channels (OCTA, structural OCT, and thickness), the network can differentiate NPA from shadows correctly. However, the larger field of view and montaging introduces new potential artifacts that required new training and validation. We demonstrated that the network could exclude low-signal artifacts in the widefield OCTA images successfully, and the NPA output showed independence from scan signal strength, indicating robustness to scan quality. This translates to improved reliability in clinical settings. In this study, the size of the montaged OCTA image is too large (approximately 1000 × 400 pixels) to train the network. The excessive size of the image will exceed the hardware (i.e., graphics processing unit) memory limit. Thus, we use separate images (temporal side scan, central macular scan, and nasal side scan) to train the network. This approach can be regarded as dividing a large image (the montaged OCTA image) into 3 small images, which is a common method for training networks on large images.

Vessel density (VD) has been used widely to quantify retinal perfusion loss in DR.29, 30, 31, 32 However, both shadow artifacts and scan quality can affect the quantification of VD in OCTA adversely, and even in images with high signal strength, persistent dependence of VD on signal strength is seen. In addition, less is known about so-called normal VD in the periphery, whereas NPA, recognized by a contiguous area of capillary dropout, always is likely to be pathologic, regardless of variation in VD. Therefore, we expect the output from our network will be more robust compared with existing VD measurements without shadow removal.

Accurately segmented by our network, NPA showed a significant correlation to DR severity. Compared with that of macular OCTA, NPA on widefield montage OCTA imaging has higher sensitivity for distinguishing different severities of DR (Table 2). However, NPA on macular OCTA imaging has a higher diagnostic accuracy than widefield montage OCTA imaging in differentiating patients with diabetes but without retinopathy from healthy control participants. This suggests that the earliest microvascular changes in DR may occur primarily in the macula, rather than the periphery. However, the detection of peripheral microvascular changes in more advanced DR can correlate better with the clinical level of retinopathy, as shown by the higher correlation with widefield OCTA. As expected, NPA on macular scans shows the best correlation with VA among 3 regions. This of course does not exclude the need to identify NPA accurately in other areas, because NPA outside the macula still indicates relevant and significant change in DR. Only by detecting NPA in larger fields of view can we avoid missing pathologic developments outside of the small fields of view used in most commercial OCTA instruments.

Compared with published deep learning algorithms that diagnose DR based on fundus photographs, our algorithm showed a lower AUC for diagnosing referable cases., However, it should be noted that nonperfusion area and clinical retinopathy levels are not expected to be correlated perfectly because they are fundamentally different features of the disease. Therefore, a difference in the diagnostic performance is expected. In particular, the Early Treatment Diabetic Retinopathy Study severity scale is based on features from fundus photography. They were studied prospectively against the risk of proliferative disease developing in a large cohort and are believed to represent a manifestation of retinal ischemia. It is possible that OCTA-quantified NPA or other OCTA-specific features may perform as well or better than clinical features in predicting risk of progression because they are more direct measurements of retinal ischemia than photographic features. However, their validation in a large, prospective clinical study is necessary, and photographic severity levels remain the gold standard for now. Nevertheless, the fundus photograph-based methods cannot detect microvasculopathy in diabetic eyes without clinically observable retinopathy by definition. OCT angiography, with its ability to evaluate the vasculature at the capillary level, is more sensitive than fundus photographs in detecting the earliest changes in DR.,

Compared with other studies that examined OCTA quantitative metrics against clinical DR, this algorithm performed very well. In general, AUC is a much more rigorous metric for diagnostic purposes compared with a statistical test that looks at population means or a test for significant correlation between a metric and severity. That is, achieving high sensitivity and specificity at a discrete severity of DR is more difficult to achieve than demonstrating a correlation or a statistically significant difference in means. It should also be noted that many of the previous studies of OCTA metrics that examined diagnostic accuracy compared patients with DR against normal control participants. However, a more clinically relevant concern is being able to distinguish eyes above and below a threshold, rather than the ability to distinguish them from normal control participants.,37, 38, 39 We hope that this more realistic diagnostic scheme can be adopted by other research groups in testing the robustness of algorithms.

Study Limitations

Limitations of the study include possible errors associated with montaging and relatively small sample sizes. In this study, we montaged 3 separate OCTA scans to obtain the widefield OCTA image. In the montage step, the montage algorithm may not detect enough pair key points in the overlapping area between 2 low-quality images, which may cause images to be aligned imprecisely, reducing the accuracy of NPA segmentation. A larger dataset could improve training of the algorithm and could improve the performance and generalizability of our algorithm. Another limitation is the relatively lower accuracy in the nasal and temporal areas. Our method achieved high performance in the central macular area, whereas in the nasal and temporal area, the accuracy showed deterioration that may be a consequence of the shadow artifact area increasing. Another reason may be that the intercapillary distance in the temporal area increases along with its variance, which caused a reduction in grader agreement. We believe that the reduced confidence in manual grading is the main reason for the performance deterioration in those 2 regions. Another limitation may lie in the definition of EAA. Excluding a 1-mm circle area centered on the fovea is a simple and valid approach to eliminate the impact of normal variation of FAZ,, but not the optimal one. The ideal way is to exclude the theoretical FAZ; however, this detection is not achievable because of the low sampling density of current 6 × 6-mm OCT and OCTA scans used in this study. We believe that with the increase of OCT scanning speed, the image resolution will increase, and this limitation also will be eliminated. A final limitation for this study is that we focused on just the SVC because it contains the fewest artifacts. For the deep plexus, many more challenges need to be solved, like projection artifacts, lower signal-to-noise ratio, and so on. All of this makes the deep plexus significantly more challenging, and we will address this plexus in future work. Despite these limitations, to the best of our knowledge, this is the first study to quantify NPA automatically while excluding low-signal artifacts on widefield OCTA with a deep learning-based method. This study suggests that objective NPA assessment using widefield OCTA may improve the ability of OCTA to assess DR severity objectively.

In conclusion, this deep learning-based algorithm can quantify NPA automatically and can exclude low signal artifacts in widefield OCTA images. Our algorithm shows high accuracy for NPA segmentation. Nonperfusion area in widefield OCTA imaging shows higher diagnostic accuracy for detecting clinically relevant DR severity levels compared with central macular scans.