Comparison of Common Retinal Vessel Caliber Measurement Software and a Conversion Algorithm.

PURPOSE: To compare three commonly used retinal vessel caliber measurement software systems, and propose an algorithm for conversion between measurement systems.
METHODS: We used 120 retinal photographs to evaluate the agreement between three commonly used software (Retinal Analysis [RA], Integrative Vessel Analysis [IVAN], and Singapore I Vessel Assessment [SIVA]). Bland-Altman plots were used to evaluate agreement of retinal arteriolar (central retinal artery equivalent, CRAE) and venular (central retinal vein equivalent, CRVE) calibers. Pearson's correlation was used to assess the associations between systemic factors and retinal vessel calibers, and Z-test was used to compare the strength of the correlation coefficients across the three software systems. An algorithm was created to convert measurements, with paired t-test performed to evaluate the differences between SIVA-measured retinal calibers and SIVA-approximates converted from RA- and IVAN-measurements using the algorithm.
RESULTS: Differences between SIVA- and RA-measured calibers (CRAE: mean difference [MD] = -21.8 μm, 95% limits of agreement [LOA], -47.3 to 3.7 μm; CRVE: MD = -7.7 μm, 95% LOA, -28.0 to 12.6 μm), SIVA- and IVAN-measured calibers (CRAE: MD = -6.7 μm, 95% LOA, -23.8 to 10.4 μm; CRVE: MD = -18.2 μm 95% LOA, -36.7 to 0.4 μm) were large. However, the strength of correlations between systemic factors with SIVA-measured retinal calibers was not significantly different to that measured using RA and IVAN (P ≥ 0.332). SIVA-approximates converted from RA and IVAN measurements using the proposed algorithm was not significantly different from SIVA-measured calibers (P ≥ 0.20).
CONCLUSION: Absolute measurements of retinal vessel calibers vary between three common software systems but associations with systemic factors were similar. TRANSLATIONAL RELEVANCE: The proposed algorithm allowed conversions of RA and IVAN measurements to SIVA-approximates. This conversion is important for future data pooling and establishment of normative values for retinal vascular caliber measurements.

Introduction

The retina is a unique site where characteristics of the microvasculature can be viewed and detected easily and noninvasively.[1,2] Studies have suggested that abnormalities of retinal microvasculature reflect systemic microvascular damage. For example, generalized narrowing of the retinal arterioles has long been known to be a sign of hypertension and has been associated with stroke.[3-5]

Advancements in digital retinal photography and computer software technologies have allowed objective measurements of retinal vessel caliber resulting in more precise quantification of generalized retinal arteriolar narrowing. These measurements have provided new data on the associations of retinal venular widening.[2,6,7] Studies using retinal vessel caliber measurement software in different population- and clinic-based settings studies have reported good internal agreement and reliability.[8] In addition, studies in these populations have also shown that retinal vessel caliber (e.g., narrower retinal arteriolar caliber and wider retinal venular caliber) are associated with risk of retinal and systemic diseases, including diabetic retinopathy, stroke, and cardiovascular mortality.[9-11]

Over the past two decades, there have been multiple software systems developed to measure retinal vessel caliber from fundus photographs. The Retinal Analysis (RA; version 6.51; Department Ophthalmology & Visual Science, University of Wisconsin, Madison, WI)[12-14] was a pioneer computer-assisted software program, that was originally used in studies such as the Atherosclerosis Risk in Communities study,[15] Cardiovascular Health Study,[16] Rotterdam study,[17] Beaver Dam Eye Study,[18] Blue Mountains Eye Study,[19] Wisconsin Epidemiologic Study of Diabetic Retinopathy,[9] the Handan Eye Study[20] (Supplementary Table S1). This software program, however, requires significant user input and manual tracing of vessels (Fig. 1A). Another widely used system that was subsequently developed with automated detection and identification of arterioles and venules was the Integrative Vessel Analysis (IVAN; University of Wisconsin, Madison; Fig. 1B),[21,22] used also in some of the earlier studies such as the Wisconsin Epidemiologic Study of Diabetic Retinopathy[23] and the Beaver Dam Eye Study,[24] and in later studies such as the Multi-Ethnic Study of Atherosclerosis,[21] Singapore Malay Eye Study,[25] Singapore Prospective Study Program26 Age, Gene/Environment Susceptibility-Reykjavik Study[27] (Supplementary Table S1). Finally, the Singapore I Vessel Assessment (SIVA; version 3.0; National University of Singapore),[28,29] improved on more automation features, including automated detection of the optic disc center, optic disc edge, and automated detection and identification of arterioles and venules. The SIVA software also provided a more global representation of the overall retinal vascular network, with measurements over a wider measurement area (up to 2 disc diameters) and additional geometry parameters such as branching angles, bifurcation, fractal dimension, and tortuosity (Fig. 1C).[29,30] Together, these three retinal vessel software systems have been used in population- and clinic-based studies totaling more than 100,000 participants (Supplementary Table S1).

Figure 1

(A) A screenshot of RA interface; (B) a screenshot of IVAN interface; (C) a screenshot of SIVA interface using the same fundus photo.

A major gap in the research is the interchangeability of the different retinal vessel caliber measurement systems, and whether reported findings from different studies using various systems are directly comparable. This poses challenges in data pooling and metaanalysis, and in understanding the “normative value” of retinal caliber measurements. There is no existing algorithm that allows one system to be converted to another. To address this, we assessed the agreement between these three widely used retinal vessel caliber measurement systems (RA, IVAN, and SIVA), compared the strength of the associations with common systemic factors (e.g., blood pressure [BP]) and developed an algorithm that converts RA IVAN measurements to SIVA-approximates. Results of our study will allow future data pooling, metaanalysis, and establishment of normative values.

Methods

Study Population

We included a subgroup of 120 healthy persons randomly selected from two population-based studies (the Singapore Prospective Study Program [SP2] and Singapore Chinese Eye Study [SCES]) for assessment of agreement of retinal vessel caliber measurement between the three retinal vessel caliber software systems: RA, IVAN, and SIVA.

Details of both the SP2 and SCES study participants and methods have been described elsewhere.[31,32] Both studies adhered to the tenets of the Declaration of Helsinki and written informed consent was obtained from all eligible participants. For purpose of this study, we excluded the participants with any self-reported stroke, self-reported heart disease, diabetes mellitus, refractive error greater than or equal to 8 diopters (D) or less than or equal to −12 D, glaucoma, any retinal diseases (retinopathy and age-related macular degeneration), obesity, or had ungradable retinal fundus photographs. The definition of diabetes, obesity, glaucoma, and retinal diseases has been reported previously.[33]

Retinal Photography

In the 120 subjects selected from SP2 and SCES, retinal photographs were taken after dilating the pupils with 1% tropicamide and 2.5% phenylephrine hydrochloride, using a digital nonmydriatic retinal camera (CR-DGi with a 10D SLR back; Canon, Tokyo, Japan). In each photograph, the optic disc was positioned at the center of the photograph.

Images acquired among the 120 subjects from SP2 and SCES were graded using RA, IVAN, and SIVA (all grading procedures are described as below).

Measurement Procedures of Different Measurement Systems

Retinal Analysis

Vessel grading was performed in accordance with the Atherosclerosis Risk in Communities (ARIC) study grading protocol modified as the MESA-EYE Retinal Vessel Measurement Protocol.[8] Details of RA measurement protocol has been described in detail in previous study.[8] Briefly, a standardized grid (calibrated to a fixed size based on the camera resolution) was manually centered on the optic disc. Each vessel was also identified manually by the trained grader as a venule or arteriole (Fig. 1A). All vessels, coursing through a specified area of the grid (0.5–1.0 disc diameter from the disc margin) were measured by a grader. Vessels measuring less than 25 μm were not measured. Branches of arterioles were measured if the trunk measured greater than or equal to 85 μm. The Knudtson-Parr-Hubbard formula was used to standardize individual vessel calibers as central retinal artery equivalent (CRAE) and central retinal vein equivalent (CRVE).[34]

Integrative Vessel Analysis

Vessel grading was performed based on the ARIC study grading.[35] Briefly, a standardized ARIC grid (calibrated to a fixed size based on the camera resolution) was manually centered on the optic disc. Unlike RA, vessels coursing through specified area (0.5–1.0 disc diameter from the disc margin) were automatically traced and identified as arterioles or venules (Fig. 1B). A trained grader masked to participants' characteristics further made manual corrections when necessary. Based on the revised Knudtson-Parr-Hubbard formula[34] the retinal arteriolar and venular calibers were summarized as CRAE and CRVE, respectively.

Singapore I Vessel Assessment

Details of SIVA measurement protocol have been described in detail previously.[29] In brief, SIVA automatically identified the optic disc with reference to the optic disc center, and automatically identified and traced the retinal arterioles and venules. To further ascertain the accuracy of automated vessel tracing generated by programs, trained graders examined the traced vessels and further made manual corrections as necessary. Retinal vascular parameters were only measured for a specified area, defined as the region from 0.5 to 2 disc diameter away from the disc margin (Fig. 1C). Retinal arteriolar and venular caliber was summarized as CRAE and CRVE, respectively, based on the revised Knudtson-Parr–Hubbard formula.[34] For the purpose of this study, only CRAE/CRVE measurements within 0.5 to 1.0 disc diameter are used.

Systemic Variables

Information on participants' demographic characteristics and medical history was obtained by using a standardized questionnaire administered by trained personnel. Age was defined as the age at the time of clinic examination. Height was measured in centimeters using a wall-mounted measuring tape and weight was measured in kilograms using a digital scale. Body mass index (BMI) was calculated as body weight divided by height squared and expressed as kg/m2. Systolic and diastolic blood pressures (S/DBP) were measured twice using a digital BP monitor (Dinamap model Pro Series DP110X-RW, 100V2; GE Medical Systems Information Technologies Inc., Milwaukee, WI). A third measurement was made if the SBP differed by greater than 10 mm Hg or the DBP differed by greater than 5 mm Hg. Hypertension was defined as SBP greater than or equal to 140 mm Hg, DBP greater than or equal to 90 mm Hg, or use of antihypertensive medication.[36] Venous blood samples were analyzed at the National University Hospital Referral Laboratory for biochemical testing of serum lipids and glucose.

Statistical Analysis

All statistical analyses were performed using MedCalc, version 12.5 (MedCalc Software, Ostend, Belgium) and SPSS statistics version 17.0 (SPSS Inc., Chicago, IL).

Bland Altman plots[37] were used to evaluate agreement among the three different grading systems. In the Bland-Altman plot analyses, the 95% limits of agreement (LOA) were defined as mean difference ±1.96 × SD. In the plots, the difference between two measurements was plotted against the average of two measurements. Where a trend in the plot was identified, the slope of the least squares regression line was tested to examine if it significantly differed from zero to investigate the presence of any proportional bias. This was tested by Pearson's correlation coefficient.[37,38] Presence of proportional bias would indicate that the discrepancies between the two measurements were not constant throughout the range of measurements. On the other hand, the mean difference value was compared with the zero value of difference using one-sample test to investigate the presence of any systemic (fixed) bias. Presence of systemic bias would indicate that the discrepancies were constant/fixed throughout the range of measurements.

Pearson correlation test was performed to examine the strength of association between each systemic variable with SIVA-measured retinal calibers (denoted as correlation coefficient R1). This analysis was repeated to obtain the correlation coefficients between each systemic parameter with RA- and IVAN-measured retinal calibers, respectively (denoted as correlation coefficients R2 and R3, respectively). For each systemic variable, we then performed Z-test to compare the correlation coefficients between R1 and R2, and between R1 and R3. In Z-test, P value of less than 0.05 denotes a significant difference between the two comparing R-values, indicating strength of associations are different in the two comparing models.

Conversion Algorithm

To develop an algorithm that converts the retinal vessel caliber measures from IVAN and RA software to approximate SIVA measurements, we used 80% data of the above mentioned 120 healthy subjects from SP2 and SCES (n = 97) as training set and the remaining 20% percent (n = 23) as validation set.

A linear regression, with SIVA measurements as the outcome, was used to construct the algorithm. In the training set, coefficient of RA/IVAN caliber variable was obtained from the regression model and used to construct the conversion algorithm. The equations of respective algorithm are shown below:

RA-derived SIVA CRAE = (0.4523 * RA-measured CRAE) + 65.6924 (1)

RA-derived SIVA CRVE = (0.6503 * RA-measured CRVE) + 64.1136 (

IVAN-derived SIVA CRAE = (0.7176 * IVAN-measured CRAE) + 34.3984 (3)

IVAN-derived SIVA CRVE = (0.7102 * IVAN-measured CRVE) + 44.8717 (4)

Algorithm for the Calculation of IVAN-Derived SIVA Approximate

Paired t-test was performed to evaluate the differences between SIVA-measured retinal caliber with RA- and IVAN-derived SIVA retinal caliber approximates. Pearson's correlation coefficient was also performed to examine the correlation between SIVA-measured retinal caliber with RA- and IVAN-derived SIVA retinal caliber approximates.

Results

Characteristics of subjects are shown in Table 1. The mean (SD) age was 49.2 years (11.1), mean SBP and DBP was 126.3 mm Hg (22.9) and 74.4 mm Hg (10.7), respectively, and mean spherical equivalent was −1.7 D (2.9).

Table 1

Baseline Characteristics of Included Participants

Figures 2 through 4 show the agreement in retinal vessel caliber measurements between the three retinal vasculature measurement systems. The mean difference between SIVA and RA was −21.8 μm (95% LOA, −47.3 to 3.7 μm) for CRAE measurement, and −7.7 μm (95% LOA, −28.0 to 12.6 μm) for CRVE measurement. For both measurements in CRAE and CRVE, the differences in vessel measurements between SIVA and RA correlated negatively and significantly with the average of the two measurements (for CRAE, r = −0.451, P < 0.001; for CRVE, r = −0.240, P = 0.008; Table 2), indicating presence of proportional bias.

Figure 2

Bland Altman plot of agreement between SIVA and Retinal Analysis (n = 120): (A) retinal arteriolar caliber (CRAE); (B) retinal venular caliber (CRVE).

Figure 4

Bland Altman plot of agreement between RA and IVAN (n = 120): (A) retinal arteriolar caliber (CRAE); (B) retinal venular caliber (CRVE).

Table 2

Agreement Analysis between RA, IVAN, and SIVA

Bland Altman plot of agreement between SIVA and IVAN (n = 120): (A) retinal arteriolar caliber (CRAE); (B) retinal venular caliber (CRVE).

When comparing retinal vessel caliber measurements between SIVA and IVAN, we observed that the mean difference between SIVA and IVAN was −6.7 μm (95% LOA, −23.8 to 10.4 μm) for CRAE measurement, and −18.2 μm (95% LOA, −36.7 to 0.4 μm) for CRVE measurement. In addition, the differences in vessel measurements between SIVA and IVAN correlated negatively and significantly with the average of the two measurements (for CRAE, r = −0.333, P < 0.001; for CRVE, r = −0.374, P = < 0.001; Table 2), indicating presence of proportional bias.

When comparing retinal vessel caliber measurements between RA and IVAN, we observed that the mean difference between RA and IVAN was 15.1 μm (95% LOA, −8.7 to 38.9 μm) for CRAE measurement, and −10.2 μm (95% LOA, −32.6 to 11.6 μm) for CRVE measurement. In addition, the differences of CRAE measurements between RA and IVAN correlated positively and significantly with the average of the two measurements (for CRAE, r = 0.232, P value: 0.011; Table 2), indicating presence of proportional bias. The differences of CRVE measurements between RA and IVAN correlated negatively but there was no presence of proportional bias (for CRVE, r = −0.099, P value: 0.282; Table 2), only systemic bias was present (P < 0.001).

Table 3 and Figures 5 and 6 show the associations between systemic parameters with retinal vessel caliber. The strength of correlations of systemic parameters (age, sex, SBP, DBP, mean arterial blood pressure [MABP], total cholesterol, low-density lipoprotein [LDL] cholesterol) with SIVA-measured CRAE were not significantly different to that measured using RA and IVAN (all P value ≥ 0.332). Similarly, the strength of correlations of systemic parameters (age, MABP, total cholesterol, serum glucose, BMI, and estimated glomerular filtration rate [eGFR]) with SIVA-measured CRVE was not significantly different to that measured using RA and IVAN (all P value ≥ 0.395).

Table 3

Association of Retinal Vessel Caliber Measured Using RA, IVAN, and SIVA with Systemic Parameters

Figure 5

Correlation of MABP with retinal arteriolar caliber measured by (A) RA; (B) IVAN; (C) SIVA.

Figure 6

Correlation of total cholesterol with retinal venular caliber measured (A) RA; (B) IVAN; (C) SIVA.

Comparisons between SIVA-measured retinal vessel caliber with RA- and IVAN-derived SIVA approximates are shown in Table 4. The mean difference between SIVA and RA-derived SIVA approximate was −0.85 μm (95% confidence interval [CI] −3.84, 5.54 μm, P = 0.710) for CRAE measurement, and 2.54 μm (95% CI −1.82, 6.90 μm, P = 0.239) for CRVE measurement. Pearson's correlation coefficient between SIVA-measured retinal vessel caliber with RA-derived SIVA approximates were high (CRAE: 0.762; CRVE: 0.874). The mean difference between SIVA and IVAN-derived SIVA approximate was −0.96 μm (95% CI, −4.92, 3.01 μm, P = 0.622) for CRAE measurement, and −0.39 μm (95% CI, −4.19 to 3.42 μm, P = 0.835) for CRVE measurement. Pearson's correlation coefficient between SIVA-measured retinal vessel caliber with IVAN-derived SIVA approximates were high (CRAE: 0.847; CRVE: 0.895).

Table 4

Mean Difference of SIVA-Measured Retinal Caliber with RA- and IVAN-Derived Approximates

Discussion

In this study, evaluating the agreement between three commonly used retinal vessel caliber measurement software systems (RA, IVAN, and SIVA), we observed poor agreement between the RA, IVAN, and SIVA absolute measurements. This indicates that absolute measurements from the three systems should not be interpreted interchangeably. However, despite the lack of agreement between the three systems, the strength of associations between systemic parameters with retinal vessel calibers measured was similar. This suggests that the ability of the three software systems to detect associations with established systemic factors was similar. We developed a simple algorithm that converts RA and IVAN measurements to SIVA-approximate with minimal error. This might improve the ability of data pooling, metaanalysis, and comparing results of retinal vessel measurements from different studies.

While numerous population-based studies have examined the associations between retinal vessel caliber measurements and various systemic risk factors, these studies have used different retinal vessel measurement systems.[18,21,29] Importantly, it is unclear if the absolute retinal vessel caliber measurements from different population-based studies can be interpreted interchangeably. In this study, we demonstrated poor agreement between three retinal measurement software. Specifically, measurements from SIVA are consistently smaller when compared with measurements of both RA and IVAN. The differences in vessel tracing algorithm protocol and retinal vessel border definition between the three computer-assisted retinal measurement systems may explain the poor agreement in retinal vessel caliber measurements.[8,34,35] Further, the measurement area of SIVA is based on the optic disc size, whereas measurement area of RA and IVAN is not affected by the optic disc size. As such, it is possible that for the same image, SIVA may have a wider/narrower measurement zone as compared with RA/IVAN and this may inherently affect retinal vessel caliber measurements. Thus, our findings suggest that absolute retinal vessel caliber measured by different software systems should not be used interchangeably, especially in the case of longitudinal/progression evaluations on the same individual (i.e., different software cannot be used at different time points on the same individual to evaluate changes in vessel caliber). In addition, the use of different retinal systems should be taken into account when pooling data from different studies.

However, we demonstrate that while the absolute measurements were different, the strength of associations with established systemic factors (i.e., MABP, total cholesterol) with retinal vessel caliber were similar across the three software (Table 3, Figs. 5, 6). This emphasizes the validity of previous studies in different populations.

We further developed a retinal vessel caliber conversion algorithm to generate SIVA-approximate measurements from RA and IVAN systems. We observed only a small mean difference (−0.39 to 2.54 μm) in absolute values when comparing SIVA-measured retinal vessel caliber with RA- and IVAN-derived SIVA approximates. This indicates the proposed algorithm is potentially useful in converting RA- and IVAN-measured retinal vessel calibers to SIVA-approximate measurements. As our testing sample is relatively small, more images will be needed to further validate our proposed algorithm.

The strengths of this study include the comprehensive measurements of systemic parameters and the complete availability of retinal vessel caliber measurements across the systems of RA, IVAN, and SIVA on same set of images. Nevertheless, this study has limitations. We only included retinal photographs with no ocular media opacities in our study. It is possible that the agreement across the three systems may be further affected in the presence of ocular media opacities. This aspect requires further evaluation.

In conclusion, we observed significant differences between retinal vessel caliber measurements made using RA, IVAN, and SIVA. However, the strength of associations between systemic parameters and retinal vessel calibers measured using the three systems were similar. Importantly, we developed a simple algorithm that can convert RA- and IVAN-measured retinal vessel calibers to SIVA-approximate measurements with minimal error. This will aid in data pooling, metaanalysis, and future establishment of normative values of retinal vessel caliber.

Click here for additional data file.