No loss of chance of diabetic retinopathy screening by endocrinologists with a digital fundus camera.

OBJECTIVE: To compare the efficacy of the diabetic retinopathy (DR) screening with digital camera by endocrinologists with that by specialist and resident ophthalmologists in terms of sensitivity, specificity, and level of "loss of chance." RESEARCH DESIGN AND METHODS: In a cross-sectional study, 500 adult diabetic patients (1,000 eyes) underwent three-field retinal photography with a digital fundus camera following pupillary dilatation. Five endocrinologists and two ophthalmology residents underwent 40 h of training on screening and grading of DR and detection of associated retinal findings. A κ test compared the accuracy of endocrinologist and ophthalmology resident screening with that performed by experienced ophthalmologists. Screening efficiency of endocrinologists was evaluated in terms of "loss of chance," i.e., missed diagnoses that required ophthalmologist referrals.
RESULTS: The mean weighted κ of DR screening performed by endocronologists was similar to that of ophthalmology residents (0.65 vs. 0.73). Out of 456 DR eyes, both endocrinologists and ophthalmology residents misdiagnosed only stage 1 DR (36 and 14, respectively), which did not require ophthalmologist referral. There were no significant differences between endocrinologists and ophthalmology residents in terms of diabetic maculopathy and incidental findings except for papillary cupping and choroidal lesions, which were not the main purpose of the study or of the training.
CONCLUSIONS: The endocrinologist with specific training for DR detection using a three-field digital fundus camera with pupillary dilatation can perform a reliable DR screening without any loss of chance for the patients when compared with identical evaluation performed by experienced ophthalmologists.

Diabetic retinopathy (DR) is one of the main causes of blindness in industrialized nations (1). The worldwide prevalence of diabetes in adults is estimated to rise to 7.7%, affecting 439 million adults by 2030 (2). In France, the increasing number of patients with diabetes, coupled with the lack of a national screening program, results in a steady rise in the visual handicaps related to the disease (3).

Annual screening of DR is recommended as an effective approach to prevent visual loss related to diabetes (4,5). Currently, digital nonmydriatic fundus photography is increasingly used as a method of screening for ophthalmologists worldwide (5–7). According to consensus classifications (4,5), DR at a stage higher than 1 needs further ophthalmological management. Despite these recommendations, only 30% of the diabetic patients in France undergo DR screening each year. Partly, this is due to the lack of ophthalmologists and insufficient awareness about the visual consequences of the disease (3,8). The situation is slowly changing after implementation of telemedical screening networks using digital fundus photography (9–11). Further increase in screening coverage can be achieved with the involvement of allied medical professionals.

Since the 1980s, the concept of “loss of chance” has emerged in medicine and law. The misdiagnosis during DR screening can lead to a loss of chance for patients requiring referral to an ophthalmologist for further examinations and management (12,13).

Two studies have shown that screening performed by an endocrinologist using an ophthalmoscope (14) and a mydriatic camera (15), respectively, were reliable, although they didn’t evaluate the loss of chance. Furthermore, no endocrinologists’ team approach was evaluated so far.

This clinical research trial has been designed to compare the efficacy of the DR screening with digital camera by a team of previously trained endocrinologists (7) with that of residents and specialist ophthalmologists, in terms of sensitivity, specificity and level of “loss of chance.”

RESEARCH DESIGN AND METHODS

The local ethics committee approved the study protocol, and all patients signed an informed consent.

Five hundred consenting and able-to-cooperate adult patients referred to the diabetes department of the University Hospital of Saint-Etienne for diabetes during a 6-month period underwent a systematic DR screening. Medical details of each patient were recorded, including age, duration of diabetes, type of treatment, associated systemic risk factors, and history of eye treatment. All 500 patients underwent three-field retinal photography with the Topcon TRC NW6S digital camera (Topcon Europe, Rotterdam, the Netherlands) linked to a high-resolution (6.17 million pixels) Fujifilm Fine Pix S2 Pro Super CCD camera (Fujiphoto, Tokyo, Japan) in Tagged Image File Format (TIFF)-RGB recording mode with a density of 2,034 × 1,728 pixels. As previously described (7), retinal photography was performed after instillation of one drop of tropicamide 1% in each eye. Three 45-degree images of horizontal overlapping fields were captured (one central [the macula including the optic disk], one temporal [macula on the nasal edge], and one nasal [disk on the nasal edge]), and stored after compression (1:17) in JPEG format (16). Images were taken using the internal and, on rare occasions, external fixation target. The image acquisition process was repeated if the original image was judged unsatisfactory by the photographer. Automatic mosaic reconstruction was performed with ImageNet2000 software (Topcon Europe, Rotterdam, the Netherlands), followed by manual retouch to enhance the gradeability of partially obscured yet interpretable images. All the images were taken by the same photographer throughout the study.

Five endocrinologists and two ophthalmology residents underwent specific training over 40 h conducted by a consultant ophthalmologist specializing in retinopathy. The training consisted of both theoretical and practical sessions and was focused on photographic detection and gradation of DR including diabetic maculopathy (DM). Additional training was offered to detect associated fundus findings: hypertensive retinopathy (arteriolar narrowing, arterio-venous crossing, cotton wool spots, and flame shaped hemorrhages), age-related macular degeneration (ARMD), myopic chorioretinal degeneration, optic disc disorders such as edema, cupping, and atrophy, and choroidal lesions such as nevi and laser scars. Particular attention was paid to the criteria and the timing of referral to the ophthalmologists (4): noninterpretable images, severe DR stage >1, DM, papillary cupping, papillary atrophy, choroidal lesions, nevi, and all other unknown incidental findings. Upon completion of training in screening and grading the retinal images and satisfactory evaluation, the endocrinologists and ophthalmology residents were inducted into the study. A simplified DR classification adapted from American Diabetes Association and Francophone Diabetes Society recommendations was used for grading (4,5): stage 1 (mild nonproliferative DR [NPDR]) was represented by occasional microaneurysms and/or hemorrhages and/or exudates; stage 2 (moderate NPDR) by intra retinal hemorrhages and/or cotton-wool spots and/or venous anomalies in one to three quadrants; stage 3 (severe NPDR) by intra retinal hemorrhages and/or cotton-wool spots and/or venous anomalies in all quadrants; stage 4 (noncomplicated proliferative DR [PDR]) by detection of new vessels on disc or retina; stage 5 (complicated PDR) by tractional retinal detachment, preretinal or vitreous hemorrhage; DM by the presence of hard exudates within one disk diameter of the fovea, considered treated with the presence of photocoagulation scars anywhere (sectored, pan-retinal, focal, and grid). The consensus opinion of two retina specialists using the same three-field images evaluation was considered as the “gold standard.”

A total of 1,000 sets (500 patients, 1,000 eyes) of three images and the respective mosaics were recorded on a CD-ROM. Each clinician independently analyzed each of the 1,000 separate sets of images in a masked fashion (without knowing which the paired eyes were and without any clinical information) on 15-inch TFT screens of personal computers at a resolution of 1,400 × 1,050 pixels and 32-bit true-color display with the Windows XP image viewer (Microsoft Corporation, Redmond, WA) as previously described (7,17).

The observers had to note the following points for each set: 1) image quality of the individual images as well as reconstructed mosaics; 2) confidence degree for screening and grading DR depending on the quality of the images. Presence of DR, its grading, and presence of MD; and 3) presence of associated ocular findings, as previously described.

Statistical analysis

Sensitivity and specificity were evaluated for each observer for screening and grading of DR, including DM. Eyes with laser images or with noninterpretable images were referred directly to ophthalmologists. This group was used in the calculation of loss of chance. The agreement between the observers and the gold standard for the assessment of the digital images was expressed as an underweight κ index or κ value (18). Agreement of screening was classified as follows: almost perfect, κ >0.8; substantial, κ 0.8–0.6; moderate, κ 0.6–0.4; fair, κ 0.4–0.2; slight, κ 0.2–0.0; no agreement, κ <0.0. κ values were compared between endocrinologists and ophthalmology residents by a nonparametric Mann-Whitney U test (P < 0.05 considered significant). Eyes needing ophthalmologist referral for further management (referral criteria), but missed by endocrinologist and ophthalmology resident screeners were counted as loss of chance. Statistical analysis was performed by SPSS version 11.0 (SPSS, Chicago, IL).

RESULTS

Of the 500 patients, 263 (52.6%) were male. Type 1 diabetes was found in 121 patients (24.2%) with a mean age of 40 ± 22 years and a mean of 15 ± 15 years since disease onset. Type 2 diabetes was found in 368 (73.6%) patients who had a mean age of 64 ± 17 years and a mean of 12 ± 12 years since disease onset. Gestational diabetes was found in six patients (1.2%) and impaired glucose tolerance in five patients (1%). Type 2 diabetic patients were treated as follows: insulin therapy (66%), oral antidiabetic drugs only (28.7%), insulin and oral antidiabetic drugs (5.8%), or diet only (4.5%).

The flow diagram of the eyes/findings through the study is presented in Fig. 1. Out of 1,000 eyes, 38 showed noninterpretable images, and were therefore excluded from the retinal assessment and directly referred to the ophthalmologists. Of 962 eyes with analyzable images, DR was detected in 456 (47.4%). It consisted of 246 (53.9%) cases of mild NPDR (stage 1), 98 (21.5%) of moderate NPDR (stage 2), 14 (3.1%) of severe NPDR (stage 3), eight (1.8%) of uncomplicated PDR (stage 4), zero cases of complicated PDR (stage 5), 74 (16.2%) cases of inactive PDR post laser, and 16 (3.5%) cases of active PDR post laser. DM was detected in 186 eyes (2.3%). Associated ocular findings were found in 669 eyes (67%), and most frequently this consisted of hypertensive retinopathy (22%), ARMD (5.7%), and transparency media abnormalities (13%).

Figure 1

Flow diagram of the eyes/findings through the study for diabetic retinopathy screening by endocrinologists.

Image quality assessment

Out of the five endocrinologists, four had substantial agreement with the gold standard regarding appreciation of the image quality (κ = 0.69, 0.62, 0.63, 0.75, and 0.22). The agreement was substantial (κ = 0.73) for one ophthalmology resident and almost perfect (κ = 0.83) for the other.

Laser detection

The ability of all the endocrinologists and ophthalmology residents to detect laser on retina was almost perfect (κ >0.8).

DR screening

Ninety eyes with laser images were not included in the analysis of screening accuracy. Thirty-eight eyes with noninterpretable images were not included in retinal assessment either. The screening accuracy, which demonstrates the agreement between each observer and gold standard, was evaluated on the remaining 872 eyes. κ index was substantial for three endocrinologists, moderate for one, and fair for one. The adjusted κ remained substantial for the same endocrinologists (κ = 0.78, 0.71, and 0.75) and moderate (κ = 0.4 and 0.6) for the rest. The average weighted κ was substantial for the endocrinologists (κ = 0.65 [range 0.55–0.68]). The κ index and the weighted one were substantial for both ophthalmology residents (weighted κ = 0.73 [0.65–0.70]) (Table 1). The average sensitivity of the endocrinologists was 89.7% [84.1–100.0]. The average sensitivity of the ophthalmology residents was 96.4% [95.1–97.8]. The negative predictive value of the screening was 91.9 [88.8–100.0] for the endocrinologists and 96.7 [95.2–98.0] for the ophthalmology residents. Out of 456 eyes with DR, 36 (8.25%) have been missed by the endocrinologists. All of them presented with mild NPDR (stage 1), which does not require referral. Fourteen cases (2.85%), all in stage 1 NPDR too, have been also missed by ophthalmology residents.

Table 1

Screening agreements, between observers (endocrinologists and ophthalmology residents) and gold standard for diabetic retinopathy and incidental findings

	Endocrinologists	Ophthalmologists
DR	0.65 (0.55–0.68)	0.73 (0.65–0.70)
Transparency media abnormalities	0.49 (0.26–0.72)	0.79 (0.66–0.91)
Hypertension retinopathy	0.24 (0.01–0.48)	0.49 (0.47–0.50)
ARMD	0.38 (0.20–0.56)	0.76 (0.66–0.85)
High myopia	0.44 (0.00–1.23)	0.91 (0.78–1.00)
Papillary cupping	0.35 (0.09–0.40)	0.76 (0.76–0.76)
Papillary atrophy	0.00 (0.00–0.00)	0.49 (0.24–0.73)
Choriodal lesions	0.30 (0.14–0.46)	0.53 (0.48–0.57)
Nevi	0.59 (0.45–0.73)	0.82 (0.44–1.00)
Extra macular druses	0.34 (0.11–0.57)	0.61 (0.42–0.79)
Isolated retinal hemorrhage	0.08 (0.00–0.58)	0.53 (0.26–0.79)
Other incidental findings	0.17 (0.00–0.37)	0.50 (0.17–0.82)

Data are κ index mean values (95% CI).

DR grading

The grading accuracy was evaluated on 872 eyes. One hundred fifty-six eyes (10.2%) were in stage 2 NPDR, and 22 (2.3%) were in stage 3 or 4 NPDR (14 and 8, respectively). The agreement for DR grading between endocrinologists and gold standard was fair for two of them (κ = 0.40 and 0.35), moderate for two (κ = 0.44 and 0.49), and substantial for one (κ = 0.62). The κ index was substantial for the ophthalmology residents (κ = 0.76). The endocrinologists have under-classified as stage 2 NPDR several stage 3 NPDR cases (2, 6, 8, 10, and 10 respectively for each endocrinologist on 22 cases).

Maculopathy screening

The agreement between the endocrinologists and gold standard for maculopathy screening was almost perfect for two of them (κ = 0.78 and 0.75), substantial for two (κ = 0.68 and 0.70) and moderate for one (κ = 0.53). The agreement between the ophthalmology residents and gold standard was almost perfect (κ = 0.83 and 0.77). Out of 186 eyes with DM, 20 eyes have been missed by both endocrinologists and ophthalmology residents.

Detection of associated findings

The agreement between the observers and gold standard is presented in Table 1. The endocrinologists have frequently misdiagnosed ARMD as diabetic macular disease. The nonparametric comparison tests indicate trends for reduced efficacy in detecting associated findings for the endocrinologists compared with the ophthalmology residents.

Loss of chance

The loss of chance concerning the screening performed by the endocrinologists and ophthalmology residents is presented in Table 2. This loss of chance was calculated from the 555 diagnoses to refer to the ophthalmologists according to the gold standard: 186 DM cases, 90 laser cases, 38 noninterpretable images, 120 DR stage >1, and 121 incidental findings. There was no loss of chance for the 120 eyes with DR to be referred (stage >1) for both endocrinologists and ophthalmology residents because none of these diagnoses has been missed.

Table 2

Loss of chance: diagnoses to refer to the ophthalmologists but missed by the observers

	DR >1	Laser	Noninterpretable images	DM	Papillary cupping	Papillary atrophy	Choroidal lesions	Nevi	Other incidental findings
Number of diagnoses to refer	120	90	38	186	20	2	34	37	28
E 1	0	0	12	30	16*	2	17*	21	16
E 2	0	4	8	44	16*	2	27*	24	22
E 3	0	8	6	18	12*	1	18*	1	19
E 4	0	2	6	4	14*	2	29*	17	27
E 5	0	0	6	4	18*	1	27*	9	12
Mean E	0	2.8	7.6	20	15.2*	1.8	23.6*	14.4	19.2
OR 1	0	0	16	14	5	0	11	20	15
OR 2	0	2	8	26	4	1	8	4	11
Mean OR	0	1	12	20	4.5	0.5	9.5	12	13

Lines E1 to Mean OR indicate the number of misdiagnoses.

*Difference between endocrinologists and ophthalmology residents: P value ≤ 0.05.

E, endocrinologist; OR, ophthalmology residents. Average values for endocrinologists (Mean E) and ophthalmology residents (Mean OR) are presented in boldface type.

Regarding the detection of DM, laser marks, noninterpretable images, and other associated findings, the difference of loss of chance between endocrinologists and ophthalmology residents was not significant. The endocrinologists missed significantly more papillary cupping and choroidal lesions (15.2 vs. 4.5 on 20 cases and 1.8 vs. 0.5 on 2 cases, respectively, P < 0.05).

CONCLUSIONS

This study compares the efficacy and the degree of loss of chance during screening performed by trained endocrinologists in comparison with ophthalmology residents. The average sensitivity of detection of DR by endocrinologists was good (89.7%), with only stage 1 NPDR disease having been missed. This has little consequence in terms of loss of chance on patient management because rapid ophthalmologic interventions are not required at this stage. Furthermore, these missed cases are likely to be picked up during subsequent annual examinations (4,5).

The endocrinologists also had difficulties in grading DR with several stage 3 NPDR diseases being under classified as stage 2. Those misdiagnoses do not amount to any loss of chance because the timing of referral to ophthalmologists for further management remains the same as per recommendations (4,5).

We did not find any major difference between endocrinologists and ophthalmology residents for DM detection and, consequently, no significant “loss of chance.” The results show a trend of a lower efficacy in screening associated ocular findings like nevi or optic disc atrophy. The low agreement for the incidental findings arising as a result of confusion between DR, hypertensive retinopathy, and ARMD can be explained by the confounding vascular changes occurring in these pathologies. The fact that the endocrinologists could detect these subtle findings demonstrates their good sensitivity. However, all of these patients were referred to the ophthalmologists, without loss of chance.

Overall, because there is no loss of chance, the screening of the DR performed with a digital fundus camera by trained endocrinologists was as accurate and safe as that performed by ophthalmology residents.

We previously reported that dilatation significantly improved image quality and certitude of screening DR using a nonmydriatic camera without disturbing the unit’s organization (20 min to obtain the dilatation, whether planned or not, no side effect and 5 min to read the images) (7). In the current study, after systematic dilatation, almost 80% of images sets presented with good quality. The agreement between endocrinologists and gold standard on the image quality was substantial (κ = 0.61). The comparable confidence level allows trained endocrinologists to screen with accuracy, and to decide to refer to the ophthalmologist in case of poor image quality, which ensures a safe delegation with no “loss of chance.”

Besides, we have found 555 diagnoses to refer on those 1,000 eyes in an exhaustive screening of each patient attending the care unit (Fig. 1). Obviously, many patients may have several coexistent pathologies, such as DR and hypertensive retinopathy, which is in agreement with our previous study. This exhaustive screening led to discover coexistent fundus pathologies, which appears to be of cost-effective second benefit (19). Additionally, this crossover diagnosis allows multiplying the chance of being referred for an eye, thus decreasing the loss of chance if one diagnosis has been missed.

Three specific points can be noticed: the device features, the delegation, and the location. Although the DR screening by endocrinologists using the direct ophthalmoscopy reported good results (14), this technique is less sensitive than the digital camera (20). The number of images to use for screening with digital camera is another issue to discuss. Current recommendations propose the use of two images (central and nasal focused) as sufficient for safe screening (4). Other recent data indicate that one central image would be sufficient to screen but not to grade (21). However, in a recent study the use of a single image by primary care clinicians failed to refer 10.2% of the patients who would have otherwise needed referral to the ophthalmologist (22). In our study, we used three images per eye to screen and grade with no loss of chance when compared with senior ophthalmologists. However, when comparing three versus seven-field comprehensive exploration, a minimal loss of chance could exist even for senior ophthalmologists (κ = 0.88) (21). The possibility of such a delegation among medical professionals has been considered since the 1990s, involving mainly general practitioners (22,23), optometrists, and orthoptists (24). There are numerous studies that demonstrate the utility of a computer program to help nonophthalmologists in screening (25), and telemedicine is well developed (11). In our study, screening has been performed by endocrinologists, who are well versed with the disease and risk factors, which could contribute to screening efficiency without disturbing the unit organization (additional 5 min for the screening performed by the endocrinologists). In contrast to previous studies involving screening by a single endocrinologist (14,15), our study has been performed by five endocrinologists with reproducible and similar skills. This screening has been performed in the diabetic care unit and not in general health centers, which allows the maximum possible recruitment of diabetic patients. Use of the digital fundus camera improves the rapidity and precision of screening thus allowing exhaustive coverage feasible in a hospital setting, as shown in our previous study (17). In addition, contrary to telemedical systems, the patients do not have to wait for the ophthalmologist’s diagnosis, with a complete diabetes work-up, glycemic control, and complications screening in a single appointment. The endocrinologists can make a preliminary screening and refer only pathological or doubtful cases to the ophthalmologists for a comprehensive examination. Therefore ophthalmologists’ workload can be reduced, and there is an increasing availability of specialist care when requested.

However, one finding raises some concern. There is a loss of chance for only the papillary cupping and the choroidal lesions with a significant higher number of missed diagnoses for the endocrinologists compared with the ophthalmology residents (P ≤ 0.05). Although these were not the main purpose of the training, this shows that training has to be improved for the associated findings.

In conclusion, DR screening by endocrinologists with three-field digital fundus camera and pupillary dilatation placed in a diabetic care unit is safe and secure, inducing no loss of chance for the patients when compared with identical evaluation performed by experienced ophthalmologists. This program allows us to attain the recommended objective of annual retina examination for all patients attending a diabetes department over 1 year. These results enable the delegation of DR screening to the endocrinologists using a digital camera. At a national health care framework level, delegation of DR screening to endocrinologists with no loss of chance could potentially increase the proportion of patients with diabetes who undergo this important preventive intervention.