Feasibility of screening for diabetic retinopathy using artificial intelligence, Brazil.

Problem: There is currently no national strategy or standardized approach to diabetic retinopathy screening in the Brazilian public health system, and multiple socioeconomic barriers prevent access to eye examination in Brazil's poorest regions. Approach: From September 2021 to March 2022 we carried out a pilot project with an artificial intelligence system for diabetic retinopathy screening, embedded in a portable retinal camera. Patients with a diagnosis of diabetes according to the municipality registry were invited to attend nearby clinics for screening on designated days. Trained health-care technicians acquired images which were automatically evaluated by the system, with instant remote evaluation by retinal specialists in selected cases. Local setting: Our study was based in Sergipe State, located at a region with high illiteracy rates and no local availability of specialized retina care. The average number of laser treatments performed annually in the last 5 years is 126, for a total State population of 2.3 million. Relevant changes: Even though screening was performed free of charge in a convenient location for patients, from a total 2052 eligible individuals, only 1083 attended for screening. Lessons learnt: Efforts to raise awareness on the condition screened and to provide health education for patients and local health-care personnel are fundamental for increased attendance. Tailoring screening systems to the local setting, such as determining the trade-off between sensitivity and specificity, is challenging in regions with no current benchmarks. Standards for retinopathy screening based on the strategies adopted by high-income countries may not be realistic in low- and middle-income countries. (c) 2022 The authors; licensee World Health Organization.

Introduction

Major public health issues need scalable solutions, especially in low- and middle-income countries where the health workforce is scarce and public health systems are under-resourced. Recently, reports of implementation of artificial intelligence systems have been published, and the screening of diabetic retinopathy, one of the leading causes of preventable blindness in the adult working population, is one such example. Estimates show that there will be more than half a billion persons with diabetes by 2030. Given that each patient needs at least one annual retinal examination, the traditional individualized assessment by retinal specialists may not be sustainable in all health-care systems.

A recent study in Thailand showed the feasibility of using artificial intelligence for diabetic retinopathy screening in a national strategy; favourable results were found as compared with the regional standard of care. The authors emphasized the importance of socioenvironmental factors and screening workflows for successful implementation. However, in some low- and middle-income countries, there are no official validated solutions or established standards of care for several important public health conditions. This issue poses the question: what are the benchmarks against which an artificial intelligence-based strategy should be compared? In many cases, standards based on the strategies adopted by high-income countries may not be realistic in low- and middle-income countries.

We share our experience of diabetic retinopathy screening that may provide useful lessons for other low- and middle-income countries.

Local setting

Brazil is a large upper-middle-income country that hosts the sixth largest population of individuals with diabetes in the world. The country also has the largest free public health-care system, on which the majority of its population relies. There are currently no national strategies or standardized workflows for diabetic retinopathy screening in the Brazilian public health system. Multiple socioeconomic barriers prevent access to eye examination in the poorest regions, as is the case of the remote areas of the north-eastern state of Sergipe. Sergipe State is located in a region with high illiteracy rates and no availability of specialized retinal care outside of the State’s capital. According to available data on the official database of Brazil’s public health system, the average number of laser treatments performed annually in the last 5 years was 126, for a total State population of 2.3 million.

Approach

From September 2021 to March 2022 we did a pilot project in four rural villages that have no available ophthalmology care, in Sergipe State’s drylands (total population 76 000; human development index (HDI) range: 0.529–0.602). Patients with a diagnosis of diabetes according to the municipality registry were invited by local health agents to attend for screening on designated days.

The screening consisted of retinal imaging with a previously validated, highly sensitive, offline artificial intelligence system, embedded in a portable retinal camera. Image acquisition was performed by trained health-care technicians, with automatic evaluation by the system, and instant remote evaluation by retinal specialists in selected cases. Screening was performed in primary-care clinics located close to the patients’ place of residence. Due to the system’s high sensitivity, only patients identified as suitable for referral (more than mild diabetic retinopathy or images with poor quality) were sent for remote evaluation by retinal specialists, the technician being instantly notified on the need for referral. Laser treatment was offered free of charge to all individuals with vision-threatening diabetic retinopathy detected.

Relevant changes

Although screening was performed free of charge in a convenient location for patients, from a total 2052 eligible individuals, only 1083 attended the screening. Since there is no legal framework for artificial intelligence in health in Brazil ‒ such discussion is currently underway in the Brazilian Senate ‒ all examinations were eventually reassessed by specialists afterwards; no vision-threatening case was missed.

Lessons learnt

First, although screening provided by artificial intelligence systems at the point of care correlates with improved adherence to referrals,, we believe the lack of awareness about diseases, complications and prevention among the public is a major issue accounting for a low attendance in our setting. Even in countries with well-established screening programmes, only approximately 18% to 60% of individuals undergo examinations. Continuous education of patients, and health-care providers and authorities, is important for the prevention of chronic complications of diabetes. Comprehensive efforts and communication campaigns should therefore be made to engage the whole society and to increase awareness and adherence rates (Box 1). Additionally, we scheduled home visits for patients unable to go to the examination site whenever possible. Furthermore, it should be noted that this kind of screening has never been performed in this region. We believe that continuity of the programme would allow patients who missed one opportunity for examination to attend subsequent screenings.

Raising awareness about diabetic retinopathy among patients and local health-care workers is essential for increased attendance and the overall success of screening initiatives.

Tailoring screening systems to the local setting, such as determining the trade-off between sensitivity and specificity, is challenging in regions with no current benchmarks.

Cost considerations are essential for the sustainability of diabetic retinopathy programmes; health outcome metrics should be sought to evaluate the impact of such programmes.

Second, the design of the strategy should be tailored to account for existing or proposed workflows. To reduce the risk of ungradable images, the training of personnel on the acquisition of ocular images should be emphasized and measured. Operators could also be trained for image quality assessment, allowing the implementation of a staged mydriasis strategy. A protocol which also includes anterior segment images allows for identification of cataracts, which may also be considered referral criteria, as lens opacities are a frequent cause of visual impairment in such patients.

Third, cost decisions have to be customized to each setting to fit local resource constraints. These decisions could include the predefined end-points for sensitivity and specificity; the threshold for referral; the criteria for considering an image as gradable or ungradable; autonomous versus semi-autonomous workflows; systems that detect multiple diseases versus disease-specific systems; and the use of other variables such as visual acuity. When defining or calibrating the screening system’s cut-off, choices are made in the form of a trade-off between sensitivity and specificity. As an example, the Exeter protocol, put forward in 1995, proposed a minimum sensitivity of 80% and a minimum specificity of 95% for the screening of vision-threatening diabetic retinopathy. With annual screenings, this level of sensitivity could be adequate because repeated examinations tend to detect retinopathy missed at earlier examinations. Since referrals incur costs for the patient and the health-care system, the emphasis on specificity over sensitivity has been a benchmark for several diabetic retinopathy screening initiatives. In our example, the artificial intelligence system had been previously optimized from a screening perspective for a high sensitivity to yield fewer false negatives. We could have used a semi-automated workflow that maintained expert human reading only for cases identified as referable, provided the system was approved by the national regulatory agency. Such an approach has been reported to be cost-effective.

Many other challenges will arise in the deployment of artificial intelligence systems, such as the absence of universal electronic records, slow internet speeds, operator-related and camera-related technical issues, and the lack of standardization for the exchange of retinal images and clinical data. In the example from Thailand mentioned above, the reported advantages of advanced technology for diabetic retinopathy screening are not yet proven through formal cost‒benefit and cost‒utility analyses, However, the recent implementation of artificial intelligence for diabetic retinopathy screening has also been reported in other low- and middle-income countries such as India and Rwanda., In India, several studies reported high diagnostic accuracy of artificial intelligence for diabetic retinopathy screening and its feasibility in the primary-care setting. In the case of Rwanda, immediate feedback of the screening results to patients was related to better adherence to follow-up appointments. The real success of artificial intelligence systems is related not only to the diagnostic accuracy but also to real improvement in the health of the population. Further studies are necessary to evaluate such outcomes.

Ensuring subsequent specialized care is important for those patients identified by the screening programme and it is also necessary to consider the ability of the community health-care system to accommodate the additional numbers of patients identified. Underserved areas already lack sufficient treatment coverage, and the systematic identification of new cases of diabetic retinopathy will certainly increase the demand for treatment in the early phases of the programme. Our initiative included advocacy directed at the local health authorities for the adoption and implementation of a cost-effective model for the screening of diabetic retinopathy, with the aim of attaining financial sustainability for the programme. Other ethical aspects of artificial intelligence, such as non-maleficence (the expert over-reading of all images not to miss false negatives) and the promotion of equity (diagnosis and treatment were offered to all eligible patients free of charge) were also elements of the initiative.

Finally, incomplete data and the lack of representative data sets in the development of artificial intelligence may lead to bias. Public data sets of diabetic retinopathy images represent samples from only seven countries, leaving most of the global population unrepresented. Of note, the screening system used in the present initiative was previously validated with a similar sample from a neighbouring state in Brazil.

In conclusion, our experience with an artificial intelligence system for prevention of diabetic blindness highlights challenges that are certainly shared by other low- and middle-income countries. Each country will make choices when deciding on public health screening strategies; decisions will be made in the political sphere, after legislative debates, or following scientific guidelines, for example. We believe that a strategy involving scientific societies or multilateral forums can determine the characteristics of the artificial intelligence system, standardized protocols and key quality metrics. The aim would be to maximize outcomes in a cost-effective and successful manner, given the available resources to ensure the sustainability of such efforts.