Valentina Bellemo1, Zhan W Lim2, Gilbert Lim2, Quang D Nguyen1, Yuchen Xie1, Michelle Y T Yip3, Haslina Hamzah1, Jinyi Ho1, Xin Q Lee1, Wynne Hsu2, Mong L Lee2, Lillian Musonda4, Manju Chandran5, Grace Chipalo-Mutati6, Mulenga Muma7, Gavin S W Tan8, Sobha Sivaprasad9, Geeta Menon5, Tien Y Wong8, Daniel S W Ting10. 1. Singapore Eye Research Institute, Singapore National Eye Centre, Singapore. 2. School of Computing, National University of Singapore, Singapore. 3. Ophthalmology and Visual Science Academic Clinical Program, Duke-NUS Medical School, Singapore. 4. Eye Department, Kitwe Central Eye Hospital, Kitwe, Zambia. 5. Department of Ophthalmology, Frimley Park Hospital NHS Foundation Trust, Camberley, UK. 6. Ophthalmology Department, Lusaka University Teaching Hospital, Lusaka, Zambia. 7. Ministry of Health, Lusaka, Zambia. 8. Singapore Eye Research Institute, Singapore National Eye Centre, Singapore; Ophthalmology and Visual Science Academic Clinical Program, Duke-NUS Medical School, Singapore. 9. NIHR Moorfields Biomedical Research Centre, London, UK. 10. Singapore Eye Research Institute, Singapore National Eye Centre, Singapore; Ophthalmology and Visual Science Academic Clinical Program, Duke-NUS Medical School, Singapore. Electronic address: daniel.ting.s.w@singhealth.com.sg.
Abstract
BACKGROUND: Radical measures are required to identify and reduce blindness due to diabetes to achieve the Sustainable Development Goals by 2030. Therefore, we evaluated the accuracy of an artificial intelligence (AI) model using deep learning in a population-based diabetic retinopathy screening programme in Zambia, a lower-middle-income country. METHODS: We adopted an ensemble AI model consisting of a combination of two convolutional neural networks (an adapted VGGNet architecture and a residual neural network architecture) for classifying retinal colour fundus images. We trained our model on 76 370 retinal fundus images from 13 099 patients with diabetes who had participated in the Singapore Integrated Diabetic Retinopathy Program, between 2010 and 2013, which has been published previously. In this clinical validation study, we included all patients with a diagnosis of diabetes that attended a mobile screening unit in five urban centres in the Copperbelt province of Zambia from Feb 1 to June 31, 2012. In our model, referable diabetic retinopathy was defined as moderate non-proliferative diabetic retinopathy or worse, diabetic macular oedema, and ungradable images. Vision-threatening diabetic retinopathy comprised severe non-proliferative and proliferative diabetic retinopathy. We calculated the area under the curve (AUC), sensitivity, and specificity for referable diabetic retinopathy, and sensitivities of vision-threatening diabetic retinopathy and diabetic macular oedema compared with the grading by retinal specialists. We did a multivariate analysis for systemic risk factors and referable diabetic retinopathy between AI and human graders. FINDINGS: A total of 4504 retinal fundus images from 3093 eyes of 1574 Zambians with diabetes were prospectively recruited. Referable diabetic retinopathy was found in 697 (22·5%) eyes, vision-threatening diabetic retinopathy in 171 (5·5%) eyes, and diabetic macular oedema in 249 (8·1%) eyes. The AUC of the AI system for referable diabetic retinopathy was 0·973 (95% CI 0·969-0·978), with corresponding sensitivity of 92·25% (90·10-94·12) and specificity of 89·04% (87·85-90·28). Vision-threatening diabetic retinopathy sensitivity was 99·42% (99·15-99·68) and diabetic macular oedema sensitivity was 97·19% (96·61-97·77). The AI model and human graders showed similar outcomes in referable diabetic retinopathy prevalence detection and systemic risk factors associations. Both the AI model and human graders identified longer duration of diabetes, higher level of glycated haemoglobin, and increased systolic blood pressure as risk factors associated with referable diabetic retinopathy. INTERPRETATION: An AI system shows clinically acceptable performance in detecting referable diabetic retinopathy, vision-threatening diabetic retinopathy, and diabetic macular oedema in population-based diabetic retinopathy screening. This shows the potential application and adoption of such AI technology in an under-resourced African population to reduce the incidence of preventable blindness, even when the model is trained in a different population. FUNDING: National Medical Research Council Health Service Research Grant, Large Collaborative Grant, Ministry of Health, Singapore; the SingHealth Foundation; and the Tanoto Foundation.
BACKGROUND: Radical measures are required to identify and reduce blindness due to diabetes to achieve the Sustainable Development Goals by 2030. Therefore, we evaluated the accuracy of an artificial intelligence (AI) model using deep learning in a population-based diabetic retinopathy screening programme in Zambia, a lower-middle-income country. METHODS: We adopted an ensemble AI model consisting of a combination of two convolutional neural networks (an adapted VGGNet architecture and a residual neural network architecture) for classifying retinal colour fundus images. We trained our model on 76 370 retinal fundus images from 13 099 patients with diabetes who had participated in the Singapore Integrated Diabetic Retinopathy Program, between 2010 and 2013, which has been published previously. In this clinical validation study, we included all patients with a diagnosis of diabetes that attended a mobile screening unit in five urban centres in the Copperbelt province of Zambia from Feb 1 to June 31, 2012. In our model, referable diabetic retinopathy was defined as moderate non-proliferative diabetic retinopathy or worse, diabetic macular oedema, and ungradable images. Vision-threatening diabetic retinopathy comprised severe non-proliferative and proliferative diabetic retinopathy. We calculated the area under the curve (AUC), sensitivity, and specificity for referable diabetic retinopathy, and sensitivities of vision-threatening diabetic retinopathy and diabetic macular oedema compared with the grading by retinal specialists. We did a multivariate analysis for systemic risk factors and referable diabetic retinopathy between AI and human graders. FINDINGS: A total of 4504 retinal fundus images from 3093 eyes of 1574 Zambians with diabetes were prospectively recruited. Referable diabetic retinopathy was found in 697 (22·5%) eyes, vision-threatening diabetic retinopathy in 171 (5·5%) eyes, and diabetic macular oedema in 249 (8·1%) eyes. The AUC of the AI system for referable diabetic retinopathy was 0·973 (95% CI 0·969-0·978), with corresponding sensitivity of 92·25% (90·10-94·12) and specificity of 89·04% (87·85-90·28). Vision-threatening diabetic retinopathy sensitivity was 99·42% (99·15-99·68) and diabetic macular oedema sensitivity was 97·19% (96·61-97·77). The AI model and human graders showed similar outcomes in referable diabetic retinopathy prevalence detection and systemic risk factors associations. Both the AI model and human graders identified longer duration of diabetes, higher level of glycated haemoglobin, and increased systolic blood pressure as risk factors associated with referable diabetic retinopathy. INTERPRETATION: An AI system shows clinically acceptable performance in detecting referable diabetic retinopathy, vision-threatening diabetic retinopathy, and diabetic macular oedema in population-based diabetic retinopathy screening. This shows the potential application and adoption of such AI technology in an under-resourced African population to reduce the incidence of preventable blindness, even when the model is trained in a different population. FUNDING: National Medical Research Council Health Service Research Grant, Large Collaborative Grant, Ministry of Health, Singapore; the SingHealth Foundation; and the Tanoto Foundation.
Authors: Ashish Jith Sreejith Kumar; Rachel S Chong; Jonathan G Crowston; Jacqueline Chua; Inna Bujor; Rahat Husain; Eranga N Vithana; Michaël J A Girard; Daniel S W Ting; Ching-Yu Cheng; Tin Aung; Alina Popa-Cherecheanu; Leopold Schmetterer; Damon Wong Journal: JAMA Ophthalmol Date: 2022-10-01 Impact factor: 8.253
Authors: Marcel Schmude; Nahya Salim; Hila Azadzoy; Mustafa Bane; Elizabeth Millen; Lisa O'Donnell; Philipp Bode; Ewelina Türk; Ria Vaidya; Stephen Gilbert Journal: JMIR Res Protoc Date: 2022-06-07
Authors: J Peter Campbell; Praveer Singh; Travis K Redd; James M Brown; Parag K Shah; Prema Subramanian; Renu Rajan; Nita Valikodath; Emily Cole; Susan Ostmo; R V Paul Chan; Narendran Venkatapathy; Michael F Chiang; Jayashree Kalpathy-Cramer Journal: Pediatrics Date: 2021-03 Impact factor: 7.124
Authors: Fernando Korn Malerbi; Rafael Ernane Andrade; Paulo Henrique Morales; José Augusto Stuchi; Diego Lencione; Jean Vitor de Paulo; Mayana Pereira Carvalho; Fabrícia Silva Nunes; Roseanne Montargil Rocha; Daniel A Ferraz; Rubens Belfort Journal: J Diabetes Sci Technol Date: 2021-01-12