Cam-Hao Hua1, Thien Huynh-The2, Kiyoung Kim3, Seung-Young Yu4, Thuong Le-Tien5, Gwang Hoon Park6, Jaehun Bang7, Wajahat Ali Khan8, Sung-Ho Bae9, Sungyoung Lee10. 1. Department of Computer Science and Engineering, Kyung Hee University, Gyeonggi-do 17104, South Korea. Electronic address: hao.hua@oslab.khu.ac.kr. 2. ICT Convergence Research Center, Kumoh National Institute of Technology, Gumi, South Korea. Electronic address: thienht@kumoh.ac.kr. 3. Department of Ophthalmology, Kyung Hee University Medical Center, Kyung Hee University, Seoul 02447, South Korea. Electronic address: pourma@naver.com. 4. Department of Ophthalmology, Kyung Hee University Medical Center, Kyung Hee University, Seoul 02447, South Korea. Electronic address: syyu@khu.ac.kr. 5. Department of Electrical and Electronics Engineering, Ho Chi Minh City University of Technology, Ho Chi Minh City, 700000, Vietnam. Electronic address: thuongle@hcmut.edu.vn. 6. Department of Computer Science and Engineering, Kyung Hee University, Gyeonggi-do 17104, South Korea. Electronic address: ghpark@khu.ac.kr. 7. Department of Computer Science and Engineering, Kyung Hee University, Gyeonggi-do 17104, South Korea. Electronic address: jhb@oslab.khu.ac.kr. 8. Department of Computer Science and Engineering, Kyung Hee University, Gyeonggi-do 17104, South Korea. Electronic address: wajahat.alikhan@oslab.khu.ac.kr. 9. Department of Computer Science and Engineering, Kyung Hee University, Gyeonggi-do 17104, South Korea. Electronic address: shbae@khu.ac.kr. 10. Department of Computer Science and Engineering, Kyung Hee University, Gyeonggi-do 17104, South Korea. Electronic address: sylee@oslab.khu.ac.kr.
Abstract
BACKGROUND: Diabetic Retinopathy (DR) is considered a pathology of retinal vascular complications, which stays in the top causes of vision impairment and blindness. Therefore, precisely inspecting its progression enables the ophthalmologists to set up appropriate next-visit schedule and cost-effective treatment plans. In the literature, existing work only makes use of numerical attributes in Electronic Medical Records (EMR) for acquiring such kind of DR-oriented knowledge through conventional machine learning techniques, which require an exhaustive job of engineering most impactful risk factors. OBJECTIVE: In this paper, an approach of deep bimodal learning is introduced to leverage the performance of DR risk progression identification. METHODS: In particular, we further involve valuable clinical information of fundus photography in addition to the aforementioned systemic attributes. Accordingly, a Trilogy of Skip-connection Deep Networks, namely Tri-SDN, is proposed to exhaustively exploit underlying relationships between the baseline and follow-up information of the fundus images and EMR-based attributes. Besides that, we adopt Skip-Connection Blocks as basis components of the Tri-SDN for making the end-to-end flow of signals more efficient during feedforward and backpropagation processes. RESULTS: Through a 10-fold cross validation strategy on a private dataset of 96 diabetic mellitus patients, the proposed method attains superior performance over the conventional EMR-modality learning approach in terms of Accuracy (90.6%), Sensitivity (96.5%), Precision (88.7%), Specificity (82.1%), and Area Under Receiver Operating Characteristics (88.8%). CONCLUSIONS: The experimental results show that the proposed Tri-SDN can combine features of different modalities (i.e., fundus images and EMR-based numerical risk factors) smoothly and effectively during training and testing processes, respectively. As a consequence, with impressive performance of DR risk progression recognition, the proposed approach is able to help the ophthalmologists properly decide follow-up schedule and subsequent treatment plans.
BACKGROUND:Diabetic Retinopathy (DR) is considered a pathology of retinal vascular complications, which stays in the top causes of vision impairment and blindness. Therefore, precisely inspecting its progression enables the ophthalmologists to set up appropriate next-visit schedule and cost-effective treatment plans. In the literature, existing work only makes use of numerical attributes in Electronic Medical Records (EMR) for acquiring such kind of DR-oriented knowledge through conventional machine learning techniques, which require an exhaustive job of engineering most impactful risk factors. OBJECTIVE: In this paper, an approach of deep bimodal learning is introduced to leverage the performance of DR risk progression identification. METHODS: In particular, we further involve valuable clinical information of fundus photography in addition to the aforementioned systemic attributes. Accordingly, a Trilogy of Skip-connection Deep Networks, namely Tri-SDN, is proposed to exhaustively exploit underlying relationships between the baseline and follow-up information of the fundus images and EMR-based attributes. Besides that, we adopt Skip-Connection Blocks as basis components of the Tri-SDN for making the end-to-end flow of signals more efficient during feedforward and backpropagation processes. RESULTS: Through a 10-fold cross validation strategy on a private dataset of 96 diabetic mellituspatients, the proposed method attains superior performance over the conventional EMR-modality learning approach in terms of Accuracy (90.6%), Sensitivity (96.5%), Precision (88.7%), Specificity (82.1%), and Area Under Receiver Operating Characteristics (88.8%). CONCLUSIONS: The experimental results show that the proposed Tri-SDN can combine features of different modalities (i.e., fundus images and EMR-based numerical risk factors) smoothly and effectively during training and testing processes, respectively. As a consequence, with impressive performance of DR risk progression recognition, the proposed approach is able to help the ophthalmologists properly decide follow-up schedule and subsequent treatment plans.