Jian Xiong1, Fei Li2, Diping Song3, Guangxian Tang4, Junjun He5, Kai Gao2, Hengli Zhang4, Weijing Cheng2, Yunhe Song2, Fengbin Lin2, Kun Hu2, Peiyuan Wang2, Ji-Peng Olivia Li6, Tin Aung7, Yu Qiao8, Xiulan Zhang9, Daniel Ting10. 1. State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, People's Republic of China; Department of Ophthalmology, The Second Affiliated Hospital of Nanchang University, Nanchang, People's Republic of China. 2. State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, People's Republic of China. 3. ShenZhen Key Lab of Computer Vision and Pattern Recognition, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, People's Republic of China; University of Chinese Academy of Sciences, Beijing, People's Republic of China. 4. The First Hospital of Shijiazhuang City, Shijiazhuang, People's Republic of China. 5. ShenZhen Key Lab of Computer Vision and Pattern Recognition, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, People's Republic of China; Shanghai AI Laboratory, Shanghai, People's Republic of China; Shanghai Jiao Tong University, Shanghai, People's Republic of China. 6. Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom. 7. Singapore Eye Research Institute, Singapore National Eye Center, and Duke-NUS Medical School, Singapore, Republic of Singapore; Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Republic of Singapore. Electronic address: aung.tin@singhealth.com.sg. 8. ShenZhen Key Lab of Computer Vision and Pattern Recognition, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, People's Republic of China; Shanghai AI Laboratory, Shanghai, People's Republic of China. Electronic address: yu.qiao@siat.ac.cn. 9. State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, People's Republic of China. Electronic address: zhangxl2@mail.sysu.edu.cn. 10. Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Republic of Singapore.
Abstract
PURPOSE: To develop and validate a multimodal artificial intelligence algorithm, FusionNet, using the pattern deviation probability plots from visual field (VF) reports and circular peripapillary OCT scans to detect glaucomatous optic neuropathy (GON). DESIGN: Cross-sectional study. SUBJECTS: Two thousand four hundred sixty-three pairs of VF and OCT images from 1083 patients. METHODS: FusionNet based on bimodal input of VF and OCT paired data was developed to detect GON. Visual field data were collected using the Humphrey Field Analyzer (HFA). OCT images were collected from 3 types of devices (DRI-OCT, Cirrus OCT, and Spectralis). Two thousand four hundred sixty-three pairs of VF and OCT images were divided into 4 datasets: 1567 for training (HFA and DRI-OCT), 441 for primary validation (HFA and DRI-OCT), 255 for the internal test (HFA and Cirrus OCT), and 200 for the external test set (HFA and Spectralis). GON was defined as retinal nerve fiber layer thinning with corresponding VF defects. MAIN OUTCOME MEASURES: Diagnostic performance of FusionNet compared with that of VFNet (with VF data as input) and OCTNet (with OCT data as input). RESULTS: FusionNet achieved an area under the receiver operating characteristic curve (AUC) of 0.950 (0.931-0.968) and outperformed VFNet (AUC, 0.868 [95% confidence interval (CI), 0.834-0.902]), OCTNet (AUC, 0.809 [95% CI, 0.768-0.850]), and 2 glaucoma specialists (glaucoma specialist 1: AUC, 0.882 [95% CI, 0.847-0.917]; glaucoma specialist 2: AUC, 0.883 [95% CI, 0.849-0.918]) in the primary validation set. In the internal and external test sets, the performances of FusionNet were also superior to VFNet and OCTNet (FusionNet vs VFNet vs OCTNet: internal test set 0.917 vs 0.854 vs 0.811; external test set 0.873 vs 0.772 vs 0.785). No significant difference was found between the 2 glaucoma specialists and FusionNet in the internal and external test sets, except for glaucoma specialist 2 (AUC, 0.858 [95% CI, 0.805-0.912]) in the internal test set. CONCLUSIONS: FusionNet, developed using paired VF and OCT data, demonstrated superior performance to both VFNet and OCTNet in detecting GON, suggesting that multimodal machine learning models are valuable in detecting GON.
PURPOSE: To develop and validate a multimodal artificial intelligence algorithm, FusionNet, using the pattern deviation probability plots from visual field (VF) reports and circular peripapillary OCT scans to detect glaucomatous optic neuropathy (GON). DESIGN: Cross-sectional study. SUBJECTS: Two thousand four hundred sixty-three pairs of VF and OCT images from 1083 patients. METHODS: FusionNet based on bimodal input of VF and OCT paired data was developed to detect GON. Visual field data were collected using the Humphrey Field Analyzer (HFA). OCT images were collected from 3 types of devices (DRI-OCT, Cirrus OCT, and Spectralis). Two thousand four hundred sixty-three pairs of VF and OCT images were divided into 4 datasets: 1567 for training (HFA and DRI-OCT), 441 for primary validation (HFA and DRI-OCT), 255 for the internal test (HFA and Cirrus OCT), and 200 for the external test set (HFA and Spectralis). GON was defined as retinal nerve fiber layer thinning with corresponding VF defects. MAIN OUTCOME MEASURES: Diagnostic performance of FusionNet compared with that of VFNet (with VF data as input) and OCTNet (with OCT data as input). RESULTS: FusionNet achieved an area under the receiver operating characteristic curve (AUC) of 0.950 (0.931-0.968) and outperformed VFNet (AUC, 0.868 [95% confidence interval (CI), 0.834-0.902]), OCTNet (AUC, 0.809 [95% CI, 0.768-0.850]), and 2 glaucoma specialists (glaucoma specialist 1: AUC, 0.882 [95% CI, 0.847-0.917]; glaucoma specialist 2: AUC, 0.883 [95% CI, 0.849-0.918]) in the primary validation set. In the internal and external test sets, the performances of FusionNet were also superior to VFNet and OCTNet (FusionNet vs VFNet vs OCTNet: internal test set 0.917 vs 0.854 vs 0.811; external test set 0.873 vs 0.772 vs 0.785). No significant difference was found between the 2 glaucoma specialists and FusionNet in the internal and external test sets, except for glaucoma specialist 2 (AUC, 0.858 [95% CI, 0.805-0.912]) in the internal test set. CONCLUSIONS: FusionNet, developed using paired VF and OCT data, demonstrated superior performance to both VFNet and OCTNet in detecting GON, suggesting that multimodal machine learning models are valuable in detecting GON.
Authors: Joel S Schuman; Maria De Los Angeles Ramos Cadena; Rebecca McGee; Lama A Al-Aswad; Felipe A Medeiros Journal: Ophthalmol Glaucoma Date: 2021-12-22