Zhong Liu1, Shaobin Zhong2,3, Qiang Liu1, Chenxi Xie1, Yunzhu Dai2, Chuan Peng2, Xin Chen4, Ruhai Zou5. 1. National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Shenzhen University, 1066 Xueyuan Road, Nanshan District, Shenzhen, 518055, Guangdong, People's Republic of China. 2. State Key Laboratory of Oncology in South China, Collaborative Innovation Center of Cancer Medicine, Department of Ultrasound, Sun Yat-sen University Cancer Center, 651 Dongfeng East Road, Guangzhou, 510060, Guangdong, People's Republic of China. 3. Department of Ultrasound, Dongguan People's Hospital, Dongguan, 523039, Guangdong, People's Republic of China. 4. National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Shenzhen University, 1066 Xueyuan Road, Nanshan District, Shenzhen, 518055, Guangdong, People's Republic of China. chenxin@szu.edu.cn. 5. State Key Laboratory of Oncology in South China, Collaborative Innovation Center of Cancer Medicine, Department of Ultrasound, Sun Yat-sen University Cancer Center, 651 Dongfeng East Road, Guangzhou, 510060, Guangdong, People's Republic of China. zourh@sysucc.org.cn.
Abstract
OBJECTIVE: To develop a deep learning-based method with information fusion of US images and RF signals for better classification of thyroid nodules (TNs). METHODS: One hundred sixty-three pairs of US images and RF signals of TNs from a cohort of adult patients were used for analysis. We developed an information fusion-based joint convolutional neural network (IF-JCNN) for the differential diagnosis of malignant and benign TNs. The IF-JCNN contains two branched CNNs for deep feature extraction: one for US images and the other one for RF signals. The extracted features are fused at the backend of IF-JCNN for TN classification. RESULTS: Across 5-fold cross-validation, the accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC) obtained by using the IF-JCNN with both US images and RF signals as inputs for TN classification were respectively 0.896 (95% CI 0.838-0.938), 0.885 (95% CI 0.804-0.941), 0.910 (95% CI 0.815-0.966), and 0.956 (95% CI 0.926-0.987), which were better than those obtained by using only US images: 0.822 (0.755-0.878; p = 0.0044), 0.792 (0.679-0.868, p = 0.0091), 0.866 (0.760-0.937, p = 0.197), and 0.901 (0.855-0.948, p = .0398), or RF signals: 0.767 (0.694-0.829, p < 0.001), 0.781 (0.685-0.859, p = 0.0037), 0.746 (0.625-0.845, p < 0.001), 0.845 (0.786-0.903, p < 0.001). CONCLUSIONS: The proposed IF-JCNN model filled the gap of just using US images in CNNs to characterize TNs, and it may serve as a promising tool for assisting the diagnosis of thyroid cancer. KEY POINTS: • Raw radiofrequency signals before ultrasound imaging of thyroid nodules provide useful information that is not carried by ultrasound images. • The information carried by raw radiofrequency signals and ultrasound images for thyroid nodules is complementary. • The performance of deep convolutional neural network for diagnosing thyroid nodules can be significantly improved by fusing US images and RF signals in the model as compared with just using US images.
OBJECTIVE: To develop a deep learning-based method with information fusion of US images and RF signals for better classification of thyroid nodules (TNs). METHODS: One hundred sixty-three pairs of US images and RF signals of TNs from a cohort of adult patients were used for analysis. We developed an information fusion-based joint convolutional neural network (IF-JCNN) for the differential diagnosis of malignant and benign TNs. The IF-JCNN contains two branched CNNs for deep feature extraction: one for US images and the other one for RF signals. The extracted features are fused at the backend of IF-JCNN for TN classification. RESULTS: Across 5-fold cross-validation, the accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC) obtained by using the IF-JCNN with both US images and RF signals as inputs for TN classification were respectively 0.896 (95% CI 0.838-0.938), 0.885 (95% CI 0.804-0.941), 0.910 (95% CI 0.815-0.966), and 0.956 (95% CI 0.926-0.987), which were better than those obtained by using only US images: 0.822 (0.755-0.878; p = 0.0044), 0.792 (0.679-0.868, p = 0.0091), 0.866 (0.760-0.937, p = 0.197), and 0.901 (0.855-0.948, p = .0398), or RF signals: 0.767 (0.694-0.829, p < 0.001), 0.781 (0.685-0.859, p = 0.0037), 0.746 (0.625-0.845, p < 0.001), 0.845 (0.786-0.903, p < 0.001). CONCLUSIONS: The proposed IF-JCNN model filled the gap of just using US images in CNNs to characterize TNs, and it may serve as a promising tool for assisting the diagnosis of thyroid cancer. KEY POINTS: • Raw radiofrequency signals before ultrasound imaging of thyroid nodules provide useful information that is not carried by ultrasound images. • The information carried by raw radiofrequency signals and ultrasound images for thyroid nodules is complementary. • The performance of deep convolutional neural network for diagnosing thyroid nodules can be significantly improved by fusing US images and RF signals in the model as compared with just using US images.
Authors: Eoin F Cleere; Matthew G Davey; Shane O'Neill; Mel Corbett; John P O'Donnell; Sean Hacking; Ivan J Keogh; Aoife J Lowery; Michael J Kerin Journal: Diagnostics (Basel) Date: 2022-03-24