Huanhuan Liu1, Yanhong Chen1, Yuzhen Zhang1, Lijun Wang1, Ran Luo1, Haoting Wu1, Chenqing Wu1, Huiling Zhang2, Weixiong Tan2, Hongkun Yin2, Dengbin Wang3. 1. Department of Radiology, Xinhua Hospital, Shanghai Jiao Tong University School of Medicine, No. 1665 Kongjiang Road, Shanghai, 200092, China. 2. Beijing Infervision Technology Co. Ltd., Beijing, 100025, China. 3. Department of Radiology, Xinhua Hospital, Shanghai Jiao Tong University School of Medicine, No. 1665 Kongjiang Road, Shanghai, 200092, China. wangdengbin@xinhuamed.com.cn.
Abstract
OBJECTIVES: To investigate the value of full-field digital mammography-based deep learning (DL) in predicting malignancy of Breast Imaging Reporting and Data System (BI-RADS) 4 microcalcifications. METHODS: A total of 384 patients with 414 pathologically confirmed microcalcifications (221 malignant and 193 benign) were randomly allocated into the training, validation, and testing datasets (272/71/71 lesions) in this retrospective study. A combined DL model was developed incorporating mammography and clinical variables. Model performance was evaluated by using areas under the receiver operating characteristic curve (AUC) and compared with the clinical model, stand-alone DL image model, and BI-RADS approach. The predictive performance for malignancy was also compared between the combined model and human readers (2 juniors and 2 seniors). RESULTS: The combined DL model demonstrated favorable AUC, sensitivity, and specificity of 0.910, 85.3%, and 91.9% in predicting BI-RADS 4 malignant microcalcifications in the testing dataset, which outperformed the clinical model, DL image model, and BI-RADS with AUCs of 0.799, 0.841, and 0.804, respectively. The combined model achieved non-inferior performance as senior radiologists (p = 0.860, p = 0.800) and outperformed junior radiologists (p = 0.155, p = 0.029). The diagnostic performance of two junior radiologists was improved after artificial intelligence assistance with AUCs increased to 0.854 and 0.901 from 0.816 (p = 0.556) and 0.773 (p = 0.046), while the interobserver agreement was improved with a kappa value increased to 0.843 from 0.331. CONCLUSIONS: The combined deep learning model can improve the malignancy prediction of BI-RADS 4 microcalcifications in screening mammography and assist junior radiologists to achieve better performance, which can facilitate clinical decision-making. KEY POINTS: • The combined deep learning model demonstrated high diagnostic power, sensitivity, and specificity for predicting malignant BI-RADS 4 mammographic microcalcifications. • The combined model achieved similar performance with senior breast radiologists, while it outperformed junior breast radiologists. • Deep learning could improve the diagnostic performance of junior radiologists and facilitate clinical decision-making.
RCT Entities:
OBJECTIVES: To investigate the value of full-field digital mammography-based deep learning (DL) in predicting malignancy of Breast Imaging Reporting and Data System (BI-RADS) 4 microcalcifications. METHODS: A total of 384 patients with 414 pathologically confirmed microcalcifications (221 malignant and 193 benign) were randomly allocated into the training, validation, and testing datasets (272/71/71 lesions) in this retrospective study. A combined DL model was developed incorporating mammography and clinical variables. Model performance was evaluated by using areas under the receiver operating characteristic curve (AUC) and compared with the clinical model, stand-alone DL image model, and BI-RADS approach. The predictive performance for malignancy was also compared between the combined model and human readers (2 juniors and 2 seniors). RESULTS: The combined DL model demonstrated favorable AUC, sensitivity, and specificity of 0.910, 85.3%, and 91.9% in predicting BI-RADS 4 malignant microcalcifications in the testing dataset, which outperformed the clinical model, DL image model, and BI-RADS with AUCs of 0.799, 0.841, and 0.804, respectively. The combined model achieved non-inferior performance as senior radiologists (p = 0.860, p = 0.800) and outperformed junior radiologists (p = 0.155, p = 0.029). The diagnostic performance of two junior radiologists was improved after artificial intelligence assistance with AUCs increased to 0.854 and 0.901 from 0.816 (p = 0.556) and 0.773 (p = 0.046), while the interobserver agreement was improved with a kappa value increased to 0.843 from 0.331. CONCLUSIONS: The combined deep learning model can improve the malignancy prediction of BI-RADS 4 microcalcifications in screening mammography and assist junior radiologists to achieve better performance, which can facilitate clinical decision-making. KEY POINTS: • The combined deep learning model demonstrated high diagnostic power, sensitivity, and specificity for predicting malignant BI-RADS 4 mammographic microcalcifications. • The combined model achieved similar performance with senior breast radiologists, while it outperformed junior breast radiologists. • Deep learning could improve the diagnostic performance of junior radiologists and facilitate clinical decision-making.