Tomohiro Wataya1, Masahiro Yanagawa2, Mitsuko Tsubamoto3, Tomoharu Sato4, Daiki Nishigaki1, Kosuke Kita1, Kazuki Yamagata1, Yuki Suzuki1, Akinori Hata2, Shoji Kido5, Noriyuki Tomiyama2. 1. Department of Artificial Intelligence Diagnostic Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka, 565-0871, Japan. 2. Department of Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka, 565-0871, Japan. 3. Department of Radiology, Nishinomiya Municipal Central Hospital, 8-24, Hayashida Cho, Nishinomiya, Hyogo, 663-8014, Japan. 4. Department of Biostatistics & Data Science, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka, 565-0871, Japan. 5. Department of Artificial Intelligence Diagnostic Radiology, Osaka University Graduate School of Medicine, 2-2, Yamadaoka, Suita, Osaka, 565-0871, Japan. kido@radiol.med.osaka-u.ac.jp.
Abstract
OBJECTIVES: To compare the performance of radiologists in characterizing and diagnosing pulmonary nodules/masses with and without deep learning (DL)-based computer-aided diagnosis (CAD). METHODS: We studied a total of 101 nodules/masses detected on CT performed between January and March 2018 at Osaka University Hospital (malignancy: 55 cases). SYNAPSE SAI Viewer V1.4 was used to analyze the nodules/masses. In total, 15 independent radiologists were grouped (n = 5 each) according to their experience: L (< 3 years), M (3-5 years), and H (> 5 years). The likelihoods of 15 characteristics, such as cavitation and calcification, and the diagnosis (malignancy) were evaluated by each radiologist with and without CAD, and the assessment time was recorded. The AUCs compared with the reference standard set by two board-certified chest radiologists were analyzed following the multi-reader multi-case method. Furthermore, interobserver agreement was compared using intraclass correlation coefficients (ICCs). RESULTS: The AUCs for ill-defined boundary, irregular margin, irregular shape, calcification, pleural contact, and malignancy in all 15 radiologists, irregular margin and irregular shape in L and ill-defined boundary and irregular margin in M improved significantly (p < 0.05); no significant improvements were found in H. L showed the greatest increase in the AUC for malignancy (not significant). The ICCs improved in all groups and for nearly all items. The median assessment time was not prolonged by CAD. CONCLUSIONS: DL-based CAD helps radiologists, particularly those with < 5 years of experience, to accurately characterize and diagnose pulmonary nodules/masses, and improves the reproducibility of findings among radiologists. KEY POINTS: • Deep learning-based computer-aided diagnosis improves the accuracy of characterizing nodules/masses and diagnosing malignancy, particularly by radiologists with < 5 years of experience. • Computer-aided diagnosis increases not only the accuracy but also the reproducibility of the findings across radiologists.
OBJECTIVES: To compare the performance of radiologists in characterizing and diagnosing pulmonary nodules/masses with and without deep learning (DL)-based computer-aided diagnosis (CAD). METHODS: We studied a total of 101 nodules/masses detected on CT performed between January and March 2018 at Osaka University Hospital (malignancy: 55 cases). SYNAPSE SAI Viewer V1.4 was used to analyze the nodules/masses. In total, 15 independent radiologists were grouped (n = 5 each) according to their experience: L (< 3 years), M (3-5 years), and H (> 5 years). The likelihoods of 15 characteristics, such as cavitation and calcification, and the diagnosis (malignancy) were evaluated by each radiologist with and without CAD, and the assessment time was recorded. The AUCs compared with the reference standard set by two board-certified chest radiologists were analyzed following the multi-reader multi-case method. Furthermore, interobserver agreement was compared using intraclass correlation coefficients (ICCs). RESULTS: The AUCs for ill-defined boundary, irregular margin, irregular shape, calcification, pleural contact, and malignancy in all 15 radiologists, irregular margin and irregular shape in L and ill-defined boundary and irregular margin in M improved significantly (p < 0.05); no significant improvements were found in H. L showed the greatest increase in the AUC for malignancy (not significant). The ICCs improved in all groups and for nearly all items. The median assessment time was not prolonged by CAD. CONCLUSIONS: DL-based CAD helps radiologists, particularly those with < 5 years of experience, to accurately characterize and diagnose pulmonary nodules/masses, and improves the reproducibility of findings among radiologists. KEY POINTS: • Deep learning-based computer-aided diagnosis improves the accuracy of characterizing nodules/masses and diagnosing malignancy, particularly by radiologists with < 5 years of experience. • Computer-aided diagnosis increases not only the accuracy but also the reproducibility of the findings across radiologists.