Sohee Park1, Sang Min Lee2, Kyung Hee Lee3, Kyu-Hwan Jung4, Woong Bae4, Jooae Choe1, Joon Beom Seo1. 1. Department of Radiology, University of Ulsan College of Medicine, Asan Medical Center, 88 Olympic-ro 43 Gil, Songpa-gu, Seoul, 138-736, South Korea. 2. Department of Radiology, University of Ulsan College of Medicine, Asan Medical Center, 88 Olympic-ro 43 Gil, Songpa-gu, Seoul, 138-736, South Korea. sangmin.lee.md@gmail.com. 3. Department of Radiology, Seoul National University Bundang Hospital, Seoul National University College of Medicine, 300 Gumi-dong, Bundang-gu, Seongnam-si, Gyeonggi-do, 13620, South Korea. kyung8404@gmail.com. 4. VUNO Inc., 736-8 Banpo-dong, Seocho-gu, Seoul, South Korea.
Abstract
OBJECTIVE: To investigate the feasibility of a deep learning-based detection (DLD) system for multiclass lesions on chest radiograph, in comparison with observers. METHODS: A total of 15,809 chest radiographs were collected from two tertiary hospitals (7204 normal and 8605 abnormal with nodule/mass, interstitial opacity, pleural effusion, or pneumothorax). Except for the test set (100 normal and 100 abnormal (nodule/mass, 70; interstitial opacity, 10; pleural effusion, 10; pneumothorax, 10)), radiographs were used to develop a DLD system for detecting multiclass lesions. The diagnostic performance of the developed model and that of nine observers with varying experiences were evaluated and compared using area under the receiver operating characteristic curve (AUROC), on a per-image basis, and jackknife alternative free-response receiver operating characteristic figure of merit (FOM) on a per-lesion basis. The false-positive fraction was also calculated. RESULTS: Compared with the group-averaged observations, the DLD system demonstrated significantly higher performances on image-wise normal/abnormal classification and lesion-wise detection with pattern classification (AUROC, 0.985 vs. 0.958; p = 0.001; FOM, 0.962 vs. 0.886; p < 0.001). In lesion-wise detection, the DLD system outperformed all nine observers. In the subgroup analysis, the DLD system exhibited consistently better performance for both nodule/mass (FOM, 0.913 vs. 0.847; p < 0.001) and the other three abnormal classes (FOM, 0.995 vs. 0.843; p < 0.001). The false-positive fraction of all abnormalities was 0.11 for the DLD system and 0.19 for the observers. CONCLUSIONS: The DLD system showed the potential for detection of lesions and pattern classification on chest radiographs, performing normal/abnormal classifications and achieving high diagnostic performance. KEY POINTS: • The DLD system was feasible for detection with pattern classification of multiclass lesions on chest radiograph. • The DLD system had high performance of image-wise classification as normal or abnormal chest radiographs (AUROC, 0.985) and showed especially high specificity (99.0%). • In lesion-wise detection of multiclass lesions, the DLD system outperformed all 9 observers (FOM, 0.962 vs. 0.886; p < 0.001).
OBJECTIVE: To investigate the feasibility of a deep learning-based detection (DLD) system for multiclass lesions on chest radiograph, in comparison with observers. METHODS: A total of 15,809 chest radiographs were collected from two tertiary hospitals (7204 normal and 8605 abnormal with nodule/mass, interstitial opacity, pleural effusion, or pneumothorax). Except for the test set (100 normal and 100 abnormal (nodule/mass, 70; interstitial opacity, 10; pleural effusion, 10; pneumothorax, 10)), radiographs were used to develop a DLD system for detecting multiclass lesions. The diagnostic performance of the developed model and that of nine observers with varying experiences were evaluated and compared using area under the receiver operating characteristic curve (AUROC), on a per-image basis, and jackknife alternative free-response receiver operating characteristic figure of merit (FOM) on a per-lesion basis. The false-positive fraction was also calculated. RESULTS: Compared with the group-averaged observations, the DLD system demonstrated significantly higher performances on image-wise normal/abnormal classification and lesion-wise detection with pattern classification (AUROC, 0.985 vs. 0.958; p = 0.001; FOM, 0.962 vs. 0.886; p < 0.001). In lesion-wise detection, the DLD system outperformed all nine observers. In the subgroup analysis, the DLD system exhibited consistently better performance for both nodule/mass (FOM, 0.913 vs. 0.847; p < 0.001) and the other three abnormal classes (FOM, 0.995 vs. 0.843; p < 0.001). The false-positive fraction of all abnormalities was 0.11 for the DLD system and 0.19 for the observers. CONCLUSIONS: The DLD system showed the potential for detection of lesions and pattern classification on chest radiographs, performing normal/abnormal classifications and achieving high diagnostic performance. KEY POINTS: • The DLD system was feasible for detection with pattern classification of multiclass lesions on chest radiograph. • The DLD system had high performance of image-wise classification as normal or abnormal chest radiographs (AUROC, 0.985) and showed especially high specificity (99.0%). • In lesion-wise detection of multiclass lesions, the DLD system outperformed all 9 observers (FOM, 0.962 vs. 0.886; p < 0.001).
Entities:
Keywords:
Automated pattern recognition; Classification; Deep learning; Thoracic radiography
Authors: Steven Schalekamp; Bram van Ginneken; Emmeline Koedam; Miranda M Snoeren; Audrey M Tiehuis; Rianne Wittenberg; Nico Karssemeijer; Cornelia M Schaefer-Prokop Journal: Radiology Date: 2014-03-12 Impact factor: 11.105
Authors: Eui Jin Hwang; Sunggyun Park; Kwang-Nam Jin; Jung Im Kim; So Young Choi; Jong Hyuk Lee; Jin Mo Goo; Jaehong Aum; Jae-Joon Yim; Julien G Cohen; Gilbert R Ferretti; Chang Min Park Journal: JAMA Netw Open Date: 2019-03-01
Authors: Ali Pirasteh; E Aleks Sorra; Hector Marquez; Robert C Sibley; Julia R Fielding; Abhinav Vij; Nicole E Rich; Ana Arroyo; Adam C Yopp; Gaurav Khatri; Amit G Singal; Takeshi Yokoo Journal: Abdom Radiol (NY) Date: 2021-03-23
Authors: Alan A Peters; Amanda Decasper; Jaro Munz; Jeremias Klaus; Laura I Loebelenz; Maximilian Korbinian Michael Hoffner; Cynthia Hourscht; Johannes T Heverhagen; Andreas Christe; Lukas Ebner Journal: J Thorac Dis Date: 2021-05 Impact factor: 3.005
Authors: Ravi Aggarwal; Viknesh Sounderajah; Guy Martin; Daniel S W Ting; Alan Karthikesalingam; Dominic King; Hutan Ashrafian; Ara Darzi Journal: NPJ Digit Med Date: 2021-04-07
Authors: Jong Seok Ahn; Shadi Ebrahimian; Shaunagh McDermott; Sanghyup Lee; Laura Naccarato; John F Di Capua; Markus Y Wu; Eric W Zhang; Victorine Muse; Benjamin Miller; Farid Sabzalipour; Bernardo C Bizzo; Keith J Dreyer; Parisa Kaviani; Subba R Digumarthy; Mannudeep K Kalra Journal: JAMA Netw Open Date: 2022-08-01