Hyo-Eun Kim1, Hak Hee Kim2, Boo-Kyung Han3, Ki Hwan Kim1, Kyunghwa Han4, Hyeonseob Nam1, Eun Hye Lee5, Eun-Kyung Kim6. 1. Lunit, Seoul, South Korea. 2. Department of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, South Korea. 3. Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, South Korea. 4. Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Severance Hospital, Yonsei University College of Medicine, Seoul, South Korea. 5. Department of Radiology, Soonchunhyang University Hospital Bucheon, Soonchunhyang University College of Medicine, Bucheon, South Korea. 6. Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Severance Hospital, Yonsei University College of Medicine, Seoul, South Korea. Electronic address: ekkim@yuhs.ac.
Abstract
BACKGROUND: Mammography is the current standard for breast cancer screening. This study aimed to develop an artificial intelligence (AI) algorithm for diagnosis of breast cancer in mammography, and explore whether it could benefit radiologists by improving accuracy of diagnosis. METHODS: In this retrospective study, an AI algorithm was developed and validated with 170 230 mammography examinations collected from five institutions in South Korea, the USA, and the UK, including 36 468 cancer positive confirmed by biopsy, 59 544 benign confirmed by biopsy (8827 mammograms) or follow-up imaging (50 717 mammograms), and 74 218 normal. For the multicentre, observer-blinded, reader study, 320 mammograms (160 cancer positive, 64 benign, 96 normal) were independently obtained from two institutions. 14 radiologists participated as readers and assessed each mammogram in terms of likelihood of malignancy (LOM), location of malignancy, and necessity to recall the patient, first without and then with assistance of the AI algorithm. The performance of AI and radiologists was evaluated in terms of LOM-based area under the receiver operating characteristic curve (AUROC) and recall-based sensitivity and specificity. FINDINGS: The AI standalone performance was AUROC 0·959 (95% CI 0·952-0·966) overall, and 0·970 (0·963-0·978) in the South Korea dataset, 0·953 (0·938-0·968) in the USA dataset, and 0·938 (0·918-0·958) in the UK dataset. In the reader study, the performance level of AI was 0·940 (0·915-0·965), significantly higher than that of the radiologists without AI assistance (0·810, 95% CI 0·770-0·850; p<0·0001). With the assistance of AI, radiologists' performance was improved to 0·881 (0·850-0·911; p<0·0001). AI was more sensitive to detect cancers with mass (53 [90%] vs 46 [78%] of 59 cancers detected; p=0·044) or distortion or asymmetry (18 [90%] vs ten [50%] of 20 cancers detected; p=0·023) than radiologists. AI was better in detection of T1 cancers (73 [91%] vs 59 [74%] of 80; p=0·0039) or node-negative cancers (104 [87%] vs 88 [74%] of 119; p=0·0025) than radiologists. INTERPRETATION: The AI algorithm developed with large-scale mammography data showed better diagnostic performance in breast cancer detection compared with radiologists. The significant improvement in radiologists' performance when aided by AI supports application of AI to mammograms as a diagnostic support tool. FUNDING: Lunit.
BACKGROUND: Mammography is the current standard for breast cancer screening. This study aimed to develop an artificial intelligence (AI) algorithm for diagnosis of breast cancer in mammography, and explore whether it could benefit radiologists by improving accuracy of diagnosis. METHODS: In this retrospective study, an AI algorithm was developed and validated with 170 230 mammography examinations collected from five institutions in South Korea, the USA, and the UK, including 36 468 cancer positive confirmed by biopsy, 59 544 benign confirmed by biopsy (8827 mammograms) or follow-up imaging (50 717 mammograms), and 74 218 normal. For the multicentre, observer-blinded, reader study, 320 mammograms (160 cancer positive, 64 benign, 96 normal) were independently obtained from two institutions. 14 radiologists participated as readers and assessed each mammogram in terms of likelihood of malignancy (LOM), location of malignancy, and necessity to recall the patient, first without and then with assistance of the AI algorithm. The performance of AI and radiologists was evaluated in terms of LOM-based area under the receiver operating characteristic curve (AUROC) and recall-based sensitivity and specificity. FINDINGS: The AI standalone performance was AUROC 0·959 (95% CI 0·952-0·966) overall, and 0·970 (0·963-0·978) in the South Korea dataset, 0·953 (0·938-0·968) in the USA dataset, and 0·938 (0·918-0·958) in the UK dataset. In the reader study, the performance level of AI was 0·940 (0·915-0·965), significantly higher than that of the radiologists without AI assistance (0·810, 95% CI 0·770-0·850; p<0·0001). With the assistance of AI, radiologists' performance was improved to 0·881 (0·850-0·911; p<0·0001). AI was more sensitive to detect cancers with mass (53 [90%] vs 46 [78%] of 59 cancers detected; p=0·044) or distortion or asymmetry (18 [90%] vs ten [50%] of 20 cancers detected; p=0·023) than radiologists. AI was better in detection of T1 cancers (73 [91%] vs 59 [74%] of 80; p=0·0039) or node-negative cancers (104 [87%] vs 88 [74%] of 119; p=0·0025) than radiologists. INTERPRETATION: The AI algorithm developed with large-scale mammography data showed better diagnostic performance in breast cancer detection compared with radiologists. The significant improvement in radiologists' performance when aided by AI supports application of AI to mammograms as a diagnostic support tool. FUNDING: Lunit.
Authors: Mark D Halling-Brown; Lucy M Warren; Dominic Ward; Emma Lewis; Alistair Mackenzie; Matthew G Wallis; Louise S Wilkinson; Rosalind M Given-Wilson; Rita McAvinchey; Kenneth C Young Journal: Radiol Artif Intell Date: 2020-11-25
Authors: Sarah E Hickman; Ramona Woitek; Elizabeth Phuong Vi Le; Yu Ri Im; Carina Mouritsen Luxhøj; Angelica I Aviles-Rivero; Gabrielle C Baxter; James W MacKay; Fiona J Gilbert Journal: Radiology Date: 2021-10-19 Impact factor: 11.105
Authors: Serena Pacilè; January Lopez; Pauline Chone; Thomas Bertinotti; Jean Marie Grouin; Pierre Fillard Journal: Radiol Artif Intell Date: 2020-11-04
Authors: Anna W Anderson; M Luke Marinovich; Nehmat Houssami; Kathryn P Lowry; Joann G Elmore; Diana S M Buist; Solveig Hofvind; Christoph I Lee Journal: J Am Coll Radiol Date: 2022-01-20 Impact factor: 5.532
Authors: John S Chorba; Avi M Shapiro; Le Le; John Maidens; John Prince; Steve Pham; Mia M Kanzawa; Daniel N Barbosa; Caroline Currie; Catherine Brooks; Brent E White; Anna Huskin; Jason Paek; Jack Geocaris; Dinatu Elnathan; Ria Ronquillo; Roy Kim; Zenith H Alam; Vaikom S Mahadevan; Sophie G Fuller; Grant W Stalker; Sara A Bravo; Dina Jean; John J Lee; Medeona Gjergjindreaj; Christos G Mihos; Steven T Forman; Subramaniam Venkatraman; Patrick M McCarthy; James D Thomas Journal: J Am Heart Assoc Date: 2021-04-26 Impact factor: 5.501