Chunyan Yi1,2,3, Yuxing Tang4, Rushan Ouyang1,2,3, Yanbo Zhang4, Zhenjie Cao4, Zhicheng Yang4, Shibin Wu5, Mei Han4, Jing Xiao5, Peng Chang6, Jie Ma7,8,9. 1. Department of Radiology, Shenzhen People's Hospital, Shenzhen, 518020, Guangdong, China. 2. The Second Clinical Medical College, Jinan University, Shenzhen, 518020, Guangdong, China. 3. The First Affiliated Hospital, Southern University of Science and Technology, Shenzhen, 518020, Guangdong, China. 4. PAII Inc, Palo Alto, CA, 94306, USA. 5. Ping An Technology, Shenzhen, 518029, Guangdong, China. 6. PAII Inc, Palo Alto, CA, 94306, USA. pengchang@gmail.com. 7. Department of Radiology, Shenzhen People's Hospital, Shenzhen, 518020, Guangdong, China. majie688@hotmail.com. 8. The Second Clinical Medical College, Jinan University, Shenzhen, 518020, Guangdong, China. majie688@hotmail.com. 9. The First Affiliated Hospital, Southern University of Science and Technology, Shenzhen, 518020, Guangdong, China. majie688@hotmail.com.
Abstract
OBJECTIVES: To investigate the value of an artificial intelligence (AI) system in assisting radiologists to improve the assessment accuracy of BI-RADS 0 cases in mammograms. METHODS: We included 34,654 consecutive digital mammography studies, collected between January 2011 and January 2019, among which, 1088 cases from 1010 unique patients with initial BI-RADS 0 assessment who were recalled during 2 years of follow-up were used in this study. Two mid-level radiologists retrospectively re-assessed these BI-RADS 0 cases with the assistance of an AI system developed by us previously. In addition, four entry-level radiologists were split into two groups to cross-read 80 cases with and without the AI. Diagnostic performance was evaluated using the follow-up diagnosis or biopsy results as the reference standard. RESULTS: Of the 1088 cases, 626 were actually normal (BI-RADS 1 and no recall required). Assisted by the AI system, 351 (56%) and 362 (58%) normal cases were correctly identified by the two mid-level radiologists hence can be avoided for unnecessary follow-ups. However, they would have missed 12 (10 invasive cancers and 2 ductal carcinoma in situ cancers) and 6 (invasive cancers) malignant lesions respectively as a result. These missed lesions were not highly malignant tumors. The inter-rater reliability of entry-level radiologists increased from 0.20 to 0.30 (p < 0.005) by introducing the AI. CONCLUSION: The AI system can effectively assist mid-level radiologists in reducing unnecessary follow-ups of mammographically indeterminate breast lesions and reducing the benign biopsy rate without missing highly malignant tumors. KEY POINTS: • The artificial intelligence system could assist mid-level radiologists in effectively reducing unnecessary BI-RADS 0 mammogram recalls and the benign biopsy rate without missing highly malignant tumors. • The artificial intelligence system was capable of detecting low suspicion lesions from heterogeneously and extremely dense breasts that radiologists tended to miss. • The use of an artificial intelligence system may improve the inter-rater reliability and sensitivity, and reduce the reading time of entry-level radiologists in assessing potential lesions in BI-RADS 0 mammograms.
OBJECTIVES: To investigate the value of an artificial intelligence (AI) system in assisting radiologists to improve the assessment accuracy of BI-RADS 0 cases in mammograms. METHODS: We included 34,654 consecutive digital mammography studies, collected between January 2011 and January 2019, among which, 1088 cases from 1010 unique patients with initial BI-RADS 0 assessment who were recalled during 2 years of follow-up were used in this study. Two mid-level radiologists retrospectively re-assessed these BI-RADS 0 cases with the assistance of an AI system developed by us previously. In addition, four entry-level radiologists were split into two groups to cross-read 80 cases with and without the AI. Diagnostic performance was evaluated using the follow-up diagnosis or biopsy results as the reference standard. RESULTS: Of the 1088 cases, 626 were actually normal (BI-RADS 1 and no recall required). Assisted by the AI system, 351 (56%) and 362 (58%) normal cases were correctly identified by the two mid-level radiologists hence can be avoided for unnecessary follow-ups. However, they would have missed 12 (10 invasive cancers and 2 ductal carcinoma in situ cancers) and 6 (invasive cancers) malignant lesions respectively as a result. These missed lesions were not highly malignant tumors. The inter-rater reliability of entry-level radiologists increased from 0.20 to 0.30 (p < 0.005) by introducing the AI. CONCLUSION: The AI system can effectively assist mid-level radiologists in reducing unnecessary follow-ups of mammographically indeterminate breast lesions and reducing the benign biopsy rate without missing highly malignant tumors. KEY POINTS: • The artificial intelligence system could assist mid-level radiologists in effectively reducing unnecessary BI-RADS 0 mammogram recalls and the benign biopsy rate without missing highly malignant tumors. • The artificial intelligence system was capable of detecting low suspicion lesions from heterogeneously and extremely dense breasts that radiologists tended to miss. • The use of an artificial intelligence system may improve the inter-rater reliability and sensitivity, and reduce the reading time of entry-level radiologists in assessing potential lesions in BI-RADS 0 mammograms.
Authors: E G Klompenhouwer; R J P Weber; A C Voogd; G J den Heeten; L J A Strobbe; M J M Broeders; V C G Tjan-Heijnen; L E M Duijm Journal: Breast Date: 2015-10 Impact factor: 4.380
Authors: Geert Litjens; Thijs Kooi; Babak Ehteshami Bejnordi; Arnaud Arindra Adiyoso Setio; Francesco Ciompi; Mohsen Ghafoorian; Jeroen A W M van der Laak; Bram van Ginneken; Clara I Sánchez Journal: Med Image Anal Date: 2017-07-26 Impact factor: 8.545
Authors: Marilyn M Schapira; William E Barlow; Emily F Conant; Brian L Sprague; Anna N A Tosteson; Jennifer S Haas; Tracy Onega; Elisabeth F Beaber; Martha Goodrich; Anne Marie McCarthy; Sally D Herschorn; Celette Sugg Skinner; Tory O Harrington; Berta Geller Journal: Acad Radiol Date: 2018-02-09 Impact factor: 3.173
Authors: Derek L Nguyen; Eniola Oluyemi; Kelly S Myers; Susan C Harvey; Lisa A Mullen; Emily B Ambinder Journal: J Am Coll Radiol Date: 2020-04-28 Impact factor: 5.532
Authors: Daniel Shu Wei Ting; Carol Yim-Lui Cheung; Gilbert Lim; Gavin Siew Wei Tan; Nguyen D Quang; Alfred Gan; Haslina Hamzah; Renata Garcia-Franco; Ian Yew San Yeo; Shu Yen Lee; Edmund Yick Mun Wong; Charumathi Sabanayagam; Mani Baskaran; Farah Ibrahim; Ngiap Chuan Tan; Eric A Finkelstein; Ecosse L Lamoureux; Ian Y Wong; Neil M Bressler; Sobha Sivaprasad; Rohit Varma; Jost B Jonas; Ming Guang He; Ching-Yu Cheng; Gemmy Chui Ming Cheung; Tin Aung; Wynne Hsu; Mong Li Lee; Tien Yin Wong Journal: JAMA Date: 2017-12-12 Impact factor: 56.272
Authors: Andre Esteva; Brett Kuprel; Roberto A Novoa; Justin Ko; Susan M Swetter; Helen M Blau; Sebastian Thrun Journal: Nature Date: 2017-01-25 Impact factor: 49.962
Authors: Derek L Nguyen; Susan C Harvey; Eniola T Oluyemi; Kelly S Myers; Lisa A Mullen; Emily B Ambinder Journal: J Am Coll Radiol Date: 2020-07-30 Impact factor: 5.532