Adam Yala1,2, Peter G Mikhael1,2, Fredrik Strand3,4, Gigin Lin5, Siddharth Satuluru6, Thomas Kim7, Imon Banerjee8, Judy Gichoya9, Hari Trivedi9, Constance D Lehman10, Kevin Hughes11, David J Sheedy12, Lisa M Matthis12, Bipin Karunakaran12, Karen E Hegarty13, Silvia Sabino14, Thiago B Silva14, Maria C Evangelista14, Renato F Caron14, Bruno Souza14, Edmundo C Mauad14, Tal Patalon15, Sharon Handelman-Gotlib15, Michal Guindy16, Regina Barzilay1,2. 1. Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA. 2. Jameel Clinic, Massachusetts Institute of Technology, Cambridge, MA. 3. Breast Radiology Unit, Department of Imaging and Physiology, Karolinska University Hospital, Stockholm, Sweden. 4. Department of Oncology-Pathology, Karolinska Institute, Stockholm, Sweden. 5. Department of Medical Imaging and Intervention, Chang Gung Memorial Hospital at Linkou, Taoyuan, Taiwan. 6. Department of Computer Science, University of California Los Angeles, Los Angeles, CA. 7. Department of Computer Science, Georgia Institute of Technology, Atlanta, GA. 8. Department of Biomedical Informatics, Emory University, Atlanta, GA. 9. Department of Radiology, Emory University, Atlanta, GA. 10. Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA. 11. Division of Surgical Oncology, Massachusetts General Hospital, Harvard Medical School, Boston, MA. 12. Analytics and Informatics Department, Novant Health, Winston-Salem, NC. 13. Digital Product and Services, Novant Health, Winston-Salem, NC. 14. Department of Cancer Prevention, Barretos Cancer Hospital, Barretos, Brazil. 15. Maccabitech, Maccabi Health Services, Tel Aviv, Israel. 16. Department of Imaging, Assuta Medical Centers, Tel Aviv, Israel.
Abstract
PURPOSE: Accurate risk assessment is essential for the success of population screening programs in breast cancer. Models with high sensitivity and specificity would enable programs to target more elaborate screening efforts to high-risk populations, while minimizing overtreatment for the rest. Artificial intelligence (AI)-based risk models have demonstrated a significant advance over risk models used today in clinical practice. However, the responsible deployment of novel AI requires careful validation across diverse populations. To this end, we validate our AI-based model, Mirai, across globally diverse screening populations. METHODS: We collected screening mammograms and pathology-confirmed breast cancer outcomes from Massachusetts General Hospital, USA; Novant, USA; Emory, USA; Maccabi-Assuta, Israel; Karolinska, Sweden; Chang Gung Memorial Hospital, Taiwan; and Barretos, Brazil. We evaluated Uno's concordance index for Mirai in predicting risk of breast cancer at one to five years from the mammogram. RESULTS: A total of 128,793 mammograms from 62,185 patients were collected across the seven sites, of which 3,815 were followed by a cancer diagnosis within 5 years. Mirai obtained concordance indices of 0.75 (95% CI, 0.72 to 0.78), 0.75 (95% CI, 0.70 to 0.80), 0.77 (95% CI, 0.75 to 0.79), 0.77 (95% CI, 0.73 to 0.81), 0.81 (95% CI, 0.79 to 0.82), 0.79 (95% CI, 0.76 to 0.83), and 0.84 (95% CI, 0.81 to 0.88) at Massachusetts General Hospital, Novant, Emory, Maccabi-Assuta, Karolinska, Chang Gung Memorial Hospital, and Barretos, respectively. CONCLUSION: Mirai, a mammography-based risk model, maintained its accuracy across globally diverse test sets from seven hospitals across five countries. This is the broadest validation to date of an AI-based breast cancer model and suggests that the technology can offer broad and equitable improvements in care.
PURPOSE: Accurate risk assessment is essential for the success of population screening programs in breast cancer. Models with high sensitivity and specificity would enable programs to target more elaborate screening efforts to high-risk populations, while minimizing overtreatment for the rest. Artificial intelligence (AI)-based risk models have demonstrated a significant advance over risk models used today in clinical practice. However, the responsible deployment of novel AI requires careful validation across diverse populations. To this end, we validate our AI-based model, Mirai, across globally diverse screening populations. METHODS: We collected screening mammograms and pathology-confirmed breast cancer outcomes from Massachusetts General Hospital, USA; Novant, USA; Emory, USA; Maccabi-Assuta, Israel; Karolinska, Sweden; Chang Gung Memorial Hospital, Taiwan; and Barretos, Brazil. We evaluated Uno's concordance index for Mirai in predicting risk of breast cancer at one to five years from the mammogram. RESULTS: A total of 128,793 mammograms from 62,185 patients were collected across the seven sites, of which 3,815 were followed by a cancer diagnosis within 5 years. Mirai obtained concordance indices of 0.75 (95% CI, 0.72 to 0.78), 0.75 (95% CI, 0.70 to 0.80), 0.77 (95% CI, 0.75 to 0.79), 0.77 (95% CI, 0.73 to 0.81), 0.81 (95% CI, 0.79 to 0.82), 0.79 (95% CI, 0.76 to 0.83), and 0.84 (95% CI, 0.81 to 0.88) at Massachusetts General Hospital, Novant, Emory, Maccabi-Assuta, Karolinska, Chang Gung Memorial Hospital, and Barretos, respectively. CONCLUSION: Mirai, a mammography-based risk model, maintained its accuracy across globally diverse test sets from seven hospitals across five countries. This is the broadest validation to date of an AI-based breast cancer model and suggests that the technology can offer broad and equitable improvements in care.
Authors: Adam Yala; Peter G Mikhael; Fredrik Strand; Gigin Lin; Kevin Smith; Yung-Liang Wan; Leslie Lamb; Kevin Hughes; Constance Lehman; Regina Barzilay Journal: Sci Transl Med Date: 2021-01-27 Impact factor: 17.956
Authors: Rayna K Matsuno; Joseph P Costantino; Regina G Ziegler; Garnet L Anderson; Huilin Li; David Pee; Mitchell H Gail Journal: J Natl Cancer Inst Date: 2011-05-11 Impact factor: 13.506
Authors: Marije F Bakker; Stéphanie V de Lange; Ruud M Pijnappel; Ritse M Mann; Petra H M Peeters; Evelyn M Monninkhof; Marleen J Emaus; Claudette E Loo; Robertus H C Bisschops; Marc B I Lobbes; Matthijn D F de Jong; Katya M Duvivier; Jeroen Veltman; Nico Karssemeijer; Harry J de Koning; Paul J van Diest; Willem P T M Mali; Maurice A A J van den Bosch; Wouter B Veldhuis; Carla H van Gils Journal: N Engl J Med Date: 2019-11-28 Impact factor: 91.245
Authors: Gustav Mårtensson; Daniel Ferreira; Tobias Granberg; Lena Cavallin; Ketil Oppedal; Alessandro Padovani; Irena Rektorova; Laura Bonanni; Matteo Pardini; Milica G Kramberger; John-Paul Taylor; Jakub Hort; Jón Snædal; Jaime Kulisevsky; Frederic Blanc; Angelo Antonini; Patrizia Mecocci; Bruno Vellas; Magda Tsolaki; Iwona Kłoszewska; Hilkka Soininen; Simon Lovestone; Andrew Simmons; Dag Aarsland; Eric Westman Journal: Med Image Anal Date: 2020-05-01 Impact factor: 8.545
Authors: Robert A Smith; Kimberly S Andrews; Durado Brooks; Stacey A Fedewa; Deana Manassaram-Baptiste; Debbie Saslow; Richard C Wender Journal: CA Cancer J Clin Date: 2019-03-15 Impact factor: 508.702
Authors: Debra L Monticciolo; Mary S Newell; Linda Moy; Bethany Niell; Barbara Monsees; Edward A Sickles Journal: J Am Coll Radiol Date: 2018-01-19 Impact factor: 5.532
Authors: John R Zech; Marcus A Badgeley; Manway Liu; Anthony B Costa; Joseph J Titano; Eric Karl Oermann Journal: PLoS Med Date: 2018-11-06 Impact factor: 11.069
Authors: Constance D Lehman; Sarah Mercaldo; Leslie R Lamb; Tari A King; Leif W Ellisen; Michelle Specht; Rulla M Tamimi Journal: J Natl Cancer Inst Date: 2022-10-06 Impact factor: 11.816
Authors: Graham A Colditz; Debbie L Bennett; Jennifer Tappenden; Courtney Beers; Nicole Ackermann; Ningying Wu; Jingqin Luo; Sarah Humble; Erin Linnenbringer; Kia Davis; Shu Jiang; Adetunji T Toriola Journal: Cancer Causes Control Date: 2022-01-21 Impact factor: 2.506