Seyedeh Neelufar Payrovnaziri1, Zhaoyi Chen2, Pablo Rengifo-Moreno3,4, Tim Miller5, Jiang Bian2, Jonathan H Chen6,7, Xiuwen Liu8, Zhe He1. 1. School of Information, Florida State University, Tallahassee, Florida, USA. 2. Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, Florida, USA. 3. College of Medicine, Florida State University, Tallahassee, Florida, USA. 4. Tallahassee Memorial Hospital, Tallahassee, Florida, USA. 5. School of Computing and Information Systems, The University of Melbourne, Melbourne, Victoria, Australia. 6. Center for Biomedical Informatics Research, Department of Medicine, Stanford University, Stanford, California, USA. 7. Division of Hospital Medicine, Department of Medicine, Stanford University, Stanford, California, USA. 8. Department of Computer Science, Florida State University, Tallahassee, Florida, USA.
Abstract
OBJECTIVE: To conduct a systematic scoping review of explainable artificial intelligence (XAI) models that use real-world electronic health record data, categorize these techniques according to different biomedical applications, identify gaps of current studies, and suggest future research directions. MATERIALS AND METHODS: We searched MEDLINE, IEEE Xplore, and the Association for Computing Machinery (ACM) Digital Library to identify relevant papers published between January 1, 2009 and May 1, 2019. We summarized these studies based on the year of publication, prediction tasks, machine learning algorithm, dataset(s) used to build the models, the scope, category, and evaluation of the XAI methods. We further assessed the reproducibility of the studies in terms of the availability of data and code and discussed open issues and challenges. RESULTS: Forty-two articles were included in this review. We reported the research trend and most-studied diseases. We grouped XAI methods into 5 categories: knowledge distillation and rule extraction (N = 13), intrinsically interpretable models (N = 9), data dimensionality reduction (N = 8), attention mechanism (N = 7), and feature interaction and importance (N = 5). DISCUSSION: XAI evaluation is an open issue that requires a deeper focus in the case of medical applications. We also discuss the importance of reproducibility of research work in this field, as well as the challenges and opportunities of XAI from 2 medical professionals' point of view. CONCLUSION: Based on our review, we found that XAI evaluation in medicine has not been adequately and formally practiced. Reproducibility remains a critical concern. Ample opportunities exist to advance XAI research in medicine.
OBJECTIVE: To conduct a systematic scoping review of explainable artificial intelligence (XAI) models that use real-world electronic health record data, categorize these techniques according to different biomedical applications, identify gaps of current studies, and suggest future research directions. MATERIALS AND METHODS: We searched MEDLINE, IEEE Xplore, and the Association for Computing Machinery (ACM) Digital Library to identify relevant papers published between January 1, 2009 and May 1, 2019. We summarized these studies based on the year of publication, prediction tasks, machine learning algorithm, dataset(s) used to build the models, the scope, category, and evaluation of the XAI methods. We further assessed the reproducibility of the studies in terms of the availability of data and code and discussed open issues and challenges. RESULTS: Forty-two articles were included in this review. We reported the research trend and most-studied diseases. We grouped XAI methods into 5 categories: knowledge distillation and rule extraction (N = 13), intrinsically interpretable models (N = 9), data dimensionality reduction (N = 8), attention mechanism (N = 7), and feature interaction and importance (N = 5). DISCUSSION: XAI evaluation is an open issue that requires a deeper focus in the case of medical applications. We also discuss the importance of reproducibility of research work in this field, as well as the challenges and opportunities of XAI from 2 medical professionals' point of view. CONCLUSION: Based on our review, we found that XAI evaluation in medicine has not been adequately and formally practiced. Reproducibility remains a critical concern. Ample opportunities exist to advance XAI research in medicine.
Authors: Tim G J de Meij; Andries E Budding; Evelien F J de Groot; Fenna M Jansen; C M Frank Kneepkens; Marc A Benninga; John Penders; Adriaan A van Bodegraven; Paul H M Savelkoul Journal: FASEB J Date: 2015-12-11 Impact factor: 5.191
Authors: Jinsung Yoon; William R Zame; Amitava Banerjee; Martin Cadeiras; Ahmed M Alaa; Mihaela van der Schaar Journal: PLoS One Date: 2018-03-28 Impact factor: 3.240
Authors: Zhe He; Shubo Tian; Ankita Singh; Shayok Chakraborty; Shenghao Zhang; Mia Liza A Lustria; Neil Charness; Nelson A Roque; Erin R Harrell; Walter R Boot Journal: Inf Process Manag Date: 2022-07-21 Impact factor: 7.466
Authors: Thomas P Quinn; Manisha Senadeera; Stephan Jacobs; Simon Coghlan; Vuong Le Journal: J Am Med Inform Assoc Date: 2021-03-18 Impact factor: 4.497
Authors: Arlene Casey; Emma Davidson; Michael Poon; Hang Dong; Daniel Duma; Andreas Grivas; Claire Grover; Víctor Suárez-Paniagua; Richard Tobin; William Whiteley; Honghan Wu; Beatrice Alex Journal: BMC Med Inform Decis Mak Date: 2021-06-03 Impact factor: 2.796