Julien Meyer1, April Khademi2, Bernard Têtu3, Wencui Han4, Pria Nippak1, David Remisch1. 1. School of Health Services Management, Ted Rogers School of Management, Toronto, Ontario, Canada. 2. Department of Electrical, Computer, and Biomedical Engineering, Ryerson University, Toronto, Ontario, Canada. 3. Départment de biologie médicale, Université Laval, Québec City, Quebec, Canada. 4. Department of Business administration, Gies College of Business, University of Illinois at Urbana-Champaign, Champaign, Illinois, USA.
Abstract
OBJECTIVE: The accuracy of artificial intelligence (AI) in medicine and in pathology in particular has made major progress but little is known on how much these algorithms will influence pathologists' decisions in practice. The objective of this paper is to determine the reliance of pathologists on AI and to investigate whether providing information on AI impacts this reliance. MATERIALS AND METHODS: The experiment using an online survey design. Under 3 conditions, 116 pathologists and pathology students were tasked with assessing the Gleason grade for a series of 12 prostate biopsies: (1) without AI recommendations, (2) with AI recommendations, and (3) with AI recommendations accompanied by information about the algorithm itself, specifically algorithm accuracy rate and algorithm decision-making process. RESULTS: Participant responses were significantly more accurate with the AI decision aids than without (92% vs 87%, odds ratio 13.30, P < .01). Unexpectedly, the provision of information on the algorithm made no significant difference compared to AI without information. The reliance on AI correlated with general beliefs on AI's usefulness but not with particular assessments of the AI tool offered. Decisions were made faster when AI was provided. DISCUSSION: These results suggest that pathologists are willing to rely on AI regardless of accuracy or explanations. Generalization beyond the specific tasks and explanations provided will require further studies. CONCLUSION: This study suggests that the factors that influence the reliance on AI differ in practice from beliefs expressed by clinicians in surveys. Implementation of AI in prospective settings should take individual behaviors into account.
OBJECTIVE: The accuracy of artificial intelligence (AI) in medicine and in pathology in particular has made major progress but little is known on how much these algorithms will influence pathologists' decisions in practice. The objective of this paper is to determine the reliance of pathologists on AI and to investigate whether providing information on AI impacts this reliance. MATERIALS AND METHODS: The experiment using an online survey design. Under 3 conditions, 116 pathologists and pathology students were tasked with assessing the Gleason grade for a series of 12 prostate biopsies: (1) without AI recommendations, (2) with AI recommendations, and (3) with AI recommendations accompanied by information about the algorithm itself, specifically algorithm accuracy rate and algorithm decision-making process. RESULTS: Participant responses were significantly more accurate with the AI decision aids than without (92% vs 87%, odds ratio 13.30, P < .01). Unexpectedly, the provision of information on the algorithm made no significant difference compared to AI without information. The reliance on AI correlated with general beliefs on AI's usefulness but not with particular assessments of the AI tool offered. Decisions were made faster when AI was provided. DISCUSSION: These results suggest that pathologists are willing to rely on AI regardless of accuracy or explanations. Generalization beyond the specific tasks and explanations provided will require further studies. CONCLUSION: This study suggests that the factors that influence the reliance on AI differ in practice from beliefs expressed by clinicians in surveys. Implementation of AI in prospective settings should take individual behaviors into account.
Authors: Philipp Tschandl; Christoph Rinner; Zoe Apalla; Giuseppe Argenziano; Noel Codella; Allan Halpern; Monika Janda; Aimilios Lallas; Caterina Longo; Josep Malvehy; John Paoli; Susana Puig; Cliff Rosendahl; H Peter Soyer; Iris Zalaudek; Harald Kittler Journal: Nat Med Date: 2020-06-22 Impact factor: 53.440
Authors: Florian Jungmann; Tobias Jorg; Felix Hahn; Daniel Pinto Dos Santos; Stefanie Maria Jungmann; Christoph Düber; Peter Mildenberger; Roman Kloeckner Journal: Acad Radiol Date: 2020-05-13 Impact factor: 3.173
Authors: Jane Scheetz; Philip Rothschild; Myra McGuinness; Xavier Hadoux; H Peter Soyer; Monika Janda; James J J Condon; Luke Oakden-Rayner; Lyle J Palmer; Stuart Keel; Peter van Wijngaarden Journal: Sci Rep Date: 2021-03-04 Impact factor: 4.379