Stephanie Helman1, Martha Ann Terry2, Tiffany Pellathy3, Andrew Williams4, Artur Dubrawski5, Gilles Clermont6, Michael R Pinsky7, Salah Al-Zaiti8, Marilyn Hravnak9. 1. The Department of Acute and Tertiary Care Nursing, University of Pittsburgh, Pittsburgh, PA, United States. Electronic address: smh178@pitt.edu. 2. The Department of Behavioral and Community Health Sciences, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA, United States. Electronic address: materry@pitt.edu. 3. The Veterans Administration Center for Health Equity Research and Promotion, Pittsburgh, PA, United States. Electronic address: tiffany.pellathy@va.gov. 4. The Auton Lab, School of Computer Science at Carnegie Mellon University, Pittsburgh, PA, United States. Electronic address: awillia2@andrew.cmu.edu. 5. The Auton Lab, School of Computer Science at Carnegie Mellon University, Pittsburgh, PA, United States. Electronic address: awd@cs.cmu.edu. 6. The Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, PA, United States. Electronic address: cler@pitt.edu. 7. The Department of Critical Care Medicine, University of Pittsburgh, Pittsburgh, PA, United States. Electronic address: pinsky@pitt.edu. 8. The Department of Acute and Tertiary Care Nursing, University of Pittsburgh, Pittsburgh, PA, United States; The Department of Emergency Medicine, University of Pittsburgh, Pittsburgh, PA, United States; The Division of Cardiology, University of Pittsburgh, Pittsburgh, PA, United States. Electronic address: ssa33@pitt.edu. 9. The Department of Acute and Tertiary Care Nursing, University of Pittsburgh, Pittsburgh, PA, United States. Electronic address: mhra@pitt.edu.
Abstract
BACKGROUND: Artificial Intelligence (AI) is increasingly used to support bedside clinical decisions, but information must be presented in usable ways within workflow. Graphical User Interfaces (GUI) are front-facing presentations for communicating AI outputs, but clinicians are not routinely invited to participate in their design, hindering AI solution potential. PURPOSE: To inform early user-engaged design of a GUI prototype aimed at predicting future Cardiorespiratory Insufficiency (CRI) by exploring clinician methods for identifying at-risk patients, previous experience with implementing new technologies into clinical workflow, and user perspectives on GUI screen changes. METHODS: We conducted a qualitative focus group study to elicit iterative design feedback from clinical end-users on an early GUI prototype display. Five online focus group sessions were held, each moderated by an expert focus group methodologist. Iterative design changes were made sequentially, and the updated GUI display was presented to the next group of participants. RESULTS: 23 clinicians were recruited (14 nurses, 4 nurse practitioners, 5 physicians; median participant age ∼35 years; 60% female; median clinical experience 8 years). Five themes emerged from thematic content analysis: trend evolution, context (risk evolution relative to vital signs and interventions), evaluation/interpretation/explanation (sub theme: continuity of evaluation), clinician intuition, and clinical operations. Based on these themes, GUI display changes were made. For example, color and scale adjustments, integration of clinical information, and threshold personalization. CONCLUSIONS: Early user-engaged design was useful in adjusting GUI presentation of AI output. Next steps involve clinical testing and further design modification of the AI output to optimally facilitate clinician surveillance and decisions. Clinicians should be involved early and often in clinical decision support design to optimize efficacy of AI tools.
BACKGROUND: Artificial Intelligence (AI) is increasingly used to support bedside clinical decisions, but information must be presented in usable ways within workflow. Graphical User Interfaces (GUI) are front-facing presentations for communicating AI outputs, but clinicians are not routinely invited to participate in their design, hindering AI solution potential. PURPOSE: To inform early user-engaged design of a GUI prototype aimed at predicting future Cardiorespiratory Insufficiency (CRI) by exploring clinician methods for identifying at-risk patients, previous experience with implementing new technologies into clinical workflow, and user perspectives on GUI screen changes. METHODS: We conducted a qualitative focus group study to elicit iterative design feedback from clinical end-users on an early GUI prototype display. Five online focus group sessions were held, each moderated by an expert focus group methodologist. Iterative design changes were made sequentially, and the updated GUI display was presented to the next group of participants. RESULTS: 23 clinicians were recruited (14 nurses, 4 nurse practitioners, 5 physicians; median participant age ∼35 years; 60% female; median clinical experience 8 years). Five themes emerged from thematic content analysis: trend evolution, context (risk evolution relative to vital signs and interventions), evaluation/interpretation/explanation (sub theme: continuity of evaluation), clinician intuition, and clinical operations. Based on these themes, GUI display changes were made. For example, color and scale adjustments, integration of clinical information, and threshold personalization. CONCLUSIONS: Early user-engaged design was useful in adjusting GUI presentation of AI output. Next steps involve clinical testing and further design modification of the AI output to optimally facilitate clinician surveillance and decisions. Clinicians should be involved early and often in clinical decision support design to optimize efficacy of AI tools.
Authors: Michael A Devita; Rinaldo Bellomo; Kenneth Hillman; John Kellum; Armando Rotondi; Dan Teres; Andrew Auerbach; Wen-Jon Chen; Kathy Duncan; Gary Kenward; Max Bell; Michael Buist; Jack Chen; Julian Bion; Ann Kirby; Geoff Lighthall; John Ovreveit; R Scott Braithwaite; John Gosbee; Eric Milbrandt; Mimi Peberdy; Lucy Savitz; Lis Young; Maurene Harvey; Sanjay Galhotra Journal: Crit Care Med Date: 2006-09 Impact factor: 7.598
Authors: Lujie Chen; Olufunmilayo Ogundele; Gilles Clermont; Marilyn Hravnak; Michael R Pinsky; Artur W Dubrawski Journal: Ann Am Thorac Soc Date: 2017-03
Authors: Katherine Bristowe; Dimitrios Siassakos; Helen Hambly; Jo Angouri; Andrew Yelland; Timothy J Draycott; Robert Fox Journal: Qual Health Res Date: 2012-07-17
Authors: Marzyeh Ghassemi; Tristan Naumann; Peter Schulam; Andrew L Beam; Irene Y Chen; Rajesh Ranganath Journal: AMIA Jt Summits Transl Sci Proc Date: 2020-05-30
Authors: Matthew M Churpek; Trevor C Yuen; Christopher Winslow; Ari A Robicsek; David O Meltzer; Robert D Gibbons; Dana P Edelson Journal: Am J Respir Crit Care Med Date: 2014-09-15 Impact factor: 21.405