Sally L Baxter1,2, Jeremy S Bass1,3, Amy M Sitapati1,4. 1. Health Department of Biomedical Informatics, University of California San Diego, La Jolla, California, United States. 2. Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego, La Jolla, California, United States. 3. Department of Psychiatry, University of California San Diego, La Jolla, California, United States. 4. Department of Medicine, University of California San Diego, La Jolla, California, United States.
Abstract
BACKGROUND: Electronic health record (EHR) vendors now offer "off-the-shelf" artificial intelligence (AI) models to client organizations. Our health system faced difficulties in promoting end-user utilization of a new AI model for predicting readmissions embedded in the EHR. OBJECTIVES: The aim is to conduct a case study centered on identifying barriers to uptake/utilization. METHODS: A qualitative study was conducted using interviews with stakeholders. The interviews were used to identify relevant stakeholders, understand current workflows, identify implementation barriers, and formulate future strategies. RESULTS: We discovered substantial variation in existing workflows around readmissions. Some stakeholders did not perform any formal readmissions risk assessment. Others accustomed to using existing risk scores such as LACE+ had concerns about transitioning to a new model. Some stakeholders had existing workflows in place that could accommodate the new model, but they were not previously aware that the new model was in production. Concerns expressed by end-users included: whether the model's predictors were relevant to their work, need for adoption of additional workflow processes, need for training and change management, and potential for unintended consequences (e.g., increased health care resource utilization due to potentially over-referring discharged patients to home health services). CONCLUSION: AI models for risk stratification, even if "off-the-shelf" by design, are unlikely to be "plug-and-play" in health care settings. Seeking out key stakeholders and defining clear use cases early in the implementation process can better facilitate utilization of these models.
BACKGROUND: Electronic health record (EHR) vendors now offer "off-the-shelf" artificial intelligence (AI) models to client organizations. Our health system faced difficulties in promoting end-user utilization of a new AI model for predicting readmissions embedded in the EHR. OBJECTIVES: The aim is to conduct a case study centered on identifying barriers to uptake/utilization. METHODS: A qualitative study was conducted using interviews with stakeholders. The interviews were used to identify relevant stakeholders, understand current workflows, identify implementation barriers, and formulate future strategies. RESULTS: We discovered substantial variation in existing workflows around readmissions. Some stakeholders did not perform any formal readmissions risk assessment. Others accustomed to using existing risk scores such as LACE+ had concerns about transitioning to a new model. Some stakeholders had existing workflows in place that could accommodate the new model, but they were not previously aware that the new model was in production. Concerns expressed by end-users included: whether the model's predictors were relevant to their work, need for adoption of additional workflow processes, need for training and change management, and potential for unintended consequences (e.g., increased health care resource utilization due to potentially over-referring discharged patients to home health services). CONCLUSION: AI models for risk stratification, even if "off-the-shelf" by design, are unlikely to be "plug-and-play" in health care settings. Seeking out key stakeholders and defining clear use cases early in the implementation process can better facilitate utilization of these models.
Entities:
Keywords:
artificial intelligence; case management; clinical informatics; electronic health records; health system; machine learning; predictive analytics; predictive models; readmissions
Authors: Ian F Caplan; Patricia Zadnik Sullivan; David Kung; Donald M O'Rourke; Omar Choudhri; Gregory Glauser; Benjamin Osiemo; Stephen Goodrich; Scott D McClintock; Neil R Malhotra Journal: World Neurosurg Date: 2019-03-27 Impact factor: 2.104
Authors: Rachael B Zuckerman; Steven H Sheingold; E John Orav; Joel Ruhter; Arnold M Epstein Journal: N Engl J Med Date: 2016-02-24 Impact factor: 91.245
Authors: Malvika Sharma; Carl Savage; Monika Nair; Ingrid Larsson; Petra Svedberg; Jens M Nygren Journal: J Med Internet Res Date: 2022-10-05 Impact factor: 7.076