Emily M Powers1,2, Richard N Shiffman1,2, Edward R Melnick1,3, Andrew Hickner4, Mona Sharifi1,2. 1. Yale Center for Medical Informatics, Yale University School of Medicine, New Haven, Connecticut, USA. 2. Department of Pediatrics, Yale University School of Medicine, New Haven, Connecticut, USA. 3. Department of Emergency Medicine, Yale University School of Medicine, New Haven, Connecticut, USA. 4. Cushing/Whitney Medical Library, Yale University School of Medicine, New Haven, Connecticut, USA.
Abstract
Objective: Clinical decision support (CDS) hard-stop alerts-those in which the user is either prevented from taking an action altogether or allowed to proceed only with the external override of a third party-are increasingly common but can be problematic. To understand their appropriate application, we asked 3 key questions: (1) To what extent are hard-stop alerts effective in improving patient health and healthcare delivery outcomes? (2) What are the adverse events and unintended consequences of hard-stop alerts? (3) How do hard-stop alerts compare to soft-stop alerts? Methods and Materials: Studies evaluating computerized hard-stop alerts in healthcare settings were identified from biomedical and computer science databases, gray literature sites, reference lists, and reviews. Articles were extracted for process outcomes, health outcomes, unintended consequences, user experience, and technical details. Results: Of 32 studies, 15 evaluated health outcomes, 16 process outcomes only, 10 user experience, and 4 compared hard and soft stops. Seventy-nine percent showed improvement in health outcomes and 88% in process outcomes. Studies reporting good user experience cited heavy user involvement and iterative design. Eleven studies reported on unintended consequences including avoidance of hard-stopped workflow, increased alert frequency, and delay to care. Hard stops were superior to soft stops in 3 of 4 studies. Conclusions: Hard stops can be effective and powerful tools in the CDS armamentarium, but they must be implemented judiciously with continuous user feedback informing rapid, iterative design. Investigators must report on associated health outcomes and unintended consequences when implementing IT solutions to clinical problems.
Objective: Clinical decision support (CDS) hard-stop alerts-those in which the user is either prevented from taking an action altogether or allowed to proceed only with the external override of a third party-are increasingly common but can be problematic. To understand their appropriate application, we asked 3 key questions: (1) To what extent are hard-stop alerts effective in improving patient health and healthcare delivery outcomes? (2) What are the adverse events and unintended consequences of hard-stop alerts? (3) How do hard-stop alerts compare to soft-stop alerts? Methods and Materials: Studies evaluating computerized hard-stop alerts in healthcare settings were identified from biomedical and computer science databases, gray literature sites, reference lists, and reviews. Articles were extracted for process outcomes, health outcomes, unintended consequences, user experience, and technical details. Results: Of 32 studies, 15 evaluated health outcomes, 16 process outcomes only, 10 user experience, and 4 compared hard and soft stops. Seventy-nine percent showed improvement in health outcomes and 88% in process outcomes. Studies reporting good user experience cited heavy user involvement and iterative design. Eleven studies reported on unintended consequences including avoidance of hard-stopped workflow, increased alert frequency, and delay to care. Hard stops were superior to soft stops in 3 of 4 studies. Conclusions: Hard stops can be effective and powerful tools in the CDS armamentarium, but they must be implemented judiciously with continuous user feedback informing rapid, iterative design. Investigators must report on associated health outcomes and unintended consequences when implementing IT solutions to clinical problems.
Authors: Vartan M Vartanians; Christopher L Sistrom; Jeffrey B Weilburg; Daniel I Rosenthal; James H Thrall Journal: Radiology Date: 2010-06 Impact factor: 11.105
Authors: Robert A Peterfreund; William D Driscoll; John L Walsh; Aparna Subramanian; Shaji Anupama; Melissa Weaver; Theresa Morris; Sarah Arnholz; Hui Zheng; Eric T Pierce; Stephen F Spring Journal: Anesth Analg Date: 2011-03-17 Impact factor: 5.108
Authors: Glenn K Geeting; Michael Beck; Michael A Bruno; Rickhesvar P Mahraj; Gregory Caputo; Christopher DeFlitch; Christopher S Hollenbeak Journal: AJR Am J Roentgenol Date: 2016-05-04 Impact factor: 3.959
Authors: Eric A Larson; Paul A Thompson; Zachary K Anderson; Keith A Anderson; Roxana A Lupu; Vicki Tigner; Wendell W Hoffman Journal: Int J Gen Med Date: 2016-06-03
Authors: Mark R Kilgore; Carrie A McIlwain; Rodney A Schmidt; Barbara M Norquist; Elizabeth M Swisher; Rochelle L Garcia; Mara H Rendi Journal: J Pathol Inform Date: 2016-11-29
Authors: Rebecca L Curran; Polina V Kukhareva; Teresa Taft; Charlene R Weir; Thomas J Reese; Claude Nanjo; Salvador Rodriguez-Loya; Douglas K Martin; Phillip B Warner; David E Shields; Michael C Flynn; Jonathan P Boltax; Kensaku Kawamoto Journal: J Am Med Inform Assoc Date: 2020-08-01 Impact factor: 4.497
Authors: Bryan D Steitz; Gen Li; Adam Wright; Brent Dunworth; Robert E Freundlich; Jonathan P Wanderer Journal: J Med Syst Date: 2022-01-26 Impact factor: 4.460
Authors: Patrick M Carter; Rebecca M Cunningham; Andria B Eisman; Ken Resnicow; Jessica S Roche; Jennifer Tang Cole; Jason Goldstick; Amy M Kilbourne; Maureen A Walton Journal: J Emerg Med Date: 2021-10-28 Impact factor: 1.484
Authors: Evan L Reynolds; James F Burke; Lacey Evans; Faiz I Syed; Eric Liao; Remy Lobo; Wade Cooper; Larry Charleston; Brian C Callaghan Journal: Headache Date: 2022-01 Impact factor: 5.311
Authors: Adam Wright; Dustin S McEvoy; Skye Aaron; Allison B McCoy; Mary G Amato; Hyun Kim; Angela Ai; James J Cimino; Bimal R Desai; Robert El-Kareh; William Galanter; Christopher A Longhurst; Sameer Malhotra; Ryan P Radecki; Lipika Samal; Richard Schreiber; Eric Shelov; Anwar Mohammad Sirajuddin; Dean F Sittig Journal: J Am Med Inform Assoc Date: 2019-10-01 Impact factor: 4.497