Armando D Bedoya1, Meredith E Clement2, Matthew Phelan3, Rebecca C Steorts4,5, Cara O'Brien6, Benjamin A Goldstein3,4. 1. Division of Pulmonary, Allergy, and Critical Care Medicine, Duke University, Durham, NC. 2. Division of Infectious Disease, Duke University, Durham, NC. 3. Center for Predictive Medicine, Duke Clinical Research Institute, Durham, NC. 4. Department of Biostatistics and Bioinformatics, Duke University, Durham, NC. 5. Department of Statistical Science and Computer Science, Duke University, Durham, NC. 6. Department of Medicine, Duke University, Durham, NC.
Abstract
OBJECTIVES: Previous studies have looked at National Early Warning Score performance in predicting in-hospital deterioration and death, but data are lacking with respect to patient outcomes following implementation of National Early Warning Score. We sought to determine the effectiveness of National Early Warning Score implementation on predicting and preventing patient deterioration in a clinical setting. DESIGN: Retrospective cohort study. SETTING: Tertiary care academic facility and a community hospital. PATIENTS: Patients 18 years old or older hospitalized from March 1, 2014, to February 28, 2015, during preimplementation of National Early Warning Score to August 1, 2015, to July 31, 2016, after National Early Warning Score was implemented. INTERVENTIONS: Implementation of National Early Warning Score within the electronic health record and associated best practice alert. MEASUREMENTS AND MAIN RESULTS: In this study of 85,322 patients (42,402 patients pre-National Early Warning Score and 42,920 patients post-National Early Warning Score implementation), the primary outcome of rate of ICU transfer or death did not change after National Early Warning Score implementation, with adjusted hazard ratio of 0.94 (0.84-1.05) and 0.90 (0.77-1.05) at our academic and community hospital, respectively. In total, 175,357 best practice advisories fired during the study period, with the best practice advisory performing better at the community hospital than the academic at predicting an event within 12 hours 7.4% versus 2.2% of the time, respectively. Retraining National Early Warning Score with newly generated hospital-specific coefficients improved model performance. CONCLUSIONS: At both our academic and community hospital, National Early Warning Score had poor performance characteristics and was generally ignored by frontline nursing staff. As a result, National Early Warning Score implementation had no appreciable impact on defined clinical outcomes. Refitting of the model using site-specific data improved performance and supports validating predictive models on local data.
OBJECTIVES: Previous studies have looked at National Early Warning Score performance in predicting in-hospital deterioration and death, but data are lacking with respect to patient outcomes following implementation of National Early Warning Score. We sought to determine the effectiveness of National Early Warning Score implementation on predicting and preventing patient deterioration in a clinical setting. DESIGN: Retrospective cohort study. SETTING: Tertiary care academic facility and a community hospital. PATIENTS: Patients 18 years old or older hospitalized from March 1, 2014, to February 28, 2015, during preimplementation of National Early Warning Score to August 1, 2015, to July 31, 2016, after National Early Warning Score was implemented. INTERVENTIONS: Implementation of National Early Warning Score within the electronic health record and associated best practice alert. MEASUREMENTS AND MAIN RESULTS: In this study of 85,322 patients (42,402 patients pre-National Early Warning Score and 42,920 patients post-National Early Warning Score implementation), the primary outcome of rate of ICU transfer or death did not change after National Early Warning Score implementation, with adjusted hazard ratio of 0.94 (0.84-1.05) and 0.90 (0.77-1.05) at our academic and community hospital, respectively. In total, 175,357 best practice advisories fired during the study period, with the best practice advisory performing better at the community hospital than the academic at predicting an event within 12 hours 7.4% versus 2.2% of the time, respectively. Retraining National Early Warning Score with newly generated hospital-specific coefficients improved model performance. CONCLUSIONS: At both our academic and community hospital, National Early Warning Score had poor performance characteristics and was generally ignored by frontline nursing staff. As a result, National Early Warning Score implementation had no appreciable impact on defined clinical outcomes. Refitting of the model using site-specific data improved performance and supports validating predictive models on local data.
Authors: Matthew M Churpek; Trevor C Yuen; Michael T Huber; Seo Young Park; Jesse B Hall; Dana P Edelson Journal: Chest Date: 2011-11-03 Impact factor: 9.410
Authors: Matthew M Churpek; Blair Wendlandt; Frank J Zadravecz; Richa Adhikari; Christopher Winslow; Dana P Edelson Journal: J Hosp Med Date: 2016-06-28 Impact factor: 2.960
Authors: Benjamin A Goldstein; Ann Marie Navar; Michael J Pencina; John P A Ioannidis Journal: J Am Med Inform Assoc Date: 2016-05-17 Impact factor: 4.497
Authors: Gary B Smith; David R Prytherch; Paul Meredith; Paul E Schmidt; Peter I Featherstone Journal: Resuscitation Date: 2013-01-04 Impact factor: 5.262
Authors: Timothy J Hodgetts; Gary Kenward; Ioannis G Vlachonikolis; Susan Payne; Nicolas Castle Journal: Resuscitation Date: 2002-08 Impact factor: 5.262
Authors: Daniel T Linnen; Gabriel J Escobar; Xiao Hu; Elizabeth Scruth; Vincent Liu; Caroline Stephens Journal: J Hosp Med Date: 2019-03 Impact factor: 2.960
Authors: Gabriel J Escobar; Vincent X Liu; Alejandro Schuler; Brian Lawson; John D Greene; Patricia Kipnis Journal: N Engl J Med Date: 2020-11-12 Impact factor: 91.245
Authors: Armando D Bedoya; Nicoleta J Economou-Zavlanos; Benjamin A Goldstein; Allison Young; J Eric Jelovsek; Cara O'Brien; Amanda B Parrish; Scott Elengold; Kay Lytle; Suresh Balu; Erich Huang; Eric G Poon; Michael J Pencina Journal: J Am Med Inform Assoc Date: 2022-08-16 Impact factor: 7.942
Authors: Marco A F Pimentel; Oliver C Redfern; James Malycha; Paul Meredith; David Prytherch; Jim Briggs; J Duncan Young; David A Clifton; Lionel Tarassenko; Peter J Watkinson Journal: Am J Respir Crit Care Med Date: 2021-07-01 Impact factor: 21.405
Authors: Chieh-Liang Wu; Chen-Tsung Kuo; Sou-Jen Shih; Jung-Chen Chen; Ying-Chih Lo; Hsiu-Hui Yu; Ming-De Huang; Wayne Huey-Herng Sheu; Shih-An Liu Journal: Int J Environ Res Public Health Date: 2021-04-25 Impact factor: 3.390