Heather M Giannini1, Jennifer C Ginestra1, Corey Chivers2, Michael Draugelis2, Asaf Hanish2, William D Schweickert2,3, Barry D Fuchs2,3, Laurie Meadows4, Michael Lynch4, Patrick J Donnelly5, Kimberly Pavan6, Neil O Fishman2, C William Hanson2, Craig A Umscheid2,7,8. 1. Department of Medicine, Hospital of the University of Pennsylvania, Philadelphia, PA. 2. University of Pennsylvania Health System, Philadelphia, PA. 3. Division of Pulmonary, Allergy, and Critical Care Medicine, Department of Medicine, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA. 4. Department of Nursing, Hospital of the University of Pennsylvania, Philadelphia, PA. 5. Department of Clinical Informatics, Pennsylvania Hospital, Philadelphia, PA. 6. Penn Presbyterian Medical Center, Philadelphia, PA. 7. Division of General Internal Medicine, Department of Medicine, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA. 8. Center for Evidence-based Practice, University of Pennsylvania Health System, Philadelphia, PA.
Abstract
OBJECTIVES: Develop and implement a machine learning algorithm to predict severe sepsis and septic shock and evaluate the impact on clinical practice and patient outcomes. DESIGN: Retrospective cohort for algorithm derivation and validation, pre-post impact evaluation. SETTING: Tertiary teaching hospital system in Philadelphia, PA. PATIENTS: All non-ICU admissions; algorithm derivation July 2011 to June 2014 (n = 162,212); algorithm validation October to December 2015 (n = 10,448); silent versus alert comparison January 2016 to February 2017 (silent n = 22,280; alert n = 32,184). INTERVENTIONS: A random-forest classifier, derived and validated using electronic health record data, was deployed both silently and later with an alert to notify clinical teams of sepsis prediction. MEASUREMENT AND MAIN RESULT: Patients identified for training the algorithm were required to have International Classification of Diseases, 9th Edition codes for severe sepsis or septic shock and a positive blood culture during their hospital encounter with either a lactate greater than 2.2 mmol/L or a systolic blood pressure less than 90 mm Hg. The algorithm demonstrated a sensitivity of 26% and specificity of 98%, with a positive predictive value of 29% and positive likelihood ratio of 13. The alert resulted in a small statistically significant increase in lactate testing and IV fluid administration. There was no significant difference in mortality, discharge disposition, or transfer to ICU, although there was a reduction in time-to-ICU transfer. CONCLUSIONS: Our machine learning algorithm can predict, with low sensitivity but high specificity, the impending occurrence of severe sepsis and septic shock. Algorithm-generated predictive alerts modestly impacted clinical measures. Next steps include describing clinical perception of this tool and optimizing algorithm design and delivery.
OBJECTIVES: Develop and implement a machine learning algorithm to predict severe sepsis and septic shock and evaluate the impact on clinical practice and patient outcomes. DESIGN: Retrospective cohort for algorithm derivation and validation, pre-post impact evaluation. SETTING: Tertiary teaching hospital system in Philadelphia, PA. PATIENTS: All non-ICU admissions; algorithm derivation July 2011 to June 2014 (n = 162,212); algorithm validation October to December 2015 (n = 10,448); silent versus alert comparison January 2016 to February 2017 (silent n = 22,280; alert n = 32,184). INTERVENTIONS: A random-forest classifier, derived and validated using electronic health record data, was deployed both silently and later with an alert to notify clinical teams of sepsis prediction. MEASUREMENT AND MAIN RESULT: Patients identified for training the algorithm were required to have International Classification of Diseases, 9th Edition codes for severe sepsis or septic shock and a positive blood culture during their hospital encounter with either a lactate greater than 2.2 mmol/L or a systolic blood pressure less than 90 mm Hg. The algorithm demonstrated a sensitivity of 26% and specificity of 98%, with a positive predictive value of 29% and positive likelihood ratio of 13. The alert resulted in a small statistically significant increase in lactate testing and IV fluid administration. There was no significant difference in mortality, discharge disposition, or transfer to ICU, although there was a reduction in time-to-ICU transfer. CONCLUSIONS: Our machine learning algorithm can predict, with low sensitivity but high specificity, the impending occurrence of severe sepsis and septic shock. Algorithm-generated predictive alerts modestly impacted clinical measures. Next steps include describing clinical perception of this tool and optimizing algorithm design and delivery.
Authors: Jennifer C Ginestra; Heather M Giannini; William D Schweickert; Laurie Meadows; Michael J Lynch; Kimberly Pavan; Corey J Chivers; Michael Draugelis; Patrick J Donnelly; Barry D Fuchs; Craig A Umscheid Journal: Crit Care Med Date: 2019-11 Impact factor: 7.598
Authors: Joyce Ji; Jeff Klaus; Jason P Burnham; Andrew Michelson; Colleen A McEvoy; Marin H Kollef; Patrick G Lyons Journal: Chest Date: 2020-06-17 Impact factor: 9.410
Authors: Santiago Romero-Brufau; Daniel Whitford; Matthew G Johnson; Joel Hickman; Bruce W Morlan; Terry Therneau; James Naessens; Jeanne M Huddleston Journal: J Am Med Inform Assoc Date: 2021-06-12 Impact factor: 4.497
Authors: Jessica M Schwartz; Amanda J Moy; Sarah C Rossetti; Noémie Elhadad; Kenrick D Cato Journal: J Am Med Inform Assoc Date: 2021-03-01 Impact factor: 4.497
Authors: Jussi Pirneskoski; Joonas Tamminen; Antti Kallonen; Jouni Nurmi; Markku Kuisma; Klaus T Olkkola; Sanna Hoppu Journal: Resusc Plus Date: 2020-12-05