Fran Balamuth1,2, Elizabeth R Alpern3, Robert W Grundmeier1,4, Marianne Chilutti4, Scott L Weiss5,6, Julie C Fitzgerald5,6, Katie Hayes2, Warren Bilker7, Ebbing Lautenbach7,8. 1. Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA. 2. Division of Emergency Medicine and Center for Pediatric Clinical Effectiveness, Children's Hospital of Philadelphia, Philadelphia, PA. 3. Department of Pediatrics, Northwestern University Feinberg School of Medicine and Division of Emergency Medicine, Ann and Robert H. Lurie Children's Hospital of Chicago, Chicago, IL. 4. Center for Biomedical Informatics, Children's Hospital of Philadelphia, Philadelphia, PA. 5. Department of Anesthesia and Critical Care, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA. 6. Division of Critical Care, Children's Hospital of Philadelphia, Philadelphia, PA. 7. Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA. 8. Department of Medicine, Division of Infectious Diseases, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA.
Abstract
OBJECTIVES: The objective was to compare the effectiveness of physician judgment and an electronic algorithmic alert to identify pediatric patients with severe sepsis/septic shock in a pediatric emergency department (ED). METHODS: This was an observational cohort study of patients older than 56 days with fever or hypothermia. All patients were evaluated for potential sepsis in real time by the ED clinical team. An electronic algorithmic alert was retrospectively applied to identify patients with potential sepsis independent of physician judgment. The primary outcome was the proportion of patients correctly identified with severe sepsis/septic shock defined by consensus criteria. Test characteristics were determined and receiver operating characteristic (ROC) curves were compared. RESULTS: Of 19,524 eligible patient visits, 88 patients developed consensus-confirmed severe sepsis or septic shock. Physician judgment identified 159 and the algorithmic alert identified 3,301 patients with potential sepsis. Physician judgment had sensitivity of 72.7% (95% confidence interval [CI] = 72.1% to 73.4%) and specificity of 99.5% (95% CI = 99.4% to 99.6%); the algorithmic alert had sensitivity of 92.1% (95% CI = 91.7% to 92.4%) and specificity of 83.4% (95% CI = 82.9% to 83.9%) for severe sepsis/septic shock. There was no significant difference in the area under the ROC curve for physician judgment (0.86, 95% CI = 0.81 to 0.91) or the algorithm (0.88, 95% CI = 0.85 to 0.91; p = 0.54). A combination method using either positive physician judgment or an algorithmic alert improved sensitivity to 96.6% and specificity to 83.3%. A sequential approach, in which positive identification by the algorithmic alert was then confirmed by physician judgment, achieved 68.2% sensitivity and 99.6% specificity. Positive and negative predictive values for physician judgment versus algorithmic alert were 40.3% versus 2.5% and 99.88% versus 99.96%, respectively. CONCLUSIONS: The electronic algorithmic alert was more sensitive but less specific than physician judgment for recognition of pediatric severe sepsis and septic shock. These findings can help to guide institutions in selecting pediatric sepsis recognition methods based on institutional needs and priorities.
OBJECTIVES: The objective was to compare the effectiveness of physician judgment and an electronic algorithmic alert to identify pediatric patients with severe sepsis/septic shock in a pediatric emergency department (ED). METHODS: This was an observational cohort study of patients older than 56 days with fever or hypothermia. All patients were evaluated for potential sepsis in real time by the ED clinical team. An electronic algorithmic alert was retrospectively applied to identify patients with potential sepsis independent of physician judgment. The primary outcome was the proportion of patients correctly identified with severe sepsis/septic shock defined by consensus criteria. Test characteristics were determined and receiver operating characteristic (ROC) curves were compared. RESULTS: Of 19,524 eligible patient visits, 88 patients developed consensus-confirmed severe sepsis or septic shock. Physician judgment identified 159 and the algorithmic alert identified 3,301 patients with potential sepsis. Physician judgment had sensitivity of 72.7% (95% confidence interval [CI] = 72.1% to 73.4%) and specificity of 99.5% (95% CI = 99.4% to 99.6%); the algorithmic alert had sensitivity of 92.1% (95% CI = 91.7% to 92.4%) and specificity of 83.4% (95% CI = 82.9% to 83.9%) for severe sepsis/septic shock. There was no significant difference in the area under the ROC curve for physician judgment (0.86, 95% CI = 0.81 to 0.91) or the algorithm (0.88, 95% CI = 0.85 to 0.91; p = 0.54). A combination method using either positive physician judgment or an algorithmic alert improved sensitivity to 96.6% and specificity to 83.3%. A sequential approach, in which positive identification by the algorithmic alert was then confirmed by physician judgment, achieved 68.2% sensitivity and 99.6% specificity. Positive and negative predictive values for physician judgment versus algorithmic alert were 40.3% versus 2.5% and 99.88% versus 99.96%, respectively. CONCLUSIONS: The electronic algorithmic alert was more sensitive but less specific than physician judgment for recognition of pediatric severe sepsis and septic shock. These findings can help to guide institutions in selecting pediatric sepsis recognition methods based on institutional needs and priorities.
Authors: Scott L Weiss; Julie C Fitzgerald; Fran Balamuth; Elizabeth R Alpern; Jane Lavelle; Marianne Chilutti; Robert Grundmeier; Vinay M Nadkarni; Neal J Thomas Journal: Crit Care Med Date: 2014-11 Impact factor: 7.598
Authors: Andrea T Cruz; Andrew M Perry; Eric A Williams; Jeanine M Graf; Elizabeth R Wuestner; Binita Patel Journal: Pediatrics Date: 2011-02-21 Impact factor: 7.124
Authors: Cláudio F de Oliveira; Débora S F de Oliveira; Adriana F C Gottschald; Juliana D G Moura; Graziela A Costa; Andréa C Ventura; José Carlos Fernandes; Flávio A C Vaz; Joseph A Carcillo; Emanuel P Rivers; Eduardo J Troster Journal: Intensive Care Med Date: 2008-03-28 Impact factor: 17.440
Authors: Joe Brierley; Joseph A Carcillo; Karen Choong; Tim Cornell; Allan Decaen; Andreas Deymann; Allan Doctor; Alan Davis; John Duff; Marc-Andre Dugas; Alan Duncan; Barry Evans; Jonathan Feldman; Kathryn Felmet; Gene Fisher; Lorry Frankel; Howard Jeffries; Bruce Greenwald; Juan Gutierrez; Mark Hall; Yong Y Han; James Hanson; Jan Hazelzet; Lynn Hernan; Jane Kiff; Niranjan Kissoon; Alexander Kon; Jose Irazuzta; Jose Irazusta; John Lin; Angie Lorts; Michelle Mariscalco; Renuka Mehta; Simon Nadel; Trung Nguyen; Carol Nicholson; Mark Peters; Regina Okhuysen-Cawley; Tom Poulton; Monica Relves; Agustin Rodriguez; Ranna Rozenfeld; Eduardo Schnitzler; Tom Shanley; Saraswati Kache; Sara Skache; Peter Skippen; Adalberto Torres; Bettina von Dessauer; Jacki Weingarten; Timothy Yeh; Arno Zaritsky; Bonnie Stojadinovic; Jerry Zimmerman; Aaron Zuckerberg Journal: Crit Care Med Date: 2009-02 Impact factor: 7.598
Authors: Raina Paul; Elliot Melendez; Anne Stack; Andrew Capraro; Michael Monuteaux; Mark I Neuman Journal: Pediatrics Date: 2014-04-07 Impact factor: 7.124
Authors: Fran Balamuth; Elizabeth R Alpern; Mengyuan Kan; Maya Shumyatcher; Katie Hayes; Ebbing Lautenbach; Blanca E Himes Journal: Ann Emerg Med Date: 2020-01-23 Impact factor: 5.721
Authors: Scott L Weiss; Mark J Peters; Waleed Alhazzani; Michael S D Agus; Heidi R Flori; David P Inwald; Simon Nadel; Luregn J Schlapbach; Robert C Tasker; Andrew C Argent; Joe Brierley; Joseph Carcillo; Enitan D Carrol; Christopher L Carroll; Ira M Cheifetz; Karen Choong; Jeffry J Cies; Andrea T Cruz; Daniele De Luca; Akash Deep; Saul N Faust; Claudio Flauzino De Oliveira; Mark W Hall; Paul Ishimine; Etienne Javouhey; Koen F M Joosten; Poonam Joshi; Oliver Karam; Martin C J Kneyber; Joris Lemson; Graeme MacLaren; Nilesh M Mehta; Morten Hylander Møller; Christopher J L Newth; Trung C Nguyen; Akira Nishisaki; Mark E Nunnally; Margaret M Parker; Raina M Paul; Adrienne G Randolph; Suchitra Ranjit; Lewis H Romer; Halden F Scott; Lyvonne N Tume; Judy T Verger; Eric A Williams; Joshua Wolf; Hector R Wong; Jerry J Zimmerman; Niranjan Kissoon; Pierre Tissieres Journal: Intensive Care Med Date: 2020-02 Impact factor: 17.440
Authors: Fran Balamuth; Elizabeth R Alpern; Mary Kate Abbadessa; Katie Hayes; Aileen Schast; Jane Lavelle; Julie C Fitzgerald; Scott L Weiss; Joseph J Zorc Journal: Ann Emerg Med Date: 2017-06-02 Impact factor: 5.721
Authors: Kimberly S Denicolo; Jacqueline B Corboy; Norma-Jean E Simon; Kate J Balsley; Daniel J Skarzynski; Emily C Roben; Elizabeth R Alpern Journal: Pediatr Qual Saf Date: 2021-06-23