Haijun Zhai1, Patrick Brady2, Qi Li1, Todd Lingren1, Yizhao Ni1, Derek S Wheeler3, Imre Solti4. 1. Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA. 2. Division of Hospital Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA; James M. Anderson Center for Health Systems Excellence, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA. 3. Division of Critical Care Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA. 4. Division of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA; James M. Anderson Center for Health Systems Excellence, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA. Electronic address: imre.solti@cchmc.org.
Abstract
BACKGROUND: Early warning scores (EWS) are designed to identify early clinical deterioration by combining physiologic and/or laboratory measures to generate a quantified score. Current EWS leverage only a small fraction of Electronic Health Record (EHR) content. The planned widespread implementation of EHRs brings the promise of abundant data resources for prediction purposes. The three specific aims of our research are: (1) to develop an EHR-based automated algorithm to predict the need for Pediatric Intensive Care Unit (PICU) transfer in the first 24h of admission; (2) to evaluate the performance of the new algorithm on a held-out test data set; and (3) to compare the effectiveness of the new algorithm's with those of two published Pediatric Early Warning Scores (PEWS). METHODS: The cases were comprised of 526 encounters with 24-h Pediatric Intensive Care Unit (PICU) transfer. In addition to the cases, we randomly selected 6772 control encounters from 62516 inpatient admissions that were never transferred to the PICU. We used 29 variables in a logistic regression and compared our algorithm against two published PEWS on a held-out test data set. RESULTS: The logistic regression algorithm achieved 0.849 (95% CI 0.753-0.945) sensitivity, 0.859 (95% CI 0.850-0.868) specificity and 0.912 (95% CI 0.905-0.919) area under the curve (AUC) in the test set. Our algorithm's AUC was significantly higher, by 11.8 and 22.6% in the test set, than two published PEWS. CONCLUSION: The novel algorithm achieved higher sensitivity, specificity, and AUC than the two PEWS reported in the literature.
BACKGROUND: Early warning scores (EWS) are designed to identify early clinical deterioration by combining physiologic and/or laboratory measures to generate a quantified score. Current EWS leverage only a small fraction of Electronic Health Record (EHR) content. The planned widespread implementation of EHRs brings the promise of abundant data resources for prediction purposes. The three specific aims of our research are: (1) to develop an EHR-based automated algorithm to predict the need for Pediatric Intensive Care Unit (PICU) transfer in the first 24h of admission; (2) to evaluate the performance of the new algorithm on a held-out test data set; and (3) to compare the effectiveness of the new algorithm's with those of two published Pediatric Early Warning Scores (PEWS). METHODS: The cases were comprised of 526 encounters with 24-h Pediatric Intensive Care Unit (PICU) transfer. In addition to the cases, we randomly selected 6772 control encounters from 62516 inpatient admissions that were never transferred to the PICU. We used 29 variables in a logistic regression and compared our algorithm against two published PEWS on a held-out test data set. RESULTS: The logistic regression algorithm achieved 0.849 (95% CI 0.753-0.945) sensitivity, 0.859 (95% CI 0.850-0.868) specificity and 0.912 (95% CI 0.905-0.919) area under the curve (AUC) in the test set. Our algorithm's AUC was significantly higher, by 11.8 and 22.6% in the test set, than two published PEWS. CONCLUSION: The novel algorithm achieved higher sensitivity, specificity, and AUC than the two PEWS reported in the literature.
Authors: Jeffrey H Silber; Robert Kaestner; Orit Even-Shoshan; Yanli Wang; Laura J Bressler Journal: Health Serv Res Date: 2010-09-28 Impact factor: 3.402
Authors: Vinay M Nadkarni; Gregory Luke Larkin; Mary Ann Peberdy; Scott M Carey; William Kaye; Mary E Mancini; Graham Nichol; Tanya Lane-Truitt; Jerry Potts; Joseph P Ornato; Robert A Berg Journal: JAMA Date: 2006-01-04 Impact factor: 56.272
Authors: Mary-Ann J Robson; Carole L Cooper; Lori A Medicus; Mary J Quintero; Stephen A Zuniga Journal: J Pediatr Nurs Date: 2012-12-28 Impact factor: 2.145
Authors: Karen M Tucker; Tracy L Brewer; Rachel B Baker; Brenda Demeritt; Michael T Vossmeyer Journal: J Spec Pediatr Nurs Date: 2009-04 Impact factor: 1.260
Authors: Christopher S Parshuram; Heather P Duncan; Ari R Joffe; Catherine A Farrell; Jacques R Lacroix; Kristen L Middaugh; James S Hutchison; David Wensley; Nadeene Blanchard; Joseph Beyene; Patricia C Parkin Journal: Crit Care Date: 2011-08-03 Impact factor: 9.097
Authors: Peter Wingrove; Winston Liaw; Jeremy Weiss; Stephen Petterson; John Maier; Andrew Bazemore Journal: Ann Fam Med Date: 2020-07 Impact factor: 5.166
Authors: Benjamin A Goldstein; Ann Marie Navar; Michael J Pencina; John P A Ioannidis Journal: J Am Med Inform Assoc Date: 2016-05-17 Impact factor: 4.497
Authors: Anoop Mayampurath; L Nelson Sanchez-Pinto; Emma Hegermiller; Amarachi Erondu; Kyle Carey; Priti Jani; Robert Gibbons; Dana Edelson; Matthew M Churpek Journal: Pediatr Crit Care Med Date: 2022-04-21 Impact factor: 3.971
Authors: Ben Wellner; Joan Grand; Elizabeth Canzone; Matt Coarr; Patrick W Brady; Jeffrey Simmons; Eric Kirkendall; Nathan Dean; Monica Kleinman; Peter Sylvester Journal: JMIR Med Inform Date: 2017-11-22
Authors: Anoop Mayampurath; Priti Jani; Yangyang Dai; Robert Gibbons; Dana Edelson; Matthew M Churpek Journal: Pediatr Crit Care Med Date: 2020-09 Impact factor: 3.971
Authors: Yizhao Ni; Andrew F Beck; Regina Taylor; Jenna Dyas; Imre Solti; Jacqueline Grupp-Phelan; Judith W Dexheimer Journal: J Am Med Inform Assoc Date: 2016-04-27 Impact factor: 4.497