Benjamin A Goldstein1,2, Ann Marie Navar2,3, Michael J Pencina4,2, John P A Ioannidis5,6. 1. Department of Biostatistics & Bioinformatics, Duke University, Durham, NC 27710, USA ben.goldstein@duke.edu. 2. Center for Predictive Medicine, Duke Clinical Research Institute, Duke University, Durham, NC 27710, USA. 3. Division of Cardiology at Duke University Medical Center, Duhram, NC 27710, USA. 4. Department of Biostatistics & Bioinformatics, Duke University, Durham, NC 27710, USA. 5. Department of Medicine, Stanford University, Palo Alto, CA 94305, USA. 6. Department of Health Research and Policy, and Statistics and Meta-Research Innovation Center at Stanford, Stanford University, Palo Alto, CA 94305, USA.
Abstract
OBJECTIVE: Electronic health records (EHRs) are an increasingly common data source for clinical risk prediction, presenting both unique analytic opportunities and challenges. We sought to evaluate the current state of EHR based risk prediction modeling through a systematic review of clinical prediction studies using EHR data. METHODS: We searched PubMed for articles that reported on the use of an EHR to develop a risk prediction model from 2009 to 2014. Articles were extracted by two reviewers, and we abstracted information on study design, use of EHR data, model building, and performance from each publication and supplementary documentation. RESULTS: We identified 107 articles from 15 different countries. Studies were generally very large (median sample size = 26 100) and utilized a diverse array of predictors. Most used validation techniques (n = 94 of 107) and reported model coefficients for reproducibility (n = 83). However, studies did not fully leverage the breadth of EHR data, as they uncommonly used longitudinal information (n = 37) and employed relatively few predictor variables (median = 27 variables). Less than half of the studies were multicenter (n = 50) and only 26 performed validation across sites. Many studies did not fully address biases of EHR data such as missing data or loss to follow-up. Average c-statistics for different outcomes were: mortality (0.84), clinical prediction (0.83), hospitalization (0.71), and service utilization (0.71). CONCLUSIONS: EHR data present both opportunities and challenges for clinical risk prediction. There is room for improvement in designing such studies.
OBJECTIVE: Electronic health records (EHRs) are an increasingly common data source for clinical risk prediction, presenting both unique analytic opportunities and challenges. We sought to evaluate the current state of EHR based risk prediction modeling through a systematic review of clinical prediction studies using EHR data. METHODS: We searched PubMed for articles that reported on the use of an EHR to develop a risk prediction model from 2009 to 2014. Articles were extracted by two reviewers, and we abstracted information on study design, use of EHR data, model building, and performance from each publication and supplementary documentation. RESULTS: We identified 107 articles from 15 different countries. Studies were generally very large (median sample size = 26 100) and utilized a diverse array of predictors. Most used validation techniques (n = 94 of 107) and reported model coefficients for reproducibility (n = 83). However, studies did not fully leverage the breadth of EHR data, as they uncommonly used longitudinal information (n = 37) and employed relatively few predictor variables (median = 27 variables). Less than half of the studies were multicenter (n = 50) and only 26 performed validation across sites. Many studies did not fully address biases of EHR data such as missing data or loss to follow-up. Average c-statistics for different outcomes were: mortality (0.84), clinical prediction (0.83), hospitalization (0.71), and service utilization (0.71). CONCLUSIONS: EHR data present both opportunities and challenges for clinical risk prediction. There is room for improvement in designing such studies.
Authors: Christie M Atchison; Shilpa Arlikar; Ernest Amankwah; Irmel Ayala; Laurie Barrett; Brian R Branchford; Michael Streiff; Clifford Takemoto; Neil A Goldenberg Journal: J Pediatr Date: 2014-07-23 Impact factor: 4.406
Authors: Elizabeth A O'Leary; Sameer Y Desale; William S Yi; Kari A Fujita; Conor F Hynes; Suma K Chandra; Jack A Sava Journal: Am Surg Date: 2014-06 Impact factor: 0.688
Authors: John R Saltzman; Ying P Tabak; Brian H Hyett; Xiaowu Sun; Anne C Travis; Richard S Johannes Journal: Gastrointest Endosc Date: 2011-09-10 Impact factor: 9.427
Authors: Wuyang Dai; Theodora S Brisimi; William G Adams; Theofanie Mela; Venkatesh Saligrama; Ioannis Ch Paschalidis Journal: Int J Med Inform Date: 2014-10-16 Impact factor: 4.046
Authors: Amit G Singal; Robert S Rahimi; Christopher Clark; Ying Ma; Jennifer A Cuthbert; Don C Rockey; Ruben Amarasingham Journal: Clin Gastroenterol Hepatol Date: 2013-04-13 Impact factor: 11.382
Authors: Sunil Gupta; Truyen Tran; Wei Luo; Dinh Phung; Richard Lee Kennedy; Adam Broad; David Campbell; David Kipp; Madhu Singh; Mustafa Khasraw; Leigh Matheson; David M Ashley; Svetha Venkatesh Journal: BMJ Open Date: 2014-03-17 Impact factor: 2.692
Authors: Yirong Wu; Jun Fan; Peggy Peissig; Richard Berg; Ahmad Pahlavan Tafti; Jie Yin; Ming Yuan; David Page; Jennifer Cox; Elizabeth S Burnside Journal: Proc SPIE Int Soc Opt Eng Date: 2018-03-07
Authors: Gary E Weissman; Rebecca A Hubbard; Lyle H Ungar; Michael O Harhay; Casey S Greene; Blanca E Himes; Scott D Halpern Journal: Crit Care Med Date: 2018-07 Impact factor: 7.598
Authors: Nicholas M Pajewski; Kristin Lenoir; Brian J Wells; Jeff D Williamson; Kathryn E Callahan Journal: J Gerontol A Biol Sci Med Sci Date: 2019-10-04 Impact factor: 6.053