Robert Chen1,2, Walter F Stewart3, Jimeng Sun2, Kenney Ng4, Xiaowei Yan1. 1. Research, Sutter Health Research, Walnut Creek, CA (R.C., X.Y.). 2. School of Computational Science and Engineering, Georgia Institute of Technology, Atlanta, (R.C., J.S.). 3. Step2Works, Orinda, CA (W.F.S.). 4. Center for Computational Health, IBM Research, T.J. Watson Research Center, Yorktown Heights, NY (K.N.).
Abstract
BACKGROUND: We determined the impact of data volume and diversity and training conditions on recurrent neural network methods compared with traditional machine learning methods. METHODS AND RESULTS: Using longitudinal electronic health record data, we assessed the relative performance of machine learning models trained to detect a future diagnosis of heart failure in primary care patients. Model performance was assessed in relation to data parameters defined by the combination of different data domains (data diversity), the number of patient records in the training data set (data quantity), the number of encounters per patient (data density), the prediction window length, and the observation window length (ie, the time period before the prediction window that is the source of features for prediction). Data on 4370 incident heart failure cases and 30 132 group-matched controls were used. Recurrent neural network model performance was superior under a variety of conditions that included (1) when data were less diverse (eg, a single data domain like medication or vital signs) given the same training size; (2) as data quantity increased; (3) as density increased; (4) as the observation window length increased; and (5) as the prediction window length decreased. When all data domains were used, the performance of recurrent neural network models increased in relation to the quantity of data used (ie, up to 100% of the data). When data are sparse (ie, fewer features or low dimension), model performance is lower, but a much smaller training set size is required to achieve optimal performance compared with conditions where data are more diverse and includes more features. CONCLUSIONS: Recurrent neural networks are effective for predicting a future diagnosis of heart failure given sufficient training set size. Model performance appears to continue to improve in direct relation to training set size.
BACKGROUND: We determined the impact of data volume and diversity and training conditions on recurrent neural network methods compared with traditional machine learning methods. METHODS AND RESULTS: Using longitudinal electronic health record data, we assessed the relative performance of machine learning models trained to detect a future diagnosis of heart failure in primary care patients. Model performance was assessed in relation to data parameters defined by the combination of different data domains (data diversity), the number of patient records in the training data set (data quantity), the number of encounters per patient (data density), the prediction window length, and the observation window length (ie, the time period before the prediction window that is the source of features for prediction). Data on 4370 incident heart failure cases and 30 132 group-matched controls were used. Recurrent neural network model performance was superior under a variety of conditions that included (1) when data were less diverse (eg, a single data domain like medication or vital signs) given the same training size; (2) as data quantity increased; (3) as density increased; (4) as the observation window length increased; and (5) as the prediction window length decreased. When all data domains were used, the performance of recurrent neural network models increased in relation to the quantity of data used (ie, up to 100% of the data). When data are sparse (ie, fewer features or low dimension), model performance is lower, but a much smaller training set size is required to achieve optimal performance compared with conditions where data are more diverse and includes more features. CONCLUSIONS: Recurrent neural networks are effective for predicting a future diagnosis of heart failure given sufficient training set size. Model performance appears to continue to improve in direct relation to training set size.
Entities:
Keywords:
diagnosis; electronic health records; heart failure; machine learning; mortality
Authors: Sanjiv J Shah; Daniel H Katz; Senthil Selvaraj; Michael A Burke; Clyde W Yancy; Mihai Gheorghiade; Robert O Bonow; Chiang-Ching Huang; Rahul C Deo Journal: Circulation Date: 2014-11-14 Impact factor: 29.690
Authors: Ana C Alba; Thomas Agoritsas; Milosz Jankowski; Delphine Courvoisier; Stephen D Walter; Gordon H Guyatt; Heather J Ross Journal: Circ Heart Fail Date: 2013-07-25 Impact factor: 8.790
Authors: Jerry H Gurwitz; David J Magid; David H Smith; Robert J Goldberg; David D McManus; Larry A Allen; Jane S Saczynski; Micah L Thorp; Grace Hsu; Sue Hee Sung; Alan S Go Journal: Am J Med Date: 2013-03-14 Impact factor: 4.965
Authors: Michael R Mathis; Milo C Engoren; Hyeon Joo; Michael D Maile; Keith D Aaronson; Michael L Burns; Michael W Sjoding; Nicholas J Douville; Allison M Janda; Yaokun Hu; Kayvan Najarian; Sachin Kheterpal Journal: Anesth Analg Date: 2020-05 Impact factor: 5.108