Shagun Gupta1, Dennis T Ko2, Paymon Azizi3, Mohamed Reda Bouadjenek1, Maria Koh4, Alice Chong4, Peter C Austin3, Scott Sanner1. 1. Department of Mechanical and Industrial Engineering, University of Toronto, Ontario, Canada. 2. Schulich Heart Centre, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada; Institute for Clinical Evaluative Service (ICES), Toronto, Ontario, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada. Electronic address: dennis.ko@ices.on.ca. 3. Institute for Clinical Evaluative Service (ICES), Toronto, Ontario, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada. 4. Institute for Clinical Evaluative Service (ICES), Toronto, Ontario, Canada.
Abstract
BACKGROUND: The ability to predict readmission accurately after hospitalization for acute myocardial infarction (AMI) is limited in current statistical models. Machine-learning (ML) methods have shown improved predictive ability in various clinical contexts, but their utility in predicting readmission after hospitalization for AMI is unknown. METHODS: Using detailed clinical information collected from patients hospitalized with AMI, we evaluated 6 ML algorithms (logistic regression, naïve Bayes, support vector machines, random forest, gradient boosting, and deep neural networks) to predict readmission within 30 days and 1 year of discharge. A nested cross-validation approach was used to develop and test models. We used C-statistics to compare discriminatory capacity, whereas the Brier score was used to indicate overall model performance. Model calibration was assessed using calibration plots. RESULTS: The 30-day readmission rate was 16.3%, whereas the 1-year readmission rate was 45.1%. For 30-day readmission, the discriminative ability for the ML models was modest (C-statistic 0.641; 95% confidence interval (CI), 0.621-0.662 for gradient boosting) and did not outperform previously reported methods. For 1-year readmission, different ML models showed moderate performance, with C-statistics around 0.72. Despite modest discriminatory capabilities, the observed readmission rates were markedly higher in the tenth decile of predicted risk compared with the first decile of predicted risk for both 30-day and 1-year readmission. CONCLUSIONS: Despite including detailed clinical information and evaluating various ML methods, these models did not have better discriminatory ability to predict readmission outcomes compared with previously reported methods.
BACKGROUND: The ability to predict readmission accurately after hospitalization for acute myocardial infarction (AMI) is limited in current statistical models. Machine-learning (ML) methods have shown improved predictive ability in various clinical contexts, but their utility in predicting readmission after hospitalization for AMI is unknown. METHODS: Using detailed clinical information collected from patients hospitalized with AMI, we evaluated 6 ML algorithms (logistic regression, naïve Bayes, support vector machines, random forest, gradient boosting, and deep neural networks) to predict readmission within 30 days and 1 year of discharge. A nested cross-validation approach was used to develop and test models. We used C-statistics to compare discriminatory capacity, whereas the Brier score was used to indicate overall model performance. Model calibration was assessed using calibration plots. RESULTS: The 30-day readmission rate was 16.3%, whereas the 1-year readmission rate was 45.1%. For 30-day readmission, the discriminative ability for the ML models was modest (C-statistic 0.641; 95% confidence interval (CI), 0.621-0.662 for gradient boosting) and did not outperform previously reported methods. For 1-year readmission, different ML models showed moderate performance, with C-statistics around 0.72. Despite modest discriminatory capabilities, the observed readmission rates were markedly higher in the tenth decile of predicted risk compared with the first decile of predicted risk for both 30-day and 1-year readmission. CONCLUSIONS: Despite including detailed clinical information and evaluating various ML methods, these models did not have better discriminatory ability to predict readmission outcomes compared with previously reported methods.
Authors: Dennis T Ko; Tareq Ahmed; Peter C Austin; Warren J Cantor; Paul Dorian; Michael Goldfarb; Yanyan Gong; Michelle M Graham; Jing Gu; Nathaniel M Hawkins; Thao Huynh; Karin H Humphries; Maria Koh; Yoan Lamarche; Laurie J Lambert; Patrick R Lawler; Jean-Francois Légaré; Hung Q Ly; Feng Qiu; Ata Ur Rehman Quraishi; Derek Y So; Robert C Welsh; Harindra C Wijeysundera; Graham Wong; Andrew T Yan; Yana Gurevich Journal: CJC Open Date: 2021-05-01