Sung Min Cho1, Peter C Austin2, Heather J Ross3, Husam Abdel-Qadir4, Davide Chicco5, George Tomlinson6, Cameron Taheri1, Farid Foroutan7, Patrick R Lawler8, Filio Billia9, Anthony Gramolini1, Slava Epelman8, Bo Wang10, Douglas S Lee11. 1. Ted Rogers Centre for Heart Research, Toronto, Ontario, Canada; University of Toronto, Toronto, Ontario, Canada. 2. Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada; Institute for Health Policy, Management and Evaluation, Toronto, Ontario, Canada; University of Toronto, Toronto, Ontario, Canada. 3. Ted Rogers Centre for Heart Research, Toronto, Ontario, Canada; Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada; University of Toronto, Toronto, Ontario, Canada. 4. Ted Rogers Centre for Heart Research, Toronto, Ontario, Canada; Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada; Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada; Institute for Health Policy, Management and Evaluation, Toronto, Ontario, Canada; Women's College Hospital, Toronto, Ontario, Canada; University of Toronto, Toronto, Ontario, Canada. 5. University of Toronto, Toronto, Ontario, Canada. 6. Institute for Health Policy, Management and Evaluation, Toronto, Ontario, Canada; Biostatistics Research Unit, University Health Network, Toronto, Ontario, Canada; University of Toronto, Toronto, Ontario, Canada. 7. Ted Rogers Centre for Heart Research, Toronto, Ontario, Canada. 8. Ted Rogers Centre for Heart Research, Toronto, Ontario, Canada; Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada; Toronto General Hospital Research Institute, Toronto, Ontario, Canada; University of Toronto, Toronto, Ontario, Canada. 9. Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada; Toronto General Hospital Research Institute, Toronto, Ontario, Canada; University of Toronto, Toronto, Ontario, Canada. 10. Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada; University of Toronto, Toronto, Ontario, Canada. 11. Ted Rogers Centre for Heart Research, Toronto, Ontario, Canada; Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada; Institute for Clinical Evaluative Sciences, Toronto, Ontario, Canada; Institute for Health Policy, Management and Evaluation, Toronto, Ontario, Canada; Toronto General Hospital Research Institute, Toronto, Ontario, Canada; University of Toronto, Toronto, Ontario, Canada. Electronic address: dlee@ices.on.ca.
Abstract
BACKGROUND: Machine learning (ML) methods are increasingly used in addition to conventional statistical modelling (CSM) for predicting readmission and mortality in patients with myocardial infarction (MI). However, the two approaches have not been systematically compared across studies of prognosis in patients with MI. METHODS: Following PRISMA guidelines, we systematically reviewed the literature via Medline, EPub, Cochrane Central, Embase, Inspec, ACM Digital Library, and Web of Science. Eligible studies included primary research articles published from January 2000 to March 2020, comparing ML and CSM for prognostication after MI. RESULTS: Of 7,348 articles, 112 underwent full-text review, with the final set composed of 24 articles representing 374,365 patients. ML methods included artificial neural networks (n = 12 studies), random forests (n = 11), decision trees (n = 8), support vector machines (n = 8), and Bayesian techniques (n = 7). CSM included logistic regression (n = 19 studies), existing CSM-derived risk scores (n = 12), and Cox regression (n = 2). Thirteen of 19 studies examining mortality reported higher C-indexes with the use of ML compared with CSM. One study examined readmissions at 2 different time points, with C-indexes that were higher for ML than CSM. Across all studies, a total of 29 comparisons were performed, but the majority (n = 26, 90%) found small (< 0.05) absolute differences in the C-index between ML and CSM. With the use of a modified CHARMS checklist, sources of bias were identifiable in the majority of studies, and only 2 were externally validated. CONCLUSION: Although ML algorithms tended to have higher C-indexes than CSM for predicting death or readmission after MI, these studies exhibited threats to internal validity and were often unvalidated. Further comparisons are needed, with adherence to clinical quality standards for prognosis research. (Trial registration: PROSPERO CRD42019134896).
BACKGROUND: Machine learning (ML) methods are increasingly used in addition to conventional statistical modelling (CSM) for predicting readmission and mortality in patients with myocardial infarction (MI). However, the two approaches have not been systematically compared across studies of prognosis in patients with MI. METHODS: Following PRISMA guidelines, we systematically reviewed the literature via Medline, EPub, Cochrane Central, Embase, Inspec, ACM Digital Library, and Web of Science. Eligible studies included primary research articles published from January 2000 to March 2020, comparing ML and CSM for prognostication after MI. RESULTS: Of 7,348 articles, 112 underwent full-text review, with the final set composed of 24 articles representing 374,365 patients. ML methods included artificial neural networks (n = 12 studies), random forests (n = 11), decision trees (n = 8), support vector machines (n = 8), and Bayesian techniques (n = 7). CSM included logistic regression (n = 19 studies), existing CSM-derived risk scores (n = 12), and Cox regression (n = 2). Thirteen of 19 studies examining mortality reported higher C-indexes with the use of ML compared with CSM. One study examined readmissions at 2 different time points, with C-indexes that were higher for ML than CSM. Across all studies, a total of 29 comparisons were performed, but the majority (n = 26, 90%) found small (< 0.05) absolute differences in the C-index between ML and CSM. With the use of a modified CHARMS checklist, sources of bias were identifiable in the majority of studies, and only 2 were externally validated. CONCLUSION: Although ML algorithms tended to have higher C-indexes than CSM for predicting death or readmission after MI, these studies exhibited threats to internal validity and were often unvalidated. Further comparisons are needed, with adherence to clinical quality standards for prognosis research. (Trial registration: PROSPERO CRD42019134896).