Hashim Kareemi1, Christian Vaillancourt1,2, Hans Rosenberg1, Karine Fournier3, Krishan Yadav1,2. 1. Department of Emergency Medicine, University of Ottawa, Ottawa, Ontario, Canada. 2. Ottawa Hospital Research Institute, Ottawa, Ontario, Canada. 3. Health Sciences Library, University of Ottawa, Ottawa, Ontario, Canada.
Abstract
OBJECTIVE: Having shown promise in other medical fields, we sought to determine whether machine learning (ML) models perform better than usual care in diagnostic and prognostic prediction for emergency department (ED) patients. METHODS: In this systematic review, we searched MEDLINE, Embase, Central, and CINAHL from inception to October 17, 2019. We included studies comparing diagnostic and prognostic prediction of ED patients by ML models to usual care methods (triage-based scores, clinical prediction tools, clinician judgment) using predictor variables readily available to ED clinicians. We extracted commonly reported performance metrics of model discrimination and classification. We used the PROBAST tool for risk of bias assessment. PROSPERO registration: CRD42020158129. RESULTS: The search yielded 1,656 unique records, of which 23 studies involving 16,274,647 patients were included. In all seven diagnostic studies, ML models outperformed usual care in all performance metrics. In six studies assessing in-hospital mortality, the best-performing ML models had better discrimination (AUROC 0.74-0.94) than any clinical decision tool (AUROC 0.68-0.81). In four studies assessing hospitalization, ML models had better discrimination (AUROC 0.80-0.83) than triage-based scores (AUROC 0.68-0.82). Clinical heterogeneity precluded meta-analysis. Most studies had high risk of bias due to lack of external validation, low event rates, and insufficient reporting of calibration. CONCLUSIONS: Our review suggests that ML may have better prediction performance than usual care for ED patients with a variety of clinical presentations and outcomes. However, prediction model reporting guidelines should be followed to provide clinically applicable data. Interventional trials are needed to assess the impact of ML models on patient-centered outcomes. This article is protected by copyright. All rights reserved.
OBJECTIVE: Having shown promise in other medical fields, we sought to determine whether machine learning (ML) models perform better than usual care in diagnostic and prognostic prediction for emergency department (ED) patients. METHODS: In this systematic review, we searched MEDLINE, Embase, Central, and CINAHL from inception to October 17, 2019. We included studies comparing diagnostic and prognostic prediction of ED patients by ML models to usual care methods (triage-based scores, clinical prediction tools, clinician judgment) using predictor variables readily available to ED clinicians. We extracted commonly reported performance metrics of model discrimination and classification. We used the PROBAST tool for risk of bias assessment. PROSPERO registration: CRD42020158129. RESULTS: The search yielded 1,656 unique records, of which 23 studies involving 16,274,647 patients were included. In all seven diagnostic studies, ML models outperformed usual care in all performance metrics. In six studies assessing in-hospital mortality, the best-performing ML models had better discrimination (AUROC 0.74-0.94) than any clinical decision tool (AUROC 0.68-0.81). In four studies assessing hospitalization, ML models had better discrimination (AUROC 0.80-0.83) than triage-based scores (AUROC 0.68-0.82). Clinical heterogeneity precluded meta-analysis. Most studies had high risk of bias due to lack of external validation, low event rates, and insufficient reporting of calibration. CONCLUSIONS: Our review suggests that ML may have better prediction performance than usual care for ED patients with a variety of clinical presentations and outcomes. However, prediction model reporting guidelines should be followed to provide clinically applicable data. Interventional trials are needed to assess the impact of ML models on patient-centered outcomes. This article is protected by copyright. All rights reserved.
Authors: Ben Li; Tiam Feridooni; Cesar Cuen-Ojeda; Teruko Kishibe; Charles de Mestral; Muhammad Mamdani; Mohammed Al-Omran Journal: NPJ Digit Med Date: 2022-01-19
Authors: Amin Naemi; Thomas Schmidt; Marjan Mansourvar; Mohammad Naghavi-Behzad; Ali Ebrahimi; Uffe Kock Wiil Journal: BMJ Open Date: 2021-11-02 Impact factor: 2.692