Mirjam A Jenny1, Ralph Hertwig1, Selina Ackermann2, Anna S Messmer3, Julia Karakoumis2, Christian H Nickel2, Roland Bingisser2. 1. Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin-Brandenburg, Germany. 2. Department of Emergency Medicine, University Hospital Basel, Basel-Stadt, Switzerland. 3. Division of Trauma, Emergency, and Acute Medicine, King's College Hospital, Denmark Hill, London, UK.
Abstract
OBJECTIVES: Patients presenting to the emergency department (ED) with nonspecific complaints are difficult to accurately triage, risk stratify, and diagnose. This can delay appropriate treatment. The extent to which key medical outcomes are at all predictable in these patients, and which (if any) predictors are useful, has previously been unclear. To investigate these questions, we tested an array of statistical and machine learning models in a large group of patients and estimated the predictability of mortality (which occurred in 6.6% of our sample of patients), acute morbidity (58%), and presence of acute infectious disease (28.2%). METHODS: To investigate whether the best available tools can predict the three key outcomes, we fed data from a sample of 1,278 ED patients with nonspecific complaints into 17 state-of-the-art statistical and machine learning models. The patient sample stems from a diagnostic multicenter study with prospective 30-day follow-up conducted in Switzerland. Predictability of the three key medical outcomes was quantified by computing the area under the receiver operating characteristic curve (AUC) for each model. RESULTS: The models performed at different levels but, on average, the predictability of the target outcomes ranged between 0.71 and 0.82. The better models clearly outperformed physicians' intuitive judgments of how ill patients looked (AUC = 0.67 for mortality, 0.65 for morbidity, and 0.60 for infectious disease). CONCLUSIONS: Modeling techniques can be used to derive formalized models that, on average, predict the outcomes of mortality, acute morbidity, and acute infectious disease in patients with nonspecific complaints with a level of accuracy far beyond chance. The models also predicted these outcomes more accurately than did physicians' intuitive judgments of how ill the patients look; however, the latter was among the small set of best predictors for mortality and acute morbidity. These results lay the groundwork for further refining triage and risk stratification tools for patients with nonspecific complaints. More research, informed by whether the goal of a model is high sensitivity or high specificity, is needed to develop readily applicable clinical decision support tools (e.g., decision trees) that could be supported by electronic health records.
OBJECTIVES:Patients presenting to the emergency department (ED) with nonspecific complaints are difficult to accurately triage, risk stratify, and diagnose. This can delay appropriate treatment. The extent to which key medical outcomes are at all predictable in these patients, and which (if any) predictors are useful, has previously been unclear. To investigate these questions, we tested an array of statistical and machine learning models in a large group of patients and estimated the predictability of mortality (which occurred in 6.6% of our sample of patients), acute morbidity (58%), and presence of acute infectious disease (28.2%). METHODS: To investigate whether the best available tools can predict the three key outcomes, we fed data from a sample of 1,278 ED patients with nonspecific complaints into 17 state-of-the-art statistical and machine learning models. The patient sample stems from a diagnostic multicenter study with prospective 30-day follow-up conducted in Switzerland. Predictability of the three key medical outcomes was quantified by computing the area under the receiver operating characteristic curve (AUC) for each model. RESULTS: The models performed at different levels but, on average, the predictability of the target outcomes ranged between 0.71 and 0.82. The better models clearly outperformed physicians' intuitive judgments of how ill patients looked (AUC = 0.67 for mortality, 0.65 for morbidity, and 0.60 for infectious disease). CONCLUSIONS: Modeling techniques can be used to derive formalized models that, on average, predict the outcomes of mortality, acute morbidity, and acute infectious disease in patients with nonspecific complaints with a level of accuracy far beyond chance. The models also predicted these outcomes more accurately than did physicians' intuitive judgments of how ill the patients look; however, the latter was among the small set of best predictors for mortality and acute morbidity. These results lay the groundwork for further refining triage and risk stratification tools for patients with nonspecific complaints. More research, informed by whether the goal of a model is high sensitivity or high specificity, is needed to develop readily applicable clinical decision support tools (e.g., decision trees) that could be supported by electronic health records.
Authors: Jessica M Schwartz; Amanda J Moy; Sarah C Rossetti; Noémie Elhadad; Kenrick D Cato Journal: J Am Med Inform Assoc Date: 2021-03-01 Impact factor: 4.497
Authors: Marta Fernandes; Rúben Mendes; Susana M Vieira; Francisca Leite; Carlos Palos; Alistair Johnson; Stan Finkelstein; Steven Horng; Leo Anthony Celi Journal: PLoS One Date: 2020-04-02 Impact factor: 3.240