Benjamin Y Gravesteijn1, Daan Nieboer2, Ari Ercole3, Hester F Lingsma2, David Nelson4, Ben van Calster5, Ewout W Steyerberg6. 1. Departments of Public Health, Erasmus MC - University Medical Centre Rotterdam, Postbus 2040, 3000 CA, Rotterdam, the Netherlands. Electronic address: b.gravesteijn@erasmusmc.nl. 2. Departments of Public Health, Erasmus MC - University Medical Centre Rotterdam, Rotterdam, the Netherlands. 3. Division of Anaesthesia, University of Cambridge, Cambridge, United Kingdom. 4. Department of Physiology and Pharmacology, Section of Perioperative Medicine and Intensive Care, Karolinska Institutet, Stockholm, Sweden. 5. Department of Development and Regeneration, KU Leuven, Belgium; Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, the Netherlands. 6. Departments of Public Health, Erasmus MC - University Medical Centre Rotterdam, Rotterdam, the Netherlands; Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, the Netherlands.
Abstract
OBJECTIVE: We aimed to explore the added value of common machine learning (ML) algorithms for prediction of outcome for moderate and severe traumatic brain injury. STUDY DESIGN AND SETTING: We performed logistic regression (LR), lasso regression, and ridge regression with key baseline predictors in the IMPACT-II database (15 studies, n = 11,022). ML algorithms included support vector machines, random forests, gradient boosting machines, and artificial neural networks and were trained using the same predictors. To assess generalizability of predictions, we performed internal, internal-external, and external validation on the recent CENTER-TBI study (patients with Glasgow Coma Scale <13, n = 1,554). Both calibration (calibration slope/intercept) and discrimination (area under the curve) was quantified. RESULTS: In the IMPACT-II database, 3,332/11,022 (30%) died and 5,233(48%) had unfavorable outcome (Glasgow Outcome Scale less than 4). In the CENTER-TBI study, 348/1,554(29%) died and 651(54%) had unfavorable outcome. Discrimination and calibration varied widely between the studies and less so between the studied algorithms. The mean area under the curve was 0.82 for mortality and 0.77 for unfavorable outcomes in the CENTER-TBI study. CONCLUSION: ML algorithms may not outperform traditional regression approaches in a low-dimensional setting for outcome prediction after moderate or severe traumatic brain injury. Similar to regression-based prediction models, ML algorithms should be rigorously validated to ensure applicability to new populations.
OBJECTIVE: We aimed to explore the added value of common machine learning (ML) algorithms for prediction of outcome for moderate and severe traumatic brain injury. STUDY DESIGN AND SETTING: We performed logistic regression (LR), lasso regression, and ridge regression with key baseline predictors in the IMPACT-II database (15 studies, n = 11,022). ML algorithms included support vector machines, random forests, gradient boosting machines, and artificial neural networks and were trained using the same predictors. To assess generalizability of predictions, we performed internal, internal-external, and external validation on the recent CENTER-TBI study (patients with Glasgow Coma Scale <13, n = 1,554). Both calibration (calibration slope/intercept) and discrimination (area under the curve) was quantified. RESULTS: In the IMPACT-II database, 3,332/11,022 (30%) died and 5,233(48%) had unfavorable outcome (Glasgow Outcome Scale less than 4). In the CENTER-TBI study, 348/1,554(29%) died and 651(54%) had unfavorable outcome. Discrimination and calibration varied widely between the studies and less so between the studied algorithms. The mean area under the curve was 0.82 for mortality and 0.77 for unfavorable outcomes in the CENTER-TBI study. CONCLUSION:ML algorithms may not outperform traditional regression approaches in a low-dimensional setting for outcome prediction after moderate or severe traumatic brain injury. Similar to regression-based prediction models, ML algorithms should be rigorously validated to ensure applicability to new populations.
Authors: Shubhayu Bhattacharyay; Ioan Milosevic; Lindsay Wilson; David K Menon; Robert D Stevens; Ewout W Steyerberg; David W Nelson; Ari Ercole Journal: PLoS One Date: 2022-07-05 Impact factor: 3.752
Authors: Jan Claassen; Yama Akbari; Sheila Alexander; Mary Kay Bader; Kathleen Bell; Thomas P Bleck; Melanie Boly; Jeremy Brown; Sherry H-Y Chou; Michael N Diringer; Brian L Edlow; Brandon Foreman; Joseph T Giacino; Olivia Gosseries; Theresa Green; David M Greer; Daniel F Hanley; Jed A Hartings; Raimund Helbok; J Claude Hemphill; H E Hinson; Karen Hirsch; Theresa Human; Michael L James; Nerissa Ko; Daniel Kondziella; Sarah Livesay; Lori K Madden; Shraddha Mainali; Stephan A Mayer; Victoria McCredie; Molly M McNett; Geert Meyfroidt; Martin M Monti; Susanne Muehlschlegel; Santosh Murthy; Paul Nyquist; DaiWai M Olson; J Javier Provencio; Eric Rosenthal; Gisele Sampaio Silva; Simone Sarasso; Nicholas D Schiff; Tarek Sharshar; Lori Shutter; Robert D Stevens; Paul Vespa; Walter Videtta; Amy Wagner; Wendy Ziai; John Whyte; Elizabeth Zink; Jose I Suarez Journal: Neurocrit Care Date: 2021-07-08 Impact factor: 3.210