Tjeerd van der Ploeg1, Daan Nieboer2, Ewout W Steyerberg2. 1. Department of Science, Medical Center Alkmaar, Wilhelminalaan 12, Alkmaar 1815 JD, The Netherlands; Department of Science, Inholland University, Bergerweg 200, Alkmaar 1817 MN, The Netherlands; Department of Public Health, Erasmus MC-University Medical Center Rotterdam, P.O. Box 2040, 3000 CA Rotterdam, The Netherlands. Electronic address: tvdploeg@quicknet.nl. 2. Department of Public Health, Erasmus MC-University Medical Center Rotterdam, P.O. Box 2040, 3000 CA Rotterdam, The Netherlands.
Abstract
BACKGROUND AND OBJECTIVE: Prediction of medical outcomes may potentially benefit from using modern statistical modeling techniques. We aimed to externally validate modeling strategies for prediction of 6-month mortality of patients suffering from traumatic brain injury (TBI) with predictor sets of increasing complexity. METHODS: We analyzed individual patient data from 15 different studies including 11,026 TBI patients. We consecutively considered a core set of predictors (age, motor score, and pupillary reactivity), an extended set with computed tomography scan characteristics, and a further extension with two laboratory measurements (glucose and hemoglobin). With each of these sets, we predicted 6-month mortality using default settings with five statistical modeling techniques: logistic regression (LR), classification and regression trees, random forests (RFs), support vector machines (SVM) and neural nets. For external validation, a model developed on one of the 15 data sets was applied to each of the 14 remaining sets. This process was repeated 15 times for a total of 630 validations. The area under the receiver operating characteristic curve (AUC) was used to assess the discriminative ability of the models. RESULTS: For the most complex predictor set, the LR models performed best (median validated AUC value, 0.757), followed by RF and support vector machine models (median validated AUC value, 0.735 and 0.732, respectively). With each predictor set, the classification and regression trees models showed poor performance (median validated AUC value, <0.7). The variability in performance across the studies was smallest for the RF- and LR-based models (inter quartile range for validated AUC values from 0.07 to 0.10). CONCLUSION: In the area of predicting mortality from TBI, nonlinear and nonadditive effects are not pronounced enough to make modern prediction methods beneficial.
BACKGROUND AND OBJECTIVE: Prediction of medical outcomes may potentially benefit from using modern statistical modeling techniques. We aimed to externally validate modeling strategies for prediction of 6-month mortality of patients suffering from traumatic brain injury (TBI) with predictor sets of increasing complexity. METHODS: We analyzed individual patient data from 15 different studies including 11,026 TBI patients. We consecutively considered a core set of predictors (age, motor score, and pupillary reactivity), an extended set with computed tomography scan characteristics, and a further extension with two laboratory measurements (glucose and hemoglobin). With each of these sets, we predicted 6-month mortality using default settings with five statistical modeling techniques: logistic regression (LR), classification and regression trees, random forests (RFs), support vector machines (SVM) and neural nets. For external validation, a model developed on one of the 15 data sets was applied to each of the 14 remaining sets. This process was repeated 15 times for a total of 630 validations. The area under the receiver operating characteristic curve (AUC) was used to assess the discriminative ability of the models. RESULTS: For the most complex predictor set, the LR models performed best (median validated AUC value, 0.757), followed by RF and support vector machine models (median validated AUC value, 0.735 and 0.732, respectively). With each predictor set, the classification and regression trees models showed poor performance (median validated AUC value, <0.7). The variability in performance across the studies was smallest for the RF- and LR-based models (inter quartile range for validated AUC values from 0.07 to 0.10). CONCLUSION: In the area of predicting mortality from TBI, nonlinear and nonadditive effects are not pronounced enough to make modern prediction methods beneficial.
Authors: Maxim J H L Mulder; Esmee Venema; Bob Roozenbeek; Joseph P Broderick; Sharon D Yeatts; Pooja Khatri; Olvert A Berkhemer; Yvo B W E M Roos; Charles B L M Majoie; Robert J van Oostenbrugge; Wim H van Zwam; Aad van der Lugt; Ewout W Steyerberg; Diederik W J Dippel; Hester F Lingsma Journal: BMJ Open Date: 2017-03-22 Impact factor: 2.692
Authors: Hendrikus J A van Os; Lucas A Ramos; Adam Hilbert; Matthijs van Leeuwen; Marianne A A van Walderveen; Nyika D Kruyt; Diederik W J Dippel; Ewout W Steyerberg; Irene C van der Schaaf; Hester F Lingsma; Wouter J Schonewille; Charles B L M Majoie; Silvia D Olabarriaga; Koos H Zwinderman; Esmee Venema; Henk A Marquering; Marieke J H Wermer Journal: Front Neurol Date: 2018-09-25 Impact factor: 4.003
Authors: Robson Luis Amorim; Louise Makarem Oliveira; Luis Marcelo Malbouisson; Marcia Mitie Nagumo; Marcela Simoes; Leandro Miranda; Edson Bor-Seng-Shu; Andre Beer-Furlan; Almir Ferreira De Andrade; Andres M Rubiano; Manoel Jacobsen Teixeira; Angelos G Kolias; Wellingson Silva Paiva Journal: Front Neurol Date: 2020-01-24 Impact factor: 4.003