Sabrina Siregar1, Daan Nieboer2, Yvonne Vergouwe2, Michel I M Versteegh2, Luc Noyez2, Alexander B A Vonk2, Ewout W Steyerberg2, Johanna J M Takkenberg2. 1. From the Department of Cardio-Thoracic Surgery, Leiden University Medical Center, Leiden, The Netherlands (S.S., M.I.M.V.); Department of Public Health, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands (D.N., Y.V., E.W.S.); Department of Cardio-Thoracic Surgery, Radboud University Nijmegen Medical Center, Nijmegen, The Netherlands (L.N.); Department of Cardio-Thoracic Surgery, VU Medical Center, Amsterdam, The Netherlands (A.B.A.V.); and Department of Cardio-Thoracic Surgery, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands (J.J.M.T.). s.siregar@lumc.nl. 2. From the Department of Cardio-Thoracic Surgery, Leiden University Medical Center, Leiden, The Netherlands (S.S., M.I.M.V.); Department of Public Health, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands (D.N., Y.V., E.W.S.); Department of Cardio-Thoracic Surgery, Radboud University Nijmegen Medical Center, Nijmegen, The Netherlands (L.N.); Department of Cardio-Thoracic Surgery, VU Medical Center, Amsterdam, The Netherlands (A.B.A.V.); and Department of Cardio-Thoracic Surgery, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands (J.J.M.T.).
Abstract
BACKGROUND: The predictive performance of static risk prediction models such as EuroSCORE deteriorates over time. We aimed to explore different methods for continuous updating of EuroSCORE (dynamic modeling) to improve risk prediction. METHODS AND RESULTS: Data on adult cardiac surgery from 2007 to 2012 (n=95 240) were extracted from the Netherlands Association for Cardio-Thoracic Surgery database. The logistic EuroSCORE predicting in-hospital death was updated using 6 methods: recalibrating the intercept of the logistic regression model; recalibrating the intercept and joint effects of the prognostic factors; re-estimating all prognostic factor effects, re-estimating all prognostic factor effects, and applying shrinkage of the estimates; applying a test procedure to select either of these; and a Bayesian learning strategy. Models were updated with 1 or 3 years of data, in all cardiac surgery or within operation subgroups. Performance was tested in the subsequent year according to discrimination (area under the receiver operating curve, area under the curve) and calibration (calibration slope and calibration-in-the-large). Compared with the original EuroSCORE, all updating methods resulted in improved calibration-in-the-large (range -0.17 to 0.04 versus -1.13 to -0.97, ideally 0.0). Calibration slope (range 0.92-1.15) and discrimination (area under the curve range 0.83-0.87) were similar across methods. In small subgroups, such as aortic valve replacement and aortic valve replacement+coronary artery bypass grafting, extensive updating using 1 year of data led to poorer performance than using the original EuroSCORE. The choice of updating method had little effect on benchmarking results of all cardiac surgery. CONCLUSIONS: Several methods for dynamic modeling may result in good discrimination and superior calibration compared with the original EuroSCORE. For large populations, all methods are appropriate. For smaller subgroups, it is recommended to use data from multiple years or a Bayesian approach.
BACKGROUND: The predictive performance of static risk prediction models such as EuroSCORE deteriorates over time. We aimed to explore different methods for continuous updating of EuroSCORE (dynamic modeling) to improve risk prediction. METHODS AND RESULTS: Data on adult cardiac surgery from 2007 to 2012 (n=95 240) were extracted from the Netherlands Association for Cardio-Thoracic Surgery database. The logistic EuroSCORE predicting in-hospital death was updated using 6 methods: recalibrating the intercept of the logistic regression model; recalibrating the intercept and joint effects of the prognostic factors; re-estimating all prognostic factor effects, re-estimating all prognostic factor effects, and applying shrinkage of the estimates; applying a test procedure to select either of these; and a Bayesian learning strategy. Models were updated with 1 or 3 years of data, in all cardiac surgery or within operation subgroups. Performance was tested in the subsequent year according to discrimination (area under the receiver operating curve, area under the curve) and calibration (calibration slope and calibration-in-the-large). Compared with the original EuroSCORE, all updating methods resulted in improved calibration-in-the-large (range -0.17 to 0.04 versus -1.13 to -0.97, ideally 0.0). Calibration slope (range 0.92-1.15) and discrimination (area under the curve range 0.83-0.87) were similar across methods. In small subgroups, such as aortic valve replacement and aortic valve replacement+coronary artery bypass grafting, extensive updating using 1 year of data led to poorer performance than using the original EuroSCORE. The choice of updating method had little effect on benchmarking results of all cardiac surgery. CONCLUSIONS: Several methods for dynamic modeling may result in good discrimination and superior calibration compared with the original EuroSCORE. For large populations, all methods are appropriate. For smaller subgroups, it is recommended to use data from multiple years or a Bayesian approach.
Authors: Sharon E Davis; Jeremiah R Brown; Chad Dorn; Dax Westerman; Richard J Solomon; Michael E Matheny Journal: Circ Cardiovasc Qual Outcomes Date: 2022-08-12
Authors: Lin Lawrence Guo; Stephen R Pfohl; Jason Fries; Jose Posada; Scott Lanyon Fleming; Catherine Aftandilian; Nigam Shah; Lillian Sung Journal: Appl Clin Inform Date: 2021-09-01 Impact factor: 2.762