Britt E Keuning1, Thomas Kaufmann2, Renske Wiersema1, Anders Granholm3, Ville Pettilä4, Morten Hylander Møller3,5, Christian Fynbo Christiansen6, José Castela Forte1,7, Harold Snieder8, Frederik Keus1, Rick G Pleijhuis9, Iwan C C van der Horst1,10. 1. Department of Critical Care, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands. 2. Department of Anesthesiology, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands. 3. Department of Intensive Care, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark. 4. Division of Intensive Care Medicine, Department of Anesthesiology, Intensive Care and Pain Medicine, University of Helsinki and Helsinki University Hospital, Helsinki, Finland. 5. Centre for Research in Intensive Care, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark. 6. Department of Clinical Epidemiology, Aarhus University Hospital, Aarhus, Denmark. 7. Bernoulli Institute for Mathematics, Computer Science and Artificial Intelligence, University of Groningen, Groningen, The Netherlands. 8. Department of Epidemiology, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands. 9. Department of Internal Medicine, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands. 10. Department of Intensive Care, Maastricht University Medical Center+, Maastricht University, Maastricht, The Netherlands.
Abstract
BACKGROUND: Mortality prediction models are applied in the intensive care unit (ICU) to stratify patients into different risk categories and to facilitate benchmarking. To ensure that the correct prediction models are applied for these purposes, the best performing models must be identified. As a first step, we aimed to establish a systematic review of mortality prediction models in critically ill patients. METHODS: Mortality prediction models were searched in four databases using the following criteria: developed for use in adult ICU patients in high-income countries, with mortality as primary or secondary outcome. Characteristics and performance measures of the models were summarized. Performance was presented in terms of discrimination, calibration and overall performance measures presented in the original publication. RESULTS: In total, 43 mortality prediction models were included in the final analysis. In all, 15 models were only internally validated (35%), 13 externally (30%) and 10 (23%) were both internally and externally validated by the original researchers. Discrimination was assessed in 42 models (98%). Commonly used calibration measures were the Hosmer-Lemeshow test (60%) and the calibration plot (28%). Calibration was not assessed in 11 models (26%). Overall performance was assessed in the Brier score (19%) and the Nagelkerke's R2 (4.7%). CONCLUSIONS: Mortality prediction models have varying methodology, and validation and performance of individual models differ. External validation by the original researchers is often lacking and head-to-head comparisons are urgently needed to identify the best performing mortality prediction models for guiding clinical care and research in different settings and populations.
BACKGROUND: Mortality prediction models are applied in the intensive care unit (ICU) to stratify patients into different risk categories and to facilitate benchmarking. To ensure that the correct prediction models are applied for these purposes, the best performing models must be identified. As a first step, we aimed to establish a systematic review of mortality prediction models in critically illpatients. METHODS: Mortality prediction models were searched in four databases using the following criteria: developed for use in adult ICU patients in high-income countries, with mortality as primary or secondary outcome. Characteristics and performance measures of the models were summarized. Performance was presented in terms of discrimination, calibration and overall performance measures presented in the original publication. RESULTS: In total, 43 mortality prediction models were included in the final analysis. In all, 15 models were only internally validated (35%), 13 externally (30%) and 10 (23%) were both internally and externally validated by the original researchers. Discrimination was assessed in 42 models (98%). Commonly used calibration measures were the Hosmer-Lemeshow test (60%) and the calibration plot (28%). Calibration was not assessed in 11 models (26%). Overall performance was assessed in the Brier score (19%) and the Nagelkerke's R2 (4.7%). CONCLUSIONS: Mortality prediction models have varying methodology, and validation and performance of individual models differ. External validation by the original researchers is often lacking and head-to-head comparisons are urgently needed to identify the best performing mortality prediction models for guiding clinical care and research in different settings and populations.
Authors: Orlyn C Lavilla; Khyzer B Aziz; Allison C Lure; Daniel Gipson; Diomel de la Cruz; James L Wynn Journal: Am J Respir Crit Care Med Date: 2022-01-01 Impact factor: 21.405
Authors: Salvatore Tedesco; Martina Andrulli; Markus Åkerlund Larsson; Daniel Kelly; Antti Alamäki; Suzanne Timmons; John Barton; Joan Condell; Brendan O'Flynn; Anna Nordström Journal: Int J Environ Res Public Health Date: 2021-12-04 Impact factor: 3.390
Authors: Eline G M Cox; Marisa Onrust; Madelon E Vos; Wolter Paans; Willem Dieperink; Jacqueline Koeze; Iwan C C van der Horst; Renske Wiersema Journal: Crit Care Date: 2021-11-15 Impact factor: 9.097
Authors: Lucas M Fleuren; Michele Tonutti; Daan P de Bruin; Robbert C A Lalisang; Tariq A Dam; Diederik Gommers; Olaf L Cremer; Rob J Bosman; Sebastiaan J J Vonk; Mattia Fornasa; Tomas Machado; Nardo J M van der Meer; Sander Rigter; Evert-Jan Wils; Tim Frenzel; Dave A Dongelmans; Remko de Jong; Marco Peters; Marlijn J A Kamps; Dharmanand Ramnarain; Ralph Nowitzky; Fleur G C A Nooteboom; Wouter de Ruijter; Louise C Urlings-Strop; Ellen G M Smit; D Jannet Mehagnoul-Schipper; Tom Dormans; Cornelis P C de Jager; Stefaan H A Hendriks; Evelien Oostdijk; Auke C Reidinga; Barbara Festen-Spanjer; Gert Brunnekreef; Alexander D Cornet; Walter van den Tempel; Age D Boelens; Peter Koetsier; Judith Lens; Sefanja Achterberg; Harald J Faber; A Karakus; Menno Beukema; Robert Entjes; Paul de Jong; Taco Houwert; Hidde Hovenkamp; Roberto Noorduijn Londono; Davide Quintarelli; Martijn G Scholtemeijer; Aletta A de Beer; Giovanni Cinà; Martijn Beudel; Nicolet F de Keizer; Mark Hoogendoorn; Armand R J Girbes; Willem E Herter; Paul W G Elbers; Patrick J Thoral Journal: Intensive Care Med Exp Date: 2021-06-28
Authors: Paul M E L van Dam; Noortje Zelis; Sander M J van Kuijk; Aimée E M J H Linkens; Renée A G Brüggemann; Bart Spaetgens; Iwan C C van der Horst; Patricia M Stassen Journal: Ann Med Date: 2021-12 Impact factor: 4.709