PURPOSE: Prognostic models have been developed to estimate mortality and to compare outcomes in different intensive care units. However, these models need to be validated before their use in different populations. In this study, we assessed the performance of 3 recently developed general prognostic models (Acute Physiologic and Chronic Health Evaluation [APACHE] IV, Simplified Acute Physiology Score [SAPS] 3 and Mortality Probability Model III [MPM(0)-III]) in a population admitted at 3 medical-surgical Brazilian intensive care units. MATERIALS AND METHODS: All patients admitted from July 2008 to December 2009 were evaluated for inclusion in the study. Standardized mortality ratios were calculated for all models. Calibration was assessed by the Hosmer-Lemeshow goodness-of-fit test. Discrimination was evaluated using the area under the receiver operator curve. RESULTS: A total of 5780 patients were included. Inhospital mortality was 9.1%. Discrimination was very good for all models (area under the receiver operator curve for APACHE IV, SAPS 3 and MPM(0)-III was 0.883, 0.855 and 0.840, respectively). APACHE IV showed better discrimination than SAPS 3 and MPM(0)-III (P < .001 for both comparisons). All models calibrated poorly and overestimated hospital mortality (Hosmer-Lemeshow statistic was 53.7, 134.2, 226.6 for APACHE IV, MPM(0)-III, and SAPS 3, respectively; P < .001 for all). CONCLUSIONS: In this study, all models showed poor calibration, while discrimination was very good for all of them. As this has been a common finding in validation studies, caution is warranted when using prognostic models for benchmarking.
PURPOSE: Prognostic models have been developed to estimate mortality and to compare outcomes in different intensive care units. However, these models need to be validated before their use in different populations. In this study, we assessed the performance of 3 recently developed general prognostic models (Acute Physiologic and Chronic Health Evaluation [APACHE] IV, Simplified Acute Physiology Score [SAPS] 3 and Mortality Probability Model III [MPM(0)-III]) in a population admitted at 3 medical-surgical Brazilian intensive care units. MATERIALS AND METHODS: All patients admitted from July 2008 to December 2009 were evaluated for inclusion in the study. Standardized mortality ratios were calculated for all models. Calibration was assessed by the Hosmer-Lemeshow goodness-of-fit test. Discrimination was evaluated using the area under the receiver operator curve. RESULTS: A total of 5780 patients were included. Inhospital mortality was 9.1%. Discrimination was very good for all models (area under the receiver operator curve for APACHE IV, SAPS 3 and MPM(0)-III was 0.883, 0.855 and 0.840, respectively). APACHE IV showed better discrimination than SAPS 3 and MPM(0)-III (P < .001 for both comparisons). All models calibrated poorly and overestimated hospital mortality (Hosmer-Lemeshow statistic was 53.7, 134.2, 226.6 for APACHE IV, MPM(0)-III, and SAPS 3, respectively; P < .001 for all). CONCLUSIONS: In this study, all models showed poor calibration, while discrimination was very good for all of them. As this has been a common finding in validation studies, caution is warranted when using prognostic models for benchmarking.
Authors: Romain Pirracchio; Maya L Petersen; Marco Carone; Matthieu Resche Rigon; Sylvie Chevret; Mark J van der Laan Journal: Lancet Respir Med Date: 2014-11-24 Impact factor: 30.700
Authors: Jesse D Raffa; Alistair E W Johnson; Zach O'Brien; Tom J Pollard; Roger G Mark; Leo A Celi; David Pilcher; Omar Badawi Journal: Crit Care Med Date: 2022-03-25 Impact factor: 9.296
Authors: Kris Salaveria; Simon Smith; Yu-Hsuan Liu; Richard Bagshaw; Markus Ott; Alexandra Stewart; Matthew Law; Angus Carter; Josh Hanson Journal: Am J Trop Med Hyg Date: 2021-10-18 Impact factor: 3.707