Dieter H Beck1, Gary B Smith, John V Pappachan, Brian Millar. 1. Department of Anaesthesiology and Intensive Care, Charité Hospital, Humboldt University, Schumannstrasse 20-21, 10098 Berlin, Germany. dieter.beck@rz.hu-berlin.de
Abstract
OBJECTIVE: External validation of three prognostic models in adult intensive care patients in South England. DESIGN. Prospective cohort study. SETTING: Seventeen intensive care units (ICU) in the South West Thames Region in South England. PATIENTS AND PARTICIPANTS: Data of 16646 patients were analysed. INTERVENTIONS: None. MEASUREMENTS AND RESULTS: We compared directly the predictive accuracy of three prognostic models (SAPS II, APACHE II and III), using formal tests of calibration and discrimination. The external validation showed a similar pattern for all three models tested: good discrimination, but imperfect calibration. The areas under the receiver operating characteristics (ROC) curves, used to test discrimination, were 0.835 and 0.867 for APACHE II and III, and 0.852 for the SAPS II model. Model calibration was assessed by Lemeshow-Hosmer C-statistics and was Chi(2 )=232.1 for APACHE II, Chi(2 )=443.3 for APACHE III and Chi(2 )=287.5 for SAPS II. CONCLUSIONS: Disparity in case mix, a higher prevalence of outcome events and important unmeasured patient mix factors are possible sources for the decay of the models' predictive accuracy in our population. The lack of generalisability of standard prognostic models requires their validation and re-calibration before they can be applied with confidence to new populations. Customisation of existing models may become an important strategy to obtain authentic information on disease severity, which is a prerequisite for reliably measuring and comparing the quality and cost of intensive care.
OBJECTIVE: External validation of three prognostic models in adult intensive care patients in South England. DESIGN. Prospective cohort study. SETTING: Seventeen intensive care units (ICU) in the South West Thames Region in South England. PATIENTS AND PARTICIPANTS: Data of 16646 patients were analysed. INTERVENTIONS: None. MEASUREMENTS AND RESULTS: We compared directly the predictive accuracy of three prognostic models (SAPS II, APACHE II and III), using formal tests of calibration and discrimination. The external validation showed a similar pattern for all three models tested: good discrimination, but imperfect calibration. The areas under the receiver operating characteristics (ROC) curves, used to test discrimination, were 0.835 and 0.867 for APACHE II and III, and 0.852 for the SAPS II model. Model calibration was assessed by Lemeshow-Hosmer C-statistics and was Chi(2 )=232.1 for APACHE II, Chi(2 )=443.3 for APACHE III and Chi(2 )=287.5 for SAPS II. CONCLUSIONS: Disparity in case mix, a higher prevalence of outcome events and important unmeasured patient mix factors are possible sources for the decay of the models' predictive accuracy in our population. The lack of generalisability of standard prognostic models requires their validation and re-calibration before they can be applied with confidence to new populations. Customisation of existing models may become an important strategy to obtain authentic information on disease severity, which is a prerequisite for reliably measuring and comparing the quality and cost of intensive care.
Authors: P G Metnitz; H Vesely; A Valentin; C Popow; M Hiesmayr; K Lenz; C G Krenn; H Steltzer Journal: Crit Care Med Date: 1999-08 Impact factor: 7.598
Authors: Edward Abraham; Peter Andrews; Massimo Antonelli; Laurent Brochard; Christian Brun-Buisson; Geoffrey Dobb; Jean-Yves Fagon; Johan Groeneveld; Jordi Mancebo; Philipp Metnitz; Stefano Nava; Michael Pinsky; Peter Radermacher; Marco Ranieri; Christian Richard; Robert Tasker; Benoit Vallet Journal: Intensive Care Med Date: 2004-06-26 Impact factor: 17.440
Authors: Heidi S Bazick; Domingo Chang; Karthik Mahadevappa; Fiona K Gibbons; Kenneth B Christopher Journal: Crit Care Med Date: 2011-08 Impact factor: 7.598
Authors: Daniel Baram; Feroza Daroowalla; Ruel Garcia; Guangxiang Zhang; John J Chen; Erin Healy; Syed Ali Riaz; Paul Richman Journal: Clin Med Circ Respirat Pulm Med Date: 2008-04-18
Authors: Hans A J M Kuijsten; Sylvia Brinkman; Iwan A Meynaar; Peter E Spronk; Johan I van der Spoel; Rob J Bosman; Nicolette F de Keizer; Ameen Abu-Hanna; Dylan W de Lange Journal: Intensive Care Med Date: 2010-06-15 Impact factor: 17.440