Laurent G Glance1, Andrew W Dick2, Turner M Osler3, Dana Mukamel4. 1. University of Rochester, School of Medicine and Dentistry, 601 Elmwood Avenue, Rochester, NY, 14642, USA. Laurent_Glance@urmc.rochester.edu. 2. University of Rochester, School of Medicine and Dentistry, 601 Elmwood Avenue, Rochester, NY, 14642, USA. 3. University of Vermont, Medical College, USA. 4. University of California, Department of Medicine, Irvine 100 Theory, Irvine CA 92697-5800, USA.
Abstract
OBJECTIVE: To determine whether hierarchical modeling agrees with conventional logistic regression modeling on the identity of ICU quality outliers within a large multi-institutional database. DESIGN: Retrospective database analysis. SETTING AND PATIENTS: Subset of the Project IMPACT database consisting of 40435 adult patients admitted to surgical, medical, and mixed surgical-medical ICUs ( n=55) between 1997 and 1999 who met inclusion criteria for SAPS II. MEASUREMENTS AND RESULTS: The SAPS II score was customized to this database using conventional logistic regression and using a hierarchical (random coefficients) model. Both models exhibited excellent discrimination ( Cstatistic) and calibration (Hosmer-Lemeshow statistic). The hierarchical and nonhierarchical models had C statistics of.870 and.865, and HL statistics of 3.71 ( p>.88, df=8) and 8.94 ( p>.35, df=8), respectively. Since the random effects component of the hierarchical model accounts for between-hospital variability, only the fixed-effects coefficients were used to calculate the expected mortality rate based on the hierarchical model. The ratio and 95% confidence intervals of the observed to expected mortality rate were calculated using both models for each ICU. ICUs whose observed/expected ratio was either less than 1 or greater than 1, and whose 95% confidence interval did not include 1 were labeled as either high-performance or low-performance outliers, respectively. Analysis using kappa statistic revealed almost perfect agreement between the two models (nonhierarchical vs. hierarchical) on the identity of ICU quality outliers. CONCLUSIONS: Models obtained by customizing SAPS II using a nonhierarchical and a hierarchical approach exhibit excellent agreement on the identity of ICU quality outliers.
OBJECTIVE: To determine whether hierarchical modeling agrees with conventional logistic regression modeling on the identity of ICU quality outliers within a large multi-institutional database. DESIGN: Retrospective database analysis. SETTING AND PATIENTS: Subset of the Project IMPACT database consisting of 40435 adult patients admitted to surgical, medical, and mixed surgical-medical ICUs ( n=55) between 1997 and 1999 who met inclusion criteria for SAPS II. MEASUREMENTS AND RESULTS: The SAPS II score was customized to this database using conventional logistic regression and using a hierarchical (random coefficients) model. Both models exhibited excellent discrimination ( Cstatistic) and calibration (Hosmer-Lemeshow statistic). The hierarchical and nonhierarchical models had C statistics of.870 and.865, and HL statistics of 3.71 ( p>.88, df=8) and 8.94 ( p>.35, df=8), respectively. Since the random effects component of the hierarchical model accounts for between-hospital variability, only the fixed-effects coefficients were used to calculate the expected mortality rate based on the hierarchical model. The ratio and 95% confidence intervals of the observed to expected mortality rate were calculated using both models for each ICU. ICUs whose observed/expected ratio was either less than 1 or greater than 1, and whose 95% confidence interval did not include 1 were labeled as either high-performance or low-performance outliers, respectively. Analysis using kappa statistic revealed almost perfect agreement between the two models (nonhierarchical vs. hierarchical) on the identity of ICU quality outliers. CONCLUSIONS: Models obtained by customizing SAPS II using a nonhierarchical and a hierarchical approach exhibit excellent agreement on the identity of ICU quality outliers.
Authors: R Rivera-Fernández; G Vázquez-Mata; M Bravo; E Aguayo-Hoyos; J Zimmerman; D Wagner; W Knaus Journal: Intensive Care Med Date: 1998-06 Impact factor: 17.440
Authors: Edward Abraham; Peter Andrews; Massimo Antonelli; Laurent Brochard; Christian Brun-Buisson; Geoffrey Dobb; Jean-Yves Fagon; Johan Groeneveld; Jordi Mancebo; Philipp Metnitz; Stefano Nava; Michael Pinsky; Peter Radermacher; Marco Ranieri; Christian Richard; Robert Tasker; Benoit Vallet Journal: Intensive Care Med Date: 2004-06-26 Impact factor: 17.440
Authors: Paola D'Errigo; Maria E Tosti; Danilo Fusco; Carlo A Perucci; Fulvia Seccareccia Journal: BMC Med Res Methodol Date: 2007-07-03 Impact factor: 4.615