Jeremy M Kahn1, Andrew A Kramer, Gordon D Rubenfeld. 1. Division of Pulmonary & Critical Care, Harborview Medical Center, University of Washington, Seattle WA, USA. jkahn@cceb.med.upenn.edu
Abstract
BACKGROUND:Transferring critically ill patients to other acute care hospitals may artificially impact benchmarking measures. We sought to quantify the effect of out-of-hospital transfers on the standardized mortality ratio (SMR), an outcome-based measure of ICU performance. METHODS: We performed a cohort study and Monte Carlo simulation using data from 85 ICUs participating in the acute physiology and chronic health evaluation (APACHE) clinical information system from 2002 to 2003. The SMR (observed divided by expected hospital mortality) was calculated for each ICU using APACHE IV risk adjustment. A set number of patients was randomly assigned to be transferred out alive rather than experience their original outcome. The SMR was recalculated, and the mean simulated SMR was compared to the original. RESULTS: The mean (+/- SD) baseline SMR was 1.06 +/- 0.19. In the simulation, increasing the number of transfers by 2% and 6% over baseline decreased the SMR by 0.10 +/- 0.03 and 0.14 +/- 0.03, respectively. At a 2% increase, 27 ICUs had a decrease in SMR of > 0.10, and two ICUs had a decrease in SMR of > 0.20. Transferring only one additional patient per month was enough to create a bias of > 0.1 in 27 ICUs. CONCLUSIONS: Increasing the number of acute care transfers by a small amount can significantly bias the SMR, leading to incorrect inference about ICU quality. Sensitivity to the variation in hospital discharge practices greatly limits the use of the SMR as a quality measure.
RCT Entities:
BACKGROUND:Transferring critically illpatients to other acute care hospitals may artificially impact benchmarking measures. We sought to quantify the effect of out-of-hospital transfers on the standardized mortality ratio (SMR), an outcome-based measure of ICU performance. METHODS: We performed a cohort study and Monte Carlo simulation using data from 85 ICUs participating in the acute physiology and chronic health evaluation (APACHE) clinical information system from 2002 to 2003. The SMR (observed divided by expected hospital mortality) was calculated for each ICU using APACHE IV risk adjustment. A set number of patients was randomly assigned to be transferred out alive rather than experience their original outcome. The SMR was recalculated, and the mean simulated SMR was compared to the original. RESULTS: The mean (+/- SD) baseline SMR was 1.06 +/- 0.19. In the simulation, increasing the number of transfers by 2% and 6% over baseline decreased the SMR by 0.10 +/- 0.03 and 0.14 +/- 0.03, respectively. At a 2% increase, 27 ICUs had a decrease in SMR of > 0.10, and two ICUs had a decrease in SMR of > 0.20. Transferring only one additional patient per month was enough to create a bias of > 0.1 in 27 ICUs. CONCLUSIONS: Increasing the number of acute care transfers by a small amount can significantly bias the SMR, leading to incorrect inference about ICU quality. Sensitivity to the variation in hospital discharge practices greatly limits the use of the SMR as a quality measure.
Authors: Christian P Schneider; Carol Seyboth; Markus Vilsmaier; Helmut Küchenhoff; Benjamin Hofner; Karl-Walter Jauch; Wolfgang H Hartl Journal: World J Surg Date: 2009-01 Impact factor: 3.352
Authors: Jeremy M Kahn; Nicholas S Hill; Craig M Lilly; Derek C Angus; Judith Jacobi; Gordon D Rubenfeld; Jeffrey M Rothschild; Anne E Sales; Damon C Scales; James A L Mathers Journal: Chest Date: 2011-07 Impact factor: 9.410
Authors: Christopher W Seymour; Thomas D Rea; Jeremy M Kahn; Allan J Walkey; Donald M Yealy; Derek C Angus Journal: Am J Respir Crit Care Med Date: 2012-10-18 Impact factor: 21.405
Authors: Meeta Prasad; Theodore J Iwashyna; Jason D Christie; Andrew A Kramer; Jeffrey H Silber; Kevin G Volpp; Jeremy M Kahn Journal: Crit Care Med Date: 2009-09 Impact factor: 7.598