BACKGROUND: Hospital quality in vascular surgery is often measured using mortality. We sought to determine whether adjusting mortality for statistical reliability changes hospital quality rankings for vascular surgery. METHODS: Patients undergoing five common vascular surgery procedures (open and endovascular abdominal aortic aneurysm repair, carotid endarterectomy, lower extremity bypass, and aorto-femoral bypass) in the National Surgical Quality Improvement Project (NSQIP) in 2007 were identified (n = 14,559). For each hospital, we first calculated a ratio of observed to expected mortality (O-E ratio) using standard NSQIP techniques. We then adjusted these estimates for statistical noise using empirical Bayes methods, a technique known as reliability adjustment. We then compared rankings based on the standard O-E ratio to the rankings after reliability adjustment. RESULTS: A total of 172 hospitals reported an average adjusted mortality rate of 2.4% for the five procedures, varying from 0% to 17%. After adjusting for statistical noise using reliability adjustment, hospital mortality was greatly diminished, varying only from 1.7% to 4.1%. This adjustment for reliability had a dramatic effect on hospital rankings. Overall, 43% of hospitals were reclassified into either a higher or lower quartile of performance using traditional methods of risk-adjustment. Fifty-one percent all hospitals in the "best" quartile of performance according to traditional O-E ratios are not classified in the "best" quartile after adjusting for statistical noise. Twenty-six percent of hospitals in the "worst" quartile were no longer classified as such after adjusting for noise. CONCLUSIONS: Adjusting mortality for reliability reduces statistical noise and provides more stable estimates of hospital quality. Reliability adjustment should be standard for comparing hospital quality.
BACKGROUND: Hospital quality in vascular surgery is often measured using mortality. We sought to determine whether adjusting mortality for statistical reliability changes hospital quality rankings for vascular surgery. METHODS:Patients undergoing five common vascular surgery procedures (open and endovascular abdominal aortic aneurysm repair, carotid endarterectomy, lower extremity bypass, and aorto-femoral bypass) in the National Surgical Quality Improvement Project (NSQIP) in 2007 were identified (n = 14,559). For each hospital, we first calculated a ratio of observed to expected mortality (O-E ratio) using standard NSQIP techniques. We then adjusted these estimates for statistical noise using empirical Bayes methods, a technique known as reliability adjustment. We then compared rankings based on the standard O-E ratio to the rankings after reliability adjustment. RESULTS: A total of 172 hospitals reported an average adjusted mortality rate of 2.4% for the five procedures, varying from 0% to 17%. After adjusting for statistical noise using reliability adjustment, hospital mortality was greatly diminished, varying only from 1.7% to 4.1%. This adjustment for reliability had a dramatic effect on hospital rankings. Overall, 43% of hospitals were reclassified into either a higher or lower quartile of performance using traditional methods of risk-adjustment. Fifty-one percent all hospitals in the "best" quartile of performance according to traditional O-E ratios are not classified in the "best" quartile after adjusting for statistical noise. Twenty-six percent of hospitals in the "worst" quartile were no longer classified as such after adjusting for noise. CONCLUSIONS: Adjusting mortality for reliability reduces statistical noise and provides more stable estimates of hospital quality. Reliability adjustment should be standard for comparing hospital quality.
Authors: Terry Shih; Adam I Cole; Paul M Al-Attar; Apurba Chakrabarti; Hussein A Fardous; Peter F Helvie; Michael T Kemp; Chris Lee; Eytan Shtull-Leber; Darrell A Campbell; Michael J Englesbe Journal: Ann Surg Date: 2015-05 Impact factor: 12.969
Authors: Andrew A Gonzalez; Micah E Girotti; Terry Shih; Thomas W Wakefield; Justin B Dimick Journal: J Vasc Surg Date: 2014-03-12 Impact factor: 4.268
Authors: Hayley B Gershengorn; Theodore J Iwashyna; Colin R Cooke; Damon C Scales; Jeremy M Kahn; Hannah Wunsch Journal: Crit Care Med Date: 2012-07 Impact factor: 7.598
Authors: Zain G Hashmi; Justin B Dimick; David T Efron; Elliott R Haut; Eric B Schneider; Syed Nabeel Zafar; Diane Schwartz; Edward E Cornwell; Adil H Haider Journal: J Trauma Acute Care Surg Date: 2013-07 Impact factor: 3.313