BACKGROUND: Measuring quality of care and ranking hospitals with outcome measures poses two major methodological challenges: case-mix adjustment and variation that exists by chance. AIM: To compare methods for comparing and ranking hospitals that considers these. METHODS: The Netherlands Stroke Survey was conducted in 10 hospitals in the Netherlands, between October 2002 and May 2003, with prospective and consecutive enrollment of patients with acute brain ischaemia. Poor outcome was defined as death or disability after 1 year (modified Rankin scale of > or =3). We calculated fixed and random hospital effects on poor outcome, unadjusted and adjusted for patient characteristics. We compared the hospitals using the expected rank, a novel statistical measure incorporating the magnitude and the uncertainty of differences in outcome. RESULTS: At 1 year after stroke, 268 of the total 505 patients (53%) had a poor outcome. There were substantial differences in outcome between hospitals in unadjusted analysis (chi(2) = 48, 9 df, P < 0.0001). Adjustment for 12 confounders led to halving of the chi(2) (chi(2) = 24). The same pattern was observed in random effects analysis. Estimated performance of individual hospitals changed considerably between unadjusted and adjusted analysis. Further changes were seen with random effect estimation, especially for smaller hospitals. Ordering by expected rank led to shrinkage of the original ranks of 1-10 towards the median rank of 5.5 and to a different order of the hospitals, compared to ranking based on fixed effects. CONCLUSION: In comparing and ranking hospitals, case-mix-adjusted random effect estimates and the expected ranks are more robust alternatives to traditional fixed effect estimates and simple rankings.
BACKGROUND: Measuring quality of care and ranking hospitals with outcome measures poses two major methodological challenges: case-mix adjustment and variation that exists by chance. AIM: To compare methods for comparing and ranking hospitals that considers these. METHODS: The Netherlands Stroke Survey was conducted in 10 hospitals in the Netherlands, between October 2002 and May 2003, with prospective and consecutive enrollment of patients with acute brain ischaemia. Poor outcome was defined as death or disability after 1 year (modified Rankin scale of > or =3). We calculated fixed and random hospital effects on poor outcome, unadjusted and adjusted for patient characteristics. We compared the hospitals using the expected rank, a novel statistical measure incorporating the magnitude and the uncertainty of differences in outcome. RESULTS: At 1 year after stroke, 268 of the total 505 patients (53%) had a poor outcome. There were substantial differences in outcome between hospitals in unadjusted analysis (chi(2) = 48, 9 df, P < 0.0001). Adjustment for 12 confounders led to halving of the chi(2) (chi(2) = 24). The same pattern was observed in random effects analysis. Estimated performance of individual hospitals changed considerably between unadjusted and adjusted analysis. Further changes were seen with random effect estimation, especially for smaller hospitals. Ordering by expected rank led to shrinkage of the original ranks of 1-10 towards the median rank of 5.5 and to a different order of the hospitals, compared to ranking based on fixed effects. CONCLUSION: In comparing and ranking hospitals, case-mix-adjusted random effect estimates and the expected ranks are more robust alternatives to traditional fixed effect estimates and simple rankings.
Authors: David M Shahian; David F Torchiana; Richard J Shemin; James D Rawn; Sharon-Lise T Normand Journal: Ann Thorac Surg Date: 2005-12 Impact factor: 4.330
Authors: Wilma J M Scholte op Reimer; Diederik W J Dippel; Cees L Franke; Robert J van Oostenbrugge; Gosse de Jong; Sanne Hoeks; Maarten L Simoons Journal: Stroke Date: 2006-05-25 Impact factor: 7.914
Authors: Harlan M Krumholz; Sharon-Lise T Normand; John A Spertus; David M Shahian; Elizabeth H Bradley Journal: Health Aff (Millwood) Date: 2007 Jan-Feb Impact factor: 6.301
Authors: Charlotte Paddison; Marc Elliott; Richard Parker; Laura Staetsky; Georgios Lyratzopoulos; John L Campbell; Martin Roland Journal: BMJ Qual Saf Date: 2012-05-23 Impact factor: 7.035
Authors: Jacob K Greenberg; Margaret A Olsen; John Poe; Christopher F Dibble; Ken Yamaguchi; Michael P Kelly; Bruce L Hall; Wilson Z Ray Journal: Spine (Phila Pa 1976) Date: 2021-09-01 Impact factor: 3.241
Authors: Peter C Austin; Iris E Ceyisakar; Ewout W Steyerberg; Hester F Lingsma; Perla J Marang-van de Mheen Journal: BMC Med Res Methodol Date: 2019-06-26 Impact factor: 4.615
Authors: I E Ceyisakar; N van Leeuwen; Diederik W J Dippel; Ewout W Steyerberg; H F Lingsma Journal: BMC Med Res Methodol Date: 2021-01-06 Impact factor: 4.615
Authors: Marzyeh Amini; Nikki van Leeuwen; Frank Eijkenaar; Rob van de Graaf; Noor Samuels; Robert van Oostenbrugge; Ido R van den Wijngaard; Pieter Jan van Doormaal; Yvo B W E M Roos; Charles Majoie; Bob Roozenbeek; Diederik Dippel; James Burke; Hester F Lingsma Journal: BMC Med Res Methodol Date: 2022-04-10 Impact factor: 4.615
Authors: Sanne J den Hartog; Hester F Lingsma; Pieter-Jan van Doormaal; Jeannette Hofmeijer; Lonneke S F Yo; Charles B L M Majoie; Diederik W J Dippel; Aad van der Lugt; Bob Roozenbeek Journal: J Am Heart Assoc Date: 2021-12-20 Impact factor: 5.501
Authors: Michael S Calderwood; Ken Kleinman; Susan S Huang; Michael V Murphy; Deborah S Yokoe; Richard Platt Journal: Med Care Date: 2017-01 Impact factor: 2.983