James F Reed1, Stephen A Olenchock. 1. Research Institute, St. Luke's Hospital & Health Network, Bethlehem, Pennsylvania 18015, USA.
Abstract
BACKGROUND: Quality improvement in health care, which relies on appropriate strategies to evaluate and compare provider performance, has spawned the propagation of many public report cards or provider profiles for cardiac surgery. These risk-adjusted prediction models allow for the computation of a provider's expected outcome event rate compared with observed outcome events. The aim of this study was to assess the accuracy and reliability of 5 risk-adjusted predictive models for mortality in an independent population of patients in a community hospital who underwent coronary artery bypass graft surgery. METHODS: Five nonproprietary models were selected for evaluation (Parsonnet, Canadian, Cleveland, New York, and the Northern New England). RESULTS: The C-statistic for the 5 models was 0.752, 0.693, 0.748, 0.735, and 0.722 for the Parsonnet, Canadian, Cleveland, New York, and Northern New England models respectively. The H-L c2 calibration statistics were 4.948, P =.763; 1.616, P =.899; 11.96, P =.035; 10.23, P =.249; and c2 = 12.14, P =.145 for the Parsonnet, Canadian, Cleveland, New York, and Northern New England models respectively. CONCLUSIONS: Comparing hospital-specific or surgeon-specific mortality/morbidity rates will remain a challenge. This analysis reaffirms the concept of risk-adjusting outcomes and emphasizes the importance of the risk-adjustment process for CABG surgery in a community hospital.
BACKGROUND: Quality improvement in health care, which relies on appropriate strategies to evaluate and compare provider performance, has spawned the propagation of many public report cards or provider profiles for cardiac surgery. These risk-adjusted prediction models allow for the computation of a provider's expected outcome event rate compared with observed outcome events. The aim of this study was to assess the accuracy and reliability of 5 risk-adjusted predictive models for mortality in an independent population of patients in a community hospital who underwent coronary artery bypass graft surgery. METHODS: Five nonproprietary models were selected for evaluation (Parsonnet, Canadian, Cleveland, New York, and the Northern New England). RESULTS: The C-statistic for the 5 models was 0.752, 0.693, 0.748, 0.735, and 0.722 for the Parsonnet, Canadian, Cleveland, New York, and Northern New England models respectively. The H-L c2 calibration statistics were 4.948, P =.763; 1.616, P =.899; 11.96, P =.035; 10.23, P =.249; and c2 = 12.14, P =.145 for the Parsonnet, Canadian, Cleveland, New York, and Northern New England models respectively. CONCLUSIONS: Comparing hospital-specific or surgeon-specific mortality/morbidity rates will remain a challenge. This analysis reaffirms the concept of risk-adjusting outcomes and emphasizes the importance of the risk-adjustment process for CABG surgery in a community hospital.