Cedric Manlhiot1, Vivek Rao1, Barry Rubin1, Douglas S Lee2. 1. Peter Munk Cardiac Centre, University Health Network (Manlhiot, Rao, Rubin, Lee), Divisions of Cardiac Surgery (Manlhiot, Rao), Vascular Surgery (Rubin) and Cardiology (Lee), Institute of Health Policy, Management and Evaluation (Lee), and Institute for Clinical Evaluative Sciences (Lee), University of Toronto (Manlhiot, Rao, Rubin, Lee), Toronto, Ont. 2. Peter Munk Cardiac Centre, University Health Network (Manlhiot, Rao, Rubin, Lee), Divisions of Cardiac Surgery (Manlhiot, Rao), Vascular Surgery (Rubin) and Cardiology (Lee), Institute of Health Policy, Management and Evaluation (Lee), and Institute for Clinical Evaluative Sciences (Lee), University of Toronto (Manlhiot, Rao, Rubin, Lee), Toronto, Ont. dlee@ices.on.ca.
Abstract
BACKGROUND: Outcomes for coronary artery bypass surgery are of broadening interest, but the impact of data type on quality reporting has not been fully examined. We compared the performance of administrative and clinical data-based risk adjustment models at a tertiary-quaternary care hospital. METHODS: We used a prospective study design to test two risk adjustment models, one from administrative (Canadian Institute for Health Information [CIHI] Cardiac Care Quality Indicator) and one from clinical data (Society of Thoracic Surgeons), on cardiac surgical procedures performed between 2013 and 2016 (n = 1635). Our primary outcome was in-hospital mortality within 30 days of surgery. Model performance was established by comparing predicted and observed mortality, model calibration and handling of critical covariates. RESULTS: Observed mortality was 1.96%, which was the same as that predicted by the Society of Thoracic Surgeons model (1.96%), but significantly higher than that predicted by the CIHI model (1.03%). Despite both models having similar C statistics (0.756 CIHI; 0.758 Society of Thoracic Surgeons), the CIHI model showed significant underestimation of mortality among patients at higher risk. There was significant miscalibration of risk associated with 7 covariates: New York Heart Association class IV, congestive heart failure, ejection fraction less than 20%, atrial fibrillation, acute coronary insufficiency, cardiac compromise (shock, myocardial infarction < 24 h, intra-aortic balloon pump, cardiac resuscitation or preprocedure circulatory support) and creatinine concentration of 100 mg/dL or more. Together, these factors accounted for 84% of the difference in predicted mortality between the administrative and clinical models. INTERPRETATION: Risk prediction using administrative data underestimated risk of death, potentially inflating observed-to-predicted mortality ratios at hospitals with patients who are more ill. Caution is warranted when hospital reports of cardiac surgery outcomes are based on administrative data alone. Copyright 2018, Joule Inc. or its licensors.
BACKGROUND: Outcomes for coronary artery bypass surgery are of broadening interest, but the impact of data type on quality reporting has not been fully examined. We compared the performance of administrative and clinical data-based risk adjustment models at a tertiary-quaternary care hospital. METHODS: We used a prospective study design to test two risk adjustment models, one from administrative (Canadian Institute for Health Information [CIHI] Cardiac Care Quality Indicator) and one from clinical data (Society of Thoracic Surgeons), on cardiac surgical procedures performed between 2013 and 2016 (n = 1635). Our primary outcome was in-hospital mortality within 30 days of surgery. Model performance was established by comparing predicted and observed mortality, model calibration and handling of critical covariates. RESULTS: Observed mortality was 1.96%, which was the same as that predicted by the Society of Thoracic Surgeons model (1.96%), but significantly higher than that predicted by the CIHI model (1.03%). Despite both models having similar C statistics (0.756 CIHI; 0.758 Society of Thoracic Surgeons), the CIHI model showed significant underestimation of mortality among patients at higher risk. There was significant miscalibration of risk associated with 7 covariates: New York Heart Association class IV, congestive heart failure, ejection fraction less than 20%, atrial fibrillation, acute coronary insufficiency, cardiac compromise (shock, myocardial infarction < 24 h, intra-aortic balloon pump, cardiac resuscitation or preprocedure circulatory support) and creatinine concentration of 100 mg/dL or more. Together, these factors accounted for 84% of the difference in predicted mortality between the administrative and clinical models. INTERPRETATION: Risk prediction using administrative data underestimated risk of death, potentially inflating observed-to-predicted mortality ratios at hospitals with patients who are more ill. Caution is warranted when hospital reports of cardiac surgery outcomes are based on administrative data alone. Copyright 2018, Joule Inc. or its licensors.
Authors: Yana Gurevich; Anne McFarlane; Kathleen Morris; Aleksandra Jokovic; Gail M Peterson; Gregory K Webster Journal: Can J Cardiol Date: 2010 Aug-Sep Impact factor: 5.223
Authors: Bradley G Hammill; Lesley H Curtis; Gregg C Fonarow; Paul A Heidenreich; Clyde W Yancy; Eric D Peterson; Adrian F Hernandez Journal: Circ Cardiovasc Qual Outcomes Date: 2010-12-07
Authors: David M Shahian; Treacy Silverstein; Ann F Lovett; Robert E Wolf; Sharon-Lise T Normand Journal: Circulation Date: 2007-03-12 Impact factor: 29.690
Authors: David M Shahian; Sean M O'Brien; Giovanni Filardo; Victor A Ferraris; Constance K Haan; Jeffrey B Rich; Sharon-Lise T Normand; Elizabeth R DeLong; Cynthia M Shewan; Rachel S Dokholyan; Eric D Peterson; Fred H Edwards; Richard P Anderson Journal: Ann Thorac Surg Date: 2009-07 Impact factor: 4.330
Authors: Elise H Lawson; Rachel Louie; David S Zingmond; Greg D Sacks; Robert H Brook; Bruce Lee Hall; Clifford Y Ko Journal: Ann Surg Date: 2016-01 Impact factor: 12.969
Authors: Howard S Gordon; Michael L Johnson; Nelda P Wray; Nancy J Petersen; William G Henderson; Shukri F Khuri; Jane M Geraci Journal: Med Care Date: 2005-02 Impact factor: 2.983
Authors: Jack V Tu; Linda R Donovan; Douglas S Lee; Julie T Wang; Peter C Austin; David A Alter; Dennis T Ko Journal: JAMA Date: 2009-11-18 Impact factor: 56.272
Authors: S Siregar; M E Pouw; K G M Moons; M I M Versteegh; M L Bots; Y van der Graaf; C J Kalkman; L A van Herwerden; R H H Groenwold Journal: Heart Date: 2013-12-13 Impact factor: 5.994