Sharon-Lise T Normand1, Katya Zelevinsky2, Meena Nathan3, Haley K Abing2, Joseph A Dearani4, Mark Galantowicz5, J William Gaynor6, Robert H Habib7, Frank L Hanley8, Jeffrey P Jacobs9, S Ram Kumar10, Donna E McDonald7, Sara K Pasquali11, David M Shahian12, James S Tweddell13, David F Vener14, John E Mayer15. 1. Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts; Department of Biostatistics, Harvard Chan School of Public Health, Boston, Massachusetts. 2. Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts. 3. Department of Cardiac Surgery, Boston Children's Hospital, Boston, Massachusetts; Department of Surgery, Harvard Medical School, Boston, Massachusetts. 4. Department of Cardiovascular Surgery, Mayo Clinic, Rochester, Minnesota. 5. Department of Cardiothoracic Surgery, Nationwide Children's Hospital, Columbus, Ohio. 6. Division of Cardiothoracic Surgery, Department of Surgery, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania; Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania. 7. STS Research Center, The Society of Thoracic Surgeons, Chicago, Illinois. 8. Division of Pediatric Cardiac Surgery, Department of Cardiothoracic Surgery, Stanford University, School of Medicine, Stanford, California. 9. Congenital Heart Center, Departments of Surgery and Pediatrics, University of Florida, Gainesville, Florida. 10. Division of Cardiac Surgery, Department of Surgery, Keck School of Medicine of the University of Southern California, Los Angeles, California; Department of Pediatrics, Keck School of Medicine of the University of Southern California, Los Angeles, California; Heart Institute, Children's Hospital Los Angeles, Los Angeles, California. 11. Division of Cardiology, Department of Pediatrics, University of Michigan C.S. Mott Children's Hospital, Ann Arbor, Michigan. 12. Department of Surgery, Harvard Medical School, Boston, Massachusetts; Division of Cardiac Surgery, Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts. 13. Department of Surgery, Cincinnati Children's Medical Center, Cincinnati, Ohio. 14. Department of Anesthesiology, Baylor College of Medicine, Houston, Texas; and; Pediatric and Congenital Cardiac Anesthesia, Texas Children's Hospital, Houston, Texas. 15. Department of Cardiac Surgery, Boston Children's Hospital, Boston, Massachusetts; Department of Surgery, Harvard Medical School, Boston, Massachusetts. Electronic address: john.mayer@cardio.chboston.org.
Abstract
BACKGROUND: The Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database (CHSD) provides risk-adjusted operative mortality rates to approximately 120 North American congenital heart centers. Optimal case-mix adjustment methods for operative mortality risk prediction in this population remain unclear. METHODS: A panel created diagnosis-procedure combinations of encounters in the CHSD. Models for operative mortality using the new diagnosis-procedure categories, procedure-specific risk factors, and syndromes or abnormalities included in the CHSD were estimated using Bayesian additive regression trees and least absolute shrinkage and selector operator (lasso) models. Performance of the new models was compared with the current STS CHSD risk model. RESULTS: Of 98 825 operative encounters (69 063 training; 29 762 testing), 2818 (2.85%) STS-defined operative mortalities were observed. Differences in sensitivity, specificity, and true and false positive predicted values were negligible across models. Calibration for mortality predictions at the higher end of risk from the lasso and Bayesian additive regression trees models was better than predictions from the STS CHSD model, likely because of the new models' inclusion of diagnosis-palliative procedure variables affecting <1% of patients overall but accounting for 27% of mortalities. Model discrimination varied across models for high-risk procedures, hospital volume, and hospitals. CONCLUSIONS: Overall performance of the new models did not differ meaningfully from the STS CHSD risk model. Adding procedure-specific risk factors and allowing diagnosis to modify predicted risk for palliative operations may augment model performance for very high-risk surgical procedures. Given the importance of risk adjustment in estimating hospital quality, a comparative assessment of surgical program quality evaluations using the different models is warranted.
BACKGROUND: The Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database (CHSD) provides risk-adjusted operative mortality rates to approximately 120 North American congenital heart centers. Optimal case-mix adjustment methods for operative mortality risk prediction in this population remain unclear. METHODS: A panel created diagnosis-procedure combinations of encounters in the CHSD. Models for operative mortality using the new diagnosis-procedure categories, procedure-specific risk factors, and syndromes or abnormalities included in the CHSD were estimated using Bayesian additive regression trees and least absolute shrinkage and selector operator (lasso) models. Performance of the new models was compared with the current STS CHSD risk model. RESULTS: Of 98 825 operative encounters (69 063 training; 29 762 testing), 2818 (2.85%) STS-defined operative mortalities were observed. Differences in sensitivity, specificity, and true and false positive predicted values were negligible across models. Calibration for mortality predictions at the higher end of risk from the lasso and Bayesian additive regression trees models was better than predictions from the STS CHSD model, likely because of the new models' inclusion of diagnosis-palliative procedure variables affecting <1% of patients overall but accounting for 27% of mortalities. Model discrimination varied across models for high-risk procedures, hospital volume, and hospitals. CONCLUSIONS: Overall performance of the new models did not differ meaningfully from the STS CHSD risk model. Adding procedure-specific risk factors and allowing diagnosis to modify predicted risk for palliative operations may augment model performance for very high-risk surgical procedures. Given the importance of risk adjustment in estimating hospital quality, a comparative assessment of surgical program quality evaluations using the different models is warranted.