Liliana Bordeianou1, Christy E Cauley, Donna Antonelli, Sarah Bird, David Rattner, Matthew Hutter, Sadiqa Mahmood, Deborah Schnipper, Marc Rubin, Ronald Bleday, Pardon Kenney, David Berger. 1. 1 Division of General and GI Surgery, Massachusetts General Hospital, Boston, Massachusetts 2 Codman Center, Massachusetts General Hospital, Boston, Massachusetts 3 Department of Quality, Safety, and Value, Partners Healthcare, Boston, Massachusetts 4 Department of Surgery, Newton Wellesley Hospital, Newton, Massachusetts 5 Department of Surgery, North Shore Medical Center, Salem, Massachusetts 6 Department of Surgery, Brigham and Women's Hospital, Boston, Massachusetts 7 Department of Surgery, Faulkner Hospital, Boston, Massachusetts.
Abstract
BACKGROUND: Two systems measure surgical site infection rates following colorectal surgeries: the American College of Surgeons National Surgical Quality Improvement Program and the Centers for Disease Control and Prevention National Healthcare Safety Network. The Centers for Medicare & Medicaid Services pay-for-performance initiatives use National Healthcare Safety Network data for hospital comparisons. OBJECTIVE: This study aimed to compare database concordance. DESIGN: This is a multi-institution cohort study of systemwide Colorectal Surgery Collaborative. The National Surgical Quality Improvement Program requires rigorous, standardized data capture techniques; National Healthcare Safety Network allows 5 data capture techniques. Standardized surgical site infection rates were compared between databases. The Cohen κ-coefficient was calculated. SETTING: This study was conducted at Boston-area hospitals. PATIENTS: National Healthcare Safety Network or National Surgical Quality Improvement Program patients undergoing colorectal surgery were included. MAIN OUTCOME MEASURES: Standardized surgical site infection rates were the primary outcomes of interest. RESULTS: Thirty-day surgical site infection rates of 3547 (National Surgical Quality Improvement Program) vs 5179 (National Healthcare Safety Network) colorectal procedures (2012-2014). Discrepancies appeared: National Surgical Quality Improvement Program database of hospital 1 (N = 1480 patients) routinely found surgical site infection rates of approximately 10%, routinely deemed rate "exemplary" or "as expected" (100%). National Healthcare Safety Network data from the same hospital and time period (N = 1881) revealed a similar overall surgical site infection rate (10%), but standardized rates were deemed "worse than national average" 80% of the time. Overall, hospitals using less rigorous capture methods had improved surgical site infection rates for National Healthcare Safety Network compared with standardized National Surgical Quality Improvement Program reports. The correlation coefficient between standardized infection rates was 0.03 (p = 0.88). During 25 site-time period observations, National Surgical Quality Improvement Program and National Healthcare Safety Network data matched for 52% of observations (13/25). κ = 0.10 (95% CI, -0.1366 to 0.3402; p = 0.403), indicating poor agreement. LIMITATIONS: This study investigated hospitals located in the Northeastern United States only. CONCLUSIONS: Variation in Centers for Medicare & Medicaid Services-mandated National Healthcare Safety Network infection surveillance methodology leads to unreliable results, which is apparent when these results are compared with standardized data. High-quality data would improve care quality and compare outcomes among institutions.
BACKGROUND: Two systems measure surgical site infection rates following colorectal surgeries: the American College of Surgeons National Surgical Quality Improvement Program and the Centers for Disease Control and Prevention National Healthcare Safety Network. The Centers for Medicare & Medicaid Services pay-for-performance initiatives use National Healthcare Safety Network data for hospital comparisons. OBJECTIVE: This study aimed to compare database concordance. DESIGN: This is a multi-institution cohort study of systemwide Colorectal Surgery Collaborative. The National Surgical Quality Improvement Program requires rigorous, standardized data capture techniques; National Healthcare Safety Network allows 5 data capture techniques. Standardized surgical site infection rates were compared between databases. The Cohen κ-coefficient was calculated. SETTING: This study was conducted at Boston-area hospitals. PATIENTS: National Healthcare Safety Network or National Surgical Quality Improvement Program patients undergoing colorectal surgery were included. MAIN OUTCOME MEASURES: Standardized surgical site infection rates were the primary outcomes of interest. RESULTS: Thirty-day surgical site infection rates of 3547 (National Surgical Quality Improvement Program) vs 5179 (National Healthcare Safety Network) colorectal procedures (2012-2014). Discrepancies appeared: National Surgical Quality Improvement Program database of hospital 1 (N = 1480 patients) routinely found surgical site infection rates of approximately 10%, routinely deemed rate "exemplary" or "as expected" (100%). National Healthcare Safety Network data from the same hospital and time period (N = 1881) revealed a similar overall surgical site infection rate (10%), but standardized rates were deemed "worse than national average" 80% of the time. Overall, hospitals using less rigorous capture methods had improved surgical site infection rates for National Healthcare Safety Network compared with standardized National Surgical Quality Improvement Program reports. The correlation coefficient between standardized infection rates was 0.03 (p = 0.88). During 25 site-time period observations, National Surgical Quality Improvement Program and National Healthcare Safety Network data matched for 52% of observations (13/25). κ = 0.10 (95% CI, -0.1366 to 0.3402; p = 0.403), indicating poor agreement. LIMITATIONS: This study investigated hospitals located in the Northeastern United States only. CONCLUSIONS: Variation in Centers for Medicare & Medicaid Services-mandated National Healthcare Safety Network infection surveillance methodology leads to unreliable results, which is apparent when these results are compared with standardized data. High-quality data would improve care quality and compare outcomes among institutions.
Authors: Krista D Atchley; Janine M Pappas; Andrea T Kennedy; Susan E Coffin; Jeffrey S Gerber; Stephanie M Fuller; Thomas L Spray; Kenneth McCardle; J William Gaynor Journal: Ann Thorac Surg Date: 2013-12-21 Impact factor: 4.330
Authors: Bruce L Hall; Barton H Hamilton; Karen Richards; Karl Y Bilimoria; Mark E Cohen; Clifford Y Ko Journal: Ann Surg Date: 2009-09 Impact factor: 12.969
Authors: Yi Mu; Jonathan R Edwards; Teresa C Horan; Sandra I Berrios-Torres; Scott K Fridkin Journal: Infect Control Hosp Epidemiol Date: 2011-09-01 Impact factor: 3.254
Authors: Elise H Lawson; Rachel Louie; David S Zingmond; Greg D Sacks; Robert H Brook; Bruce Lee Hall; Clifford Y Ko Journal: Ann Surg Date: 2016-01 Impact factor: 12.969
Authors: Robert L Smith; Jamie K Bohl; Shannon T McElearney; Charles M Friel; Margaret M Barclay; Robert G Sawyer; Eugene F Foley Journal: Ann Surg Date: 2004-05 Impact factor: 12.969
Authors: Avinash Bhakta; Marcel Tafen; Owen Glotzer; Ashar Ata; A David Chismark; Brian T Valerian; Steven C Stain; Edward C Lee Journal: Dis Colon Rectum Date: 2016-04 Impact factor: 4.585
Authors: Vikas N O'Reilly-Shah; Katherine R Gentry; Wil Van Cleve; Samir M Kendale; Craig S Jabaley; Dustin R Long Journal: Anesth Analg Date: 2020-08 Impact factor: 5.108