Do claims-based comorbidities adequately capture case mix for surgical site infections?

BACKGROUND: There is increasing interest in using administrative claims data for surveillance of surgical site infections in THAs and TKAs, but the performance of claims-based models for case-mix adjustment has not been well studied. Performance of claims-based models can be improved with the addition of clinical risk factors for surgical site infections. QUESTIONS/PURPOSES: We assessed (1) discrimination and calibration of claims-based risk-adjustment models for surgical site infections; and (2) the incremental value of adding clinical risk factors to claims-based risk-adjustment models for surgical site infections. PATIENTS AND METHODS: Our study included all THAs and TKAs performed at a large tertiary care hospital from January 1, 2002 to December 31, 2009 (total n = 20,171 procedures). Revision procedures for infections were excluded. Comorbidity data were ascertained through administrative records and classified by the Charlson comorbidity index. Clinical details were obtained from the institutional joint registry and patients' electronic health records. Cox proportional hazards regression models were used to estimate the 1-year risk of surgical site infections with a robust sandwich covariance estimator to account for within-subject correlation of individuals with multiple surgeries. The performance of claims-based risk models with and without the inclusion of four clinical risk factors (morbid obesity, prior nonarthroplasties on the same joint, American Society of Anesthesiologists score, operative time) was assessed using measures of discrimination (C statistic, Somers' D xy rank correlation, and the Nagelkerke R(2) index). Furthermore, calibrations of claims-based risk models with and without clinical factors were assessed graphically by plotting the smoothed trends between model predictions and empirical rates from Kaplan-Meier.
RESULTS: Discrimination of the claims-based risk models was moderate for the THA (C statistic = 0.662, D xy = 0.325, R(2) = 0.028) and TKA (C statistic = 0.621, D xy = 0.241, R(2) = 0.017) cohorts. Inclusion of four clinical risk factors improved discrimination in both cohorts with significant improvement in the C statistic in the THA cohort (C statistic = 0.043; 95% CI, 0.012-0.074) and in the TKA cohort (C statistic = 0.027; 95% CI, 0.007-0.047). Visual inspection suggested that calibration of the claims-based risk models was adequate and comparable to that of models which included the four additional clinical factors.
CONCLUSIONS: Claims-based risk-adjustment models for surgical site infections in THA and TKA appear to be adequately calibrated but lack predictive discrimination, particularly with TKAs. The addition of clinical risk factors improves the discriminative ability of the models to a moderate degree; however, addition of clinical factors did not change calibrations, as the models showed reasonable degrees of calibration. When used in the clinical setting, the predictive performance of claims-based risk-adjustment models may be improved further with inclusion of additional clinical data elements.