Are administrative data valid when measuring patient safety in hospitals? A comparison of data collection methods using a chart review and administrative data.

OBJECTIVE: To evaluate the validity and reliability of German Diagnosis Related Group administrative data to measure indicators of patient safety in comparison to clinical records.
DESIGN: A cross-sectional study was conducted using chart review (CR) as gold standard and screening of associated administrative data based on DRG coding.
SETTING: Three German somatic acute care hospitals for adults. PARTICIPANTS: A total of 3000 cases treated between May and December, 2010. MAIN OUTCOME MEASURES: Eight indicators were used to analyse the incidence of associated adverse events (AEs): pressure ulcers, catheter-related infections, respiratory failure, deep vein thromboses, hospital-acquired pneumonia, acute renal failure, acute myocardial infarction and wound infections. We calculated sensitivity, specificity, positive predictive value (PPV) and Cohen's Kappa with 95% confidence intervals.
RESULTS: Screening of administrative data identified 171 AEs and 456 were identified by CR. A number of 135 identical events were identified by both methods. Sensitivities for the detection of AEs using administrative data ranged from 6 to 100%. Specificities ranged from 99 to 100%. PPV were 33 to 100% and reliabilities were 12 to 85%.
CONCLUSIONS: Indicators based on German administrative data deviate widely from indicators based on clinical data. Therefore, hospitals should be cautious to use indicators based on administrative data for quality assurance. However, some might be useful for case findings and quality improvement. The precision of the evaluated indicators needs further development to detect AEs by the valid use of administrative data.