Using present-on-admission coding to improve exclusion rules for quality metrics: the case of failure-to-rescue.

BACKGROUND: The Agency for Healthcare Research and Quality (AHRQ) patient safety indicator "death among surgical inpatients with serious treatable complications" (failure-to-rescue) uses rules to exclude complications presumed to be present-on-admission (POA). Like other administrative data-based quality measures, exclusion rules were developed with limited information on whether complications were POA. We examine whether the accuracy of failure-to-rescue exclusion rules can be improved with data with good POA indicators.
METHODS: POA-coded data from 243,825 discharges from a large academic medical center were used to develop 3 failure-to-rescue exclusion rules. Data from 82,871 discharges from California hospitals screened for good POA coding practices was used as a validation sample. The AHRQ failure-to-rescue measure and 3 new measures based on alternative exclusion rules were compared on sensitivity, specificity, and C-statistics for prediction of POA status. Using data from the AHRQ HCUP National Inpatient Sample, the alternative specifications were tested for sensitivity to nurse staffing.
RESULTS: The AHRQ exclusion rules had sensitivity of 18.5%, specificity 92.1%, and a C-statistic of 0.553. All POA-informed specifications of exclusion rules improved the C-statistic of the failure-to-rescue measure and its sensitivity, with modest losses of specificity. For all tested specifications, higher licensed hours and proportions of registered nurse were statistically significant and associated with lower risk of death.
CONCLUSIONS: Failure-to-rescue is a robust quality measure, sensitive to nursing across alternative exclusion rule specifications. Despite expanded POA coding, exclusion-based rules are needed to analyze datasets not coded for POA, legacy datasets, and datasets with poor POA coding. POA-informed construction of exclusions significantly improves rules identifying POA complications.