Validation of a discharge summary term search method to detect adverse events.

OBJECTIVE: Adverse events are poor health outcomes caused by medical care. Measuring them is necessary for quality improvements, but current detection methods are inadequate. We performed this study to validate a previously derived method of adverse event detection using term searching in physician-dictated discharge summaries.
DESIGN: This was a retrospective, chart review study of a random sample of 245 adult medicine and surgery patients admitted to a multicampus academic medical center in 2002. MEASUREMENTS: The authors used a commercially available search engine to scan discharge summaries for the presence of 104 terms that potentially indicate an adverse event. Summaries with any of these terms were reviewed by a physician to determine the term's context. Screen-positive summaries had a term that was contextually indicative of an adverse event. We used a two-stage chart review as the gold standard to determine the true presence or absence of an adverse event.
RESULTS: The average patient age was 62 years (standard deviation 18.6) and 55% were admitted to a medical service. By gold standard criteria, 48 of 245 patients had an adverse event. Term searching classified 27 cases with an adverse event, with 11 true positives; 218 cases were classified as not having an adverse event, with 181 true negatives. The sensitivity, specificity, and positive and negative predictive values were 0.23 (95% confidence interval [CI]=0.11-0.35), 0.92 (95% CI=0.88-0.96), 0.41 (95% CI=0.25-0.59), and 0.83 (95% CI=95% 0.77-0.97), respectively.
CONCLUSION: Although the sensitivity of the method is low, its high specificity means that the method could be used to replace expensive manual chart reviews by nurses.