Natural Language Processing for Real-Time Catheter-Associated Urinary Tract Infection Surveillance: Results of a Pilot Implementation Trial.

BACKGROUND: Incidence of catheter-associated urinary tract infection (CAUTI) is a quality benchmark. To streamline conventional detection methods, an electronic surveillance system augmented with natural language processing (NLP), which gathers data recorded in clinical notes without manual review, was implemented for real-time surveillance.
OBJECTIVE: To assess the utility of this algorithm for identifying indwelling urinary catheter days and CAUTI.
SETTING: Large, urban tertiary care Veterans Affairs hospital.
METHODS: All patients admitted to the acute care units and the intensive care unit from March 1, 2013, through November 30, 2013, were included. Standard surveillance, which includes electronic and manual data extraction, was compared with the NLP-augmented algorithm.
RESULTS: The NLP-augmented algorithm identified 27% more indwelling urinary catheter days in the acute care units and 28% fewer indwelling urinary catheter days in the intensive care unit. The algorithm flagged 24 CAUTI versus 20 CAUTI by standard surveillance methods; the CAUTI identified were overlapping but not the same. The overall positive predictive value was 54.2%, and overall sensitivity was 65% (90.9% in the acute care units but 33% in the intensive care unit). Dissimilarities in the operating characteristics of the algorithm between types of unit were due to differences in documentation practice. Development and implementation of the algorithm required substantial upfront effort of clinicians and programmers to determine current language patterns.
CONCLUSIONS: The NLP algorithm was most useful for identifying simple clinical variables. Algorithm operating characteristics were specific to local documentation practices. The algorithm did not perform as well as standard surveillance methods.