Measuring compliance with transmission-based isolation precautions: comparison of paper-based and electronic data collection.

BACKGROUND: Decreasing the transmission of resistant organisms in hospitals is a key goal of infection prevention plans. Studies have consistently shown inadequate health care worker (HCW) compliance with isolation precautions. Evaluating adherence through direct observation of HCW behavior is considered the "gold standard" but is labor-intensive, requiring the collection and analysis of a large volume of observations.
METHODS: Two methods of data collection to assess HCW compliance were evaluated: a manual method using a paper form (PF), with subsequent data entry into a database, and an electronic method using a Web-based form (WBF) with real-time data recording. Observations were conducted at 4 hospitals (a total of 2,065 beds) to assess the availability of gloves, gowns, and masks; isolation sign postings; and HCW isolation practices.
RESULTS: A total of 13,878 isolation rooms were observed in 2009. The median number of rooms observed per day was 61 for PF and 60 for WBF, and the respective mean observation times per room were 149 seconds and 60 seconds. Thus, use of the WBF provided a time savings of 89 seconds per room.
CONCLUSION: Simple electronic forms can significantly decrease the required resources for monitoring HCW adherence to hospital policies. Use of the WBF decreased the observation time by 60%, allowing for increases in the frequency and intensity of surveillance activities.

Antimicrobial resistance in pathogens that cause health care–associated infections (HAIs) is a growing problem in health care institutions worldwide. Up to 16% of all HAIs reported through the US National Health Care Safety Network in 2006-2007 were associated with 9 multidrug-resistant organisms (MDROs), and these continue to pose serious therapeutic challenges. Other studies have reported increases in antimicrobial resistance in some organisms by as much as 47% since the 1990s.1, 2, 3 Recently, novel pathogens, such as the H1N1 influenza virus and coronavirus (severe acute respiratory syndrome), as well as an increased virulence of known pathogens, such as Clostridium difficile, have become a concern in hospitalized patients. The Healthcare Infection Control Advisory Committee of the Centers for Disease Control and Prevention (CDC) recommends isolation precautions for patients with MDROs and C difficile infection (contact isolation), influenza and meningococcal meningitis (droplet isolation), and tuberculosis and measles (airborne isolation). The CDC also recommends that health care workers (HCWs) and visitors perform hand hygiene before and after contact with all patients or patient equipment/environment. Breaches in technique may lead to the spread of organisms directly to patients via HCWs’ hands or by contact with the contaminated environment or equipment.

Previous studies have shown that HCWs’ compliance with isolation precautions and hand hygiene is often inadequate, and that some form of behavior monitoring is necessary.5, 6, 7, 8, 9, 10, 11 Direct observation of HCW behavior is considered the gold standard for measuring adherence to these standards, but this is labor-intensive and costly, and standardized data collection tools are lacking.5, 6, 12 Electronic data collection tools show promise for automating the process of identifying patients who require isolation and providing a venue for efficient collection of real-time data. The aim of the present study was to evaluate the data quality and time efficiency of 2 methods of data collection—one using a paper form (PF) and one using a Web-based form (WBF)—for assessing hospital-wide adherence to transmission-based isolation precautions and hand hygiene.

Methods

Setting

This study was conducted at 4 sites within the New York–Presbyterian Hospital system in New York City: (a) a 221-bed community hospital; (b) a 283-bed freestanding pediatric acute care facility, (c) a 647-bed adult academic tertiary care facility, and (d) a 914-bed pediatric/adult academic tertiary care facility. All inpatient units at these sites except psychiatric and maternity wards were included in the study. The Institutional Review Boards of Weill Cornell Medical Center and Columbia University Medical Center approved the study design.

Infection Prevention System

The Infection Prevention System (IPS) is a Web-based epidemiology decision support system developed by members of the Departments of Infection Prevention and Control and Information Services of New York–Presbyterian Hospital and the Department of Biomedical Informatics of Columbia University. The IPS electronically identifies patients who require isolation by capturing demographic information from the admission/discharge/transfer system and merging it with microbiology data, physicians’ isolation orders from the hospital’s electronic medical record, and isolation information from previous admissions. Rules and logic are applied to monitor the need for isolation precautions; for example, organisms of interest are automatically linked to the correct category of isolation based on Centers for Disease Control and Prevention criteria and institutional policy. In addition, patients with a previous history of an infection requiring isolation are automatically placed back on the isolation patient list on readmission to the respective institution. Patient-specific clinical information also can be gathered from the system and displayed in a summary format on a single screen. The hospital’s infection preventionists confirm the isolation information for patients on the IPS list at least twice daily. When a patient did not have positive microbiologic results suggestive of an infectious process but clinicians nevertheless suspected that the patient had a communicable illness (eg, signs and symptoms of tuberculosis or diarrhea), clinical indications were entered by the infection preventionists into the IPS for monitoring.

Instrument and procedure

Direct observations of rooms housing isolated patients were performed by trained observers in 5-consecutive-day increments (including weekends) during varying times of the day (7 a.m. to 10 p.m.) at each of the study sites over a 10-month interval in 2009. Observer training included reviewing the observation protocols and assessing interrater reliability with research staff who had conducted similar observations in previous research. Throughout the study, study investigators monitored the quality and consistency of the observation methods. The observers recorded data on the appropriateness of isolation sign postings, the availability of personal protective equipment (PPE), and HCWs’ adherence with recommended isolation precautions as part of a larger study, the Impact of Automated Surveillance on MRSA Isolation, funded by a cooperative agreement with the Association for Prevention Training and Research and the CDC (5U50CD3000-860-21).

Two methods of recording observations of adherence to isolation precautions were evaluated. Each phase of the study included a time period for the observer to become comfortable with the study tool. For both phases, the observer followed the same pattern at each hospital site, starting at the top floor and working downward. Rooms that were known or expected to house isolated patients were reviewed, along with any other rooms in which staff had placed isolation signs but had not informed the infection preventionists. Each room was evaluated for the presence of an isolation supply cart or anteroom and for the existence of PPE, including gloves (small, medium, and large sizes), isolation gowns, and, if appropriate, masks and protective eyewear. If HCWs were seen entering or exiting a patient’s room, the observer documented whether they performed hand hygiene and wore PPE on entering the room, removed the PPE on exiting the room, and performed hand hygiene after removing gloves. If the HCW did not remove his or her PPE before leaving the room, the observer noted whether he or she touched the inanimate environment or patient care equipment outside the room. If a group of staff or visitors entered or exited an isolation room, a maximum of 3 people were observed for a total of 6 observations per room. Staff and visitor adherence to isolation precautions were recorded only when they could be observed directly.

Paper form

During the first phase of the study, a PF was used to record study observations. Each day, the observer printed a daily census of patients requiring isolation using the IPS and manually transcribed each patient’s unit, room, and bed information onto the PF. The observer visited each patient’s room listed on the isolation census. These visits were generally made on weekdays during the day or evening shift, although they were occasionally made on weekends. A schedule was developed on a monthly basis by the observer. If the observer found additional patients who had been placed on isolation by the medical team but were not listed on the IPS, then the observer added the room and bed information to the PF and observed those patients as well. If the observer identified inconsistencies between the expected isolation status for a room/bed on the IPS list (eg, an unoccupied room, a missing sign), then a staff member was consulted to confirm the patient’s information. The PF was modified based on changes in patient location, isolation status, or discharge. The observer recorded HCW and visitor compliance with hand hygiene and transmission-based precautions, and noted whether the appropriate PPE was available for each patient. Periodically, the observer transcribed the observations into an Excel (Microsoft, Redmond, WA) database, and these data were rematched to the IPS to merge additional information on visit admission and discharge dates and isolation start and end dates. Mismatches and data entry errors (eg, incorrect medical record number or isolation category) were retrospectively adjusted to ensure that observations were being made on the correct patient.

Web-based form

The WBF was developed to display room/bed and isolation categories from the IPS system. The WBF captures the same data elements listed on the PF but provides structured data fields that contain all possible responses, to eliminate variability in recording and to prompt the observer to evaluate PPE and staff compliance appropriate to the specific isolation category. Radio buttons are used to collect binary data elements, check boxes are used for the fields requiring multiple responses, and text fields are used to document notes. A wireless-networked, hospital-approved tablet computer (Lenovo ThinkPad X200, Lenovo, Morrisville, NC); is used to collect data in real time while rounding on the unit. The tablet computer was chosen over a nonnetworked electronic device, such as a palm pilot, because the WBF was available through the hospital’s intranet. Accessing the WBF via the tablet computer also allows for more rapid recording of observations using a direct input method compared with the use of a cursor button or mouse, which is ergonomically challenging.

The WBF is accessed by logging on to the application’s password-protected Web site. The form is linked to the IPS and is prepopulated with rooms identified as housing isolated patients. The WBF displays the correct category of isolation as determined by the IPS logic and the institution’s infection preventionists. Initially, the WBF was defaulted to load all beds within the building being observed. Because loading the information onto the tablet takes considerable time, the system was adjusted to allow the observer to select only the units to be observed. The observer conducts rounds as in the PF phase, and observations are recorded directly into the database. Blank forms are included for the observer to add patients on isolation but not yet listed in the IPS system. Observations and date/time stamps are automatically linked to patient information in the isolation system and archived in the IPS database.

Analysis

During the 4-month PF phase, the observer recorded the overall amount of time spent performing observations each day, as well as any time spent entering the information into the Excel database. Observations recorded during the 6-month WBF phase were archived in the IPS structured tables, and date/time stamps were used to evaluate the time spent conducting observations for each room. This information was input into the Excel database for comparison to the PF data.

The total number of observation days, unique patients observed, number of observations, and time spent on observations was calculated for each phase. For the PF phase, the total observation time was added to the total data entry time, and the mean observation time per room was then determined by dividing the total time by the total number of observations per day. The WBF phase required no additional time for data entry, so the time required for each individual observation was used to calculate the mean observation time per room. The time saved per room was calculated by subtracting the mean observation time per room for the WBF from the mean observation time per room for the PF. The total time saved in hours per year was calculated by multiplying the time saved per room by the total number of observations conducted in the 10-month study period dividing by 10 and multiplying by 12 to extrapolate from 10 months to 1 year.

Results

A total of 13,878 isolation rooms were observed for 3,969 unique patients between January and November 2009. The total days of observation were 85 days for the PF and 123 days for the WBF, the total number of patient rooms observed were 5,207 for the PF and 8,671 for the WBF, the median number of rooms observed per day were 60 for the PF (PF range, 19-128) and 61 for the WBF (range, 15-154), and the average observation time per room was 149 seconds for the PF and 60 seconds for the WBF. Overall, the WBF provided a savings of observation and data entry time of 89 seconds per room, which for this project extrapolates to 412 hours per year—a 60% savings. Using a salary of $31/hour for an observer or $50/hour for a nurse, using the WBF for observations would provide an annual savings of $12,772 or $20,600, respectively.

Discussion

To decrease the spread of MDROs and other communicable conditions, infection preventionists must be able to identify patients who are colonized or infected with these organisms in a timely fashion and to communicate information for isolation precautions to the appropriate teams. Two important factors for adequate isolation are (a) placement of patients in single rooms, or if necessary because of limited availability of single rooms, cohorting patients with similar organisms, and (b) consistent use of proper protective attire and hand hygiene by HCWs when providing care. The responsibility for compliance with wearing PPE and performing hand hygiene ultimately rests with the direct caregiver.

Surveillance of HAIs and MDROs is time-consuming and limits the ability of infection preventionists to focus on prevention activities such as education, current practice assessment, and performance improvement initiatives. In fact, a recent survey concluded that most health care epidemiology and infection prevention and control programs are understaffed and lack adequate resources to address the mandate for more thorough reporting of HAIs and prevention efforts. Several external factors have put increasing pressure on hospital leadership to reduce HAIs without increasing infection prevention and control staff levels. These factors include mandatory public reporting of HAIs to federal and state agencies, adherence to performance standards associated with Joint Commission patient safety goals, decreases in Medicare reimbursement, and increased public accountability.

These increased demands are forcing each institution to evaluate current workflow and process patterns and identify new ways to streamline their activities. This involves carefully evaluating the advantages and disadvantages of performing surveillance using a PF versus a WBF (Table 1 ). Each method has its advantages and disadvantages. For example, a PF requires extra transcription time and data entry time and has the potential for data loss from misplaced data collection forms or transcription errors from the PF to an electronic database or software package. Moreover, the PF cannot control for user variability in documenting observations, because only free text is allowed. On the other hand, the PF does not require technical expertise, a portable tablet computer, or special software, and it can be easily created and adjusted as needed. Thus, the initial development time is minimal.

Table 1

Comparison of data collection tools

Category	Detail	Greater for PF	Greater for WF	PF‒WF equivalent
Design and implementation	• Initial development needed		∗
	• Technical expertise needed		∗
	• Instrument cost		∗
	• Adaptability of data collection tool			∗
	• Ability to reprocess previously entered data			∗
User features	• Computer training required		∗
	• Issues with network connectivity/speed		∗
	• Weight of data collection tool		∗
	• Problem with network connections/speed		∗
	• Time required to manually copy information from IPS to PF	∗
	• Time needed to perform observations	∗
	• Potential for data entry errors	∗
Data entry/processing	• Potential for data loss			∗
	• Postobservation manual data entry/processing time	∗
Special features	• Level of detail on data collection time provided		∗
	• Ability to prevent incorrect or missing data points		∗
	• Alert for missed/omitted observation data		∗
	• Automated compliance report generation		∗

The WBF requires information technology expertise or support, and there is the potential for slow network connections or equipment failures. The tablet may be costly, heavy, and have a short battery life. The design of the WBF requires forethought to ensure adequate and appropriate output. The WBF has some significant advantages, however, in that it does not require either transcription or additional data entry time and minimizes incorrect or incomplete entries by providing structured choices. The WBF avoids data loss and increased security, because the records can be saved on a hospital server. Initial startup costs for the WBF, including the cost of the tablet, an extra battery, replacement batteries, and per month charge for a wireless card, is approximately $3,200. Other incidental costs associated with the WBF were consistent with those associated with any computer.

In this study, use of the WBF decreased the observation time by 60%, with a commensurate savings in salary of $12,772 per year. These savings in both time and salary can allow for an increased frequency of observations and expansion of surveillance activities.

A possible limitation of this study is related to the fact that the PF was used first, and thus the observer could have gained expertise in navigating the hospital units and conducting the observations during the first phase, which might have decreased the time spent on each unit/observation in the WBF phase. Also, although the total observation times per day and per data entry session were documented for the PF, observation times for individual patients were not documented. For the WBF, time stamps of all activities can be reviewed. It also is important to note that the surveillance activities described in this article were conducted for research purposes. For clinical purposes, surveillance must be accompanied by feedback to allow behavior change. The time saved in surveillance activities using automated methods could be used for such feedback and other educational activities.

In conclusion, electronic solutions, such as the WBF, can significantly decrease resources needed to monitor HCW adherence to hospital policies. The methods described here can be replicated by other institutions using an electronic spreadsheet or IPS. Additional systematic and objective comparative studies of the costs and data accuracy of these methods are needed to help guide infection preventionists in choosing the optimal data collection tool and method for their institution. Standard forms could be developed to facilitate uniform data collection practices and allow comparisons across settings. The cost-effectiveness and clinical outcomes of such data collection systems merit further study.