Moving beyond hand hygiene monitoring as a marker of infection prevention performance: Development of a tailored infection control continuous quality improvement tool.

BACKGROUND: Infection control practice compliance is commonly monitored by measuring hand hygiene compliance. The limitations of this approach were recognized in 1 acute health care organization that led to the development of an Infection Control Continuous Quality Improvement tool.
METHODS: The Pronovost cycle, Barriers and Mitigation tool, and Hexagon framework were used to review the existing monitoring system and develop a quality improvement data collection tool that considered the context of care delivery.
RESULTS: Barriers and opportunities for improvement including ambiguity, consistency and feasibility of expectations, the environment, knowledge, and education were combined in a monitoring tool that was piloted and modified in response to feedback. Local adaptations enabled staff to prioritize and monitor issues important in their own workplace. The tool replaced the previous system and was positively evaluated by auditors. Challenges included ensuring staff had time to train in use of the tool, time to collect the audit, and the reporting of low scores that conflicted with a target-based performance system.
CONCLUSIONS: Hand hygiene compliance monitoring alone misses other important aspects of infection control compliance. A continuous quality improvement tool was developed reflecting specific organizational needs that could be transferred or adapted to other organizations.

Preventing infection in health care settings depends on the practices and behaviors of health care workers (HCW) and organizational factors that influence practice. Hand hygiene has long been considered the most important infection prevention precaution, and hand hygiene audit data are reported at a senior level as part of quality assurance, overlooking the contribution of other important practices, for example isolation of infectious patients and sterilization and disinfection of equipment. The value of hand hygiene remains undisputed, but despite regular monitoring and performance feedback, compliance is suboptimal.4, 5 Hand hygiene forms only part of an overall infection prevention program, and its use as the overall indicator of infection prevention excellence is questionable, especially as monitoring is fraught with pitfalls. Hand hygiene compliance monitoring is commonly undertaken by directly observing practice, but this is flawed because HCWs are aware of scrutiny and transient improvement in performance may occur, generating inflated scores and not reflecting usual behavior.8, 9, 10

A review of the hand hygiene monitoring throughout 1 organization found the data that were based on observation of practice did not accurately reflect infection control compliance, contributed little to improving practice, were not considered the best use of time, and lacked local credibility. These factors are likely to influence the value of these data in practice improvement.

Rationale

Achieving and maintaining high levels of infection control compliance is challenging, and few studies consider the context of care or barriers and opportunities to improve compliance. Recognition of the context of and constraints on practice can provide insights into the opportunities for practice improvements. The collection of related data15, 16 includes the variability of activities, associated infection risks, and the importance of clarifying expectations of compliance. A data collection tool was required to provide credible information relating to a range of infection prevention practices, reflecting the operational risks and constraints encountered in different clinical settings and generating data of value for practice improvement. The aim of this work was to develop and implement an infection control performance and quality improvement data collection tool to meet the needs of a large, acute health care provider and to improve the credibility and use of infection control performance monitoring.

Methods

The Infection Control Continuous Quality Improvement (IC-CQI) tool was developed in an acute teaching hospital in London, with over 1,200 inpatient beds and 8,000 staff spread across 7 hospitals on separate sites, providing emergency, general medicine, surgery, critical care, maternity, neonatal, and cancer services. Hand hygiene compliance monitoring was established, but other aspects of infection control practice were not systematically monitored.

To create an IC-CQI tool and reporting framework the infection prevention team used the Pronovost Knowledge Translation Cycle to review the current hand hygiene monitoring tool, and to develop a quality improvement data collection tool. The Barriers and Mitigation tool was used to identify workplace improvement barriers and potential solutions that involved “walking the process”: observing clinical processes and compliance measurement as they occurred in different clinical areas. A double loop learning cycle was used to ensure that the context, values, assumptions, and culture of the whole organization were included in proposed quality improvement intervention using the Hexagon tool framework to assess feasibility and how to engage with stakeholders.

A variety of arrangements including questionnaires, day to day contacts with auditors, feedback from users via the IC-CQI data input system, discussion groups, and IC-CQI training sessions were made for providing feedback on the data collection tool and process, to meet the operational needs, including time constraints, of different practitioners and clinical areas. Modifications to the tool and implementation of the change were made in response to feedback.

No other routine infection control performance data were collected apart from monthly hand hygiene compliance data collection and reporting that took place continuously across all clinical areas until this was replaced by the IC-CQI system. Intermittent validation was then undertaken of IC-CQI results including hand hygiene product availability, isolation practices, appropriateness of use of personal protective equipment (PPE), and compliance with standards of invasive devices insertion and management throughout the implementation period.

Results

Results are reported using the Pronovost Knowledge Translation Cycle.

Summarize the science

A literature review focused on the current evidence for opportunities and barriers to compliance in infection control, including hand hygiene that is the most researched infection control intervention, and in addition evidence from related fields such as psychology. Five themes emerged that are summarized.

Knowledge, education, and training

Improving infection control knowledge, education, and training can enhance compliance and potentially reduce infection acquisition,21, 22, 23 although its impact and value is disputed.24, 25 The effect of education is difficult to gauge as it is one of a bundle of interventions in studies in which it has been judged to be beneficial.26, 27, 28, 29

Further, improvements after education may not be sustained,30, 31, 32 and may require continuous renewal to sustain reported effectiveness, although a limited ongoing residual effect has been reported. The “stickiness” and memorability of ideas have influenced key concepts such as the “My Five Moments of Hand Hygiene” concept promoted by the World Health Organization, which has been widely adopted as an approach to indicate when hands should be cleaned.

Lack of understanding of expectations and ambiguity is a significant barrier to compliance, particularly with guidelines. This is compounded by the considerable variation in the scope, approach, content, expectations, and terminology in infection control guidance and recommendations.37, 38 Improving and clarifying infection control information and expectations may improve compliance, knowledge and high levels of self-efficacy are recognized to improve performance, whereas poor self-efficacy, despite good theoretical knowledge, is more likely to be associated with lower compliance.

Promotion of infection control requirements

The promotion of infection control compliance has been used widely in acute care settings with variable outcomes, and has included marketing,42, 43, 44 campaigns,45, 46, 47 stimulating,48, 49, 50 and reminding staff of infection control requirements.26, 51 Motivation associated with infection control is complex and appears to be related to culture, beliefs, and values. Emotion, habit/routine, and incentives affect behavior, and whereas pride in work, empathy, automatic habits, and rewards may have a positive impact on compliance, sanctions could have a negative effect. Other factors such as the protection of patients or self53, 54, 55, 56 or the perception of risk may also affect performance. , 57, 58, 59

Environmental and human factors

The health care environment influences infection control compliance including equipment design, position, and workflow.61, 62, 63 Provision, availability, and accessibility of hand hygiene facilities and products are important factors in hand hygiene compliance.64, 65, 66, 67, 68, 69 Inadequate hand hygiene may be mitigated by reducing the environmental contamination of the patient environment including computer keyboards and telephones. Improving cleaning efficacy may reduce hand contamination.71, 72 However, this requires an environment designed to expedite cleaning, competent cleaning staff, and sufficient time and opportunity to clean in busy clinical environments.73, 74

Other significant barriers to infection control compliance include pressure of work, understaffing, overcrowding, high bed occupancy and patient turnover, high patient-to-nurse ratio, and lack of time for education,22, 25 , 75, 76, 77, 78 which may contribute to infection transmission and outbreaks.79, 80, 81

Organizational priorities and culture

In the United Kingdom, health care organizational performance is closely monitored, and achievement of quality standards and performance targets are important. Health care infection acquisitions are an important marker for the quality of care delivered and organizational management.83, 84 Effective infection prevention is typically a bundle of interventions including isolation provision and practice, the use of PPE, cleanliness including environment, and hand hygiene.

Culture, commitment, and leadership at unit or ward level affect the implementation of best practice,12, 86, 87 whereas organizational culture can influence, improve, and sustain infection control compliance.81, 88, 89 Strong leadership, good role models, local ownership, champions, empowerment, commitment, and role have been found to be important.86, 87, 90

Feedback of observation of practice

Performance feedback is widely used to improve hand hygiene performance, , 91, 92, 93 but has been found in other settings to be a potentially destructive process that demotivates and diminishes performance if not done well. Behavior changes related to observation may promote “good” behavior such as increasing hand hygiene compliance, with more pronounced effects when observers are known to the staff.95, 96 Another benefit of observation is the opportunity to check technique,27, 97, 98 praise, recognize constraints, or offer advice and support. Clarifying and standardizing expectations prior to observation may reduce inconsistency in expectations.

Observing sequences of care has been used as an alternative approach to simply observing hand hygiene practice.100, 101, 102 The HCW is observed for the duration of the care activity, such as mobilizing a patient or measuring vital signs, to enable the observer to put the actions observed into the context and constraints of practice.

Measure performance

The review of organizational data in an acute NHS Trust from 2008 to 2012, established that although reported levels of compliance were high (Fig 1 ) and met the performance target of >90% compliance, the data collection method lacked validity and relaibility. Nurses were responsible for data collection, and it was perceived by some as “ticking boxes” with no expectation that improvement would occur. Many nurses had not received training in auditing hand hygiene, no time was allocated to undertake it, and providing feedback was difficult particularly if the results were poor. Noncompliant staff were seldom challenged because of fear of a negative response. Factors impeding compliance, such as empty soap dispensers or ambiguity of expectations, were not resolved when observed as they should have been. Other issues reported by staff were inconsistent feedback about performance, inadequate facilities, dissatisfaction with products and supplies, lack of appropriate knowledge, ambiguity related to definitions, expectations and standards, and difficulty in observing single rooms without causing disruption. Infection control risks varied across the organization and specialties. Misconceptions such as when to wear disposable gloves were not managed or monitored, and there was a focus on identifying failures in compliance, whereas good practice was unrecognized. Some staff, including medical staff, were largely disengaged from the process of monitoring and improving infection control practice.

Fig 1

Hand hygiene compliance results averaged across all sites 2008-2015.

Understand the current process and context of work

The context is summarized in 6 categories of the Hexagon framework that examines the current process and context of the proposed change.

Needs

Infection control performance data provides assurance of compliance with national standards.

Related audits including cleaning and environmental monitoring were not collated or widely disseminated.

The organization was often blind to infection control issues until a significant problem emerged.

Fit

The proposal reflected the core values of the organization.

The organization supported quality improvement and this initiative created opportunities for continuous improvement based on known and locally identified opportunities and barriers.

It was an organizational priority that staff were competent and regularly updated in infection prevention and control.

Some managers perceived that reporting low scores was an admission of failure rather than an opportunity for improvement which was a potential barrier.

The potential to save time auditing aligned with an organizational strategy to reduce costs and improve efficacy.

Resource

No additional resource was required as this replaced the established data collection, reporting, and dissemination infrastructure. Although it was recognized that in the existing system staff were often untrained and given no additional time to undertake this work.

There was potential for a decreased dependence on nurses as data collection could be shared with other team members and undertaken throughout the 24-hour period.

Evidence

There was evidence that the current system lacked credibility and had little effect on improving reported compliance.

There was evidence of systematic defects and barriers to infection control compliance including ambiguity, disengagement, and unreliable hand hygiene product provision.

Readiness

Issues such as isolation were increasingly problematic and required improvement, particularly with the emergence of Ebola, severe acute respiratory syndrome, and Middle East respiratory syndrome, when considerable resource was required to ensure staff were educated and resourced to be able to manage these emerging viruses safely.

Capacity

There was capacity within the infection control department to support the changes in the data collection and reporting, training, communication, and validation of the data collected.

A number of other competing changes and initiatives prevalent in the organization including building, reorganization of services, and staffing structure were potential barriers.

Ensure all patients reliably receive the intervention

Five key themes emerged from the literature review, Barriers and Mitigation work,15, 16 feedback from auditors, and observations from stakeholders suggesting barriers and opportunities for improvement that were combined to produce a draft IC-CQI tool. This was pilot tested in several areas while the existing hand hygiene monitoring arrangements continued throughout the remainder of the organization (Fig 2 ). The draft IC-CQI tool was clarified and simplified in response to feedback. The rationale for including criteria in the final IC-CQI tool is summarized in Table 1 . Results of the pilot studies and the final IC-CQI tool was presented to senior managers, infection control staff, and auditors who agreed that the results would be used to provide a monthly score of infection control performance (Table 2 ).

Fig 2

Hand hygiene audit compliance averaged across all sites (line) compared with the distribution of reporting: traditional audit tool (light gray bars), replaced by pilot (dark gray bars), and the final version of the Infection Control Continuous Quality Improvement tool (intermediate gray bars).

Table 1

Rationale for including criteria in the final Infection Control Continuous Quality Improvement tool

Criteria	Rationale for inclusion
Knowledge including education and training
Mandatory infection control training	1. This was readily available data.2. Training at entry to the organization (induction) and regular updates ensure the health care worker is aware of the expected standards of practice, policies, and guidance.3. The level of staff training and education reflects the local and organizational commitment to the prevention of infection.
Knowledge	1. Regular assessment of current infection control knowledge of staff identifies knowledge gaps.2. Provides evidence of success or failure of education.3. Identifies areas of confusion or inconsistent practice, which provides an opportunity to remove ambiguity and reinforce consistency in practice.
Local education	1. Local practice and facilities may vary.2. Regular local updates may clarify ambiguity.3. The presence of a visible, credible advisor ensures queries are promptly managed and learning is not delayed.
Promotion and awareness	1. To maintain awareness and increase knowledge at a local level.2. Reflects local ownership of infection control.
Facilities	1. Provides information on environmental issues that are not resolved, deteriorating, or where performance was suboptimal.2. Provides managers with information and evidence to assist in requests for improvement or investment.3. Clarifies standards expected.
Area specific factors	1. An opportunity to focus on issues important to each area and not included in organization-wide improvement strategies.2. Identification of an issue in one area could lead organization-wide learning.
Observation of single room practice and sequences of care	1. Observation of sequences included the context of care in the assessment.2. Isolation practice of single rooms could be observed.3. A visible monitoring presence potentially improved performance.4. Inclusion in the tool legitimized observation of practice (permission to stop and watch).5. Observation enabled assessment of competence.6. Agreeing expectations prior to observing removed ambiguity.

Table 2

Simplified example of the Infection Control Continuous Quality Improvement tool

	Examples of questions	Score
Training	Have all staff received infection control training at induction and updates every 2 years?
Knowledge	Select 5 staff each month and check knowledge of hand hygiene (or infection control issue relevant to your area), for example, 5 questions for each member of staff.
Education	Is ward-level training in hand hygiene established and underway?(This could include access to a training aid/trainer and dedicated time or sessions.)
Promotion/awareness/information	Are hand hygiene posters and other signage in place? Is there clear and enough information displayed in the ward or department about how to wash and gel hands?
Facilities	Is there alcohol gel at each bed end that is filled and working?
	Are soap and hand towel dispensers filled, clean, and working at each sink?
	Are the alcohol gel dispensers at entrance and wall-mounted dispensers filled and working?
	Check taps–are they correctly adjusted for elbow operation?
	Is gel available on desks, next to keyboards, and by notes trolleys?
	Check keyboards–are they being cleaned regularly?
Area specific criteria	Examples include:Are patients provided with hand wipes at mealtimes?Is the environment clean and cleaned to a high standard?Is glove use appropriate?
Observation	Single room/sequence of care observation.

It took more than 3 years to develop the IC-CQI tool, educate staff in the purpose, methods, and implications for practice and integrate the data into the established quality measurement system in the organization. Over 100 hours of training in the final tool was delivered across the organization, and more than 150 people attended training in 2016-2017.

The use of the IC-CQI tool was finally established throughout the organization in August 2016, and the existing hand hygiene audit discontinued. All areas were expected to report using the new tool from September 2016 (Fig 2). Progress developing and establishing the process is summarized in the following text.

Knowledge including education and training

Mandatory training

Basic infection control training including hand hygiene is mandatory in the first month at work, with online updating every 2 years. Initially, externally employed staff were excluded from this training and senior medical staff often opted out, but during the development of the new tool this training became mandatory for all staff. Electronic training records were reported monthly to managers and the executive team.

Knowledge

A list of common infection control questions was developed to assess the knowledge of HCWs and identify education requirements. Each ward or department were required to ask a representative sample of staff working in the area monthly either standard or locally developed and agreed questions.

Examples could include:

When do you need to isolate a patient with diarrhea?

Describe how a spillage of blood should be cleaned up?

The infection control knowledge of auditors was a limiting factor, and initially some auditors restricted questions to the ones they could answer. The range of questions and potential for improving infection control knowledge increased when answer sheets were provided.

Local education

An infection control link personnel system was already established to provide local induction and refresh basic skills and knowledge in the workplace, which included hand hygiene techniques, cleaning equipment, and the use of PPE. Staff turnover was high particularly in junior doctors and the burden of local induction and support was onerous in some areas. Delivery varied across the organization and reflected local commitment to the induction of new team members and the energy and commitment of the link staff.

Promotion and awareness

It was envisaged that prompts and reminders would increase awareness and that posters and screen savers could provide useful information such as the actions to take after a needle stick injury or how to clean equipment. However, audible reminders confused some patients, irritated some staff, and were rapidly removed or sabotaged by detractors. Some senior managers removed infection control notice boards as they found them “untidy,” and plans to install monitors failed as there was no space or electrical supply or funds. The most enduring promotion was hand hygiene technique stickers on hand hygiene product dispensers. In addition, regular supplies of posters were delivered to wards and departments.

Facilities (environmental and human factors)

Local managers were responsible for arranging maintenance, but the process could be onerous and protracted. Metrics relating to minor repairs were not collated and recurring problems were largely invisible. An issue mentioned frequently was empty or broken soap and alcohol hand gel dispensers. Organization-wide audits in 2013 found 18% of soap dispensers were broken or empty and there were examples of delays of several days before they were repaired. In 2013-2014, a project was undertaken to replace all soap and alcohol hand gel dispensers with standardized products and dispensers that staff had positively evaluated. The condition of soap and gel dispensers was subsequently included in the IC-CQI tool, and subsequent validated scores indicated a sustained improvement across the organization.

Area specific factors

The range of specialties, workflows, client groups, facilities, and infection risks provided a wide variety of area-specific issues that emerged from staff observations, audits, feedback, complaints, and root cause analysis. These included correcting air pressure in isolation rooms, staff refusing to remove wrist watches, parents visiting neonates while contagious, inappropriate disposable glove use, and poor patient hand hygiene. Staff identified local issues requiring improvement and agreed on expectations and actions. These were included in the daily handover, local education, knowledge assessment, and the progress audited. Once improvement was demonstrated, monitoring could stop and switch to other issues of concern.

Some areas readily used this opportunity to identify, improve, and monitor issues, whereas others were reluctant to highlight problems as there was anxiety about producing a low score even for a short period. Sometimes, encouragement was required to tackle an area of practice that was recognized as requiring improvement.

Observation of single room practice and sequences of care

Feedback indicated that observing practice was valued by staff as an opportunity to look at practice delivery and the environment. There was also a recognition that staff disliked covert observation and wanted to understand what was expected and how they could improve practice. Two types of observation: observing sequences of care and single room isolation practice were identified in pilots as potentially useful and acceptable to HCWs and were subsequently adopted. Both required training auditors and clarification and agreement of expectations with practitioners and subsequent inclusion in local education.

Sequence of care monitoring requires the observer to compare the infection control expectation with performance, records, data, and offers an opportunity to provide feedback. Examples include observing doctors on a ward round or a nurse preparing and administering intravenous drugs. Sometimes, it was difficult to understand what was monitored as documentation was often limited and it was not always possible to validate these observations.

Single room observation monitored isolation practice using an audit tool. An observer records the infection control practice expectations, and then while positioned outside the room, observes, records, and offers feedback of performance. This may include infection control precautions taken by people entering and leaving the room, if the isolation sign was accurate, and if adequate PPE was available. These data were simple to validate, and the tool was also used to check isolation practice compliance ad hoc to clarify expectations of new patients requiring isolation.

Evaluation of the IC-CQI tool

The IC-CQI tool (Table 2) was evaluated to assess acceptability to local auditors and managers and if the results were perceived to be a fair reflection of performance. At this stage, the assessment focused on the perspective of the auditors and those using the information rather than those being assessed.

In December 2016, 27% (6 of 22) of data collectors responding to online questionnaires had time allocated to undertake data collection. This had increased to 55% (27 of 49) by June 2017. Those trained in the use of the tool had also increased slightly from 44% (23 of 52) in 2013 to 48% (24 of 50) in 2017.

Use of the tool had increased to 100% (50 of 50 wards sampled) in 2017, and 64% (32 of 50) believed the tool had helped improve infection control practice in their area. A total of 70% (35) did not believe the tool had led to a decline in infection control practice standards, and 59% (29 of 49) believed the data were an accurate reflection of practice. In addition, auditors assessed observation, knowledge, education, and promotion of awareness as the most valuable components of the tool, while at the same time the most difficult to collect (Fig 3 ). Data collectors/auditors consistently requested more training in the use of the tool, more prepared questions rather than locally developed questions, and simplification of the data collection process.

Fig 3

Feedback of the average value of tool components versus average difficulty of data collection across 50 auditors (scale of 1-5 in which 5 relates to the greatest value and difficulty). Error bars are 95% credible intervals derived via bootstrap sampling.

Validation of the scores obtained were undertaken by the infection control team, although some observations were difficult to validate particularly in the presence of local interpretations.

The use of the tool and validity of the data collected continue to be evaluated. Infection reduction data are not reported here as it is unlikely that outcomes will be directly attributable to the use of the tool as other improvements and changes in care occur frequently such as increased isolation provision and increases in robotic surgery.

Discussion

Broadening the scope of monitoring to include other aspects of infection control practice beyond hand hygiene was hampered by a lack of a robust evidence base for some common infection control practices and inconsistent opinions from subject matter experts. This created some ambiguity of expectations of infection control practice, but consensus was often achieved when the rationale for practice was examined and options explored. The approach was sometimes uncomfortable for senior infection control staff but liberating for junior staff who were empowered to question entrenched habitual practice.

Changes in the focus of monitoring considered the value of knowledge, education, training, and human factors and were readily accepted, but, despite evidence that local ownership and participation is beneficial, this was difficult to achieve. Staff were seldom allocated time to attend training or collect information and sometimes lost the momentum to identify new areas for local improvement. Lack of energy and resilience has been recognized previously in NHS staff, and it has been suggested that this may be related to an underlying lack of engagement and absence of positive reinforcement for previous efforts. These issues were particularly problematic when there was potential for reporting lower scores conflicting with the established organizational aspiration of reporting high performance.

Resistance to acknowledging and monitoring areas that required improvement was a recurring issue during the development and implementation of this tool. Although the fallibility of the previous monitoring system was recognized within the organization, the use of soft intelligence provided by local observations and feedback was a major departure from the normal practice of collecting data for assurance to one of improving practice. Consequently, the assimilation of this change within the organization was slow, and auditors reported persistent pressure from managers to achieve targets and avoid highlighting areas for improvement. At times this perpetuated the organizational blindness to problems and it was unclear if this influenced the priority given for time for training and data collection.

Establishing rational and feasible expectations of HCWs within the context of care delivery and providing information including data that was useful locally had a positive impact on engagement and acceptance of the changes introduced. However, the flexibility and adaptability of the tool created inconsistencies and anomalies that hampered standardization of practice and validation of results. The use of observation of practice was valued by auditors but standardization of practice was simpler to achieve and validate when an unambiguous audit tool was provided, for example, the single room audit tool as this was less reliant of staff knowledge and local variations.

Summary

A widely used hand hygiene compliance monitoring system produced data that did not contribute to quality and safety improvement in 1 organization, and an alternative quality improvement tool was developed and implemented. Consideration of the context of care delivery led to the creation of a flexible and pragmatic tool that could be adapted. The previous focus on hand hygiene compliance was replaced by monitoring performance in a range of infection control-related factors.

Limitations

This work was undertaken in 1 organization that may limit generalizability. The barriers and opportunities identified may vary in other organizations and facilities that may affect replication,114, 115 although issues such as ambiguity, poor role models, and knowledge are likely to be common. The tool is now established as a performance metric in the organization and has been adopted and adapted in other health care organizations.

Conclusions

The removal of ambiguity, realistic expectations, and local engagement contributed to the successful introduction and continued use of this tool. Further evaluation is required to establish the impact of this system in improving infection control practice.