Assessing the Hawthorne effect on hand hygiene compliance in an intensive care unit.

BACKGROUND: The influence of the Hawthorne effect on hand hygiene compliance in an intensive care unit was assessed using covert and overt direct observation.
METHODS: The observational study was conducted from February to November 2018 in a 24-bed adult intensive care unit in a 243-bed tertiary care hospital, in four periods (P): P-1, February 5-March 3, 29 h (covert) and P-2, March 15-April 16, 33 h (overt), prior to an educational campaign on hand hygiene; and P-3, August 27-September 28, 33 h (covert) and P-4, October 3-November 6, 35 h (overt), after the campaign. Three 20-min observation sessions were performed daily, randomly distributed in the morning, afternoon and evening shifts, including holidays and weekends. Hand hygiene compliance rates observed in Periods 2 and 4 were displayed on an electronic panel installed in the unit. Hand hygiene compliance was assessed according to the World Health Organization "My Five Moments for Hand Hygiene" guidelines.
RESULTS: Before the campaign, the overall hand hygiene compliance rate was 31.95% (340/1064, covert) versus 68.10% (790/1160, overt), and afterwards was 56.11% (615/1096, covert) versus 80.98% (1086/1341, overt). The infection rate was reduced by 22.62% (18.87% versus 14.60%).
CONCLUSIONS: The Hawthorne effect and educational campaign markedly influenced compliance with hand hygiene recommendations. The results suggest that combining overt and covert observation methods, including regular feedback on hand hygiene compliance displayed on an electronic panel, may be a valid alternative to increase real hand hygiene compliance rates in hospital practice.

Introduction

Direct observation of hand hygiene practices in healthcare settings is currently considered the gold standard for monitoring hand hygiene compliance [1]. However, a major limitation of this method is the Hawthorne effect, i.e., subjects who knows they are being observed might behave differently, affecting study outcomes [2]. Although the Hawthorne effect often leads to overestimation of hand hygiene compliance in hospital practice, only a few studies have assessed the extent of the Hawthorne effect by direct observation [3,4]. This study assessed the influence of the Hawthorne effect on hand hygiene compliance in an adult intensive care unit by means of covert and overt observation methods, before and after an educational campaign on hand hygiene.

Methods

Prior to the study period, one closed-format, self-administered questionnaire by the World Health Organization (WHO), the Hand Hygiene Self-Assessment Framework (HHSAF), was applied in our hospital to assess the infrastructure available to promote hand hygiene and practice [5]. Briefly, the HHSAF is structured in five sections with 27 indicators that reflect the five components of the WHO Multimodal Hand Hygiene Improvement Strategy: system change, training and education, evaluation and feedback, reminders in the workplace, and institutional safety climate for hand hygiene. Based on a scoring system, the indicators are assigned values totalling 100 points within each five-element section, for a maximum overall score of 500 points. Based on the score achieved for the five components, the facility is assigned to one of four levels of hand hygiene promotion and practice: 1, inadequate (score 0–125); 2, basic (126–250); 3, intermediate (251–375); or 4, advanced (376–500) [1,5,6].

A prospective direct observational study was conducted from February to November, 2018 in a 24-bed adult intensive care unit (AICU) in a Brazilian 243-bed tertiary care hospital. The AICU is staffed by 18 physicians, 12 nurses, 48 nursing technicians, and 12 other health workers including physiotherapists, laboratory technicians, and radiology technicians. During the study period, data on infection rates and alcohol gel consumption in the AICU were obtained from the hospital infection control committee. The study was approved by the hospital ethics committee.

Direct observation was performed by three of the authors of the present study, who are AICU staff members (SAB and WEF are nurses, and ACSF is a physiotherapist). Before beginning the observational study SAB, WEF, and ACSF were trained in the WHO-recommended method for direct observation, according to the “WHO Hand Hygiene Reference Technical Manual” [7].

The observational study was conducted in four periods: briefly, Period 1, February 5-March 3, 2018, 29h (SAB, covert observer) and Period 2, March 15-April 16, 2018, 33h (SAB, overt observer), which were performed before the educational campaign on hand hygiene; and Period 3, August 27-September 28, 33h (ACSF and WEF, covert observers) and Period 4, October 3-November 6, 2018, 35h (SAB, overt observation), which were performed after the campaign. Observation sessions lasted for 20 min and were performed in three daily sessions, randomly distributed in the morning, afternoon and evening shifts, including holidays and weekends.

Hand hygiene compliance was assessed according to the WHO “My Five Moments for Hand Hygiene” (5MHH) guidelines, which includes the following moments (M): M-1, before touching a patient; M-2, before clean/aseptic procedure; M-3, after body fluid exposure risk; M-4, after touching a patient; and M-5, after touching patient surroundings [1,7]. In general, the observer monitored hand hygiene at 6 of the 24 AICU beds.

Hand hygiene compliance rates observed in Periods 2 and 4 (overt observation) were displayed on an electronic light-emitting diode board (electronic panel) installed in the AICU. The panel was lighted from 7:00 to 21:00 h and displayed hand hygiene compliance rates (general and by professional category) and the message “Save lives − Wash your hands".

The educational campaign on hand hygiene was carried out from August 1–15, 2018, and consisted of lectures to small groups of AICU personnel, to avoid interfering with activities of the unit. A 3-min video on the 6-step hand hygiene technique was shown, combined with a 10-slide session emphasizing the importance of hand hygiene in hospital practice. A WHO pocket leaflet on when and how to conducted hand hygiene was also distributed to each campaign participant.

To determine if there was a significant difference between compliance with hand hygiene recorded by the overt and covert observation methods, the Z statistical test to compare proportions using the program Statistica (version 13.2) was performed. A P value less than 0.05 was considered significant.

Results and discussion

According to HHSAF, our hospital was classified at the intermediate level and received 257.5 of 500 points (i.e., system change, 100 points; training and education, 40 points; evaluation and feedback, 35 points; reminders in the workplace, 42.5 points; institutional safety climate for hand hygiene, 40 points) [5]. This was somewhat surprising, since we were not aware that systematic educational programs to promote hand hygiene practices have been performed at our hospital for the last 15 years. However, the score of 257.5 achieved by our hospital is at the lower limit of the intermediate range (score 251–375) and requires improvement in comparison to other hospitals around the world [6]. For example, a WHO study conducted two global surveys in 2011 and 2015 using the HHSAF. In 2011, 2119 health facilities from 69 countries participated, and in 2015, 807 health facilities from 91 countries participated. The average score of 292.5 in 2011 and 372.9 in 2015 ranked health facilities at the intermediate level (i.e., 251 to 375 points) in relation to promotion and hand hygiene practices [6]. These scores exceed that of our hospital.

A total of 4,661 indications for hand hygiene were recorded during the 130 h (390 sessions lasting 20 min each) of direct observation. Overall hand hygiene compliance rates were: P-1, 31.95% (340/1064; i.e., 340 hand hygiene actions performed versus 1,064 indications for hand hygiene); P-2, 68.10% (790/1160); P-3, 56.11% (615/1096); and P-4, 80.98% (1086/1341) (Table 1).

Table I

Hand hygiene compliance by the healthcare workers of the adult intensive care unit, in general and according to professional category; hand hygiene indications according to the World Health Organization's “My Five Moments for Hand Hygiene” guidelines; and covert and overt observation before and after the hand hygiene educational campaign

Hand hygiene indication and professional category	Hand hygiene before the campaign		P-value∗	Hand hygiene after the campaign		P-value∗
	P-1 Covert observer	P-2 Overt observer		P-3 Covert observer	P-4 Overt observer
M-1. Before touching a patient
All	13.31% (61/458)a	34.74% (148/426)	0.0001	46.39% (238/513)	71.87% (391/544)	0.0001
Physician	24.32% (18/74)	38.63% (34/88)	0.0001	51.88% (55/106)	72.72% (96/132)	0.0001
Nurse	26.77% (25/97)	27.38% (23/84)	0.7464	46.04% (64/139)	81.81% (108/132)	0.0001
Nursing technician	5.77% (13/225)	37.50% (75/200)	0.0001	41.13% (65/158)	71.33% (112/157)	0.0001
Otherb	8.06% (5/62)	29.62% (16/54)	0.0001	49.09% (54/110)	60.97% (75/123)	0.0001
M-2. Before clean/aseptic procedure
All	77.77% (14/18)	100% (26/26)	0.0001	100% (18/18)	100% (26/26)	0.9999
Physician	100% (6/6)	100% (3/3)	0.9999	100% (9/9)	100% (17/17)	0.9999
Nurse	100% (5/5)	100% (22/22)	0.9999	100% (9/9)	100% (9/9)	0.9999
Nursing technician	0% (0/3)	100% (1/1)	0.0001	−	−	NA
Other	75% (3/4)	−	NA	−	−	NA
M-3. After body fluid exposure risk
All	100% (7/7)	85.71% (6/7)	0.0001	100% (12/12)	100% (6/6)	0.9999
Physician	−	−	NA	−	−	NA
Nurse	100% (1/1)	−	NA	100% (1/1)	−	NA
Nursing technician	100% (3/3)	83.33% (5/6)	0.0001	100% (11/11)	100% (1/1)	0.9999
Other	100% (3/3)	100% (1/1)	0.9999	−	100% (5/5)	NA
M-4. After touching a patient
All	54.65% (229/419)	90.24% (546/605)	0.0001	66.44% (295/444)	90.55% (537/593)	0.0001
Physician	66.26% (55/83)	92.36% (133/144)	0.0001	81.31% (74/91)	93.54% (116/124)	0.0001
Nurse	68.36% (67/98)	94.44% (119/126)	0.0001	74.13% (86/116)	89.40% (135/151)	0.0001
Nursing technician	40.85% (67/164)	88.40% (221/250)	0.0001	44.44% (64/144)	85.85% (170/198)	0.0001
Other	54.05% (40/74)	85.88% (73/85)	0.0001	76.34% (71/93)	96.66% (116/120)	0.0001
M-5. After touching patient surroundings
All	17.90% (29/162)	66.66% (64/96)	0.0001	47.70% (52/109)	73.25% (126/172)	0.0001
Physician	23.07% (3/13)	100% (5/5)	0.0001	47.82% (12/23)	74.35% (29/39)	0.0001
Nurse	34.28% (12/35)	33.33% (3/9)	0.6361	48.64% (18/37)	76.78% (43/56)	0.0001
Nursing technician	9.09% (9/99)	67.50% (54/80)	0.0001	45.23 (19/42)	70.83% (51/72)	0.0001
Other	33.33% (5/15)	100% (2/2)	0.0001	42.85% (3/7)	60% (3/5)	0.0001

P, period; M, moment; −, no observation; NA, not applicable.

∗Significant at P<0.05.

Hand hygiene compliance rate was calculated as percentage (numerator, number of hand hygiene actions performed; denominator, number of hand hygiene opportunities).

Physiotherapists, laboratory technicians, radiology technicians.

The Hawthorne effect, estimated by the difference between overt (P-2) and covert (P-1) observations, revealed a 113% increase in the hand hygiene compliance rate amongst AICU workers before the campaign (31.95% for covert observation versus 68.10% for overt observation), and a 44.32% increase after the campaign (P-3, 56.11% for covert observation versus P-4, 80.98% for overt observation). Overall, with only two exceptions in the nurse category (M-1, 26.77% versus 27.38%, P =0.7464; and M-5, 34.28% versus 33.33%, P = 0,6361), hand hygiene compliance rates differed significantly between covert and overt observations for all moments and professional categories assessed (Table 1). These findings suggest that the Hawthorne effect can be used deliberately to improve compliance with hand hygiene in hospital practice. In this case, it is important for hospitals to establish their own Hawthorne effect and use it to estimate the ‘real’ frequency of hand hygiene [8].

Feedback on hand hygiene adherence is essential to change health worker behaviour and results in increased hand hygiene compliance [1]. For example, one study demonstrated that third-part remote video auditing combined with real-time feedback produced a significant and sustained improvement in hand hygiene in a 17-bed medical intensive care unit. During the 4-month pre-feedback period, the hand hygiene compliance rate was 6.5% versus 81.6% in the 4-month post-feedback period, and 87.9% in the 17-month follow-up period [9]. An interesting aspect of this study was that performance feedback was continuously displayed on electronic boards mounted in the hallway of the unit for continuous viewing by the healthcare workers in the course of their work [9]. This motivated us to use the electronic panel in our study.

Although it is widely recognised that physicians' compliance with hand hygiene is often lower than that of nurses [1], our study showed that during the times when we obtained complete data for all professional categories (i.e., M-1, M-4 and M-5), physicians' level of compliance was similar to that of nurses (Table 1). Amongst other factors, regular feedback on hand hygiene compliance rates displayed on the electronic panel may have contributed to this unusual finding.

The educational campaign lecture on hand hygiene was delivered to groups each comprised of three AICU workers. Seven, nine and 12 lectures were delivered in the morning, afternoon and evening shifts, respectively. A total of 28 lectures were delivered within two weeks, and 84 (93%) AICU workers were involved. Overall compliance according to 5MHH guidelines was highest for M-4 (i.e., after touching a patient) on overt observation (90.24% before the campaign and 90.55% after the campaign) (Table 1). This finding highlights the universal tendency of self-protection of health professionals as described in other studies [1].

During the study period, the infection rate in our AICU was reduced by 22.6% (18.8% before the hand hygiene campaign versus 14.6% afterward), and use alcohol-based hand gel increased 132% (16.93 L versus 39.20 L per 1,000 patient-days) (data not shown). These findings demonstrate the effectiveness of the hand hygiene education campaign.

Our study had the limitations inherent to observational studies, such as observation bias (Hawthorne effect), observer bias, and selection bias. To reduce or eliminate the influence of the Hawthorne effect on covert observation, SAB (observer in P-1 and P-2) was replaced in P-3 by WEF and ASCF. In an attempt to reduce observer bias we sought to make observations in all shifts and on every day of the week during the four periods of the observational study.

Another limitation was the small sample size. Direct observation is estimated to capture only a small number of all hand hygiene opportunities that are occurring simultaneously under the study conditions [8]. For example, one study showed that in a 60-min observation period per day only 0.5–1.7% of the total number of opportunities for hand hygiene was captured [10]. In our study, during the observation periods, we estimated that 447,456 opportunities for hand hygiene occurred, of which we captured 1.04%. In brief, 4661 opportunities were observed at 6 of 24 beds in 130 h; 447,456 opportunities were estimated at 24 beds over 3120 h (i.e., 20min morning+20min afternoon+20min evening = 1 h/day × 130 h × 24 h/day] of activities in the AICU].

Despite these limitations, the direct observation method used in our study is considered the gold standard for hand hygiene compliance in healthcare facilities [1,8]. We believe that our data reflect an overall improvement in compliance with hand hygiene in the AICU during the study period. However, this study must be continued to determine the long-term effectiveness of the intervention.

In conclusion, the Hawthorne effect, associated with overt observation, and the educational campaign had a marked influence on hand hygiene compliance in our adult intensive care unit. The results suggest that combining overt and covert observation methods, including regular feedback on hand hygiene compliance displayed on an electronic panel, may be a valid alternative to increase real hand hygiene compliance rates in hospital practice.

Declaration of Competing Interest

None.

Funding source

This study was financed in part by the ; Finance Code 001.