Changing personnel behavior to promote quality care practices in an intensive care unit.

The delivery of safe high quality patient care is a major issue in clinical settings. However, the implementation of evidence-based practice and educational interventions are not always effective at improving performance. A staff-led behavioral management process was implemented in a large single-site acute (secondary and tertiary) hospital in the North of England for 26 weeks. A quasi-experimental, repeated-measures, within-groups design was used. Measurement focused on quality care behaviors (ie, documentation, charting, hand washing). The results demonstrate the efficacy of a staff-led behavioral management approach for improving quality-care practices. Significant behavioral change (F [6, 19] = 5.37, p < 0.01) was observed. Correspondingly, statistically significant (t-test [t] = 3.49, df = 25, p < 0.01) reductions in methicillin-resistant Staphylococcus aureus (MRSA) were obtained. Discussion focuses on implementation issues.

Introduction

Adverse events that harm patients is thought to cost the British National Health Service (NHS) an estimated £2 billion a year in additional hospital stays alone, without taking any account of human or wider economic costs (DOH 2000). Moreover, hospital-acquired infections (HAI) are estimated to cost the NHS a further £1 billion per annum (Plowman et al 2001), 15%–30% of which are estimated to be avoidable. In accordance with the findings of the United States Institute of Medicine (Kohn et al 1999), many of these problematic issues surrounding healthcare emanate from human error or failure of people to do the right things.

It is clear from the work of Reason (1998) and many others (eg, Gross et al 2001) that the immediate antecedents for human error actions often include underlying management system faults (a simple healthcare example being a failure to communicate changes in brand of temperature probe to anesthetists which resulted in a child's death) and the prevailing organizational safety culture (Cooper 2000). In addition, some human actions with adverse consequences are attributable to “behaviors” of staff rather than management system faults or error-producing conditions in the workplace.

This study examines the impact of deliberately changing such behavior patterns upon the outcomes of some key healthcare activities within a hospital – especially hand washing and nursing documentation to monitor patient condition. Hand washing is known to reduce patient infection in a multitude of healthcare disciplines (Jenner et al 2002), yet doctors have been reported to wash their hands on only 8.6% of appropriate occasions (Tibballs 1996). Such actions may reflect aspects of professional cultural communities (Hong 2001) where doctors operate within their own autonomous culture and feel the hand washing rule does not apply to them. Jenner and colleagues (2002) argue it is imperative that ways are found to increase adherence to hand washing practice. Cooper, Medley, and colleagues (1999) suggest that small increases in the frequency of effective hand washes can have an impact upon the spread of hand-borne hospital pathogens.

Nursing documentation of patient condition is a critical component of good healthcare. Documentation provides an important source of reference for monitoring purposes and is a vital communication link between healthcare professionals: the main objective is to promote consistency and continuity of patient care (Benner et al 2002). Incomplete documentation can lead to medication errors, which is one of the highest risk areas of nursing practice (Gladstone 1995). Documentation errors resulting from management system faults include staffing shortages that result in a heavy patient load for remaining staff, a lack of time, and burdensome charting formats (Brooks 1998). An estimated 15%–20% of a nurse's time is spent on documentation and is one of the most common reasons for overtime (Moody and Snyder 1995). Staff often find themselves in a situation where they are so busy that they just forget to document various details that are essential to accurate documentation and care. Unless the nursing documentation is satisfactory and adequate, there is an obvious risk of compromised patient safety, security, and well-being (Smith 1998).

Organizational behavior management

Many of the human errors and unsatisfactory healthcare outcomes highlighted in the examples above are rooted in the behaviors of healthcare providers. The issue, therefore, is how to successfully improve these behaviors – what people actually do – where deficiencies exist.

One approach to improving performance at work that has been used for about three decades is organizational behavior management (OBM) (Komaki et al 2000). This approach has been successfully applied to occupational safety (Grindle et al 2000), quality performance (Welsh et al 1992), productivity improvement (Jessup and Stahelski 1999), absenteeism (Orpen 1978), sales (Fellows and Mawhinney 1997), and patient infection control (Babcock et al 1992).

Organizational behavior management is a motivational process aimed at directing people's attentions and actions to perform desired behaviors on a daily basis. The features that theoretically distinguish OBM from other types of managerial interventions are its:

focus on current determinants of behavior, not prior history,

emphasis on overt behavior change as the criteria for treatment evaluation,

careful targeting of critical behaviors,

emphasis on measuring behaviors and monitoring their outcomes,

emphasis on the involvement of all staff in its development and application.

In OBM the unit of analysis is staff behavior, which is determined by direct measurement of critical behaviors or their proxies. Critical behaviors are defined as that small proportion of behaviors responsible for the lion's share of undesired outcomes. Identifying critical behaviors is often achieved via functional analyses of incident records which examine the antecedents that drive undesired/desired behaviors and the consequences that maintain such behaviors. Thus, OBM is a highly focused problem-solving process that adopts a systematic approach to improving organizational performance. The intervention process is based on the following methodological rules:

Tasks are divided into their constituent “observable” behaviors.

The desired behaviors for improving performance are clearly specified and are able to be labeled as being performed either correctly or incorrectly.

Improvement goals are set by all those involved.

The performance of the desired behaviors is regularly monitored.

Based on the monitoring results, there is regular and continued feedback to all.

These rules mean tasks must be divided into specific, but observable behaviors to facilitate the monitoring process. Once agreed upon by staff, these behaviors are placed on checklists which trained observers use to monitor and record people's actual performance during 10–20 minute tours of the workplace. A sufficient sample of behaviors has to be observed on a regular basis (eg, daily) to provide reliable feedback. The monitoring results are scored and computed to provide percentage scores (ie, number of correct behaviors, divided by the total number of behaviors observed, and multiplied by 100). These scores are used to give feedback so that employees may track their progress against implicit self-set or explicit assigned or participative improvement goals (Locke and Latham 1990). Feedback may be given verbally at the point of observation (Zohar et al 1980). This could mean praise given to an employee seen to behave in the desired manner during an observation or an exploration of why an observed person is behaving in an undesired manner. Graphical charts visualizing the observed percent scores results are placed in prominent positions in the workplace, where they can be seen by all employees, and are updated weekly (Duff et al 1994). Sometimes the observation results are analyzed and condensed into written performance summaries. These focus specifically on those behaviors that have improved and those that remain problematic. These summaries are distributed and discussed at weekly 30-minute briefings (Cooper et al 1994). As a whole, the methodological rules comprise an OBM intervention. Over a period of time, significant culture changes take place (Cooper and Phillips 2004), in which continuing improvement of standards progressively becomes the embedded norm.

Method

Participants and setting

The study was conducted in two adjacent 8-bed intensive care wards at James Cook University Hospital, a large single site acute (secondary and tertiary) hospital in the North of England. The Intensive Care Unit (ICU) wards employ approximately 140 personnel operating a mixture of continuous and flexible shift patterns. One of the two wards (ICU2) cares for longer term critical care patients (eg, renal), and the other (ICU3) cares primarily for short-term neurotrauma patients. Participants in the study included doctors, nurses, healthcare assistants, administrative members of the care team, and visitors (physicians, hospital staff, family members, and friends).

Intervention design

Conducted over 26 weeks, this study utilized a quasi-experimental, within-group, repeated-measures design within 2 ICUs. The study is deemed quasi-experimental because it uses an internal intervention control (ie, the units average baseline scores as a comparison point) rather than an external control group (Komaki et al 2000).

Study background

As part of the general response within the local health economy to the new statutory duty of quality on NHS providers (DOH 1998) a group of Clinical Governance leaders in the Co Durham and Tees Valley Strategic Health Authority (SHA) area visited a local petrochemical plant that had been implementing an OBM approach to occupational safety for 7 years. Their reaction was summarized as “If only we could get our people to behave like that!” This was a reaction that led to the decision to pilot the OBM approach within the SHA area.

The ICU chosen for the pilot had previously used conventional approaches to improvement, including more intensive monitoring, training, and propaganda exercises to raise staff awareness, the development of improved policies and protocols. For methicillin-resistant Staphylococcus aureus (MRSA) specifically, all patients were screened on admission, there were investigations when the level of infection rose, and additional infection control measures (eg, barrier nursing) were implemented at a clinical level. This intervention was additional to these activities.

Procedure

Obtaining staff participation

Initial briefings were conducted for as many personnel as possible at 1-hour “orientation” meetings to seek staff participation. The briefing covered the reasons for wanting to implement the project, how the process would be implemented, and what staff would be required to do to help. Volunteer observers were also sought. After some discussions about practicalities, staff agreed to participate. A 2-item questionnaire was distributed asking staff (1) to identify the most serious areas of concern they had in their work in general, and (2) to identify the most common undesired behaviors they engaged in or knew of in others. The idea was to engage staff in problem-solving with regard to lack of resources, management systems, etc, and to try and identify the impact of these on people's day-to-day behavior. For example, staff shortages often led to documentation being completed later in the shift, as staff moved on to deal with another patient's needs.

Staff were informed that management would address the issues arising as quickly as possible (eg, examine staffing issues). To demonstrate management's commitment to improving quality care practices, many of the issues arising were pursued soon after the briefing sessions and then publicized. One specific example was installing a sink near the entrance doors to the unit so all visitors (medical and family members) could wash their hands before proceeding into the unit. The undesired behaviors identified were used later to guide discussions when developing the observation checklists.

Project team and management training

A small project team was formed, starting with the two “champions” from management (ie, the Head of the ICU and the Clinical Matron); their role being to provide leadership and motivation, and ensure that time spent by others on the project was “protected”. A coordinator was appointed (from within the ICU team) and trained in the basic principles and practice of this behavioural approach. This consisted of 1 day's training and several 1-hour follow-up sessions on practical aspects (how to do it) and problems. The training was provided by the experienced behavioral safety coordinator from the local petrochemical plant nearby. The coordinator/champion training covered a 6-stage process encompassing: (1) behavior analysis applied to incident records; (2) development of behavioral observation checklists; (3) observer training; (4) baseline establishment; (5) participative goal-setting (Cooper et al 1992); and (6) feedback mechanisms. The 1-hour follow up sessions concentrated on administration aspects to facilitate tracking of the projects progress.

Behavioral performance checklist

Front-line staff in a location or team – together with their line management – came together in groups to brainstorm and identify areas of concern where they considered they needed to be successful as a group. The undesired behaviors and concerns staff had previously identified at the initial briefing sessions were used to guide these discussions. Three areas of concern were considered very important by most staff and were categorized as (1) nursing documentation; (2) chart; and (3) hand washing. Within each of these, specific behaviors (eg, staff verbally instructing visiting teams to wash hands) or outcomes of behavior that needed to be performed to achieve the desired ends were identified. Outcomes of behavior (eg, all entries delegated to others [eg, healthcare assistant (HCA), student, or new starter] are countersigned by a nurse) were used as proxies of behavior, as it could not be guaranteed that an observer would actually witness a nurse countersigning during an observation. However, the observer could examine the documentation and assess whether this was being done or not. In this way, it could be determined whether or not staff were engaging in the desired behaviors. The measures therefore contained both behaviors and “outcomes” of behavior.

Based on this input, a common behavioral checklist was developed to cover both units. This contained 36 behavioral items within 3 separate categories: (1) nursing documentation; (2) chart; and (3) hand washing (see Figure 1). The documentation category contained 10 proxy behaviors focused on the facilitation of communication between healthcare providers (ie, behaviors 1, 4, 5, 6, and 10); staff accountability (ie, behaviors 2, 3, and 9); and clarity and legibility of staff handwriting (ie, behaviors 3, 7, and 8). The chart category contained 15 proxy behavioral items focused on 4 administrative requirements (behaviors 11, 12, 13, and 14), 9 nursing care requirements (ie, behaviors 15, 16, 17, 18, 19, 20, 21, 23, and 24) and 2 accountability requirements (ie, behaviors 22 and 25). The hand wash category contained 11 behavioral items focused on cleanliness of sinks and bins (ie, behaviors 26, 27, and 31); availability of supplies (ie, behaviors 28, 29, 30, and 31); and actual hand washes (ie, behaviors 33, 34, 35, and 36).

Figure 1

Behavioral checklist used in both ICUs.

Abbreviations: HCA, healthcare assistants; ICU, intensive care unit; ID, identity; MRSA, methicillin-resistant Staphylococcus aureus.

Each checklist contained 3 columns: compliance; noncompliance, and unseen. The compliance and non-compliance columns were used to calculate an observed percent compliance score, which was used as the primary dependent variable in this study. The unseen column was used when a specific behavior did not take place during an observation session (eg, if staff were not actually seen to verbally instruct visiting people to wash hands).

The completed checklists were returned to the project coordinator for comment and, as the project progressed, some behaviors were removed or added (following review discussions with those involved) to improve the value and relevance of the observations.

Observer recruitment and training

The project team (coordinator and champions) recruited and trained eight volunteer HCAs as observers. Each was trained by the project team how to observe, how to give verbal feedback, and how to set participative improvement goals. The observers also visited the petrochemical plant to be given reassurance about the whole process by seeing it actually working. Subsequently, the HCAs were given a 1-week period (or shift cycle) to practice making observations and to reassure them they should do so without anybody questioning their veracity (ie, note what they actually saw, not what they or others thought they should see). However, checks were made by the coordinator to ensure that observers were using the scoring system correctly (eg, using frequency counts, not ticks) and that observations were actually being done. The observers were not involved in the day-to-day administration of the project, which was completed entirely by the project team.

Establishing baselines

The project was designed so that 14 observations could potentially be conducted per week (ie, 7 observations per week in each of the 2 ICUs). To establish a baseline for a work area, each observer monitored everyone in their respective units (ie, the whole group: nurses, other HCAs, doctors, and other members of the care team, and visitors, etc) for approximately 20-minute periods on a shift within their normal working time on the ICU, once everyday, for 1 week.

Each observer randomly chose the time of day during a work shift when their observation would take place. To minimize the potential impact on performance, instructions were given to the observers not to give verbal feedback about the observation results during this period, and no formal written or posted graphical feedback was provided. An online computerized behavioral tracking program (Cooper, Brown, et al 1999) was used to record and analyze observation results.

Goal-setting

At the end of the baseline period, group improvement goals were collectively set for the intervention (Cooper et al 1994) by the unit staff, who were led by the workgroup observers, all of whom worked from the same goal-setting script (Cooper 1993). Each ICU's goal was then posted on the ICU's graphical feedback chart as a line at the appropriate percent goal level.

Monitoring and feedback.

After each unit's goal-setting session the observers continued to monitor their colleagues' behavior on a daily basis for 20 minutes at randomly chosen times of day. All behaviors on the checklists were observed during this 20-minute period. Instead of walking around (as would normally be the case), observers tended to stay at the nurses station as this was a central vantage point in each ICU. In terms of charting and documentation, the previous 24 hour's charts and documents were also assessed during this period. Observation data were passed to the project coordinator for data entry at the end of each working day.

The observation data for each ICU were analyzed weekly by computer to provide the percent compliance which was posted on their graphical feedback chart. A written analysis that reported results by category of behavior (eg, documentation, charts, and hand washing) was discussed at weekly 30-minute group feedback meetings. Monthly reports that summarized the ICU's average percent compliance score and a percentage of observations missed were also produced for senior management meetings. This monitoring and feedback process was followed for the period of the pilot, 26 weeks.

Patient infection data

The outcome data used to assess the impact of the OBM process were the weekly prevalence of MRSA in the critical care unit. These data are routinely collected and monitored as an integral aspect of the hospitals' management procedures. Prevalence rates are calculated each Friday by the number of patients colonized with MRSA on each critical care ward.

Archival data were examined to test the effects of the intervention on patient infection rates. MRSA rates at the hospital were not “bad” compared with other similar units, so this pilot was not about fixing poor performance, it was about adding new and additional capability to an already well-performing unit so that it will do even better. In terms of hospital-acquired infections of MRSA, 6-month mean incidence rates were calculated for the 18 months prior to the study and for the 6-month behavioral intervention period.

Statistical analyses

Testing the statistical significance of any behavioral improvements is not a simple matter. The type of experimental design employed in this study violates major assumptions of typical factorial designs and the number of data points is insufficient for times–series analysis (Pritchard et al 1989). Nonetheless, the data were subjected to an independent group's one-way analysis of variance (ANOVA) procedure to ascertain whether any behavioral improvements were due to chance variation.

ANOVAs are designed to test differences between several groups of mean average scores and are based on the ratio of between-group variability and within-group variability. A significant F statistic signals only that the group means are unequal (ie, different): it does not pinpoint where the differences are. This requires the use of post-hoc analyses such as the “Scheffe” test.

In this study, the levels of the factor were computed as sequential 4-week time intervals to create groups of mean average behavioral scores. However, an independent group's design results in inflated error terms. This, in combination with the small number of data points in each 4-week period, signals that large mean differences between the time periods are required to achieve significance. As such, the statistical results will be considered conservative.

In terms of hospital-acquired infections (ie, MRSA), because of the relatively small data sets (n = –26) there is a need to eliminate the possibility of a type II error (ie, detecting and accepting any significant differences that do not really exist). This is usually achieved via power analysis which tests the probability of detecting a particular effect with different sample sizes (Witte 1989). Power analysis makes use of the level of significance (ie, Alpha), sample size, and treatment effect size (ie, the gain in scores divided by the spread of scores). Cohen's d is calculated using the mean of pre-treatment scores minus the mean of post-treatment scores, and divided by the pooled sample standard deviation. The average treatment effect size for most organizational interventions is 0.44 (Guzzo et al 1985). Borenstein and Cohen's (1988) computer program was used to conduct power analysis on the statistical data obtained from t-tests comparing the means of 6-monthly MRSA frequency rates. Alpha was set at 0.05, using 2-tails and power of 0.80 (Bausell 1986). The output is a treatment effect size (Cohen's d) and beta (β) statistic which represents the probability of retaining a false null hypothesis.

Results

A steady overall improvement in behavior was observed across the unit (see Figure 2). Global performance increased by approximately 15 points. From a baseline average of 72% (range: 66%–80%) performance increased to an average 86% (range: 69%–89%) by week 26.

Figure 2

Degree of behavior change in both intensive care units (ICUs).

The ANOVA procedure revealed statistically significant behavioral change (F[6,19] = 5.37, p < 0.01) for the combined data from the ICUs. One-way ANOVAs were also conducted on the category data (ie, nursing documentation, chart, and hand-washes) for the individual wards. In Table 1, analysis reveals statistically significant changes in ICU2's hand-washing behavior (F[6,19] = 5.46, p < 0.01) only. In ICU3, statistically significant behavioral changes were obtained only for completion of nursing documentation (F[6,19] = 2.99, p < 0.05). Although not statistically significant, behavioral change was moving in the right direction for each of the remaining categories. Overall, therefore, the ANOVA results suggest that the OBM procedures helped to significantly change behavior over the study period

Table 1

One-way ANOVA results by behavioral category by ICU

Behavioral category	Source of variation	Sum of squares	DF	Mean square	F	Sig
ICU2
Documentation	Between groups	586.40	6	97.73	1.72	n.s
	Within groups	1080.25	19	56.86
	Total	1666.65	25
Charting	Between groups	537.38	6	89.56	2.03	n.s
	Within groups	840.50	19	44.24
	Total	1377.88	25
Hand washing	Between groups	2316.63	6	386.11	5.46	0.01
	Within groups	1344.75	19	70.78
	Total	3661.38	25
ICU3
Documentation	Between groups	639.54	6	106.59	2.99	0.05
	Within groups	676.00	19	35.58
	Total	1315.54	25
Charting	Between groups	292.13	6	48.69	0.85	ns
	Within groups	1085.75	19	57.14
	Total	1377.88	25
Hand washing	Between groups	587.96	6	97.99	0.99	ns
	Within groups	1867.00	19	98.26
	Total	2454.96	25

Abbreviations: DF, degree of freedom; F, F ratio; ICU, intensive care unit; Sig, significance; ns, nonsignificant.

Patient infection results

Paired sample t-tests revealed no significant differences in MRSA reduction between the first (mean [m] = 1.92, standard deviation [sd] = 1.41, degrees of freedom [df] = 26) and second (m = 1.58, sd = 1.27, df = 26) 6-month periods. A significant difference (t-test [t] = 2.15, df = 25, p < 0.05) was obtained between the second and third (m = 0.92, sd = 0.94, df = 26) 6-month periods (see Figure 3). Thus in the 6 months prior to the behavioral intervention, MRSA had significantly reduced due to existing infection control measures. A paired samples t-test between the immediate (third) pre-intervention period and the study intervention period (m = 0.23, sd = 0.43, df = 26) again revealed a significant pre- and post-intervention difference (t = 3.49, df = 25, p < 0.01), suggesting the OBM intervention helped to significantly reduce the total incidence of MRSA in the two ICUs. Figure 3 also shows the standard deviation shrank quite dramatically during the intervention period compared with earlier 3, 6-month pre-intervention periods. Thus, the OBM procedures also appear to exert an impact on the consistency of reduction in patient-acquired infections.

Figure 3

Six-monthly means and standard deviations for MRSA from December 2002 to October 2004.

Abbreviations: MRSA, methicillin-resistant Staphylococcus aureus.

In terms of statistical power, the actual mean values and standard deviations from the pre- and post-intervention periods were entered into the power analysis computer program (Borenstein and Cohen 1988). This revealed that 18 was the minimum sample size to avoid a type II error. With a sample size of 26 weeks in each group, β was 0.08 (power of 0.92). In other words, there is a 92% chance that the statistically significant differences are real.

The program also calculated a treatment effect size of 0.95 (Cohen's d), which is considered large (Cohen 1988). The effect size was multiplied by the pooled sample sd of 0.73 (Aamodt 2004) to ascertain its practical significance. The product (0.70) indicates that adding a behavioral intervention of the type described here to existing patient infection controls would be expected to help reduce patient infection of MRSA by approximately 70%.

Discussion

Behavior change

The OBM intervention described here appears to have been very successful in helping to change the quality care behaviors of personnel ranging from ICU staff to visiting teams. Behavior changes were observed in both wards for all 3 categories. Only 1 category of behaviors in each ward exhibited statistical significance. Unlike educational interventions (Oxman et al 1995), which tend to produce mixed results, the magnitude of behavior change reported here is in accordance with the wider behavioral management literature (Stajkovic and Luthans 2003) which has repeatedly demonstrated the utility of OBM procedures for improving behavioral performance in a wide range of organizational settings. To a large degree, the efficacy of OBM can be attributed to the joint effects of motivation (eg, goals) and cognition (ie, feedback) controlling action (behavior) within a clearly defined measurement structure.

From the motivational perspective, some evidence (Ambrose and Kulik 1999) suggests that compared with assigned goals, individual's self-set goals increase commitment to goal-achievement. Higher levels of commitment lead to higher levels of performance (Locke and Latham 1990). Given that nurses in critical care wards are often assigned to 1 patient at a time, it may prove useful to compare the effects of specific group goals (ie, for an ICU) against specific individual self-set goals (Seijts and Latham 2000). Some evidence suggests participative goals are more effective for groups than individuals due to the joint effects of psychological and sociological processes (Erez and Arad 1986). As such, it may also prove useful to examine the joint effects of individual self-set goals operating in conjunction with group goals. This may reveal larger effects on behavior than either goal-setting method alone

From a cognitive perspective, feedback is known to be a key variable in OBM (eg, Alvero et al 2001) and in most other types of performance improvement initiatives (eg, Six Sigma, Total Quality Management). The detailed feedback provided in this study about each ICU's behavioral performance on a weekly basis, inevitably contributed to the behavior change exhibited. It is doubtful most medical settings do provide such detailed feedback on a sufficiently regular basis (eg, weekly) for desired behavioral performance to improve or be maintained. Some work (Babcock et al 1992) has shown nurses prefer verbal over written feedback, but no work has examined different feedback frequency regimes or feedback types for groups and individuals to try to establish the optimum for a critical care setting.

In terms of a clearly defined structure, behavioral measurement is an essential component of OBM, as “what gets measured, gets done” (Deming 1986). The development of behavioral checklists allows staff to explore problematic issues in their sphere of activity, which leads to a common understanding and ownership of the improvement process. Consistent with goal-setting theory (Locke and Latham 1990), the actual monitoring focuses people's attentions and actions on improving specific behaviors. The measurement data provides evidence about actual levels of desired behavior, which facilitates the provision of feedback about performance and the tracking of goal-achievement. In turn, this helps reset group norms, whereby social processes induce “peer pressure” to conform (Mullen and Copper 1994). Although difficult, separating out the structural effects of the process from the motivational and cognitive components of OBM could provide a fruitful avenue of research. Staff reactivity to the observation process was generally positive. However, behavioral compliance may have increased primarily due to the presence of the observer rather than the through the motivational effects of goal-setting or informational effects of feedback. The influence of observers on the observed (Alvero and Austin 2004), observation frequency, and the optimum number of behaviors to be observed at any one time, are important issues awaiting scientific enquiry in the wider OBM literature. Conceivably, such structural variables moderate or mediate the effects of either motivation or cognitive components of OBM on behavior change. For example, observer presence is likely to mediate the goal–performance relationship. However, only future research will be able to shed light on such issues.

Outcome change

The intervention also appears to have contributed to reductions in MRSA rates, suggesting OBM provides a valuable addition to other forms of HAI intervention such as screening (Boyce 2001), isolation procedures (Chaix et al 1999), and cleaning (Griffith et al 2000). The exact magnitude of impact is unknown due to the presence of other infection controls measures. MRSA rates had already declined from December 2002 to November 2003 by some 18%, and by a further 41% between December 2003 and April 2004. At an annual rate of decline of 18%–23%, it could be argued that the intervention only contributed an additional 11% reduction to what might have been achieved anyway. Although not inconsequential, perhaps the major contribution of the intervention was helping to ensure the consistency of impact of the existing measures (as demonstrated by the large shrinkage in the standard deviation during the intervention period). This shrinkage in the standard deviation also accounts for the large effect size (Cohen's d = 0.95) suggesting a behavioral intervention will help to reduce rates by about 70% (compared with the previous 6-month period). Future research comparing the effects of existing infection control strategies with OBM techniques (separately and in combination) may provide some useful insights that help to improve both types of HAI eradication strategies.

Costs and benefits

The intervention was not overly resource-intensive. Cost expenditures amounted to only several hundred pounds for clerical materials and some additional cleaning items. There were also the costs for the training time of staff as detailed in the training sections above. The other costs involved were those associated with the time of the staff involved (ie, observation time, coordination time, and feedback meetings). The activities of all participants, except the coordinator, were built into their normal time at work without great difficulty. The coordinator was occupied with this role for about one third of the normal working day, so was able to continue with a large proportion of other work as a clinical auditor. However, the requirement to devote sufficient attention to the project was respected by the ICU management, who “protected” this necessary time. While this cost analysis is simplistic and may understate the true costs somewhat because it does not include opportunity costs (eg, the value of the work the staff could otherwise have been doing were they not engaged in this study), which are low relative to those expected for many interventions aimed at changing culture.

Based on the findings of Chaix and colleagues (1999), which indicated ICU isolation interventions become cost-effective when HAI is reduced by 14%, the degree of MRSA reduction in this study was sufficient to provide cost and capacity/benefits. Even if the OBM impact was only an 11% reduction in MRSA, it appears to become cost-effective when MRSA is reduced by only 1 or 2 cases. This is very simply illustrated by taking the daily average rate of occupancy by MRSA patients in the two 26-week periods before and after the introduction of the intervention, which fell from about 1 to about 0.25. That released the equivalent of three quarters of a bed every day, which might have been available to other patients. At a typical ICU-bed cost of £2000 per day, this corresponds to over £500 000 per annum of extra capacity value. Eleven percent of this figure is £55 000. As the NHS moves to the planned “Payment by Results” regime, such additional capability for little or no extra revenue cost (and zero new capital) will represent a major, significant, and realizable financial opportunity. Other financial benefits include reduced expenditures on screening and lab requirements, eradication therapy, overtime/agency costs on the wards, and reduced costs from claims and litigation. If similar results to those achieved in this study were widely replicated and sustained, it is estimated the United Kingdom's NHS could save a significant part of the costs of avoidable HAI.

Similarly, all the evidence from other applications of OBM suggest that there will be corresponding improvements expected in those areas of clinical performance that arise from better charting and the other behavioral improvements observed in this pilot. There are no data to demonstrate this on this occasion (because the pilot was not set up to provide it), but beneficial outcomes such as fewer accidents and untoward incidents, reduced medication errors would be expected to appear as this OBM approach became embedded in the organization.

Feasibility and acceptability of the behavioral approach

For other ICUs to adopt a similar developmental intervention approach, resource requirements can be assumed to follow the pattern shown by this pilot. For comparison, the time required of the coordinator in this case (such as a third of 1 persons' working day for the 140 staff involved) is reasonably analogous to the full time involvement of a single person in the coordinator role for the corresponding activity at the nearby petrochemicals complex (where several hundred staff are involved). There is ample evidence from other applications of the methodology that these resource requirements are typical.

The importance of ongoing managerial support, however, must not be underestimated (Cooper, 2005). There must be an expected and agreed level of commitment and support from ICU managers if the OBM intervention is to succeed in their own areas. Their commitment must include provision of resources to allow the staff to work within the OBM framework, which may be identified as extra or new equipment and certainly requires protected time for staff to do training and observations. Although, this pilot has, so far, been implemented without any additional resources to the wards, a stage will be reached where additional support is required. The project is currently relatively small scale in ICU, but as it grows so will the demands on ICU staff time.

Study limitations

It is possible that the behavior of those being monitored differed during times when observations did not take place. To some extent, this was controlled by the random observation schedule adopted by observers. Although it remains a real possibility, it would have been difficult for staff to mask their normal behavior specifically for the observation periods or from other staff. A further potential scientific limitation stems from the lack of any inter-observer reliability checks. However, this was not a “classical” experimental study, where independent variables were manipulated (which would require reliability checks). The purpose of this study was merely to ascertain if the OBM process was feasible and practical in a critical care setting. Observations are obviously the lynch-pin of any OBM intervention. As such, it was considered more important to maintain commitment to the process by encouraging staff to conduct observations. Reliability checks could be perceived to question observer integrity, which could have resulted in no observations being completed at all. Such issues present very real obstacles to overcome in the workplace when introducing a behavioral approach. Persuading employees to conduct behavioral observations can be fraught with difficulty. Often perceived as “spying”, some American labor unions (eg, United Auto Workers [UAW], Transport Workers Union of America [TWU]) officially disapprove of behavioral approaches in the workplace (Frederick and Lessin 2000) as they can generate conflict among workers and drive problems underground. Although, patently untrue in the majority of cases (Cooper, 2003), poorly implemented cases can reinforce this argument. In fact, the lack of inter-reliability checks is viewed as a positive strength of the study, as it means the method can be transferred to a multitude of settings without a scientific bias, which may deter some. Certainly, the vast majority of applications in industry do not use inter-observer reliability checks, but still exert their intended impact (eg, injury reduction, productivity improvement). The extent to which behavior improved and MRSA infection decreased strongly suggests that the above limitations did not present major problems here.

Summary

This study has demonstrated OBM approaches to improving quality care practices are feasible, practical, and relatively low cost. However, much more work is required to identify the optimum. It is hoped the work described here will stimulate others to adopt and research the approach across a wide scope of medical settings.