Sustaining reductions in catheter related bloodstream infections in Michigan intensive care units: observational study.

OBJECTIVES: To evaluate the extent to which intensive care units participating in the initial Keystone ICU project sustained reductions in rates of catheter related bloodstream infections. Design Collaborative cohort study to implement and evaluate interventions to improve patients' safety.
SETTING: Intensive care units predominantly in Michigan, USA. INTERVENTION: Conceptual model aimed at improving clinicians' use of five evidence based recommendations to reduce rates of catheter related bloodstream infections rates, with measurement and feedback of infection rates. During the sustainability period, intensive care unit teams were instructed to integrate this intervention into staff orientation, collect monthly data from hospital infection control staff, and report infection rates to appropriate stakeholders. MAIN OUTCOME MEASURES: Quarterly rate of catheter related bloodstream infections per 1000 catheter days during the sustainability period (19-36 months after implementation of the intervention).
RESULTS: Ninety (87%) of the original 103 intensive care units participated, reporting 1532 intensive care unit months of data and 300 310 catheter days during the sustainability period. The mean and median rates of catheter related bloodstream infection decreased from 7.7 and 2.7 (interquartile range 0.6-4.8) at baseline to 1.3 and 0 (0-2.4) at 16-18 months and to 1.1 and 0 (0.0-1.2) at 34-36 months post-implementation. Multilevel regression analysis showed that incidence rate ratios decreased from 0.68 (95% confidence interval 0.53 to 0.88) at 0-3 months to 0.38 (0.26 to 0.56) at 16-18 months and 0.34 (0.24-0.48) at 34-36 months post-implementation. During the sustainability period, the mean bloodstream infection rate did not significantly change from the initial 18 month post-implementation period (-1%, 95% confidence interval -9% to 7%).
CONCLUSIONS: The reduced rates of catheter related bloodstream infection achieved in the initial 18 month post-implementation period were sustained for an additional 18 months as participating intensive care units integrated the intervention into practice. Broad use of this intervention with achievement of similar results could substantially reduce the morbidity and costs associated with catheter related bloodstream infections.

Introduction

Catheter related bloodstream infections cause considerable morbidity, mortality, and healthcare costs.1 2 An estimated 82 000 catheter related bloodstream infections and up to 28 000 attributable deaths occur in intensive care units annually,3 and each infection costs about $45 000 (£28 000; €31 000).4 In an ongoing quality improvement project, known as the Michigan Health & Hospital Association (MHA) Keystone ICU project, these infections were substantially reduced in 103 participating intensive care units.5 The median infection rate per 1000 catheter days dropped from 2.7 at baseline to 0 within three months after implementation of an evidence based intervention. Eighteen months after implementation, infection rates had decreased by 66% from baseline. However, whether these initial results were sustained was not known.

Limited evidence assessing the sustainability of quality improvement projects beyond the initial implementation and evaluation period is available.6 7 To evaluate sustainability, a quality improvement project must have an adequate infrastructure to sustain activities beyond its initial phase. After the 18 month post-implementation evaluation period, most hospitals participating in the Keystone ICU project continued to submit data on infection rates. The objective of this study was to evaluate the extent to which intensive care units participating in the initial Keystone ICU project sustained reductions in rates of catheter related bloodstream infection. We hypothesised that the bloodstream infection rate would remain low during the sustainability period relative to baseline.

Methods

Study design

The Keystone ICU project was designed as a prospective cohort study and used a collaborative model involving the Johns Hopkins Quality and Safety Research Group, the MHA Keystone Center for Patient Safety and Quality, and participating hospitals. All hospitals in Michigan with an intensive care unit were invited to participate in the project. To participate, they had to form an intensive care unit team, agree to complete the project work, and send a signed letter of commitment from a senior hospital executive with a list of team members. At a minimum, the team had to include a senior executive, the intensive care unit director and nurse manager, and a physician and nurse in the intensive care units who could both dedicate 20% of their effort to project activities. The initial implementation and evaluation phase was funded by the Agency for Healthcare Research and Quality. During this phase, individual intensive care units implemented two cultural interventions first and then two interventions targeting patients’ safety (in any order).5 The four interventions were implemented at three month intervals. As a result, the intervention targeting catheter related bloodstream infection had a staggered timing for its implementation.

At the end of the initial funding, intensive care unit teams were instructed to sustain the bloodstream infection intervention. They were to maintain a team, integrate the intervention into staff orientation, continue collecting monthly data on catheter related bloodstream infections and catheter days in collaboration with hospital infection control staff, and continue reporting infection rates per 1000 catheter days to appropriate stakeholders. We lacked funding to formally evaluate any intensive care units’ compliance with each of these components. Informally, however, nearly all units reported that they were continuing all of these components. After the initial funding ended, intensive care units paid MHA to continue working with the Johns Hopkins team on new quality improvement interventions not related to catheter related bloodstream infections. Thirteen intensive care units declined further participation specifically because of the MHA fee. Compared with all intensive care units that continued to participate, the 13 units that dropped out of this project were more likely to be at a teaching hospital (93% v 65 %, P=0.04), but they did not vary in the median number of hospital beds (383 v 338, P=0.70) and did not have significantly different rates of catheter related bloodstream infection in the first and final quarters of the initial 18 month post-implementation period or quarterly rates of improvement over that period.

Intervention

We used a conceptual model to increase use of evidence based interventions and improve safety culture. This model has been described elsewhere.8 9 10 The intervention evaluated in this study targeted clinicians’ use of five evidence based recommendations to reduce catheter related bloodstream infections from the Centers for Disease Control and Prevention (CDC).2 5 Four recommendations related to insertion of the catheter: hand washing, using full barrier precautions, cleaning the skin with chlorhexidine, and avoiding the femoral site when possible. The fifth recommendation was to remove unnecessary catheters. Strategies to increase the use of these evidence based recommendations are described elsewhere.5 11 Teams reported all catheter related bloodstream infections and number of days of use of catheters (catheter days). For the remainder of this paper we will use the term bloodstream infection to indicate catheter related bloodstream infection.

Outcomes

The primary outcome was the quarterly rate of bloodstream infections during the 18 month sustainability period. To test sustainability of the intervention, we analysed trends in infection rates for the 18 month initial evaluation period compared with the 18 month sustainability period. Specifically, a total of 103 intensive care units from 67 hospitals contributed data to the initial evaluation period. Some intensive care units did not contribute to all time periods, and 13 of the 103 units declined to participate in the sustainability period, leaving 90 for which we evaluated the primary outcome. All hospitals used standard definitions of catheter related bloodstream infections provided by the CDC.12 Our analysis included hospital teaching status (binary variable) and bed size (continuous variable) to show any effect on distribution of catheter days. These variables came from the American Hospital Association database.

Statistical analysis

We summarised rates of bloodstream infection as medians and interquartile ranges and as means and standard deviations. We used generalised linear latent and mixed models,13 14 with a Poisson distribution, to compare the quarterly bloodstream infection rate from baseline to the end of the initial 16-18 month evaluation period and for the subsequent 18 month sustainability period (from 19-21 months to 33-36 months post-implementation). In the models, we used robust estimation of variance, adjusted for hospital teaching status and bed number, and included two level random effects to account for nested clustering within the data13 15: intensive care units over time and hospitals within regions. We also evaluated whether the relative change in quarterly bloodstream infection rate was different during the initial evaluation period compared with the sustainability period by using a linear spline model, which allowed the relative change in quarterly infection rate to change at the end of the initial 18 month period.16

To assess the sensitivity of our results to missing data and to potential non-reporting bias, we replicated the analysis including only those 43 intensive care units that reported continuous data from baseline to the end of the sustainability period. To assess the sensitivity of our results to a few units that reported quarters with large rates of bloodstream infections (primarily driven by small numbers of catheter days), we replicated the analysis excluding the quarters of data in which catheter days were in the lowest 10% across all quarters. Finally, we replicated our models assuming an over-dispersed Poisson distribution (known as a negative binomial distribution) for the number of bloodstream infections to assess the sensitivity of our results to the Poisson assumption. All P values are two sided; we considered a P value of ≤0.05 to be statistically significant. We used Stata (version 9.1) for all analyses.

Results

During the sustainability period, 90 (87%) of the original 103 intensive care units from 61 (91%) of the original 67 hospitals reported 1532 intensive care unit months of data and 300 310 catheter days. Thirteen units dropped out because they chose not to pay the MHA’s fee. The distribution of total catheter days by hospital teaching status and bed number was consistent between the initial evaluation period and the sustainability period (table 1).

Table 1

 Number of intensive care units (ICUs) and catheter days by hospital type and size for each study period

Hospital type/size	Baseline period before implementation of intervention			During implementation of intervention (3 months)			Initial evaluation period (18 months)			Sustainability period (18 months)
	No of ICUs*	Catheter days (%)		No of ICUs	Catheter days (%)		No of ICUs	Catheter days (%)		No of ICUs	Catheter days (%)
All hospitals	55	41 506 (100)		96	57 033 (100)		103	300 175 (100)		90	300 310 (100)
Teaching status:
Teaching	33	29 407 (71)		66	44 752 (78)		71	231 595 (77)		60	230 836 (77)
Non-teaching	22	12 099 (29)		30	12 281 (22)		32	68 580 (23)		30	69 474 (23)
No of beds:
<200	13	3 266 (8)		17	3 066 (5)		19	16 349 (5)		17	16 398 (5)
200-299	12	13 321 (32)		19	10 792 (19)		23	67 970 (23)		22	59 122 (20)
300-399	12	10 531 (25)		20	12 314 (22)		20	66 765 (22)		17	63 650 (21)
≥400	18	14 388 (35)		40	30 861 (54)		41	149 091 (50)		34	161 140 (54)

*Of 103 ICUs participating in initial evaluation, 48 did not contribute baseline data—40 implemented intervention during baseline period of study; 8 did not report baseline data.

Overall, 1394 possible quarters of bloodstream infection data existed for the entire study (baseline data for 55 intensive care units, plus 13 subsequent quarters for the three month implementation period and the 36 month post-implementation period for 103 units) (table 2). Seventy-eight quarters (roughly 5%) of data were lost when the 13 units left the collaboration, and 65 quarters (roughly 5%) of bloodstream infection rates were missing for the remaining units.

Table 2

 Catheter related bloodstream infection rates from baseline until 36 months after quality improvement intervention

Study period	No of ICUs	Median (IQR) No of infections	Median (IQR) catheter days	Infection rate		Incidence rate ratio* (95% CI)
				Median (IQR)	Mean (SD)
Baseline	55	2 (1-3)	551 (220-1091)	2.7 (0.6-4.8)	7.7 (28.9)	Reference
During implementation	96	1 (0-2)	447 (237-710)	1.6 (0-4.4)	2.8 (4.0)	0.81 (0.61 to 1.08)
After implementation—initial evaluation period:
0-3 months	95	0 (0-2)	436 (246-771)	0 (0-3.0)	2.3 (4.0)	0.68 (0.53 to 0.88)
4-6 months	95	0 (0-1)	460 (228-743)	0 (0-2.7)	1.8 (3.2)	0.62 (0.42 to 0.90)
7-9 months	96	0 (0-1)	467 (252-725)	0 (0-2.0)	1.4 (2.8)	0.52 (0.38 to 0.71)
10-12 months	95	0 (0-1)	431 (249-743)	0 (0-2.1)	1.2 (1.9)	0.48 (0.33 to 0.70)
13-15 months	95	0 (0-1)	404 (158-695)	0 (0-1.9)	1.5 (4.0)	0.48 (0.31 to 0.76)
16-18 months	95	0 (0-1)	367 (177-682)	0 (0-2.4)	1.3 (2.4)	0.38 (0.26 to 0.56)
After implementation—sustainability period:
19-21 months	89	0 (0-1)	399 (230-680)	0 (0-1.4)	1.8 (5.2)	0.34 (0.23 to 0.50)
22-24 months	89	0 (0-1)	450 (254-817)	0 (0-1.6)	1.4 (3.5)	0.33 (0.23 to 0.48)
25-27 months	88	0 (0-1)	481 (266-769)	0 (0-2.1)	1.6 (3.9)	0.44 (0.34 to 0.57)
28-30 months	90	0 (0-1)	479 (253-846)	0 (0-1.6)	1.3 (3.7)	0.40 (0.30 to 0.53)
31-33 months	88	0 (0-1)	495 (265-779)	0 (0-1.1)	0.9 (1.9)	0.31 (0.21 to 0.45)
34-36 months	85	0 (0-1)	456 (235-787)	0 (0-1.2)	1.1 (2.7)	0.34 (0.24 to 0.48)

ICU=intensive care units; IQR=interquartile range.

*Calculated with use of generalised linear latent and mixed model,13 with robust variance estimation and random effects to account for clustering of catheter related bloodstream infections within ICUs over time and clustering of hospitals within geographical regions; rates of catheter related bloodstream infections during implementation, initial evaluation, and sustainability periods compared with baseline (pre-implementation) values, adjusted for hospital’s teaching status and number of beds.

Total catheter days were consistent during each quarter in the initial evaluation period and the sustainability period. The mean and median rates of bloodstream infection decreased from 7.7 and 2.7 (interquartile range 0.6-4.8) per 1000 catheter days at baseline to 2.3 and 0 (0.0-3.0) at 0-3 months after implementation of the intervention and to 1.3 and 0 (0-2.4) at 16-18 months after implementation, and they were sustained at 1.1 and 0 (0.0-1.2) at 34-36 months post-implementation (table 2). The multilevel Poisson regression model showed a significant decrease in bloodstream infection rates during all study periods compared with baseline rates; incidence rate ratios decreased from 0.68 (95% confidence interval 0.53 to 0.88) at 0-3 months to 0.38 (0.26 to 0.56) at 16-18 months after implementation, with a sustained improvement of 0.34 (0.24 to 0.48) at 34-36 months (table 2).

From baseline to the end of the initial 18 month evaluation period, the mean rate of bloodstream infection decreased significantly by 12% (95% confidence interval 9% to 15%) per quarter (figure). During the 19-36 month sustainability period, the quarterly bloodstream infection rate did not significantly change from the rate achieved at the end of the initial evaluation period (1% decrease, 95% confidence interval 9% decrease to 7% increase).

Catheter related bloodstream infection rate as a function of time. Circles represent mean infection rate per quarter; thick blue line represents estimated mean rate of infection modelled as a linear-spline function in which rate of change in mean infection rate is allowed to change after 16-18 month period; thin red lines represent changes in observed infection rates over time within a random sample of 50 intensive care units

Forty-three intensive care units reported data continuously from baseline to the end of the sustainability period. A sensitivity analysis including only these 43 units resulted in similar findings to our main analysis including all units: an estimated 13% (9% to 16%) decrease in the mean rate of bloodstream infection per quarter during the initial evaluation period followed by a less than 1% decrease (4% decrease to 5% increase) per quarter during the sustainability period. The primary results were not substantially changed by exclusion of infection data for quarters in which the total catheter days were in the lowest 10% (<67 days). Finally, the results changed little when we used regression models assuming an over-dispersed Poisson distribution for the number of infections.

Discussion

Our findings show that the markedly reduced rates of bloodstream infection achieved in the initial evaluation period of the Keystone ICU project were sustained for an additional 18 months. The median rate of infection remained at zero for the entire 18 month sustainability period, with a greater than 60% reduction in infection rates from baseline sustained at the end of the 36 month period.

Our project is one of relatively few robustly designed and evaluated large scale quality improvement projects that have shown substantial improvements. Even fewer projects have evaluated the sustainability of such results. Achieving sustainability has been a major challenge for the quality improvement field.17 18 19 The Keystone ICU project coupled a formal model to translate evidence into practice with a comprehensive patient safety intervention to improve culture, educate staff, learn from mistakes, and involve senior leaders.8 10 During interviews, intensive care unit teams noted several factors that were important in sustaining this project, including continued feedback of infection data that the team perceived as valid, improvements in safety culture that occurred as part of the overall Keystone ICU project, an unremitting belief in the preventability of bloodstream infections, involvement of senior leaders who reviewed infection data and provided teams with the resources needed, and a shared goal rather than a competition to reduce infection rates throughout the state of Michigan. Other studies describe similar methods to facilitate sustainability, such as active support from leaders and an infrastructure that supports quality improvement methods, training of staff, involving key internal and external stakeholders,6 20 21 22 23 alignment of project and organisational goals,21 multidisciplinary teams and collaborations,6 and inclusion of change management that also encourages local adaptation and rewards innovation and change.21 Quality improvement initiatives that demonstrate value are more likely to be sustained.21 24 We did not formally evaluate the factors associated with continued improvement. However, the quality improvement model used in the Keystone ICU project and our recommendations to teams at the end of the initial evaluation period to maintain a team, orient new staff, collect monthly data, and report infection rates to appropriate stakeholders target many of the recommendations described in the literature.

Limitations of study

This research has several new limitations not reported in our evaluation of the initial evaluation period.5 Firstly, we cannot evaluate whether the results would have been sustainable if intensive care units were blinded to their infection rates, because our model to change practice included feedback of data so units could evaluate their performance.8 Secondly, we had insufficient funds to formally evaluate mechanisms that contributed to the success of this sustainability, so we cannot evaluate the relative importance during the sustainability phase of periodic meetings between participating units and the Keystone Center. Although these meetings focused on implementing new safety interventions, they may have influenced the sustainability of the bloodstream infection intervention. Also unknown was the impact of Blue Cross Blue Shield of Michigan’s quality payment incentive on a hospital’s decision to continue to participate in the Keystone ICU project. In the first year, hospitals received an incentive payment if they submitted 90% of the required bloodstream infection data. In subsequent years, the incentive payment was based on performance thresholds (statewide infection rate compared against CDC pooled mean) for continued reduction of infection rates. Thirdly, we did not evaluate the use of new technologies, such as impregnated dressings or catheters and chlorhexidine baths, on rates of infection.25 26 27 28

Implications and future research

These findings have important public health consequences. If the multifaceted quality improvement intervention and collaborative model were implemented in all intensive care units across the United States, and the results were similar to those achieved in Michigan, substantial and persistent reductions could be made in the 82 000 infections, 28 000 deaths, and $2.3 billion costs attributed to these infections annually.3 Moreover, the use of this quality improvement model to reduce other complications may further improve quality and reduce costs of care.

A prominent item on the research agenda for quality improvement initiatives is identifying methods that sustain a successful project. Sustainability is often vague and may not be formally separated from the initial implementation and evaluation of the project.22 Although we do not have empirical data to support this statement, both our experience and theoretical reasons support the idea that sustainability should be examined separately from implementation. Weick and Quinn noted that organisations and teams vary in whether they are best suited for episodic or continuous change.29 Moreover, early, mid, and late implementers have differing characteristics and motivations for implementing quality improvement interventions.30 Some implementers focus efforts on short term results, whereas sustainers generally focus on long term results. Teams that are good “implementers” may not be the same as those that are good “sustainers.” More research is needed to better understand when implementation ends and sustaining begins, as well as the attributes of teams that are good implementers and good sustainers.

Bloodstream infections from central venous catheters are preventable to some extent

Sustainability of quality improvement initiatives is a major challenge

A multifaceted quality improvement project can sustain reductions in bloodstream infection rates to 36 months post-implementation