'Matching Michigan': a 2-year stepped interventional programme to minimise central venous catheter-blood stream infections in intensive care units in England.

BACKGROUND: Bloodstream infections from central venous catheters (CVC-BSIs) increase morbidity and costs in intensive care units (ICUs). Substantial reductions in CVC-BSI rates have been reported using a combination of technical and non-technical interventions.
METHODS: We conducted a 2-year, four-cluster, stepped non-randomised study of technical and non-technical (behavioural) interventions to prevent CVC-BSIs in adult and paediatric ICUs in England. Random-effects Poisson regression modelling was used to compare infection rates. A sample of ICUs participated in data verification.
RESULTS: Of 223 ICUs in England, 215 (196 adult, 19 paediatric) submitted data on 2479 of 2787 possible months and 147 (66%) provided complete data. The exposure rate was 438 887 (404 252 adult and 34 635 paediatric) CVC-patient days. Over 20 months, 1092 CVC-BSIs were reported. Of these, 884 (81%) were ICU acquired. For adult ICUs, the mean CVC-BSI rate decreased over 20 months from 3.7 in the first cluster to 1.48 CVC-BSIs/1000 CVC-patient days (p<0.0001) for all clusters combined, and for paediatric ICUs from 5.65 to 2.89 (p=0.625). The trend for infection rate reduction did not accelerate following interventions training. CVC utilisation rates remained stable. Pre-ICU infections declined in parallel with ICU-acquired infections. Criterion-referenced case note review showed high agreement between adjudicators (κ 0.706) but wide variation in blood culture sampling rates and CVC utilisation. Generic infection control practices varied widely.
CONCLUSIONS: The marked reduction in CVC-BSI rates in English ICUs found in this study is likely part of a wider secular trend for a system-wide improvement in healthcare-associated infections. Opportunities exist for greater harmonisation of infection control practices. Future studies should investigate causal mechanisms and contextual factors influencing the impact of interventions directed at improving patient care.

Introduction

Blood stream infections (BSIs) from central venous catheters (CVCs) increase morbidity and are estimated to increase mortality risk by 25% and costs of care in the USA by US$16 550 on average per patient1 2 (box 1). A substantial body of evidence suggests that rates of CVC-BSIs are modifiable.3–13 The Michigan-Keystone project13 in 103 intensive care units (ICUs) in the USA reported a major reduction in CVC-BSIs from 7.7 to 1.4 CVC-BSIs per 1000 CVC-patient days using a complex intervention targeting specific technical practices (box 2), combined with support for cultural, behavioural and systemic change.14 A 3-year follow-up study reported sustained improvement15 and accelerated the trend for a reduction in case mix-adjusted mortality rates.16

Central venous catheters (CVCs) are widely used in patients in intensive care units (ICUs) and other hospital locations for monitoring, drug delivery, and dialysis

CVCs increase the risk of blood stream infections (BSIs) which increase mortality and costs of care

CVC-BSIs can substantially be prevented when clinicians use best practice guidance during catheter insertion and subsequent maintenance

CVC-BSI rates in the NHS in England are unknown

This study examined the impact of benchmarking and best practice guidance on minimising CVC-BSIs in English ICUs

Hand hygiene, gown, gloves, hat, mask. Eye protection when indicated

Skin antisepsis: 2% chlorhexidine gluconate in 70% isopropyl alcohol

Maximal sterile precautions including full barrier drapes

Site of insertion: avoid the femoral route

CVC maintenance: aseptic access technique, daily site review, and remove CVCs at earliest opportunity

The NHS Next Stage Review in 200817 announced that the National Patient Safety Agency (NPSA) would run a ‘national patient safety initiative to tackle central line catheter-related blood stream infections, drawing lessons from a remarkably successful Michigan initiative’. This 2-year programme, known as Matching Michigan, ran in England from April 2009 to the end of March 2011. It aimed to minimise CVC-BSI rates in adult and paediatric ICUs in England to at least the mean level (1.4 per 1000 CVC-patient days) seen in the Michigan-Keystone project. It involved three components: technical interventions, which sought to ensure consistent use of evidence-based measures for reducing risks of CVC-BSIs; non-technical interventions, which sought to intervene in culture and systems; and establishment of a standardised national reporting system for CVC-BSIs. All participating sites were invited to take part in two training sessions, the first focused on data collection and the second focused on the technical and non-technical interventions.

Matching Michigan followed, and took place during, heightened media interest and policy initiatives focused on healthcare-associated infections and BSIs (table 1) including the introduction by the Department of Health (DoH) in 2007 of best practice guidance on CVC insertion and management18 through its multicomponent ‘Saving Lives’ programme.19 Other improvement activities relevant to CVC-BSIs included the Health Foundation's Safer Patients Initiative, which ran in two phases from 2004 to 2008,20 and the Patient Safety First campaign, which began in 2008.21 However, in the absence of a national reporting system, it was not possible to assess the impact of any of these or any other efforts on CVC-BSI rates.

Table 1

The context: national infection control initiatives in England before and during Matching Michigan

2001	Mandatory reporting to the Health Protection Agency (HPA) of MRSA bacteraemia.
	//www.hpa.org.uk/web/HPAweb&HPAwebStandard/HPAweb_C/1244763936373
2003	Report of the Chief Medical Officer: Winning ways: guidance to reduce healthcare associated infection in England.
	//www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_4064682
2004	Mandatory reporting of Clostridium difficile infection (HPA website)
2004	Hospital in Europe Link for Infection Control through Surveillance of Nosocomial Infections in ICUs protocol.
	http://helics.univ-lyon1.fr/helicshome.htm
2004 to 2008	Health Foundation's Safer Patients Initiative (24 hospitals): includes CVC bundle. http://www.health.org.uk/areas-of-work/programmes/safer-patients-initiative/
2005	DoH Saving Lives programme—NHS High Impact Interventions (NHS-HII), modelled on Institute for Healthcare Improvement bundles.
	http://webarchive.nationalarchives.gov.uk/20120118164404/hcai.dh.gov.uk/whatdoido/high-impact-interventions/
2006	Health Act 2006: Department of Health Code of Practice gives new powers of inspection to the Healthcare Commission. Superseded by the Health & Social Care Act 2008
2008	Health and Social Care Act 2008: required registration with the Care Quality Commission: duty to protect patients against HCAIs. New code of practice.
	http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_081927
2008	Patient Safety First sponsored by National Patient Safety Agency (NPSA), NHS HII, and Health Foundation, includes interventions to reduce CVC-BSIs
	http://www.patientsafetyfirst.nhs.uk/content.aspx?path=/
2008	High Quality Care For All: NHS Next Stage Review (Darzi report) states that the NPSA will run an ‘initiative to tackle central line catheter-related bloodstream infections’.
	http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_085825
04/2009 to 03/2011	Matching Michigan project. http://www.patientsafetyfirst.nhs.uk/Content.aspx?path=/interventions/relatedprogrammes/matchingmichigan/
2011	Mandatory reporting of MRSA and Escherichia coli bacteraemia (HPA website)

BSI, blood stream infections; CVC, central venous catheter; HPA, Health Protection Agency; ICU, intensive care unit; MRSA, methicillin-resistant Staphylococcus aureus.

In this article, we report an analysis of the impact of Matching Michigan on rates of reported CVC-BSIs in adult and paediatric ICUs in England.

Methods

Design

This was a prospective, interventional, non-randomised, stepped, four-cluster, 2-year quality improvement project with continuous feedback of results to participating ICUs. The National Research Ethics Committee waived the requirement for informed patient consent on the basis that the intent was to improve uptake of established best practice care, and no patient-identifiable information would be collected centrally.

Delivery and recruitment

The NPSA established a national project team and an External Reference Group representing professional and governmental organisations. The scientific leads from the original Michigan-Keystone project acted as advisors and provided their improvement tools. Chief executive officers (CEOs) of all acute hospitals in England with ICUs were invited to participate in the programme. Participating hospitals agreed to appoint a local project team comprising an ICU physician, an ICU nurse, a microbiologist or infection control specialist and an executive or non-executive director.

Clusters

ICUs were grouped into four clusters with stepped implementation (table 2). Cluster 1 (North-Eastern Strategic Health Authority) allowed piloting of data collection, training and interventions. Clusters 2 and 3 comprised ICUs in southern and northern England respectively. Cluster 4 consisted of ICUs unable to join the project in the earlier phases.

Table 2

ICU clusters, duration in project, training day attendance and reliability of submission of infection data

				Training day dates and no. (%) of total ICUs attending					Maximum opportunity to submit data and ICU-months submitted				No. ICUs submitting data
				Data collection		Interventions
Cluster	Adult ICUs	Paediatric ICUs	Duration in project (months)	Date	No. (%) attended	Date	No. (%) attended	Attended both	Max ICU-months	All	Adult	Paediatric	All	Adult	Paediatric	No. (%) ICUs with 100% submission
1	15	4	20	30/3/09	17 (8)	18/09/09	19 (9)	17 (8)	380	350	273	77	19	15	4	13 (68%)
2	73	7	12	20/10/09	70 (31)	16/03/10	57 (23)	54 (24)	2015	1776	1642	134	150	139	11	103 (66%)
3	70	5	12	3/11/09	71 (32)	18/03/10	72 (32)	69 (31)
4	44	5	9	29/4/10	46 (21)	15/07/10	41 (18)	39 (17)	392	353	319	34	46	42	4	31 (63%)
Total	202	21	53		204 (91)		183 (82)	179 (80.3)	2787	2479	2234	245	215	196	19	147 (66%)

ICU, intensive care unit.

Definitions

Definitions of CVC, BSI, catheter-related (CRBSI) and catheter-associated BSI (CABSI) and measures of exposure are not straightforward. There is considerable evidence of variability in these definitions or a lack of clarity in their application in prior publications.22–25 The definitions we used, which were current in 2009, were from the Hospital In Europe Link for Infection Control through Surveillance programme,26 and the US National Nosocomial Infection Surveillance System from the Centre for Disease Control & Prevention,27 28 and were piloted and refined to ensure applicability and ease of understanding for an English context (see electronic supplementary material 1 (ESM 1)). The definitions distinguish between the surveillance definition of CRBSI and the clinical definition of CABSI. The key distinction between these definitions lies in the type of microbiological analysis undertaken to determine whether the source of any individual BSI can be attributed to a CVC.

ICUs were asked to submit data monthly to a specially created web-based system and to identify which definition they used for each infection at the time of reporting. Infections reported as either CRBSI or CABSI were summed to calculate infection rates. Measures of exposure were recorded through a daily census in each ICU involving a count of the number of CVCs in situ at a set time each day. ICUs were asked to complete a survey on generic infection control practices (table 3). Infection data could be submitted until 31 March 2011. However, to permit data cleaning before project closure, analysis was limited to the 20-month period from May 2009 to December 2010.

Table 3

ICU infection control practices (127 respondents of 223 ICUs, response rate 57%)

	No. (%) of respondents
Joint ward round with microbiology/infection control
Daily weekday round	56 (44%)
Less frequent	54 (43%)
Never	17 (13%)
Chlorhexidine bed baths
Routine	19 (15%)
If MRSA positive	63 (50%)
Never	27 (21%)
Information not given	18 (14%)
Oral hygiene
Chlorhexidine mouthwash	25 (20%)
Corsodyl gel	31 (24%)
Corsodyl mouthwash	10 (8%)
Toothpaste	41 (32%)
None of above	2 (2%)
Information not given	18 (14%)
Antimicrobial-coated CVCs	35 (28%)
Antiseptic-coated CVCs	37 (29%)
Bionnector valve use
Yes	86 (68%)
No	26 (20%)
Information not given	15 (12%)
Three-way tap use
Routine	55 (43%)
Sometimes or rare	34 (27%)
Never	23 (18%)
Information not given	15 (12%)
Chlorhexidine-impregnated patch at CVC insertion site
Yes	21 (17%)
No	90 (71%)
Information not given	16 (13%)

CVC, central venous catheter; ICU, intensive care unit; MRSA, methicillin-resistant Staphylococcus aureus.

Training and support

Each cluster was invited to attend two training days, the first on the data definitions developed for the programme (ESM 1) and the second some months later on the technical and non-technical interventions (table 4) adapted from the Michigan-Keystone project.14 Training was held in a centralised location and involved plenary and small group interactive sessions. ICUs started baseline data collection as soon as possible after the first training day.

Table 4

Technical and non-technical interventions

Resource or tool	Content, format
Technical
Evidence based
Effective hand hygiene	CVC insertion checklist DoH high-impact interventions Technical interventions to prevent CVC-BSIs evidence summary Frequently asked questions
2% chlorhexidine skin antiseptic
Full-barrier precautions
Avoidance of the femoral route
Review and prompt removal
Facilitators
CVC insertion checklist	Printable example CVC insertion trolley or pack
Colocated materials
Non-technical
Science of safety
Guidance and teaching resources on safety	PowerPoint presentation WebEx sessions
Clinical stories and safety incidents	Videos
Attendance at training sessions	Document
Identifying and learning from incidents
Identifying hazards, learning from safety incidents	Guidance for identifying and learning from incidents Assessment of potential patient safety incident
LFD framework/root cause analysis	Web tools (National Patient Safety Agency)
Staff safety assessment	Short survey
Executive–clinician partnerships
Senior executive/shadowing partnership	Guidance note Executive Leadership Webex
Safety issues worksheet for executive partnership	‘How to’ guide for leadership walk-rounds Video
Teamwork and communication
Establishing a unit safety team	Guidance note
Safety ‘climate’ and teamworking	Guide and framework for observing patient rounds and handovers Shadowing another professional
Safety culture survey	AHRQ
Daily goals checklist	Three examples of daily goals charts offered

Also available via: http://www.patientsafetyfirst.nhs.uk/content.aspx?path=/

AHRQ, Agency for Healthcare Research and Quality; BSI, blood stream infections; CVC, central venous catheter; DoH, Department of Health; LFD, Learning from Defects.

Teleconference calls and internet-based teaching sessions were offered over the course of the programme. Guidance was provided by telephone and email and, if appropriate, on-site visits by two quality improvement facilitators (ICU nurses). The Patient Safety First website was used to host information on the interventions and on the programme more generally.21 The project clinical leads provided additional ad hoc support and guidance when required.

Data verification

Data limits and rules programmed into the software allowed erroneously entered data to be detected and corrected through the web-based tool. Extreme values were examined by clinical members of the project team, and discussed with local project leads. We also undertook verification of consistency between ICUs in identifying and reporting CVC-BSIs in a purposive sample of ICUs. To conduct the verification, we used on-site criterion-referenced case note review and contemporaneous telephone discussion with a second remote and blinded reviewer. Following institutional approval, each ICU in the verification sample provided a list of all blood cultures (BCs) performed over 3 months, and the case records of 5–20 patients with positive BCs. The number of BCs performed and the number of CVC-patient days were compared with the number of patient days to determine the frequency of sampling for BCs, and the CVC-utilisation ratio. Local adjudication and reporting of each CVC-BSI was compared with external review. Inter-observer agreement was determined using the κ statistic. ICUs were not asked to provide self-reported data on compliance or implementation of the technical and non-technical interventions because there was no method of assuring data reliability or completeness.

Statistical analysis

Random-effects Poisson regression modelling was used for the primary outcome, based on mean monthly CVC-BSIs related to CVC-patient days, anchored by time since the second training day for each cluster (zero pre-intervention, number of months from month of intervention onwards), and using as covariates the time trend (months from May 2009), teaching status, size of unit, random effect of unit, and cluster. This tests the hypothesis that the intervention (the second training day) will change the slope of an underlying secular trend. To explore whether changes in ICU infection rates were independent of, or potentially part of, a whole-hospital trend, and in the absence of a measure of pre-ICU exposure rates, we compared quarterly pre-ICU with ICU-acquired infection rates expressed as the proportion of all CVC-BSIs which were ICU acquired (ICU-acquired CVC-BSIs divided by the sum of ICU-acquired and pre-ICU CVC-BSIs). A stable ratio over time would suggest ICU trends were part of a wider whole-hospital effect. A χ2 test for trend was performed to evaluate changes in this ratio. All p values are two sided, with p≤0.05 considered statistically significant. Stata (V.9) was used for all analyses.

Results

Participant characteristics

Chief executives of all (139) acute hospitals in England with ICUs agreed that their organisations would participate. Of these, 32 (23%) were university hospitals. The study sample represented 223 ICUs, of which 176 (79%) were general adult ICUs, 21 (9%) paediatric, and 26 (11.6%) subspeciality. The mean (range) number of ICU beds per unit was 12 (3–43); the mean (range) annual admissions was 685 (166–2423). More than 80% of ICUs attended both training days (table 2), though the size of the team attending training ranged from single individuals (doctor or nurse) to large groups including executive leads.

Most (96.4%, 215) ICUs submitted at least some infection data to Matching Michigan. Responses (57%) to the survey of generic infection control practices demonstrated wide variation between ICUs (table 3).

Infection rates

Infection data were submitted on 2479 ICU-months of a maximum 2787, giving a reliability rate of 0.89. Complete data were submitted for every possible month by 147 (66%) ICUs (range between clusters 63–68%) (table 2). The first cluster of 19 ICUs (15 adult, 4 paediatric) provided baseline comparator infection data for subsequent clusters. Clusters 2 and 3 received their training a few weeks apart and their infection data were merged into a single cluster for analysis.

Of 1092 CVC-BSIs reported over 20 months, 884 (81%) were ICU acquired. A majority (66.7%) were diagnosed using the catheter-associated definition (table 5). Paediatric CVC-BSIs accounted for 14.6% of total declared infections, but only 7.89% of CVC-patient days. A total of 438 887 (404 252 adult and 34 635 paediatric) CVC-patient days were reported, giving a mean ICU-acquired infection rate for the entire project of 2.01 CVC-BSIs per 1000 CVC-patient days (adult ICUs 1.88, paediatric ICUs 3.58). Detailed monthly infection and CVC utilisation rates are given in ESM 2.

Table 5

1092 CVC-BSIs by infection classification and location

	Pre-ICU acquired			ICU acquired
Infection classification	CVC associated	CVC related	Total pre-ICU	CVC associated	CVC related	Total in ICU	CVC-patient days	ICU CVC-BSI rate/1000 CVC-patient days
Adult	114	57	171	503	258	761	404252	1.88
Paediatric	28	9	37	84	39	123	34635	3.55
Total	142	66	208	587	297	884	438887	2.01

BSI, blood stream infection; CVC, central venous catheter; ICU, intensive care unit.

Changes in infection rates

Aggregated adult and paediatric ICU infection rates diminished with time from a first month rate of 4.4 CVC-BSIs/1000 CVC-patient days for cluster 1, to 1.7 CVC-BSIs in December 2010 (all clusters) (ESM 2 monthly, figure 1A quarterly). The ratio between ICU-acquired CVC-BSIs and all CVC-BSIs remained stable during the project (test of homogeneity χ2=16.11, p=0.6497; test for trend of odds χ2=0.12, p=0.7237), suggesting a possible common cause for the reduction in infection rates in ICU and non-ICU locations (figure 1B).

Figure 1

Central venous catheter (CVC)-blood stream infection (BSI) rates. (A) Total adult and paediatric CVC-BSI infection rate (——) and CVC utilisation ratio % (……) by quarter. (B) Ratio of intensive care unit (ICU)-acquired to (pre-ICU+ICU-acquired) CVC-BSIs. (C) Adult CVC-BSI infection rate (——) and CVC utilisation ratio % (……) by quarter. (D) Adult ICU CVC-BSI rates by cluster. (E) Paediatric CVC-BSI infection rate (——) and CVC utilisation ratio % (……) by quarter. (F) Paediatric CVC-BSI rates by cluster.

Mean adult ICU CVC-BSIs diminished from 3.7 CVC-BSIs/1000 CVC-patient days in the first quarter (inception of cluster 1), to 1.48 in the last quarter (figure 1C), and for paediatric ICUs from 5.65 (four paediatric ICUs) to 2.89 (18 paediatric ICUs) (figure 1E). The progressive reduction in infection rates was statistically highly significant for adult ICUs (Z statistic −4.45, χ2 p<0.0001), but not paediatric ICUs (Z statistic −0.79, χ2 p=0.625).

The rate of change in reduction in infection rates did not accelerate following the second training day. For adult ICUs, each successive cluster to join the project had an entry-level infection rate close to the post-intervention level of the preceding cluster (figure 1D) (Z statistic 1.29 and 0.87, χ2 probability 0.19 and 0.38 for clusters 2 and 3 and cluster 4 respectively). Late engagement (cluster 4) was not associated with poorer performance in any metric. Numbers were too small, and the variation in infection rates too great, to draw secure conclusions from the paediatric data (figure 1F).

Associations

The trend for reduction in infection rates was not associated with hospital type or the number of CVC-patient days for either adult or paediatric ICUs. CVC utilisation ratios could only be determined from December 2009; utilisation rates remained stable (66.3/100 patient days for December 2009–February 2010, 64.6/100 for October–December 2010) (ESM 2 and figure 1A,C,E), despite the continuing fall in pre-ICU and ICU-acquired CVC-BSI rates for this period.

Attendance at both training days was achieved by 179 ICUs (80.3%), 127 of which also provided 100% complete infection data (of 147 ICUs achieving this). Training day attendance was strongly associated with more reliable data submission (χ2 10.2187, p<0.005), but not with infection rates (Z statistic −0.29, p=0.773).

Twenty-eight of 45 ICUs responded to an invitation to participate in data verification and 17 actually participated (one paediatric ICU, two university, 14 adult general). Reasons for non-participation included no response to further contacts (10), clinical workload (3), inadequate administrative support (4), absence of timely authority to access medical records (7), and inadequate project team resources (4).

The 17 ICUs participating in the verification sub-study performed 2357 BCs during 17 020 patient-days and 10 601 CVC-patient days, of which 328 (13.9%) BCs were positive (ICU range 5.7–23%). Frequency of sampling and CVC use varied widely: the BC:patient-days ratio was 2357/17 020=13.8 BCs/100 patient-days (range 4.8–39.6) and the CVC utilisation ratio was 0.62 (range 0.42–0.78).

Criterion-referenced case note review was conducted in 177 patients with 187 positive BCs; in 54 patients (30.5%) no CVC was in situ within 48 hours of the positive BC, which excluded potential CVC-BSIs. Of the 177 patients with positive BCs, 17 had been declared as CVC-BSIs and 160 as non-attributable. External adjudication agreed with local adjudication in 167 instances (seven reclassified as attributable, three as non-attributable, overall correct classification 94.3%). The kappa for agreement between local and external adjudicators was 0.706 (SE of kappa=0.088; 95% CI 0.534 to 0.877). The method did not permit determination of CVC infection in the absence of a blood culture.

Discussion

On initial examination, and using the metrics employed by the majority of studies in this area, Matching Michigan was a success. The programme demonstrated a 60% reduction in reported CVC-BSIs in adult ICUs in England, despite starting with less headroom for improvement than the original Keystone-Michigan project13 (baseline 4.4 CVC-BSIs per 1000 patient catheter days in the first Matching Michigan cluster compared with 7.7 at baseline in Michigan). For paediatric ICUs the 48% reduction did not achieve statistical significance; the difficulty of reducing CVC-BSIs in paediatric intensive care is well recognised.29–32 A conventional narrative might run thus: training in technical and non-technical interventions to improve patient safety combined with measurement and performance feedback stimulated a change in behaviour which resulted in a reduction in BSIs from CVCs.

Closer examination of the data reveals a more complex picture requiring a nuanced interpretation. Attributing the impressive reduction in adult ICU CVC-BSIs rates solely to programme participation is complicated by two novel insights. First, each successive cluster joined the project on the trend line for the post-intervention level of the preceding cluster, thus indicating a strong secular trend. Second, pre-ICU infections (which were not targeted by Matching Michigan) diminished in line with ICU-acquired infections, indicating that the secular trend was not limited to the ICU. These findings suggest the possibility that the reduction in infection rates could be attributable as much to concurrent and preceding improvement efforts and to the consciousness-raising effect of a nationwide programme as to any specific component of the Matching Michigan programme itself.

This study is an example of the challenges of conducting field evaluations of complex interventions to improve care in real time in rapidly moving fields. It illustrates in particular the challenges of identifying causal mechanisms during ‘rising tides’ when multiple policy pressures and the emergence of professional and scientific consensus combine to produce improvements across the board.33–35 Falling rates of CVC-BSIs have been reported in a number of studies worldwide36 37 and our study was undertaken during a period of intense national activity in England directed towards reducing hospital-acquired infections, including methicillin-resistant Staphylococcus aureus BSI rates (which fell by 22% between April 2009 and March 2011, and by 50% since 2008).38 For example, many hospitals had already introduced 2% alcoholic chlorhexidine skin disinfectant, full-barrier drapes were becoming more widely available, and alcohol hand rub had become universally available.

Our stepped before and after design reduces the risk of bias,39 and the analysis therefore emphasises the need for caution in attributing the reduction in infection rates to specific elements in the programme. Lack of a specific causative link between complex behavioural interventions and improved outcomes has been reported for end-of-life care,40 stroke care,33 coronary balloon angioplasty34 and multifaceted safety programmes,35 while others have reported strong secular trends for improvement in CVC-BSI rates in conjunction with national reporting but in the absence of specific targeted interventions.36 Financial penalties as a further stimulus for improvement do not appear to have had an additional impact on the adoption of self-reported CVC-BSI prevention measures in the USA.41

Study designs involving randomisation, which could help to determine quality improvement programme effects more precisely, are challenged by ethical considerations when best practice is already well established, and practical considerations of isolating intervention from controls. Cluster-randomised designs are particularly important for interventions involving behavioural change,40 42 since the component elements may be rooted in specific cultures, locations and periods, and require testing in the same way as a pharmaceutical intervention in a new population.43 44

A design such as that used in our study—involving clusters joining in a pre-determined sequence, with each successive cluster acting as a de facto control for the preceding cluster—although not formally randomised is one of the more robust approaches that can feasibly be deployed. However, it is subject to a number of threats to internal validity. The ‘waiting’ clusters were exposed to diffusion of treatment effects, as the interventions were widely publicised on the Patient Safety First website from the beginning of the study, and the original Michigan-Keystone project had received widespread attention. ICUs in ‘waiting’ clusters may also have engaged in ‘compensatory rivalry’,45 and increased their efforts to reduce CVC-BSIs while waiting to join the programme. It is also possible that the reduction in reported rates of infections may to some extent have been an artefact of reporting behaviours, since data were collected and reported by ICUs themselves and may have been influenced by perceptions of external scrutiny and performance management.46 How far any trend in reported infection rates may reflect changes in reporting behaviour over time is not easy to establish. A further limitation of our study was the absence of measures of adoption of the interventions and compliance with best practice. Several studies have reported an association between higher compliance and lower infection rates,47–49 but data completeness and the methods chosen for compliance monitoring are rarely described in detail, and the literature on hand hygiene demonstrates poor correlation between self-reported and observed compliance.50–52

The data verification sub-study provides some reassurance of validity in relation to reporting behaviours, but also demonstrates considerable variability in local practices in relation to CVC use and intensity of sampling blood for culture. Variability in surveillance techniques is well recognised and substantially alters reported infection rates.25 The survey of generic infection control practices (not compliance with the technical interventions) demonstrates wide variation, including the level of interaction between intensive care physicians and microbiologists. These factors make direct comparison between ICUs challenging. Harmonisation of practice would reduce the risk of confounding, and could bring additional benefits in reducing nosocomial infection rates.

Despite the difficulties of identifying specific programme effects, it is unlikely that the contribution of large-scale programmes such as Matching Michigan to the ‘rising tide’ is trivial. Such programmes may have a particular role in raising awareness, increasing the intensity of focus and stimulating managerial support for professional activities. Feedback of infection rates may have promoted more reliable provision of and adherence to the well known technical aspects of infection prevention for CVCs. Understanding more precisely how such programmes work remains an important task, since such understanding is likely to avoid inappropriate and ineffective interventions, optimise delivery and improve effectiveness.53 This is especially important when elements of programme design vary from the original: Matching Michigan was not exactly the same as the original Michigan-Keystone project. Differences included amendments to some of the programme materials to ensure contextual relevance; definitions of CVC-BSIs were specified more precisely; and the programme was directed by a government agency with advisory clinician input, not as a clinician-led collaborative. Contextual variability was also evident: Matching Michigan was, unlike Michigan-Keystone, implemented following extensive prior national efforts to improve practice, in a national health system in which intensive care specialists direct infection management with input from microbiology, as opposed to this being the domain of independent infection control practitioners.

It is encouraging that reported rates of pre-ICU and ICU-acquired CVC-BSIs showed reductions over the course of Matching Michigan. Reduced rates of infection will deliver health gains for patients and benefits for health systems. The apparent trend for a reduction in CVC-BSIs acquired before ICU admission should not encourage complacency, however,54 since in the absence of a denominator, conclusions cannot be drawn about rates of infection and quality of care. CVC use in non-ICU locations requires the same intensity of focus as it has received in the ICU.55–60 A national clinician-directed system for sustained continuous CVC-BSI benchmarking, such as those in Scotland61 and Wales,62 would ensure continued attention to CVC-BSIs, and could provide a platform for monitoring other healthcare-associated infections with linkage to patient outcomes.

This study adds to the science of improvement by using a quasi-experimental design that reveals the significance of underlying secular trends but does not rule out the possibility that the programme itself was implicated in that trend. Future studies should use robust mixed-methods research methodologies to clarify causal mechanisms underpinning quality improvement interventions, and to identify those most likely to promote more reliable delivery of best practice throughout the healthcare system, as well as promoting clinician ownership.63 To this end, a separate, independent ethnographic study of culture and behaviour in relation to CVC-BSIs in England was conducted at the same time as Matching Michigan and may provide insights that will promote such understanding.