Multistate programme to reduce catheter-associated infections in intensive care units with elevated infection rates.

BACKGROUND: Preventing central line-associated bloodstream infection (CLABSI) and catheter-associated urinary tract infection (CAUTI) remains challenging in intensive care units (ICUs).
OBJECTIVE: The Agency for Healthcare Research and Quality Safety Program for ICUs aimed to reduce CLABSI and CAUTI in units with elevated rates.
METHODS: Invited hospitals had at least one adult ICU with elevated CLABSI or CAUTI rates, defined by a positive cumulative attributable difference metric (CAD >0) in the Centers for Disease Control and Prevention's Targeted Assessment for Prevention strategy. This externally facilitated programme implemented by a national project team and state hospital associations included on-demand video modules and live webinars reviewing a two-tiered approach for implementing key technical and socioadaptive factors to prevent catheter infections, using principles and tools based on the Comprehensive Unit-based Safety Program. CLABSI, CAUTI and catheter use data were collected (preintervention 13 months, intervention 12 months). Multilevel negative binomial models assessed changes in catheter-associated infection rates and catheter use.
RESULTS: Of 366 recruited ICUs from 220 hospitals in 16 states and Puerto Rico for two cohorts, 280 ICUs completed the programme including infection outcome reporting; 274 ICUs had complete outcome data for analyses. Statistically significant reductions in adjusted infection rates were not observed (CLABSI incidence rate ratio (IRR)=0.75, 95% CI 0.52 to 1.08, p=0.13; CAUTI IRR=0.79, 95% CI 0.59 to 1.06, p=0.12). Adjusted central line utilisation (IRR=0.97, 95% CI 0.93 to 1.00, p=0.09) and adjusted urinary catheter utilisation were unchanged (IRR=0.98, 95% CI 0.95 to 1.01, p=0.14).
CONCLUSION: This multistate programme targeted ICUs with elevated catheter infection rates, but yielded no statistically significant reduction in CLABSI, CAUTI or catheter utilisation in the first two of six planned cohorts. Improvements in the interventions based on lessons learnt from these initial cohorts are being applied to subsequent cohorts. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Central line-associated bloodstream infection (CLABSI) and catheter-associated urinary tract infection (CAUTI) are morbid and expensive hospital-acquired infections, particularly for patients in the intensive care unit (ICU) who often receive these devices as part of routine care to closely monitor and deliver therapies. ICU patients also commonly have serious medical comorbidities that increase their risk of acquiring drug-resistant catheter-associated infections, as well as complications from antibiotics treating these infections. Due to their high expense and capacity to harm patients, CLABSI and CAUTI have been a major focus of national and international efforts to reduce mortality and morbidity by improving quality of care delivered in hospital settings.1–5 Multiple Centers for Medicare and Medicaid Services (CMS) programmes that impact public reporting of hospital performance and link Medicare payments with hospital rates of CLABSI and CAUTI have been implemented over the past decade.3 4 6–10 Overall, both CLABSI and CAUTI have decreased nationally in recent years, although there has been notably less success in preventing CAUTI among critically ill patients as well as limited progress in reducing overall central line and urinary catheter days of use for patients who received care in the ICU, despite studies noting opportunities for reducing unnecessary catheters.5 10–18 In prior national acute care implementation projects funded by the Agency for Healthcare Research and Quality (AHRQ)—and administered by the Health Research & Educational Trust (HRET), the research and education affiliate of the American Hospital Association (AHA)—a Comprehensive Unit-based Safety Program (CUSP) has been used to reduce CLABSI and CAUTI. The CLABSI project produced a greater than 40%19 20 relative reduction in CLABSI in ICUs. The CAUTI project yielded a 32%14 relative reduction in CAUTI in non-ICUs, but did not produce a significant CAUTI reduction in ICUs. Even in the highly successful CLABSI project, not all ICUs performed equally well. Nationally, CLABSI and CAUTI also remain a persistent challenge for many ICUs, despite numerous ICUs reporting success in published studies.18 To help ICUs with elevated CLABSI and CAUTI rates, AHRQ initiated a multistate collaborative, known as the AHRQ Safety Program for ICUs: Preventing CLABSI and CAUTI.21 Here we describe this programme’s development, implementation and results for the first two cohorts of this collaborative, which informed important programme changes for subsequent cohorts which are in progress.

Methods

Overview of the programme

The programme objective was to reduce CLABSI and CAUTI in ICUs with persistently elevated rates through state or regional consortia in 4 of 10 US Department of Health and Human Services (HHS) regions of the country as the first phase of a planned nationwide rollout of the project. The programme was funded and guided by AHRQ, and led and developed by HRET, the University of Michigan and other members of the National Program Team, including the American Nurses Association, the Association for Professionals in Infection Control and Epidemiology, the Society of Critical Care Medicine, and the Society of Hospital Medicine. This National Program Team served as coaches and faculty to develop, disseminate and track programme components, as well as garner support for this programme within their respective organisations. An additional technical expert panel of experts in patient safety, CLABSI, CAUTI, teamwork and implementation convened and provided guidance at two separate meetings during the project.

HRET provided centralised coordination and oversight for this programme’s implementation via an externally facilitated model (figure 1), including facilitation of recruitment, data collection and educational resource dissemination from state hospital association partners using their pre-existing relationships with participating ICUs. A state hospital association lead in each geographical region was recruited, educated, and provided technical and funding support throughout the programme. The state leads were responsible for coordinating the programme with the ICUs in their state or region by coordinating with HRET for educational and coaching sessions and holding monthly coaching calls and one state-wide inperson meeting. State leads encouraged ICUs to submit their project data on time and provided feedback on state and individual hospital performance. In collaboration with programme faculty, coaches and state hospital association leads, hospital ICU staff could also receive support from clinical mentors in their state who provided additional coaching at the local level and mentoring ICUs in implementing their action plan. Puerto Rico and all participating states except one had at least one clinical mentor (a nurse or a physician, and ideally a dyad of a nurse and a physician).

Figure 1

External facilitation and implementation model. AHRQ, Agency for Healthcare Research and Quality; ICU, intensive care unit.*Note: AHRQ provided contract funding, set program objectives and deliverables, provided guidance throughout the project, and coordinated with other federal agencies. AHRQ was not directly responsible for the implementation of the project.

State leads were also expected to perform site visits for up to 50% of their participating hospitals, with the percentage requested by the programme varying by state size. Sites were chosen for multiple reasons, including ICU interest in a visit, challenges with the interventions, persistently high CLABSI or CAUTI rates, or had effective practices to share. The visit’s purpose was for state leads and clinical mentors to meet with ICU teams and leadership to strengthen relationships, engage in discussion about infection prevention, and facilitate unit-specific changes by selecting gaps to address by targeted, unit-specific action plans. However, how the state leads identified ICUs for a site visit varied, as well as the visit’s components and data collected.

Programme interventions

To inform this programme’s interventions, a systematic literature review18 was performed to summarise the evidence for practices to reduce CLABSI and CAUTI in the ICU setting. This literature review’s results, along with a previously successful two-tiered approach implemented in Veterans Affairs hospitals for prioritising interventions for CAUTI,22 were reviewed with the National Program Team and technical expert panel to inform the intervention. Baseline data for gaps in CLABSI and CAUTI prevention practices specific to the participating units for this project were not available at the time of recruitment or intervention development. Additionally, the intervention was informed by prior experience and resources developed in the Michigan Health and Hospital Association Keystone Center’s Bladder Bundle Initiative23–25 and prior AHRQ-funded national implementation projects14 19 20 26 for preventing CLABSI and CAUTI that applied the CUSP27 and included a combination of technical and adaptive interventions.

This multicomponent intervention addressed both the technical and socioadaptive components of infection prevention28 and was rolled out in two cohorts. Participating units were provided with an onboarding curriculum (delivered by a combination of live webinars and on-demand webinars) that gave an overview of some CUSP and basic principles units need to understand and implement in order to drive improvement, such as how to assemble their CLABSI and CAUTI prevention teams, developing shared mindsets for infection prevention goals and appropriate catheter use, how to engage senior leadership, and understanding the required resources to support their efforts, and guidance on the data to be collected. Brief on-demand video modules were developed to address the technical (eg, alternatives to catheters, aseptic insertion/maintenance, prompting catheter removal when no longer indicated) as well as the socioadaptive aspects (eg, strategies to address common barriers in changing clinician behaviour, recognising unit culture challenges and strategies to address them, building successful teams, improving accountability for device use and outcomes, engaging leaders, strategies to sustain change) of CLABSI and CAUTI prevention. These on-demand modules were specifically designed to be short in duration and available to view by busy ICU staff when convenient. Monthly ‘virtual learning group’ webinars were available to reinforce the resources and implementation principles provided by national project team members and subject matter experts to address common gaps in both technical and socioadaptive practices for prevention of CLABSI and CAUTI noted in the systematic review18 that informed this project, as well as to provide the opportunity for peer-to-peer coaching from participating ICUs. By comparison, the on-demand modules were designed to deliver the ‘what’ of the content, and the virtual learning groups were designed to focus more on the ‘how’ of implementation, by allowing participants to hear successes, challenges and strategies employed by their peers for implementing the intervention within their ICUs. Podcasts were developed to support ICU senior leaders, including topics such as why senior leadership matters, and roles and responsibilities of ICU leaders and teams.

Intervention components were prioritised and presented to participating ICUs using a tiered approach. Tier 1 combined technical and socioadaptive strategies that have been found to be impactful on infection prevention and includes fundamental infection prevention strategies, such as optimising appropriate use of catheters and ensuring aseptic placement and maintenance care to promote early success. If rates remained high despite implementing tier 1 recommendations, then units were advised to advance to tier 2. Tier 2 was a stepwise process that began with a formal reassessment of challenges known as the Guide to Patient Safety29 30 (available at http://www.improvepicc.com/gpsclabsi.html and http://www.catheterout.org/cauti-gps.html) to inform the selection of other enhanced practices that may be useful as additional interventions, and ending with a root cause analysis if rates remain elevated. For example, if CLABSI rates remain high after implementing tier 1 approaches, technological approaches such as antimicrobial-impregnated catheters or daily chlorhexidine bathing could be employed.31–37 The tier 2 interventions are not included in tier 1 because they were judged to be more costly, resource-intensive or currently supported by less evidence.

The brief on-demand video modules developed for this project introduced the tiered approach to reducing CLABSI and CAUTI in ICUs and included (1) a brief review of the risk factors and morbidity associated with unnecessary catheter use, CLABSIs and CAUTIs; (2) catheter use guidance (including the Centers for Disease Control and Prevention (CDC) Healthcare Infection Control Practices Advisory Committee consensus-based guidelines for catheter use, the Ann Arbor Criteria for Appropriate Urinary Catheter Use in Hospitalized Medical Patients, and the Michigan Appropriateness Guide for Intravenous Catheters criteria)38–41; and (3) evidence-based practices related to ‘Disrupting the Life Cycle of the Catheter’, with a focus on catheter avoidance or selection, optimal aseptic insertion, maintenance, and prompt removal when no longer clinically appropriate.18 42 Key technical focal points included proper catheter placement and maintenance, with emphasis on evaluating the necessity for a catheter. Multiple intervention components also addressed several socioadaptive elements, including how to prioritise use of the potential interventions and resources in the programme, and strategies to garner the support of all the team members using CUSP principles, as well as experience in prior implementation projects and clinical work in ICUs by National Program Team members. Some CUSP principles were included in various components of the intervention to provide guidance on improving staff engagement, teamwork, communication, as well as engaging leadership, which can facilitate units to advocate for additional hospital resources. Finally, the participants were provided education on the data collection and submission processes using brief on-demand modules and coaching calls. Unfortunately, this project was not able to provide financial or personnel resources to the participating units to support purchase of catheter insertion kits, catheter maintenance care supplies or equipment such as bladder scanners. There were also no additional resources to support additional staff (eg, nurse assistants to help manage incontinence) or dedicated time for existing staff to perform additional infection prevention activities, such as daily catheter rounds or root cause analyses. Finally, the participants were provided education on the data collection and submission processes using brief on-demand modules and coaching calls.

Programme eligibility and recruitment

ICUs were identified as eligible for participation if the unit’s hospital met two inclusion criteria. First, the hospital was located in one of the four HHS regions, including 19 states and US territories, selected because these regions had the highest number of hospitals with elevated CLABSI and/or CAUTI rates (defined as a positive cumulative attributable difference (CAD) based on the Targeted Assessment for Prevention (TAP) methodology).43–45 Second, the hospital had ‘persistently elevated rates of CLABSI and/or CAUTI’, as defined by a positive CAD, using one of two data sources: (1) National Healthcare Safety Network (NHSN)45 data assessed by the CDC (April 2014–March 2015 for cohort 1 identification, and January 2015–December 2015 for cohort 2 identification), or (2) NHSN data publicly reported on the Hospital Compare website from January to December 2014. Veterans Affairs hospitals and paediatric hospitals were excluded.

Two recruitment methods were used. First, eligible hospitals identified through the NHSN data source were sent an initial email from the CDC and one follow-up email informing them about the opportunity to join the AHRQ project and providing a link to the project website where an informational flier and narrated presentation about the project were available and hospital staff could enroll or request additional information. Additionally, the CDC called hospitals that were not participating in an NHSN group involved with the Quality Innovation Network-Quality Improvement Organizations contracted by the CMS to inform them about the opportunity. Second, the list of eligible hospitals identified with Hospital Compare data was shared with the respective state leads, who were advised to encourage each hospital to run its own TAP reports within NHSN, which would identify which ICUs within a hospital might benefit most from participation in the initiative. A supplementary recruitment method was used only for cohort 2: an informational webinar hosted by HRET highlighting the benefits of the programme as well as success stories from an ICU team lead who participated in a prior AHRQ-funded CAUTI prevention project. In both cohorts, hospitals recruited ICUs that met CAD eligibility criteria, and in some hospitals the team leader also recruited other ICUs within their hospital that may not have met CAD eligibility criteria. Interested hospitals obtained leadership support for the initiative and identified specific units for the improvements to take place.

Data collection and outcomes

The efforts of the collaborative were structured to include two different data collection periods: preintervention and intervention. The preintervention period was 13 months for cohorts 1 and 2. The intervention phase was 11 months with an optional 3-month extension for cohort 1 and 12 months for cohort 2. For consistency in the analysis, we used the first month of the 3-month extension of cohort 1, yielding a 12-month intervention period for analysis for both cohorts 1 and 2. Hospital characteristics collected during the preintervention period were obtained from the 2015 AHA Annual Survey of Hospitals. Participating ICUs were encouraged to complete an ICU Assessment Tool, developed specifically for this project, to help the ICU assess its needs in order to develop an action plan. The needs assessment was also used by the National Program Team and state hospital associations to assist with tailoring coaching and project resources. The assessment included basic unit demographic descriptions such as ICU bed size and ICU type, as well as information about current infection prevention practices and resources used for CLABSI and CAUTI prevention and other strengths and potential barriers including teamwork and communication strategies.

The outcomes were defined by and reported to the NHSN as part of the hospital’s routine surveillance, recommended monthly in this programme but required quarterly by NHSN. Primary outcomes were NHSN CLABSI and CAUTI rates calculated as the number of CLABSIs (or CAUTIs) per 1000 device days. Secondary outcomes were device utilisation ratios for both urinary catheters and central venous catheters calculated as the number of device days per 100 patient days. Non-outcome data consisted of demographic data (hospital teaching status, hospital urban/rural location, ICU type, ICU bed size and ownership) and reported ICU programme focus (CLABSI, CAUTI or both).

Measures of programme participation were also summarised to describe the available data on level of ICU participation in the specific components of the programme interventions. These measures included the percentage of units that completed the ICU Assessment Tool at baseline and follow-up, as well as the percentage of units that viewed the various online live educational webinars and on-demand modules categorised within four levels of participation: 0% webinars/modules viewed, 1%–49% viewed, 50%–99% viewed, 100% viewed.

Statistical analysis

Our analyses included participating ICUs that reported CLABSI and CAUTI outcomes and device utilisation data. Other variables included hospital characteristic data available from the 2015 AHA Annual Survey of Hospitals46 and ICU bed size and type as reported by the ICUs during programme recruitment and/or via the ICU Assessment Tool. Descriptive statistics were reported using mean (SD) and n (%) for unit characteristics for all participating ICUs. Preintervention and intervention outcome rates were calculated by aggregating the number of infections and catheter/patient days at each time period over all units. Multilevel negative binomial regression with a random intercept for each ICU was used for modelling all primary (CLABSI rate, CAUTI rate) and secondary (urinary catheter utilisation rate, central line utilisation rate) outcome measures. This approach included the main effects for continuous time, a main effect for an indicator for the start of the intervention period, plus an interaction term with time and an indicator variable for the intervention period. All models were adjusted for teaching status, urban or rural hospital location, type of hospital ownership, ICU bed size, ICU type, and whether or not the ICU focused its programme implementation efforts on the relevant infection (CLABSI, CAUTI or both). All characteristic main effects were included simultaneously in the multivariate models. Using the same modelling approach as outlined above, we also conducted sensitivity analyses examining the impact of the intervention over time on the CAUTI, CLABSI and device utilisation outcomes among ICUs with positive CAD values. SAS V.9.4 and Stata/MP V.13.1 were used for all analyses.

Study oversight

Authors with access to project data signed a confidentiality agreement with HRET. The data analysis plan for the project overall was informed by the Data and Analysis Committee, which was led by HRET and the University of Michigan, with input from the National Program Team. Analyses for this publication were prepared and conducted by two of the authors (DR, MTG).

Results

Hospital and ICU characteristics

Table 1 details the hospital and ICU characteristics. Of 366 recruited adult ICUs from 220 hospitals in 16 states and Puerto Rico (online supplementary appendix figure 1), 280 ICUs completed the programme including reporting of infection outcome data for CLABSI and/or CAUTI. Our analysis focuses on the 274 units (171 hospitals) that reported both CLABSI and CAUTI outcome data and had 2015 AHA data, number of beds and ICU type data available. Similar to participating ICUs in a prior AHRQ collaborative14 involving both ICUs and non-ICUs, the majority of the ICUs in this collaborative were from urban hospitals (89.8%) and teaching hospitals (70.8%), although less than a quarter were from major teaching hospitals (23.4%). The most common ICU type was medical/surgical (77.7%), followed by specialty ICUs including cardiology and/or cardiothoracic (12.0%), trauma/burn (5.8%), and neurology and/or neurosurgery (4.4%). Of ICUs that reported a particular infection focus for this project, the majority (82.6%) focused on both CLABSI and CAUTI.

Table 1

Hospital and ICU characteristics

Hospital characteristics,* ICUs, n (% of 274 units)
Teaching hospital	194 (70.8)
Major teaching hospital	64 (23.4)
Urban hospital	246 (89.8)
Ownership
Non-profit, non-government	150 (54.7)
For profit	47 (17.2)
Government	77 (28.1)
ICU characteristics†
ICU size in mean number of beds ±SD	16.7±12.4
ICU type, units, n (% of 279 participating units)
Medical/surgical	213 (77.7)
Cardiology and/or cardiothoracic	33 (12.0)
Trauma/burn	16 (5.8)
Neurology and/or neurosurgery	12 (4.4)
Programme focus, as identified by ICU, n (% of 274)
Focused on CAUTI and CLABSI	185 (67.5)
Focused on CLABSI	202 (73.7)
Focused on CLABSI only	17 (6.2)
Focused on CAUTI	207 (75.5)
Focused on CAUTI only	22 (8.0)
Programme focus not specified	50 (18.2)

*Hospital characteristics are as defined in the AHA Annual Survey.

†Excluded units from analysis: one unit that reported only CAUTI data, two units that could not be linked with available AHA survey data, two units that were missing data on bed size and one unit that was missing ICU type data.

AHA, American Hospital Association; CAUTI, catheter-associated urinary tract infection; CLABSI, central line-associated bloodstream infection; ICU, intensive care unit.

Changes in CLABSI rates and central line utilisation

The unadjusted CLABSI and central line utilisation rates over the continuum of the project are illustrated in figure 2. Among all participating units reporting CLABSI outcomes, the unadjusted rates of CLABSI decreased by 27.2%, from 1.08 CLABSIs per 1000 central line days at the end of the preintervention period to 0.78 per 1000 central line days at the end of the intervention period. Central line utilisation decreased 5.5%, from 50% to 47% over the course of the project. CLABSI and central line utilisation by ICU type is shared in online supplementary appendix figure 2.

Figure 2

Central line-associated bloodstream infection (CLABSI) rates and central line utilisation rates, as reported to the National Healthcare Safety Network.

In adjusted analyses (table 2), the CLABSI rate decreased from 1.25 at the end of the preintervention period to 1.06 at the end of the intervention period. There was no significant change in CLABSI rates during the preintervention period (incidence rate ratio (IRR)=0.95, 95% CI 0.77 to 1.19, p=0.67). During the intervention period, rates decreased 29% (IRR=0.71, 95% CI 0.53 to 0.96, p=0.03); however, the rates did not change significantly over time when comparing the preintervention with the intervention period (IRR=0.75, 95% CI 0.52 to 1.08, p=0.13). After adjustment, central line utilisation rates decreased from 50% to 47% during the intervention period. Central line utilisation significantly decreased during the preintervention period (IRR=0.97, 95% CI 0.94 to 0.99, p=0.004) as well as during the intervention period (IRR=0.94, 95% CI 0.91 to 0.96, p<0.001). However, there was not a statistically significant change in central line utilisation from the preintervention to the intervention period (IRR=0.97, 95% CI 0.93 to 1.00, p=0.09).

Table 2

Multilevel negative binomial regression results

	Adjusted results*
	IRR (95% CI)	P value
CLABSI rate
Preintervention slope	0.95 (0.77 to 1.19)	0.67
Shift in rates preintervention to intervention	0.90 (0.72 to 1.11)	0.32
Intervention slope	0.71 (0.53 to 0.96)	0.03
Intervention slope change	0.75 (0.52 to 1.08)	0.13
Central line utilisation rate
Preintervention slope	0.97 (0.94 to 0.99)	0.004
Shift in rates preintervention to intervention	1.01 (0.99 to 1.03)	0.51
Intervention slope	0.94 (0.91 to 0.96)	<0.001
Intervention slope change	0.97 (0.93 to 1.00)	0.09
CAUTI rate
Preintervention slope	0.96 (0.80 to 1.15)	0.66
Shift in rates preintervention to intervention	0.94 (0.79 to 1.12)	0.48
Intervention slope	0.76 (0.60 to 0.96)	0.02
Intervention slope change	0.79 (0.59 to 1.06)	0.12
Urinary catheter utilisation rate
Preintervention slope	0.95 (0.93 to 0.97)	<0.001
Shift in rates preintervention to intervention	1.00 (0.99 to 1.02)	0.73
Intervention slope	0.93 (0.91 to 0.95)	<0.001
Intervention slope change	0.98 (0.95 to 1.01)	0.14

*All models were adjusted for teaching status, urban or rural hospital location, type of hospital ownership, ICU bed size, ICU type, and whether or not the ICU focused its programme implementation efforts on the relevant infection (CLABSI, CAUTI or both).

CAUTI, catheter-associated urinary tract infection; CLABSI, central line-associated bloodstream infection; ICU, intensive care unit; IRR, incidence rate ratio.

Changes in CAUTI rates and urinary catheter utilisation

The unadjusted CAUTI and urinary catheter utilisation rates over the continuum of the project are illustrated in figure 3. Among all participating units reporting CAUTI outcomes, the unadjusted rates of CAUTI decreased by 10.7%, from 1.25 CAUTIs per 1000 urinary catheter days at the end of the preintervention period to 1.11 per 1000 urinary catheter days at the end of the intervention period. Urinary catheter utilisation decreased 6.7%, from 64% to 59% over the course of the project. CAUTI and urinary catheter utilisation by ICU type is provided in online supplementary appendix figure 3.

Figure 3

Catheter-associated urinary tract infection (CAUTI) rates and urinary catheter utilisation rate, as reported to the National Healthcare Safety Network.

In adjusted analyses (table 2), the CAUTI rate decreased from 1.47 at the end of the preintervention period to 1.28 at the end of the intervention. There was no significant change in CAUTI rates during the preintervention period (IRR=0.96, 95% CI 0.80 to 1.15, p=0.66). During the intervention period, the rates decreased 24% (IRR=0.76, 95% CI 0.60 to 0.96, p=0.02); however, the rates did not change significantly over time when comparing the preintervention with the intervention period (IRR=0.79, 95% CI 0.59 to 1.06, p=0.12). Urinary catheter utilisation decreased from 63% to 59% during the intervention period. During the preintervention period, there was a 5% reduction in catheter utilisation (IRR=0.95, 95% CI 0.93 to 0.97, p<0.0001). Rates decreased at a nearly identical rate during the intervention period (IRR=0.93, 95% CI 0.91 to 0.95, p<0.001) as the change from the preintervention to the intervention period was a non-statistically significant 2% reduction (IRR=0.98, 95% CI 0.95 to 1.01, p=0.14).

Sensitivity analyses

For CLABSI, a total of 146 (53.3%) ICUs had positive CAD values, 111 (40.5%) were negative and 17 (6.2%) were missing. Among ICUs with positive CAD values, CLABSI rates did not change significantly over time when comparing the preintervention with the intervention period (IRR=1.23, 95% CI 0.78 to 1.95, p=0.38). We did, however, observe a statistically significant reduction in central line utilisation from preintervention to postintervention (IRR=0.95, 95% CI 0.90 to 0.99, p=0.02). For CAUTI, a total of 133 (48.5%) ICUs had positive CAD values, 123 (44.9%) were negative and 18 (6.6%) were missing. Among ICUs with positive CAD values, CAUTI rates (IRR=0.93, 95% CI 0.64 to 1.35, p=0.71) and urinary catheter utilisation (IRR=0.99, 95% CI 0.96 to 1.03, p=0.61) did not change significantly over time when comparing the preintervention with the intervention period.

Measures of programme participation

The ICU Assessment Tool was submitted by 247 (90%) of the 274 ICUs at baseline and 97 (35%) of ICUs at follow-up. All state hospital association leads met the programme requirement for performing site visits; unit-specific action plans were submitted by 100 ICUs. Online supplementary appendix 4 summarises participation of the ICUs for viewing the modules and webinars; overall, participation by module and webinar viewing varied by educational product, but was higher for those with live and interactive components such as the virtual learning group webinars, as opposed to on-demand only modules.

Discussion

We report the results from the first two cohorts of this collaborative for ICUs identified as having elevated CLABSI and/or CAUTI rates at baseline. Despite expected challenges in recruiting, engaging and performing site visits for struggling units, this programme successfully recruited and engaged a large number and variety of ICUs. This project also rapidly conducted a systematic literature review18 to inform and tailor educational resources to ICUs. In addition to live educational webinars, the project created on-demand brief video modules in order to be more flexible for ICU clinicians. Despite these efforts, many of the project’s training and coaching resources were underutilised or not accessed by a number of ICUs. We anticipate this may have contributed to the lack of a statistically significant improvement in CLABSI, CAUTI, urinary catheter utilisation and central line utilisation rates in the intervention period compared with the rate of decline noted in the 12-month preintervention period.

This programme has three notable features that provide context to these findings that may contribute to why this programme did not show the same success in reductions seen in prior collaboratives for CLABSI19 20 and CAUTI.14 First, this programme specifically targeted only ICUs, which are complex hospital units that have not previously shown success in reducing urinary catheter use or CAUTI rates in the prior collaborative,14 and units in which the rate of improvement in CLABSI has slowed in the past few years after earlier successes in CLABSI prevention.5 14 Second, this programme specifically aimed to recruit ICUs that were identified as having elevated CLABSI or CAUTI rates at baseline, as opposed to all-comers recruited in prior AHRQ-funded projects14 19 20 which could have recruited units that could be described as ‘early adopters’ or ‘early majority’47 of technical strategies and behaviour changes to reduce catheter-associated infections. To our understanding, unlike the prior collaboratives,14 19 20 this programme may also be among the first to recruit hospitals and units by the relatively new CAD metric generated from NHSN data based on the CDC’s TAP strategy which was designed to rank facilities or locations (ie, units) in order to identify and target areas where the greatest prevention impact could be achieved.48 Third, this collaborative obtained and analysed preintervention data for the participating hospitals over a long period (12 months) to allow reliable calculation of the slope of change in CLABSI, CAUTI and device rates, which permitted a comparison of slope of outcome changes in both the preintervention and intervention phases, to account for secular trends in changes in slope over time. In contrast, the prior CLABSI collaborative20 compared the postintervention outcomes with baseline data that were a weighted average over four baseline quarters, and the prior CAUTI collaborative14 included only 3 months of baseline data for comparison.

Noting the secular trends that were occurring in national CLABSI and CAUTI rates in the preintervention period, it is feasible that catheter infection rates have reduced in recent years to levels for which improvement is becoming increasingly difficult, possibly reaching a ceiling effect. Comparing this collaborative’s 2014–2015 preintervention ICU CLABSI rates (1.08 per 1000 central line days) and ICU CAUTI rates (1.25 per 1000 urinary catheter days) with prior collaboratives’ data highlights that ICUs in the current project, although targeted for having excess CLABSI and CAUTI, have much lower rates than prior programmes. For example, in the AHRQ On the CUSP: Stop CAUTI project with data from 2011 to 2013, CAUTI rates at the end of the intervention were 2.50 per 1000 catheter days in ICUs and 1.52 per 1000 catheter days in non-ICUs.14 Similarly, in the AHRQ On the CUSP: Stop CLABSI programme using 2009–2011 data, CLABSI rates were 1.96 per 1000 catheter days at the project start and 1.15 per 1000 catheter days at project end 16–18 months after implementation.20 However, NHSN rates are also impacted by definition changes occurring between 2011 and 2015, such as a major NHSN CAUTI definition change that removed yeast as a qualifying organism in urine culture, which was associated with lower NHSN CAUTI rates.

In a recent systematic review of qualitative studies of healthcare organisations struggling to improve quality,49 several domains, such as inadequate infrastructure, that can impede improvements were identified that ICUs in this project may be experiencing more than units in prior projects, for which intervention by online and virtual resources, outcome feedback, and the externally facilitated implementation approach used in this project may be inadequate. For example, this project included some resources for helping units build a team, improve accountability, collaboration and leadership, and use quality improvement tools such as using data for feedback, although the project did not provide other explicit CUSP-focused training27 as compared with prior projects.14 20 27 A lesser focus on the adaptive elements of the CUSP model, such as engagement, may have had a bearing on the underutilisation of the project’s training and coaching resources. Also, the project could not address common infrastructure barriers such as insufficient staffing and high turnover, often associated with lack of staff resources to internally facilitate and champion the project within the ICU. We also anticipate (but only have limited evidence from site visits and coaching calls for this project to support this) that many of the participating ICUs and their hospitals could have been impacted by ‘system shocks’49 or ‘disruptive events’,50 described as an organisation-wide event or change that detracts from day-to-day operations; examples include new electronic health records, mergers or reorganisations, C-suite turnover and financial difficulties impacting staff and technology resources. Although formal qualitative data collection was not performed for this project, lessons learnt from the site visits and coaching calls suggested that major challenges included hospital senior leadership not being engaged, and limited staff resources and time available to focus on this project’s implementation.

We also saw opportunities to tailor the educational and peer coaching resources for ICU type for the future cohorts and the need to balance the flexibility of on-demand modules for busy clinicians with obtaining team commitment to more scheduled coaching and opportunities to reinforce and expand the education provided in the on-demand educational modules. We also recognised that this programme may have required more effort from the ICUs to assess gaps and develop their own individual ICU action plans in contrast to the prior AHRQ-funded programmes,14 20 which each implemented a single bundle of a limited number of interventions to prevent CLABSI and CAUTI, respectively, before these types of bundles became standard. Although this programme’s approach allowed for and prioritised tailoring the intervention components to the individual ICU’s strengths and needs, this design could potentially be burdensome to ICUs that are struggling at baseline due to suboptimal staffing, resources and team communication strategies.

Important limitations of this study should be considered. Because it was not a randomised trial, confounding variables could impact the results. For future similar projects, we believe it would be important to consider effectiveness-implementation hybrid trial designs (eg, stepped wedge cluster randomised trials; sequential, multiple assignment, randomised trials) to assess both the implementation strategy at the unit level and specific health outcomes. Such design considerations will need to plan for adequate allocation of resources to permit more detailed evaluations of the implementation given collection of implementation metrics is much more resource-intensive and not in the usual scope of clinicians in the participating ICUs. Programme participation was also voluntary among those invited to participate, so findings may not be generalisable to all ICUs in the USA. The measures of programme participation also have important limitations. For example, to be counted as having viewed or participated in each individual webinar or module, an ICU must have had at least one team member access the webinar or module online, for any length of time, but the programme was unable to track individuals who accessed a webinar by phone without logging into the webinar platform. Also, the reported results may under-represent ICUs’ use of the on-demand materials, as the results do not include sharing of the materials by means other than by accessing the online material directly; for example, some state leads reported downloading the module slides and distributing them to the ICU teams by email. Also, although we were able to perform a more rigorous assessment of secular trends in the preintervention period compared with similar collaboratives given our longer preintervention data collection period, we cannot compare the secular trends noted in our programme’s participants with non-participants, as we did not have access to the NHSN data for non-participating units in this same time period. Also, although the new CAD metric used to assess eligibility was designed to identify units with the greatest potential for improving infection rates, to our knowledge, this was among the first use of this metric to recruit hospitals and units into an intervention project. It is feasible that although the CAD metric measures and quantitatively ranks opportunities for infection prevention, long-term performance of the CAD metric for intervention prioritisation and as a predictor of the likelihood of intervention success remains unclear. Also, although the majority of recruited units had positive CAD values, a large subset of participating units had low or negative CAD values. As such, it is possible that inclusion of these units limited our ability to observe CLABSI and CAUTI reductions. However, we conducted sensitivity analyses only focused on units with positive CAD values and still observed no statistically significant reductions in CLABSI, CAUTI, or urinary catheter utilisation, although a significant reduction in central line utilisation was observed. Finally, the results presented here from the first two cohorts of the project are from a smaller number of units compared with prior intervention efforts14 20 and may not reflect the final impact of the programme across all participating cohorts given the project is ongoing for later cohorts. Still, the intervention slope change coefficients for CAUTI, CLABSI and device utilisation rates, while generally in the desired direction of improvement (as better illustrated by our presented 95% CIs), did not meet our criteria for statistical significance.

Despite these limitations, and lack of statistically significant improvements in these first two cohorts, we feel this project has demonstrated important success in rapidly recruiting ICUs and hospitals targeted by baseline performance challenges, developing resources specific for addressing catheter use challenges in ICU settings, as well as encouraging more collaboration between ICUs and state hospital associations which may contribute to sustaining quality improvement work beyond this project. Lessons learnt from exit interviews conducted with state leads in this project have been applied to the expansion of this project to other regions of the USA which began in September 2017. Given concerns that low participation could be a contributor to lack of statistically significant improvement in outcomes, important modifications to the programme materials and approach have been implemented in the ongoing project. First, there is a greater emphasis on adaptive components of the CUSP model to increase engagement and ownership at multiple levels to prevent infections in the ICU. All of the onboarding materials for the expansion project incorporate a more explicit emphasis of CUSP and other adaptive/cultural concepts such as reinforcing the message that this programme must be implemented by a team and not solely by the infection preventionist or a single individual in the ICU. Tools were also developed or made available to better support the use and application of CUSP, such as action planning templates to enable easier unit customisation of their intervention based on their responses to the ICU Assessment Tool, CLABSI and CAUTI event planning tools to accompany the CUSP Learn from Defects Tool, and new short CUSP training videos and companion audio files of interviews with experts to help ICUs overcome common cultural challenges. Intervention and implementation resources are being augmented to better address the challenge of engaging hospital leadership and to provide coaching and peer support specific to ICU type where possible. Enhancements to engage hospital leadership include requiring a letter of commitment from a hospital senior leader that also includes signatures from the unit lead and quality improvement department head to promote accountability and ease of follow-up. Coaching and peer support enhancements include providing additional coaching support through onsite training to the state leads, providing additional resources to state leads to host inperson meetings for their ICUs to increase engagement, peer-to-peer learning and real-time coaching, inviting National Program Team subject matter experts on site visits to provide extra support to challenged ICUs and to model coaching for state leads, and hosting affinity group webinars to ‘specialty ICUs’ (eg, neuro-ICUs, burn/trauma ICUs) to offer targeted coaching and peer-to-peer learning for ICUs with specific patient populations.