Evaluation of the association between Hospital Survey on Patient Safety Culture (HSOPS) measures and catheter-associated infections: results of two national collaboratives.

BACKGROUND: The Agency for Healthcare Research and Quality (AHRQ) has funded national collaboratives using the Comprehensive Unit-based Safety Program to reduce rates of two catheter-associated infections-central-line-associated bloodstream infection (CLABSI) and catheter-associated urinary tract infection (CAUTI), using evidence-based intervention bundles to improve technical aspects of care and socioadaptive approaches to foster a culture of safety.
OBJECTIVE: Examine the association between hospital units' results for the Hospital Survey on Patient Safety Culture (HSOPS) and catheter-associated infection rates.
METHODS: We analysed data from two prospective cohort studies from acute-care intensive care units (ICUs) and non-ICUs participating in the AHRQ CLABSI and CAUTI collaboratives. National Healthcare Safety Network catheter-associated infections per 1000 catheter-days were collected at baseline and quarterly postimplementation. The HSOPS was collected at baseline and again 1 year later. Infection rates were modelled using multilevel negative binomial models as a function of HSOPS components over time, adjusted for hospital-level characteristics.
RESULTS: 1821 units from 1079 hospitals (CLABSI) and 1576 units from 949 hospitals (CAUTI) were included. Among responding units, infection rates declined over the project periods (by 47% for CLABSI, by 23% for CAUTI, unadjusted). No significant associations were found between CLABSI or CAUTI rates and HSOPS measures at baseline or over time.
CONCLUSIONS: We found no association between results of the HSOPS and catheter-associated infection rates when measured at baseline and postintervention in two successful large national collaboratives focused on prevention of CLABSI and CAUTI. These results suggest that it may be possible to improve CLABSI and CAUTI rates without making significant changes in safety culture, particularly as measured by instruments like HSOPS. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

Introduction

Central-line-associated bloodstream infection (CLABSI) and catheter-associated urinary tract infection (CAUTI) remain common, morbid and expensive healthcare-associated complications.1–5 Success in reducing CLABSI and CAUTI is theorised to depend on improving two different types of care: ‘technical’ components of care such as aseptic catheter insertion technique and use of standard evidence-based checklists for insertion and maintenance, and ‘socioadaptive’ components of care such as teamwork, habits and willingness to change.6 ‘Safety culture’ is a term to describe the procedures, expectations, attitudes and behaviours of an organisation's personnel as they relate to safety; a team's safety culture can serve as a strength or barrier for improving technical and socioadaptive aspects of care to improve safety.7–12 Safety culture is theorised to be very important in the development and prevention of catheter-associated infections such as CLABSI and CAUTI for several reasons, described here using components measured by the Hospital Survey on Patient Safety Culture (HSOPS). The HSOPS tool is a multi-item survey assessing 12 different dimensions of safety culture such as teamwork, communication, non-punitive response to error, staffing and management support. For example, the strength of teamwork and communication between clinicians in a busy unit would be expected to influence the comfort level of clinicians to speak up and stop a procedure such as catheter placement, if the procedure was not being performed as recommended to optimise patient safety. The comfort level of an employee to self-report an error without a punitive response and acknowledge the need for additional training to improve safe catheter use is hypothesised to be important in reducing inappropriate catheter placement and care. Additionally, staffing is theorised to impact catheter-associated complications because the temptation to inappropriately use urinary and vascular catheters for clinician convenience beyond the clinical need of the patient can be influenced by nurse–patient ratios (particularly for care of patients with urinary incontinence). Management support for patient safety is hypothesised to influence how well supported clinicians feel by their multidisciplinary team in helping with time-consuming tasks such as using less convenient alternatives to indwelling catheters, including more frequent peripheral blood draws for the patient without a central venous catheter, and the need for more frequent turning and bathing for the incontinent patient without a urinary catheter.

Two large national collaboratives to reduce CLABSI13 and CAUTI,14 funded by the Agency for Healthcare Research and Quality (AHRQ), recently employed HSOPS to assess safety culture for two purposes. First, unit team members and collaborative coaches use the results to prompt team discussion, evaluate progress and highlight challenges to refocus efforts and resources. Second, collaborative leaders use HSOPS results to track changes in safety culture measures to assess how well interventions were implemented across different sites and units.

However, despite HSOPS and related safety culture surveys15–17 becoming common tools to assess safety culture in large collaboratives, the extent to which measures from these survey tools are associated with changes in catheter-associated infection rates is unclear.8 18 Furthermore, although as outlined above, safety culture is strongly hypothesised to be very important to reduce hospital-acquired complications such as catheter-associated infections, it remains unclear how important changing safety culture is in reducing catheter-associated infections compared with the importance of standardising technical components of care such as standardising procedures. For example, if educational interventions and training are indeed successful in standardising the steps of aseptic insertion, maintenance and removal of catheters, how important are additional interventions focused on improving safety culture and performing assessments of safety culture? This question is important to inform future collaboratives involving catheter-associated infections to justify the opportunity costs associated with widespread implementation and facilitation of safety culture interventions and routinely performing assessments such as HSOPS. Despite survey tools such as HSOPS being available as standard tools for collecting baseline and reassessments in different units and hospitals with limited expertise needed to administer and analyse the surveys, even these tools have important limitations such as reporting bias of those who complete the survey, and the need for busy clinicians to invest time in completing surveys and collaborative coaches to invest time in encouraging survey completion. To further study these issues, we performed an analysis of HSOPS survey and patient outcome data from the AHRQ CLABSI and CAUTI collaboratives.13 19 We hypothesised that hospital units with HSOPS results consistent with higher scores for safety culture measures would be more successful at implementing technical and socioadaptive components of CLABSI and CAUTI intervention bundles, and would achieve lower CLABSI and CAUTI rates over time.

Methods

Design and intervention

This study was a secondary analysis of a subset of healthcare worker survey and patient outcome data collected in the AHRQ CLABSI and CAUTI collaboratives. Each collaborative was a prospective cohort study in which participating hospital units implemented interventions to reduce catheter-associated infections using principles from the Comprehensive Unit-based Safety Program (CUSP).19 20 In brief, CUSP is a multistep programme, designed to promote safety culture and teamwork, which involves preintervention baseline assessment, strategies to improve engagement and implementation, and feedback to aid the implementation of evidence-based interventions such as checklists for catheter insertion and maintenance care. The HSOPS was employed to assess safety culture for the CLABSI and CAUTI collaboratives21–23 studied in this analysis.

In the online supplementary material, for interested readers, we also provide a summary of two other measures collected in these collaboratives to assess baseline exposure and experience to interventions to be prioritised in the collaborative (ie, the Readiness Assessment,19 20 see online supplementary appendix 2) and to use input from the team at baseline and serially in follow-up (ie, the Team Check-up Tool,13 19 20 see online supplementary appendix 3) with the goal to report progress and barriers in implementation.

Study participants and data sources

Analyses included data from participating adult, acute-care intensive care units (ICUs) and non-ICUs, including all six cohorts from the CLABSI collaborative with data collected from 2008 to 2011, and cohorts 1–4 from the CAUTI collaborative with data collected from 2011 to 2013. CLABSI collaborative cohorts were 30 months in duration; CAUTI collaborative cohorts were 17 months in duration. CAUTI cohorts 5–6 were excluded because of additional interventions implemented in the emergency department and ICU settings. To be included in the analyses the unit also had to have (1) non-missing values for all individual HSOPS measures; (2) hospital characteristic data from American Hospital Association Annual Survey 2010 for CLABSI and 2011 for CAUTI (eg, teaching status, bed size, rurality and critical access); and (3) at least one valid baseline and one valid postbaseline outcome reported.

Measures

Patient outcome measures

The conventional National Healthcare Safety Network infection rate of catheter-associated infections per 1000 catheter-days was collected for both collaborative projects. Unit-level aggregate infection rates were collected at baseline and monthly postimplementation for CLABSI. Unit-level aggregate infection rates were collected at baseline and quarterly postimplementation for CAUTI. Monthly CLABSI results were aggregated to quarterly numbers for our analyses.

HSOPS components

The HSOPS was scored and reported as recommended by the survey tool instructions to the participating units, without recoding of scores or creation of composite or climate profiles.23 The measure scores yielded by this analysis are in the same format provided to the collaborative members in feedback for understanding their units and performance.

The HSOPS is a 42-item survey with Likert scale responses, aligned by 12 dimensions.13 19 20 24 All staff members were encouraged to complete the HSOPS survey at baseline and in follow-up approximately 1 year later.

Of note, the HSOPS tool was employed in these national collaboratives instead of other measures such as the Safety Attitudes Questionnaire (SAQ)15 23 25–28 because at the time of these collaboratives' initiatives, HSOPS was employed more widely in US hospitals nationally, and this tool is in the public domain (as a tool developed by AHRQ) and thus did not require a fee for hospitals to use. While some studies have demonstrated improvements in safety culture measures in response to interventions such as the CUSP, there is limited evidence linking survey-based safety culture measures and patient outcomes, particularly for surveys other then the SAQ.15 23 25–28

Statistical analyses

Multilevel negative binomial models were applied to adjust for clustering within hospitals and within units over time and also for overdispersion of the outcome. Hospital characteristics of bed number, teaching status, critical access status and rurality were adjusted for in the models. Separate models were analysed for ICUs and non-ICUs and culture items were tested individually because of higher catheter use rates in ICUs, and because safety culture of ICUs was anticipated to differ from non-ICUs due to differences in team structure and rounding styles. HSOPS measures were time-varying; models with and without interaction with time were tested individually. Conservative significance testing criteria of p<0.01 was chosen to account for the higher likelihood of random association due to multiple comparisons. Model results for the culture items of interest were depicted using coefficient plots, which plot the coefficient point estimate and the corresponding 99% CI. Analyses were performed using Stata/MP13.1 (StataCorp, College Stations, Texas, USA).

Results

Participant characteristics

Table 1 details characteristics of recruited hospitals and all participating units for each collaborative before application of exclusion criteria. A total of 1821 units from 1079 hospitals (CLABSI) and 1576 units from 949 hospitals (CAUTI) were considered for analysis. Major differences included the CLABSI collaborative having a higher proportion of ICUs (75% vs 41%) and higher proportion of teaching hospitals (33% vs 7%) compared with the CAUTI collaborative. The application of inclusion and exclusion criteria for this analysis is detailed in online supplementary Appendix Figure 1, yielding 598 units from 436 hospitals (CLABSI) and 675 units from 463 hospitals (CAUTI) in the analysis.

Table 1

Recruited hospital and unit characteristics by collaborative

	CLABSI	CAUTI
Hospitals	1079	949
Critical access hospital, N (%)	82 (8)	190 (20)
Rural hospital, N (%)	259 (24)	355 (37)
Teaching hospital, N (%)	359 (33)	67 (7)
Bed size, mean (SD)	253 (229)	189 (198)
Units	1821	1576
Intensive care units, N (%)	1372 (75)	653 (41)

CAUTI, catheter-associated urinary tract infection; CLABSI, central-line-associated bloodstream infection.

Patient outcome measures

Overall, using data available from all participating units in the collaboratives (prior to application of exclusion criteria), catheter-associated infection rates declined over the project periods for the six CLABSI and four CAUTI cohorts including a 41% decline for CLABSI and 14.7% decline for CAUTI (unadjusted).13 29 For the units meeting inclusion criteria for this analysis, catheter-associated infection rates declined by 47% for CLABSI and 23% for CAUTI (unadjusted).

HSOPS components

HSOPS response rates were low overall, at 24% for the CLABSI collaborative and 43% for the CAUTI collaborative, calculated as the number of units with HSOPS data divided by the total number of active units. Summary statistics for the HSOPS dimensions at baseline, follow-up and change from baseline to follow-up are detailed in tables 2 and 3 separately for ICU and non-ICUs. In general, mean changes in HSOPS scores ranged from −3.4 to +2.9, with variation in individual hospital units showing both large improvements and declines.

Table 2

CLABSI HSOPS dimension score summary for analytic sample, mean (SD) (range)

	ICUs (n=304)			non-ICUs (n=58)
HSOPS dimension	Baseline	Follow-up	Change Score	Baseline	Follow-up	Change Score
Supervisor expectations and actions promoting safety	75 (12) (33, 100)	78 (14) (24, 100)	3 (16) (−56, 53)	74 (15) (33, 100)	79 (13) (43, 100)	5 (17) (−38, 50)
Organisational learning—continuous improvement	74 (12) (36, 100)	74 (14) (25, 100)	0.4 (14) (−43, 51)	72 (15) (23, 100)	75 (15) (38, 100)	3 (18) (−40, 50)
Teamwork within hospital units	85 (9) (38, 100)	82 (12) (41, 100)	−3 (12) (−56, 34)	76 (15) (25, 100)	73 (18) (0, 100)	−3 (18) (−63, 43)
Communication openness	63 (12) (30, 100)	63 (16) (8, 100)	0.8 (16) (−50, 54)	55 (15) (24, 84)	61 (15) (29, 100)	5 (18) (−50, 39)
Feedback and communication about error	60 (14) (20, 100)	61 (17) (11, 100)	1 (16) (−59, 71)	61 (16) (20, 100)	58 (15) (7, 85)	−3 (17) (−58, 30)
Non-punitive response to error	39 (15) (3, 100)	47 (20) (0, 100)	8 (19) (−44, 72)	39 (16) (7, 100)	55 (20) (13, 100)	16 (27) (−53, 69)
Staffing	58 (14) (18, 100)	59 (15) (13, 100)	1 (15) (−42, 59)	52 (15) (26, 100)	55 (18) (15, 100)	3 (22) (−53, 60)
Management support for patient safety	62 (16) (11, 100)	61 (18) (0, 100)	−1 (18) (−77, 59)	65 (15) (29, 100)	66 (17) (0, 100)	0.2 (20) (−52, 29)
Teamwork across hospital units	56 (14) (20, 100)	57 (16) (0, 100)	0.8 (14) (−47, 52)	55 (13) (23, 85)	52 (19) (0, 83)	−3 (20) (−52, 29)
Hospital handoffs and transitions	48 (13) (0, 100)	50 (16) (0, 100)	1 (16) (−42, 75)	43 (13) (17, 75)	43 (16) (0, 79)	−0.2 (16) (−28, 38)
Frequency of event reporting	57 (14) (0, 100)	56 (17) (0, 100)	−0.4 (17) (−75, 56)	60 (14) (29, 90)	51 (20) (0, 100)	−10 (23) (−74, 63)
Overall perceptions of safety	60 (14) (22, 92)	56 (19) (0, 100)	−4 (18) (−71, 55)	55 (15) (20, 100)	43 (20) (0, 83)	−12 (20) (−77, 24)
Patient safety grade	69 (22) (0, 100)	69 (20) (0, 100)	0.2 (20) (−62, 66)	63 (22) (0, 100)	64 (22) (0, 100)	2 (25) (−54, 66)

Baseline and follow-up scores represent the per cent of positive responses for all items in the domain at the baseline and follow-up time points. HSOPS change scores are computed as the change in per cent positive responses for all items within each dimension from baseline to the follow-up survey period (approximately 11 months later). Values greater than zero represent an increase in per cent positive responses, while values less than zero represent a decrease in per cent positive responses.

CLABSI, central-line-associated bloodstream infection; HSOPS, Hospital Survey on Patient Safety Culture; ICU, intensive care unit.

Table 3

CAUTI HSOPS dimension score summary for analytic sample, mean (SD) (range)

	ICUs (n=164)			Non-ICUs (n=276)
HSOPS dimension	Baseline	Follow-up	Change Score	Baseline	Follow-up	Change Score
Supervisor expectations and actions promoting safety	76 (16) (19, 100)	79 (15) (17, 100)	3 (18) (−58, 81)	79 (12) (40, 100)	80 (13) (8, 100)	1 (14) (−62, 40)
Organisational learning—continuous improvement	73 (15) (29, 100)	74 (16) (20, 100)	0.7 (17) (−54, 71)	76 (12) (41, 100)	77 (14) (33, 100)	1 (12) (−34, 35)
Teamwork within hospital units	86 (9) (64, 100)	87 (10) (38, 100)	0.7 (11) (−38, 33)	79 (13) (0, 100)	80 (13) (32, 100)	1 (13) (−35, 95)
Communication openness	63 (17) (0, 100)	64 (16) (0, 100)	1 (19) (−61, 67)	59 (14) (17, 100)	61 (16) (0, 100)	2 (15) (−66, 44)
Feedback and communication about error	61 (18) (11, 100)	65 (18) (0, 100)	3 (21) (−100, 67)	63 (15) (27, 100)	65 (17) (0, 100)	3 (16) (−100, 67)
Non-punitive response to error	40 (16) (0, 85)	46 (19) (0, 100)	5 (18) (−36, 70)	43 (16) (0, 100)	46 (17) (0, 100)	3 (17) (−100, 58)
Staffing	59 (17) (21, 100)	60 (18) (16, 100)	0.5 (17) (−38, 60)	55 (16) (10, 100)	57 (17) (8, 100)	2 (15) (−45, 63)
Management support for patient safety	63 (21) (0, 100)	65 (19) (4, 100)	2 (20) (−67, 75)	70 (15) (25, 100)	71 (16) (19, 100)	1 (15) (−49, 59)
Teamwork across hospital units	59 (16) (20, 100)	62 (18) (0, 100)	3 (16) (−50, 52)	59 (15) (0, 100)	61 (16) (8, 100)	2 (14) (−45, 72)
Hospital handoffs and transitions	51 (15) (0, 100)	53 (18) (0, 100)	1 (16) (−50, 67)	47 (16) (16, 100)	49 (17) (0, 100)	2 (14) (−50, 66)
Frequency of event reporting	60 (16) (0, 100)	61 (20) (0, 100)	0.6 (16) (−60, 48)	66 (14) (20, 100)	67 (15) (22, 100)	0.6 (15) (−67, 45)
Overall perceptions of safety	60 (18) (0, 100)	62 (18) (0, 100)	2 (18) (−47, 53)	60 (15) (23, 100)	62 (16) (13, 100)	2 (14) (−42, 57)
Patient safety grade	70 (21) (12, 100)	70 (20) (0, 100)	−0.4 (26) (−100, 84)	66 (20) (0, 100)	70 (20) (0, 100)	3 (20) (−100, 63)

Baseline and follow-up scores represent the per cent of positive responses for all items in the domain at the baseline and follow-up time points. HSOPS change scores are computed as the change in per cent positive responses for all items within each dimension from baseline to the follow-up survey period (approximately 11 months later). Values greater than zero represent an increase in per cent positive responses, while values less than zero represent a decrease in per cent positive responses.

CAUTI, catheter-associated urinary tract infection; HSOPS, Hospital Survey on Patient Safety Culture; ICU, intensive care unit

Association between HSOPS measures and patient outcomes

Detailed model results for baseline and longitudinal outcomes for the CLABSI and CAUTI collaboratives are found in online supplementary appendix 1 tables 1–8 with separate models for ICUs and non-ICUs. In a few cases, model results are not presented due to either lack of variation in the response for the culture item of interest, or too few units responding. Figure 1 illustrates the coefficient plots for the HSOPS models for ICUs and non-ICUs, for CLABSI (figure 1A) and CAUTI (figure 1B). Based on the conservative significance testing criteria of p<0.01, there were no statistically significant associations found between any of the measures selected for testing from each of the selected measures of safety culture and the catheter-associated infection outcomes.

Figure 1

(A) Central-line-associated bloodstream infection (CLABSI) coefficient plots, Hospital Survey on Patient Safety Culture (HSOPS) models. (B) Catheter-associated urinary tract infection (CAUTI) coefficient plots, HSOPS models. Incidence rate ratios (IRRs) and their 99% confidence intervals (CIs) are given for the HSOPS domains. CIs that cross the vertical line at the value of 1 indicate non-significant findings. Models also adjusted for hospital characteristics including bed size, teaching and critical access hospital status and rurality. For detailed model results, see online supplementary appendix. ICU, intensive care unit.

Discussion

Key results

Contrary to our hypothesis, there was no significant association between safety culture as assessed by the HSOPS with CLABSI or CAUTI outcomes when either measured at baseline or in follow-up, in two national collaboratives that were successful in reducing these infections. An important implication of this result is that it is possible to reduce CLABSI and CAUTI rates without improving safety culture, through improvements in technical components of care such as standardising procedures involving catheter insertion, maintenance and removal. This interpretation would prompt reconsideration and prioritisation of technical and safety culture components of interventions in future studies, particularly for hospitals or units with limited bandwidth to implement new interventions given competing priorities or limited resources. Another potential explanation of the findings is that safety culture did improve in these collaboratives and was instrumental for reducing CLABSI and CAUTI but the HSOPS tool did not adequately detect or assess important components of safety culture in the participating units. This could occur if the HSOPS survey was not designed to produce a valid measure of safety culture impacting care at the bedside or if it was completed by respondents who had less influence or understanding of the safety culture impacting bedside care. Safety culture can be difficult to assess, particularly given its fluid nature impacted by changes in staff, resources and competing priorities that commonly occur in hospital units in short time frames.30 31

Strengths and limitations

Strengths of this study include the use of data collected from two large national prospective cohort studies implementing evidence-based interventions with the support of CUSP strategies for optimising safety culture. These collaboratives involved a wide variety of hospital and unit types, and included analyses specific to ICU and non-ICU designations to study anticipated differences in the safety culture; ICUs are often ‘closed’ units with fewer physician teams and a stronger tradition of multidisciplinary rounds and more experience in using checklists for care.

Some important limitations should be noted. Despite considerable focus during the collaboratives to encourage the use of HSOPS, survey response rates were low for both the CLABSI collaborative at 24% and the CAUTI collaborative at 43%—much lower than the response rates of >70% in prior studies using the SAQ to assess safety culture.15 16 Such a low response rate for HSOPS from two large national collaboratives is an important finding of this analysis given the substantial resources invested in measure collection. Low culture survey response rates clearly impacted the available data and which units were included in the analysis. There are several potential explanations for the low survey response rate.32 It could reflect ‘safety culture survey fatigue’ as these surveys were overlaid on top of periodic safety culture surveys that many participants may have been asked to complete on an ongoing basis beyond these collaboratives' tasks. Low response rate could also reflect participant attitude towards the value of the survey or the value of the collaborative requesting the survey. Of note, a low overall response rate does not indicate that individual units or participants do not find value in the survey tool as a prompt for discussion and identification of areas for improvement. However, an association between culture scores and infection rates was not detected even among the units who completed the culture surveys, despite overall having moderate (23% CAUTI decline, unadjusted) to large (47% CLABSI decline, unadjusted) improvements in catheter-associated infection rates. Therefore, having a higher response rate would not be anticipated to better detect changes in safety culture. The low response rate is also an important outcome as empirical evidence of the difficulty in using these survey tools in busy inpatient units, particularly in collaboratives such as these with a strong focus on measurement and improvement of safety culture. The validity of HSOPS in the AHRQ CLABSI and CAUTI collaboratives for detecting true changes in safety culture is also unclear, particularly when assessed periodically, at the unit level, with low response rates, from a limited number of clinicians per unit, from multiple hospitals. Indeed, this study did not assess other safety culture tools collected in other collaboratives such as the SAQ, a validated and reliable survey tool that has been found to be responsive to interventions including CUSP.15 16 26

It is also important to note that there are differences of opinion in whether HSOPS results should be analysed and interpreted using individual survey dimensions or items as opposed to generating and comparing ‘climate profiles’, as described by Weaver and colleagues using a k-means cluster analysis to study the association between the HSOPS baseline data and baseline CLABSI outcomes in the AHRQ CLABSI collaborative ICUs.23 In contrast to our study, which demonstrated no statistically significant associations between the individual dimensions (in HSOPS) and CLABSI outcomes, Weaver et al report that certain climate profiles described as ‘conflicting’ or ‘non-punitive’ climates generated from baseline HSOPS dimensions are significantly associated with higher baseline CLABSI rates compared with a climate profile described as ‘generative leadership’.23 It is acknowledged that profiles of survey measures can provide additional insight into the combination of factors that yield a unit-based culture. However, generating and interpreting climate profiles is resource intensive, and climate profiles are not available in real time currently to hospital units that are implementing and feeding back results of these survey tools to clinicians. It remains concerning that of the multiple measures selected for analysis by content and coaching experts as most likely to be associated with success in device-associated infection prevention, not a single positive association was demonstrated and some associations appeared to be in the opposite direction of what was expected.

Conclusions

Analyses from two large national collaboratives involving the prevention of CLABSI and CAUTI demonstrated no association between safety culture measures collected from the HSOPS and patient outcomes of CLABSI and CAUTI. These unexpected results do not support the hypothesis that hospital units with improvements in safety culture measures would be more successful in prevention of CLABSI and CAUTI. These results have two important potential interpretations. First, it may simply be true that these collaboratives successfully reduced CLABSI and CAUTI by means other than improving safety culture, such as a strong emphasis on standardising technical components of care such as aseptic insertion, maintenance and removal. Changing safety culture may not be critical in the setting of improvements in standardised practices. A second interpretation is that the safety culture did improve and was instrumental in improving outcomes, but was inadequately measured by the HSOPS tool. Assessment of safety culture is complicated, dynamic and may simply not be captured adequately by survey tools. Considering the training, time and financial resources needed to conduct and analyse these types of surveys in busy clinical units, these results showing a lack of association between HSOPS and CLABSI and CAUTI outcomes do suggest the need to reassess and potentially reprioritise components of intervention bundles and collaborative tools that focus on improving and monitoring technical aspects of care with respect to the need to collect measures of safety culture such as HSOPS as a routinely recommended tool.