A multicentre cluster-randomized clinical trial to improve antibiotic use and reduce length of stay in hospitals: comparison of three measurement and feedback methods.

BACKGROUND: Various metrics of hospital antibiotic use might assist in guiding antimicrobial stewardship (AMS).
OBJECTIVES: To compare patient outcomes in association with three methods to measure and feedback information on hospital antibiotic use when used in developing an AMS intervention.
METHODS: Three methods were randomly allocated to 42 clusters from 21 Dutch hospitals: (1) feedback on quantity of antibiotic use [DDD, days-of-therapy (DOT) from hospital pharmacy data], versus feedback on (2) validated, or (3) non-validated quality indicators from point prevalence studies. Using this feedback together with an implementation tool, stewardship teams systematically developed and performed improvement strategies. The hospital length of stay (LOS) was the primary outcome and secondary outcomes included DOT, ICU stay and hospital mortality. Data were collected before (February-May 2015) and after (February-May 2017) the intervention period.
RESULTS: The geometric mean hospital LOS decreased from 9.5 days (95% CI 8.9-10.1, 4245 patients) at baseline to 9.0 days (95% CI 8.5-9.6, 4195 patients) after intervention (P < 0.001). No differences in effect on LOS or secondary outcomes were found between methods. Feedback on quality of antibiotic use was used more often to identify improvement targets and was preferred over feedback on quantity of use. Consistent use of the implementation tool seemed to increase effectiveness of the AMS intervention.
CONCLUSIONS: The decrease in LOS versus baseline likely reflects improvement in the quality of antibiotic use with the stewardship intervention. While the outcomes with the three methods were otherwise similar, stewardship teams preferred data on the quality over the quantity of antibiotic use.

RCT Entities:   Population Interventions Outcomes

Introduction

To curb antimicrobial resistance, better use of antibiotic agents is pivotal. Antimicrobial stewardship programmes (ASPs) have been designed to measure and improve the appropriateness of antibiotic use while minimizing the unintended consequences of antibiotic use. As not every hospital or ward needs the same level of improvement, ASP improvement strategies should be tailored to local settings.

One cornerstone of ASPs is the systematic measurement of the (appropriateness of) local use to guide tailored improvement strategies. Feedback is an important ingredient of such strategies, as it further increases the effect of enabling strategies. Various methods have been recommended to measure and feed-back on antibiotic use in hospitals, ranging from monitoring quantitative antibiotic use at an institutional level, to performing point prevalence studies (PPS) on the appropriateness of antibiotic use in individual patients. However, efforts to perform these methods, and information obtained with them, vary substantially. For stewardship teams to optimally improve local antibiotic use, the comparative effectiveness of these various options in measuring and feeding back data on antibiotic use should be evaluated.

In this cluster-randomized multicentre trial, we compared the effects on length of hospital stay and secondary patient outcomes of three recommended methods to measure and feed-back information on hospital antibiotic use, when used as a first step of a stewardship intervention to improve hospital antibiotic use.

Methods

Study design, setting and population

We performed a multicentre, cluster-randomized trial with repeated before and after measurements in 21 Dutch hospitals to compare the effects of three different methods to measure and feed-back information on hospital antibiotic use (Figure 1). The methods were: (1) extraction of, and feedback on, last year’s quantity of hospital pharmacy antibiotic use data (OVERALL USE); (2) performance of a PPS to provide feedback on validated quality indicators (QIs) for appropriate antibiotic use (PPS-QI, Table S1, available as Supplementary data at JAC Online); and (3) performance of a PPS to provide feedback on a simplified, non-validated set of indicators (PPS-ECDC, Figure S1). Measurements and feedback (phase 1) were followed by systematic development (phase 2) and performance (phase 3) of setting-specific improvement strategies (Figure 1 and Box S1). Twenty-one Dutch hospitals participated in the study. In each participating hospital, two main clusters were selected: a non-surgical cluster containing the ‘high antibiotic use’ specialties internal medicine (including geriatric patients), gastroenterology and pulmonology; and a surgical cluster containing the high use specialties surgery, urology and orthopaedics. This trial was registered with the Dutch Trial Registry, number 5933 (http://www.trialregister.nl, where the trial protocol is available).

Figure 1.

The study design.

Randomization

Using clusters as the unit of randomization, the 42 clusters (21 pairs) were randomly allocated to one of the three methods, using SAS Proc Survey select software, version 9.4. In each hospital, each method was allocated to no more than one cluster by consecutive random sampling (Table S2 and Text S1).

The Medical Ethics Research Committee of the Academic Medical Center confirmed that the Dutch Medical Research Involving Human Subjects Acts (WMO) did not apply to this study and that an official approval by the committee was not required (October 2014).

Intervention

The intervention period lasted from September 2015 to May 2017 and consisted of three phases (Box S1).

In phase 1, each local stewardship team performed a one-time (quantitative or qualitative) antibiotic use measurement for the surgical and for the non-surgical cluster in their hospital, according to the randomly assigned method (Box S2). Data were processed into a feedback report by the research team: results were summarized for each hospital cluster and benchmarked against similar clusters from other hospitals. Each team received two feedback reports, one for the surgical and one for the non-surgical cluster, with instructions to use these reports only for their allocated cluster.

In phase 2, all 21 hospital stewardship teams were trained in applying an implementation tool (Figure S2) that supported a structured approach to stewardship, i.e. the systematic development of setting-specific stewardship improvement strategies based on the feedback reports. The tool systematically guides stewardship teams through the stepwise process of identifying targets for improvement from the feedback reports; assessing local barriers that hinder appropriate use; and developing an action plan including improvement strategies to overcome these barriers. Stewardship teams were supported by the study team (i.e. the study coordinator, one infectious diseases specialist and one implementation specialist) in applying this structured approach to stewardship.

In phase 3, once instructed, stewardship teams applied this approach to locally perform the stewardship improvement strategies. All improvement strategies were initiated and executed by the local stewardship teams, the role of the study team was only to guide and advise on this.

Before and after measurements

To assess (per method) the impact of the three-phase stewardship intervention, we included for each of the 42 clusters, 100 admitted patients receiving antibiotic treatment at baseline (February–May 2015, 4 monthly sets of 25 patients), and again 100 patients per cluster after the intervention (February–May 2017). Consecutive patients were selected from a hospital-generated list containing all patients from the participating clusters receiving antibiotic treatment. Within each cluster, we aimed at a balanced distribution of medical specialties. Further details on in/exclusion criteria and data collection are provided in Text S2 and Figure S3.

Primary and secondary outcome measures

Length of stay (LOS) was used as primary outcome measure. We used LOS as a surrogate marker of quality of antibiotic use, as previous studies found an association between appropriate antibiotic use and LOS. LOS was defined as the number of days between admission and discharge for community-acquired infections. For hospital-acquired infections, LOS was defined as the number of days between start of antibiotic treatment and discharge. Day of admission and day of discharge counted as one hospital day. Secondary outcome measures were total antibiotic use, use of IV antibiotics and use of restricted antibiotics [expressed in Days-Of-Therapy (DOT) per 100 admissions and DOT per 100 patient-days; (Table S3], admission to and duration of ICU stay, and hospital mortality. If patients were admitted and discharged from the ICU on the same day, the duration of ICU stay was set at 0.5 days. If patients were admitted to the ICU more than once during their hospital stay, ICU days were summed.

Process evaluation

At the end of the intervention a process evaluation was performed, to be able to explain potential differences in effect between the various methods.

We collected information on hospital (e.g. hospital size, type) and stewardship team characteristics (e.g. presence of stewardship team, frequency of team meetings). Stewardship teams were asked to provide information on time investment for obtaining feedback data; actual use of the feedback report and the implementation tool; and on local stewardship improvement activities performed. Also, stewardship teams self-assessed the effectiveness of each measurement and feedback method [6-point Likert scale, ‘not effective’ (0) to ‘very effective’ (5)].

Statistical analyses and sample size

The effectiveness of the three-phase stewardship intervention was expressed as a difference in the primary outcome (LOS) and secondary outcomes (DOT, in-hospital mortality, ICU admission and duration) before and after intervention, overall and for each of the three methods. Mixed linear effect models were used to accommodate the hierarchical structure in the data (patients nested within clusters nested within hospitals) and to adjust for differences in case-mix between baseline and post-intervention measurements and between hospitals. LOS and ICU LOS were log-transformed due to non-normal distributions. Continuous outcomes (LOS, DOT and ICU LOS) were analysed with linear mixed models (LMM). For dichotomous outcomes (ICU admission and in-hospital mortality) we used generalized linear mixed models (GLMM).

We adjusted LOS for the secular trend in LOS using national annual reference data (estimated −0.1 patient-days/year), to ensure that a difference in LOS in the model was more likely to reflect a change due to interventions. Analyses were performed according to the intention-to-treat (ITT) approach (i.e. with clusters allocated to the method they had been randomized to) and the ‘as-treated’ (AT) approach (i.e. according to the methods they actually used). Further details are provided in Text S3, including the effects specified for the process evaluation variables, i.e. hospital and stewardship team characteristics, and performance (at cluster level) of the five steps of the implementation tool (translated into a sum score of actual steps performed by the stewardship teams).

All analyses were done using IBM SPSS Statistics, version 23.0.

Sample size calculation

We assumed a baseline LOS of 9 days (SD 6.2) and a within-cluster correlation (ICC) of 0.20, based on results of previous studies on length of hospital stay., We estimated that with 21 hospitals with 2 clusters each, 4 × 25 patients per cluster before and 4 × 25 patients per cluster after the intervention would be needed for this study to have a power of approximately 80% to demonstrate a reduction in geometric mean length of stay of 0.8 days (−9%) with an alpha of 0.05. This results in a total sample size (before and after intervention) of 8400 patients. For the power analysis we used SAS version 9.3.

Results

Three university hospitals, sixteen teaching hospitals and two non-teaching hospitals participated. Hospital size ranged between 255 and 1350 beds (median: 630 beds). Primary and secondary outcome measures were assessed in 8840 patients: 4245 before (2125 surgical/2120 non-surgical patients), and 4195 after intervention (2104 surgical/2091 non-surgical patients). Distribution of specialties within the clusters was comparable before and after intervention (Table S4). Baseline characteristics (per cluster) are shown in Table 1 and Table S5. Linear mixed models corrected for any before and after differences.

Table 1.

Patient characteristics

Baseline characteristics	Baseline (N = 4245)	Post intervention (N = 4195)	P value
Sex, male	2217 (52)	2207 (53)	0.72
Age, mean (SD)	68.5 (16)a	68.6 (16)	0.87
Infection, community-acquired/hospital-acquired	2990 (70)/1255 (30)	2939 (70)/1256 (30)	0.71
Type of diagnosis
Respiratory tract infection	1099 (26)	1096 (26)	0.80
Urinary tract infection	806 (19)	873 (21)	0.04
Skin and soft tissue infection	556 (13)	542 (13)	0.81
Orthopaedic infection	252 (6)	270 (6)	0.34
Abdominal infection	786 (19)	799 (19)	0.53
Other infection	352 (8)	268 (6)	0.001
Two or more possible infections	335 (8)	328 (8)	0.90
Diagnosis unknown	59 (1)	19 (1)	<0.001
Charlson Comorbidity Index, median (IQR)	1 (0–2)	1 (0–2)	0.01
Received antibiotics <30 days before start of treatment	1539 (36)	1574 (38)	0.23
Received also prophylactic antibiotics	747 (18)	827 (20)	0.01
Admitted from a nursing home	170 (4)	220 (5)	0.01

Numbers are n (%) unless otherwise indicated. Percentages were calculated with the denominator excluding missing cases.

Missing data in 1 patient.

Overall effectiveness of the three-phase stewardship intervention

Primary outcome measure

The geometric mean LOS was 9.5 days (95% CI 8.9–10.1, N = 4245 patients) at baseline versus 8.7 days (95% CI 8.1–9.2, N = 4195 patients) after intervention, while adjusting for dependencies within clusters and potential confounders (Table 2). After adjusting for secular trend, the estimated decrease in geometric mean LOS was 0.5 days: 9.5 days (95% CI 8.9–10.1, N = 4245 patients) at baseline versus 9.0 days after intervention (95% CI 8.5–9.6); P < 0.001, N = 4195 patients. Figure S4 graphically illustrates the decreasing trend of LOS over time.

Table 2.

Effect of the three-phase intervention on primary and secondary outcome measures, adjusted for co-variates

Outcome measures	Baseline (N = 4245)	Post intervention (N = 4195)	Δ (%)	P value
Primary
LOS geometric mean (95% CI)a,c,d,e,f,g	9.5 (8.9–10.1)	8.7 (8.1–9.2)	−0.8 (−8)	<0.001
LOS geometric mean (95% CI)–with adjustment for timea,c,d,e,f,g,j	9.5 (8.9–10.1)	9.0 (8.5–9.6)	−0.5 (−5)	<0.001
Secondary
Days of antibiotic therapy (DOT)
per 100 patient-days (95% CI)a,b,c,d,e,f,g,h,i	86 (81–90)	84 (80–89)	−2 (−2)	0.20
per 100 admissions (95% CI)d,e,f,g,h	1320 (1253–1387)	1185 (1119–1252)	−135 (−10)	<0.001
Days of IV antibiotic therapy (DOT)
per 100 patient-days (95% CI)a,c,d,e,f,g,i	53 (49–58)	54 (49–58)	+1 (+2)	0.58
per 100 admissions (95% CI)a,d,e,f,g,h,i	897 (841–952)	806 (751–862)	−91 (−10)	<0.001
Days of restricted antibiotic therapy (DOT)
per 100 patient-days (95% CI)d,e,f,h	8 (7–9)	7 (6–9)	−1 (−13)	0.11
per 100 admissions (95% CI)d,e,f,g,h	178 (152–204)	144 (118–170)	−34 (−19)	<0.001
ICU admissionb,c,d,e,f,g,h,i	251 (5.9)	201 (4.8)	−50 (−1)	0.02
ICU LOS, geometric mean (95% CI)f,g	1.9 (1.6–2.2)	1.9 (1.6–2.3)	+0.03 (+2)	0.82
In-hospital mortalitya,c,d,e,f	137 (3.2)	142 (3.4)	+5 (+0.2)	0.69

Numbers are n (%) unless otherwise indicated. There were no missing cases. LOS, length of hospital stay.

The model was evaluated at mean age = 68.6 years and mean baseline LOS = 7.0 days.

Adjusted for:

age;

sex;

comorbidity;

type of diagnosis;

community versus hospital-acquired infection;

type of admission specialty;

discharge to a nursing home;

antibiotics received 30 days prior to admission;

antibiotic; prophylaxis;

time since first included patient.

Secondary outcomes

DOT per 100 admissions decreased from 1320 (95% CI 1253–1387, N = 4245 patients) at baseline to 1185 (95% CI 1119–1252, N = 4195 patients) after the intervention (−10%; P < 0.001), while DOT per 100 patient-days remained unchanged (Table 2). Similar trends were found for days of IV antibiotic therapy. A larger decrease was found for restricted DOT per 100 admissions (−19%, P < 0.001, N = 324 versus N = 285) and for restricted DOT per 100 patient-days (−13%, P = 0.11, N = 324 versus N = 285).

The percentage of patients admitted to the ICU was lower after the intervention (4.8%, N = 201 patients) compared with at baseline (5.9%, N = 251 patients). There was no difference in ICU LOS or in-hospital mortality (Table 2). Results were comparable for surgical and non-surgical clusters.

Comparative effectiveness of the three measurement methods

Details about the comparative effects of the three methods on the primary and secondary outcome measures are presented in Table 3 and Table S6.

Table 3.

Comparative effectiveness of the three measurement methods on length of hospital stay, using the ITT and AT approach

LOSa	All clusters (N = 42)	Non-surgical clusters (N = 21)	Surgical clusters (N = 21)	Baseline (N = 4245)	Post intervention (N = 4195)	Δ (%)	P valueb
ITT approachc
Method 1: Overall use	14/42 (33)	7/21 (33)	7/21 (33)	9.5 (8.7–10.4)	9.0 (8.6–9.4)	−0.5 (−5)	Ref
Method 2: PPS-QI	14/42 (33)	7/21 (33)	7/21 (33)	9.5 (7.7–11.6)	8.9 (8.2–9.8)	−0.6 (−6)	0.97
Method 3: PPS-ECDC	14/42 (33)	7/21 (33)	7/21 (33)	9.2 (7.5–11.3)	8.9 (8.1–9.7)	−0.3 (−3)	0.69
AT approachd
Method 1: Overall use	2/42 (5)	1/21 (5)	1/21 (5)	8.8 (6.3–12.6)	8.9 (7.5–10.6)	+0.1 (+1)	0.59
Method 2: PPS-QI	15/42 (36)	8/21 (38)	7/21 (33)	9.9 (7.8–12.7)	9.3 (8.1–10.6)	−0.6 (−6)	0.19
Method 3: PPS-ECDC	12/42 (29)	7/21 (33)	5/21 (24)	9.6 (7.5–12.3)	9.0 (7.9–10.2)	−0.6 (−6)	0.22
Method 4: Other	13/42 (31)	5/21 (24)	8/21 (38)	9.2 (8.1–10.4)	9.0 (8.3–9.6)	−0.2 (−2)	Ref

Numbers are n/N (%) unless otherwise indicated. There were no missing cases. LOS, length of hospital stay. Ref indicates the reference category.

The model was evaluated at mean age = 68.6 years and mean baseline LOS = 7.0 days.

Adjusted for: age; comorbidity; type of diagnosis; community versus hospital acquired infection; type of admission specialty; discharge to a nursing home; time since first included patient.

Interaction effect=the difference in effect on outcomes (after minus before) between the measurement method and the reference method.

ITT approach: measurement methods as allocated by randomization.

AT approach: measurement methods as used by the stewardship teams.

ITT

No significant differences in effect on LOS were found between the methods (Overall use versus PPS-QI P = 0.97, Overall use versus PPS-ECDC P = 0.69). In the Overall use group LOS decreased from 9.5 days (95% CI 8.7–10.4) to 9.0 days (95% CI 8.6–9.4), P = 0.02; in the PPS-QI group LOS decreased from 9.5 days (95% CI 7.7–11.6) to 8.9 days (95% CI 8.2–9.8), P = 0.02; in the PPS-ECDC group LOS decreased from 9.2 days (95% CI 7.5–11.3) to 8.9 days (95% CI 8.1–9.7), P = 0.09.

No significant differences in effect on DOT were found between the methods, except that PPS-QI and PPS-ECDC showed a significantly larger decrease on restricted DOT per 100 admissions (P = 0.04 and P = 0.02).

AT

The Overall use feedback report was actually used by stewardship teams to develop improvement strategies for only two clusters (5%), the PPS-QI feedback report was used for fifteen clusters (36%), and the PPS-ECDC feedback report was used for twelve clusters (29%). Twenty-one percent of the feedback reports were used for a non-allocated cluster, for instance because a feedback report provided more constructive information to identify targets for ASP improvement strategies (see Process evaluation section), or because an improvement strategy based on one feedback report was executed hospital-wide. Thirteen clusters (31%) did not base their improvement strategies on any feedback report, for example if they were part of ongoing local stewardship activities.

The PPS-QI method (15 clusters) and PPS-ECDC method (12 clusters) showed a larger decrease in LOS (−0.6 days for both methods) as opposed to the Overall use method (2 clusters) and the ‘no feedback used’ group (13 clusters) (+0.1 days and −0.2 days respectively). PPS-QI showed a significantly larger decrease in restricted DOT per 100 admissions compared with the other methods (P = 0.03).

Effect of hospital and stewardship team characteristics

Table S7 shows the effects on the outcome measures of hospital factors, such as hospital type, number of beds, presence of residency programmes, and stewardship team factors, such as the presence of an officially appointed stewardship team, frequency of team meetings, and staff full-time equivalents dedicated to the stewardship team. No strong or consistent associations were found.

Team’s evaluation of the three methods (phase 1)

Stewardship teams scored the PPS-QI method as most effective for stewardship (Table 4). Targets for improvement could be easily identified from the feedback report. It provided specific information on the appropriateness of current antibiotic use, e.g. by reporting the percentage of patients in whom blood cultures were performed before start of antibiotic therapy.

Table 4.

Effectiveness of the intervention components as reported by the stewardship teams

How effective did you find the following elements to structurally develop and implement setting-specific stewardship strategies during the study?	Mean of hospital means (1–4 responses per hospital)a
Feedback report Overall use	2.9 (1.7–4.5)
Feedback report PPS-QI	3.8 (2.0–5.0)
Feedback report PPS-ECDC	3.3 (1.0–5.0)
Implementation tool to improve appropriate antibiotic use	3.0 (1.0–4.5)
Educational meeting	3.2 (1.0–5.0)
Visits of the study team	3.6 (2.3–5.0)
Guidance, advice and reminders by the study team	3.4 (2.0–5.0)

0 = not effective; 5 = very effective.

The Overall use method received the lowest effectiveness score. Stewardship teams used the Overall use feedback reports for only two clusters. The main reason for this was that the interpretation of data was difficult, requiring knowledge on data registration and extraction procedures, and resulted in extra analyses or extra information extraction from electronic patient records.

Data collection for the PPS-QI method was the most time-consuming, followed by the PPS-ECDC and Overall use method (respectively 48.5 versus 19.5 versus 6.7 h per cluster).

Use of the stewardship implementation tool (phase 2)

The stewardship implementation tool was used for 36 of 42 clusters (86%). Stewardship teams identified targets for improvement from the feedback reports for 36 clusters (step 1 of the tool). For 25 clusters a barrier analysis was performed (step 2–3) and for 29 clusters an action plan was developed, containing local improvement strategies (step 4–5). The mean time invested by stewardship teams in applying the improvement tool (for the identification of improvement targets, performance of a barrier analysis and development of an action plan) was 9.2 h per cluster.

More-consistent use of the stewardship implementation tool resulted in a larger decrease in total DOT, IV DOT and restricted DOT. A significantly larger decrease in DOT per 100 patient-days was found for a sum score equal to 1 (P = 0.03) and equal to 2 (P = 0.04), a significantly larger decrease in restricted DOT per 100 admissions was found if the sum score equalled 2 (P = 0.02) (Figure S5 and Table S8).

Stewardship activities initiated by the stewardship teams (phase 3)

A total of 52 setting-specific improvement projects were performed in 42 clusters, with a median of one project (range: 1–3) per cluster: mainly IV-to-oral switch projects (43%), and projects focusing on appropriate treatment for patients with pneumonia (21%) or the appropriate use of restricted antibiotics (19%).

Figure S6 and Table S9 illustrate the improvement activities initiated by stewardship teams during the intervention period, with the different colours reflecting the ‘as treated’ measurement and feedback methods on which these activities were based.

Discussion

This multicentre, cluster-randomized study showed that the three-phase stewardship intervention was associated with a significant reduction in LOS, in addition to the secular trend, without affecting ICU admission or in-hospital mortality. Also, a significant decrease in DOT per 100 admissions and no change in DOT per 100 patient-days was found, suggesting an absolute decrease in total hospital antibiotic use per patient. A decrease in the use of restricted antibiotics contributed to this. No significant differences in effect were found between the three measurement and feedback methods. The process evaluation showed, however, that feedback on the quality of antibiotic use was used more often to identify targets for ASP improvement strategies and was preferred over feedback on the quantity of use. Moreover, consistent use of the stewardship implementation tool resulted in a larger decrease in total DOT, IV DOT and restricted DOT.

Although our results suggest no significant differences in effect between the three methods, stewardship teams found the Overall use method the least effective for stewardship purposes, whereas the PPS-QI method was regarded to be the most effective for stewardship purposes as targets for improvement could be easily identified from the feedback report. However, data collection for the PPS-QI method was substantially more time consuming compared with the PPS-ECDC and the Overall use method. At present, a PPS still requires manual data collection, which is time consuming and labour intensive. Electronic data registration will facilitate future measurements.

The challenge for stewardship teams lies in systematically selecting improvement strategies useful for their clinical setting; the assessment of local barriers should inform the choice of the appropriate improvement strategy for their setting.,, Flottorp et al. developed a comprehensive checklist built on 12 frameworks and taxonomies, including all types of barriers that might prevent improvement in clinical practice. We provided stewardship teams with an implementation tool based on this checklist. Use of this tool encouraged stewardship teams to systematically develop improvement strategies, based on local barriers, and to make a structured action plan, providing clear focus for the teams. Consistent use of the tool seemed to increase the effectiveness of the intervention. Using a structured approach to stewardship may therefore be as important as using an optimal measurement and feedback method.

LOS was preferred as a primary outcome in this study as previous studies had found a strong association between appropriate antibiotic use (process measure) and LOS (outcome measure).,, The use of LOS has several advantages: it is easy to measure, applies to all included patients, reflects recovery time of hospitalized patients and drives hospital costs. As the parameter can be influenced by several factors and secular trends, we statistically adjusted for confounders (e.g. age, comorbidity, type of diagnosis) that have shown to potentially affect LOS. Also, we corrected for the secular trend of LOS over time using national reference data. In addition to the primary outcome LOS, we applied several secondary outcomes including process (DOT) as well as outcome measures (mortality, ICU admission and ICU duration). In our opinion, process measures can complement outcome measures as they are able to identify specific targets for quality improvement.

Our study has several strengths. To our knowledge, this is the first cluster-randomized multicentre study comparing the effectiveness of three recommended methods to measure and feed-back information on hospital antibiotic use, and assessing the effectiveness of a structured approach to stewardship. This study design is highly recommended.,, Also, we showed that even in Dutch hospitals, where antibiotic use is already low compared with other countries, antibiotic use can be decreased. The impact of our structured approach to stewardship could therefore potentially be larger in countries with higher antibiotic consumption. Finally, we used an implementation tool that has been rigorously developed. The tool guides stewardship teams to systematically develop and perform setting-specific improvement strategies. This tool can be used in other hospitals.

Our study has some limitations. First, the design based on two clusters within each hospital was prone to contamination: stewardship teams operated hospital-wide, received feedback reports on two clusters, and consequently could use information from an allocated cluster report to initiate improvement strategies in a non-allocated cluster. However, in line with the implementation tool, improvement strategies were to be adapted to the needs and context of each cluster by a systematic analysis of local barriers in the clusters. Second, the initial study design did not include a control group, as we were merely interested in comparing the three methods. In that respect we followed the conclusion of Ivers et al. in their Cochrane review that ‘Future studies of audit and feedback should directly compare different ways of providing feedback.’ However, there was a significant number of clusters for which no feedback report was used (AT approach), which can be regarded as a control group, showing no significant decrease in LOS. Also, we did correct for the secular trend of LOS over time using national reference data. Third, hospitals with a higher baseline LOS seemed to have more potential to reduce LOS (floor effect). Even though we aimed to correct for this effect in our model, it suggests that in hospitals with relatively low baseline LOS, further stewardship strategies might have no measurable return on investment: this remains to be established.

In conclusion, this multicentre cluster-randomized study suggests that using data on the quality of antibiotic use is more valuable than using data on the quantity of use to develop setting-specific ASP improvement interventions. Moreover, we strongly encourage stewardship teams to use a structured approach, as for example supported by the implementation tool, to develop their stewardship improvement strategies. Future studies should consider focusing on obtaining appropriate data from existing electronic health record systems, in order to obtain data for stewardship purposes in a timely and efficient fashion.

Click here for additional data file.