Effectiveness of interventions targeting antibiotic use in long-term aged care facilities: a systematic review and meta-analysis.

OBJECTIVES: There are high levels of inappropriate antibiotic use in long-term care facilities (LTCFs). Our objective was to examine evidence of the effectiveness of interventions designed to reduce antibiotic use and/or inappropriate use in LTCFs.
DESIGN: Systematic review and meta-analysis. DATA SOURCES: MEDLINE, Embase and CINAHL from 1997 until November 2018. ELIGIBILITY CRITERIA: Controlled and uncontrolled studies in LTCFs measuring intervention effects on rates of overall antibiotic use and/or appropriateness of use were included. Secondary outcomes were intervention implementation barriers from process evaluations. DATA EXTRACTION AND SYNTHESIS: Two reviewers independently applied the Cochrane Effective Practice and Organisation of Care group's resources to classify interventions and assess risk of bias. Meta-analyses used random effects models to pool results.
RESULTS: Of include studies (n=19), 10 had a control group and 17 had a high risk of bias. All interventions had multiple components. Eight studies (with high risk of bias) showed positive impacts on outcomes and included one of the following interventions: audit and feedback, introduction of care pathways or an infectious disease team. Meta-analyses on change in the percentage of residents on antibiotics (pooled relative risk (RR) (three studies, 6862 residents): 0.85, 95% CI: 0.61 to 1.18), appropriateness of decision to treat with antibiotics (pooled RR (three studies, 993 antibiotic orders): 1.10, 95% CI: 0.64 to 1.91) and appropriateness of antibiotic selection for respiratory tract infections (pooled RR (three studies, 292 orders): 1.15, 95% CI: 0.95 to 1.40), showed no significant intervention effects. However, meta-analyses only included results from intervention groups since most studies lacked a control group. Insufficient data prevented meta-analysis on other outcomes. Process evaluations (n=7) noted poor intervention adoption, low physician engagement and high staff turnover as barriers.
CONCLUSIONS: There is insufficient evidence that interventions employed to date are effective at improving antibiotic use in LTCFs. Future studies should use rigorous study designs and tailor intervention implementation to the setting. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

We used a broad search strategy to identify studies evaluating the impact of interventions aiming to improve antibiotic use in long-term care facilities and considered a range of detailed indicators of antibiotic use and appropriateness.

To support comparison with future reviews, intervention components were classified using the Cochrane Effective Practice and Organisation of Care group’s Taxonomy of Health System Interventions.

We report the results of process evaluations to identify reasons for intervention effects, in order to inform future intervention design and implementation.

A limitation of the meta-analyses is that these were conducted using pre-intervention and post-intervention measurements from the intervention groups of studies, as there were insufficient studies with control groups to include in the analyses.

The majority of included studies were assessed as having a high risk of bias.

Introduction

The widespread inappropriate use of antibiotics is a major contributor to antibiotic resistance. Antibiotic resistance is a growing problem globally, leading to poor patient outcomes, pressure on health system resources and significant economic impacts.1–6 Antimicrobial resistant infections are estimated to cause over 700 000 deaths annually worldwide—a rate which is projected to increase to 10 million by 2050.1

In long-term care facilities (LTCFs), antibiotic use rates are high compared with the general population, and LTCFs residents are on antibiotics for longer.7 Studies have estimated that between 47% and 70% of residents receive a course of antibiotics annually, and that between 77% to 88% of infections are treated with an antibiotic.7–9 Approximately 50% of these prescriptions are judged to be inappropriate,9 with inappropriate treatment of urinary tract infections and respiratory tract infections common in LTCF residents.7–12 Thus, there is considerable scope for improving antibiotic use in LTCFs. This goal is especially important given that residents of LTCFs are at increased risk of colonisation with resistant organisms and facilitate the spread of these organisms due to their close proximity to other residents and regular contact with hospitals, LTCF workers and the community.13–18 The widespread use of antibiotics by LTCF residents also puts them at risk of related infections, such as Clostridioides difficile, which are subsequently easily spread to contacts.19

Interventions designed to improve antibiotic prescribing in the hospital setting have been shown to confer benefits in terms of appropriate antibiotic use, such as adherence to antibiotic use policy and shorter antibiotic course length.20 Similarly, interventions targeting physicians in primary care have been shown to reduce rates of antibiotic prescribing and improve compliance with guidelines.21 22 Evidence of effective interventions in LTCFs is more limited, and the LTCF setting is complex with many factors that can influence the success of interventions. Two previous systematic reviews of interventions to improve antibiotic use in LTCFs both concluded that while there was some evidence to suggest these interventions had positive impacts, the evidence base needed strengthening.23 24 However, there is an opportunity to build on this work. One of these reviews, published in 2013, included only randomised trials of which there were only four.24 The second review’s primary aim was to assess whether antimicrobial stewardship interventions in LTCFs reduced mortality, the incidence of Clostridioides difficile infections, emergency department visits and hospitalisations.23 The impact of interventions on antibiotic use was a secondary objective. Neither review included a meta-analysis of the effectiveness of interventions to reduce and improve antibiotic use, nor examined factors influencing intervention effectiveness.

Our objective was to conduct a systematic review and meta-analysis of the effectiveness of interventions to reduce the use or improve the appropriateness of antibiotics in LTCFs. The primary outcomes of interest were changes in rates of overall antibiotic use and rates of appropriate antibiotic use. In order to gain insights into how intervention impacts were achieved, the secondary aim was to provide a qualitative review of the results from process evaluations that were conducted as part of the intervention studies.

Methods

This systematic review and meta-analysis are reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses25 (online supplementary file 1).

Information sources and search strategy

The databases MEDLINE via Ovid (1946 to February 2018) and via PubMed (1966 to February 2018), Embase (1974 to February 2018) and CINAHL (1966 to February 2018) were searched in November 2018 for original research articles reporting on the impact of interventions to improve antibiotic use in LTCFs, published in English, in the last 20 years (since 1997). The search used keywords and subject heading terms related to two main groups of concepts which were combined using the Boolean operator ‘AND’: (i) long-term care facilities and (ii) antibiotics, common infections affecting long-term care residents and antibiotic resistant organisms. The full search strategy is presented in online supplementary file 2. The reference lists of included articles and identified review articles were hand searched for further relevant citations. Additionally, the PROSPERO: International Prospective Register of Systematic Reviews26 and the Cochrane Library27 were searched.

Eligibility criteria

Original research articles reporting the effects of interventions to reduce the use of antibiotics or improve the appropriateness of antibiotic use in residents of LTCFs were eligible for inclusion. Studies in residential aged care facilities, nursing homes, skilled nursing facilities and assisted living facilities for older people were included; while those in community-dwelling populations, and long-term care hospital wards were excluded. Long-term care hospital wards have a different model of care and populations compared with stand-alone LTCFs. Studies were included if they reported intervention outcomes in terms of the rate of overall antibiotic use or rate of appropriate antibiotic use (eg, per resident, per 1000 bed days, per antibiotic prescription). Studies reporting only rates of infection, resistant organism colonisation, hospitalisation or mortality were excluded, as were studies where the intervention did not target antibiotic use for example, interventions consisting only of infection control strategies. Due to the limited literature on this topic, the study designs were not restricted to randomised trials. The study types eligible for inclusion were: randomised trials, controlled before and after studies, interrupted time series and before and after studies. Cross-sectional studies were not eligible for inclusion.

Study selection

Two authors (MR, CG) independently reviewed the title and abstracts of citations returned from the search strategy, after removal of duplicates, to identify potentially relevant citations for full-text review. The full-text of all relevant citations were then reviewed independently by two authors (MR, CG) using Covidence software to aid the process.28 Discrepancies between the two authors in the assessment of full-text articles were resolved through discussion.

Data collection and data items

Data were extracted initially by one author (CG), and then this was reviewed and refined by a second author (MR). The data extracted included study characteristics (country, number of LTCFs included, study design), outcomes of interest measured and results related to these and intervention details. For overall antibiotic use outcomes, the indicators extracted were the standard indicators used to report antibiotic use:29–31 defined daily doses per 1000 resident days (DDD/1000 days), days of antibiotic therapy per 1000 resident days (DOT/1000 days), prescriptions (equivalent to antibiotic initiations) per 1000 resident days, and percentage of residents on an antibiotic. Indicators of the appropriateness of antibiotic use extracted were: the percentage of antibiotic prescriptions that were appropriate decisions to treat an infection with an antibiotic (appropriateness of decisions to treat), and the percentage of antibiotic prescriptions where the appropriate antibiotic was selected (appropriateness of antibiotic selection). The methods for assessment of antibiotic appropriateness were recorded, and the appropriateness outcomes were stratified by infection type (all infections, urinary tract infections (UTIs), respiratory tract infections (RTIs)). The most common infection types in LTCF are UTIs, RTIs and skin and soft tissue infections, accounting for over 90% of all infections.7 32 33 If interventions targeted all three infection types, they were classified as targeting ‘all infections’. The timing of outcome measurement in relation to the beginning of intervention implementation (excluding any piloting of the intervention) was also recorded. When studies included process evaluations, the results of the process evaluation were extracted.

Intervention details extracted included the infection types and personnel who were targeted by the intervention (eg, nurses, physicians, family/residents). The Cochrane Effective Practice and Organisation of Care (EPOC) group’s Taxonomy of Health System Interventions was used as a guide to classify intervention components.34 Two authors (CG and MR) independently coded the study interventions and resolved discrepancies in coding by discussion. Online supplementary file 3 shows the intervention taxonomy sub-categories applied, along with their definitions.

Assessment of risk of bias in included studies

The risk of bias was assessed using the Cochrane EPOC group’s criteria.35 An assessment of the risk of bias of studies provides an indication of the likely issues that may bias study results. The Cochrane EPOC group’s risk of bias tool is an extension of the Cochrane Collaboration’s tool and includes items specific not only to randomised trials, but non-randomised trials and controlled before-and-after studies. The included studies were rated against each criterion as having either a low risk, high risk or unclear risk of bias.

The criterion assessing whether outcome assessment was blinded was applied to each of the two categories of outcomes of interest (overall antibiotic use and appropriate antibiotic use). Within this criterion we also considered whether the study assessed the reliability of outcome measurement. This is especially important when considering the appropriateness of antibiotic use as it requires clinical judgement, and previous studies have shown that this can vary between assessors.36–38 Studies without controls (interrupted time series or before-after designs) were rated against three additional criteria.

We considered whether the characteristics of the facility residents were similar in the before and after periods. Differences in the characteristics of the residents in the two periods could affect the outcomes of interest. We also added two additional criteria related to long follow-up periods (where other systemic changes may influence outcomes) and retrospective study designs, which increase the risk of bias of before-after study designs.39 Criteria requiring assessment of the comparability of the control and intervention groups were not considered as they are not applicable in these study designs.

The risk of bias for all studies was rated independently by two authors (MR and MB). Discrepancies in assessment were resolved through discussion to arrive at a final rating for each criterion. The risk of bias assessment tools used are presented in full in online supplementary file 4. A summary of risk of bias assessment score was assigned to each study as follows: ‘low risk of bias’ when all criteria were scored as ‘low’, ‘medium risk of bias’ if one to two criteria were scored as ‘unclear’ or ‘high’, and ‘high risk of bias’ if more than two criteria were scored as ‘unclear’ or ‘high’. RevMan40 software was used to collate and present the risk of bias results.

Meta-analysis

Authors of papers containing insufficient information to be included in the meta-analyses were sent emails requesting additional data to be provided. Where sufficient data were available, we conducted meta-analyses for the outcomes of interest using random effects models to pool the results. Since less than three studies for any given outcome included control groups, only the results from the intervention groups were included in the meta-analyses. Relative risk ratios (RR) and their 95% CIs were calculated. The conservative Knapp-Hartung approach41 was applied to account for heterogeneity between studies. The meta-analyses results are presented using forest plots. Between-study heterogeneity was evaluated using I2 statistics.42 The potential for publication bias for each meta-analysis was assessed by inspection of funnel plots and statistical tests based on weighted linear regression of the intervention effect on its SE.43 Sensitivity analyses were conducted using ORs. In addition, analyses were also conducted for studies reporting results from different post-intervention periods44 45 and different levels of appropriateness of antibiotic selection affected results.46 All statistical tests were two-sided and were evaluated at a significance level of 0.05. Analyses were carried out using R V.3.5.0.47

Patient and public involvement

There was no patient or public involvement in the design or conduct of this study.

Results

There were 19 studies (21 papers) assessed as eligible for inclusion in this review (figure 1).44–46 48–63 Table 1 presents the study characteristics. The majority of studies were undertaken in North America (n=14), four in Europe and one in Australia. Five were cluster randomised trials,46 54 57–59 61 five were controlled before-after studies,50 53 56 62 64 two were interrupted time series studies49 51 and there were seven before-after studies.46 48 52 60 61 63 65 Thirteen studies measured overall antibiotic use and 10 the appropriateness of antibiotic use. Data were received in 3 out of 10 cases. Information reported in included studies or reported by authors allowed us to perform meta-analyses for three outcome measures: percentage of residents on antibiotics, appropriateness of decision to treat (any infection) and appropriateness of antibiotic selection (for RTIs).

Figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart of search strategy and screening results. *There were two instances where two full-text articles reported results from one study.

Table 1

Characteristics of included studies

Author, year	Study design	Country	Number of LTCFs*	Infection types targeted by intervention	Target groups for intervention	Post-intervention outcome measurement time frame	Outcomes of interest measured	Overall risk of bias
Colon-Emeric, et al 200948	Before-after	USA	2	UTI, RTI	Physicians, nurses	1–6 months post	Appropriateness(antibiotic selection)	High
Doernberg, et al 201549	Interrupted time series	USA	3	UTI	Physicians	1–7 months post	No. of prescriptions/1000 resident days	High
Fleet, et al 201446	Cluster randomised trial	UK	30	All infections	Nurses	8–12 months post	DDD/1000 resident days; % of residents; appropriateness (decision to treat)	High
Hutt, et al 200650	Controlled before-after	USA	2	RTI	Physicians, nurses	4–6 months post	Appropriateness(antibiotic selection)	High
Jump, et al 201251	Interrupted time series	USA	1	All infections	Physicians	1–18 months post	Days of antibiotic therapy/1000 resident days	High
Kassett, et al 201652	Before-after	Canada	1	UTI	Physicians, nurses, pharmacists, patients and their families	1–12 months post	Days of therapy/1000 resident days (adjusted for infection rate)	High
Linnebur, et al 201153	Controlled before-after	USA	16	RTI	Physicians, pharmacists, nurses	1–7 months	Appropriateness(antibiotic selection)	High
Loeb, et al 2005 and Lohfeld, et al 200754 55	Cluster randomised trial†	Canada and USA	22	UTI	Physicians, nurses	1–12 months post	No. of prescriptions/1000 resident days	Low
McMaughan, e t al 201656	Controlled before-after	USA	12	UTI	Physicians	1–6 months post	Appropriateness(decision to treat)	High
Monette, et al 200757	Cluster randomised trial	Canada	8	All infections	Physicians	1–3 months post-intervention one, 1–3 months post-intervention two, 4–6 months post- intervention two	Appropriateness(antibiotic selection)	Medium
Naughton, et al 200158	Cluster randomised trial	USA	10	RTI	Physicians, nurses	Unclear, not stated	Appropriateness‡	High
Pettersson, et al 201159	Cluster randomised trial	Sweden	46	All infections	Physicians, nurses	5–8 months post	% of residents	High
Rahme, et al 201660	Before-after	USA	1	UTI, RTI (NHs chose which intervention to use)	Physicians, family members	1–12 months post	DDD/1000 resident days	High
Rummukainen, et al 2009 and 201244 45	Before-after	Finland	39	All infections	Physicians, nurses	12, 24 and 36 months post	% of residents	High
Sloane, et al 201461	Before-after	USA	4	All infections	Physicians, nurses, family members	1–12 intervention post	No. of prescriptions/1000 resident days; appropriateness (decision to treat; antibiotic selection)	High
Stuart, et al 201565	Before-after	Australia	2	All infections	Physicians	6–8 months post	Days of therapy/1000 resident days	High
van Buul, et al 201562	Controlled before-after	Netherlands	10	UTI, RTI	Physicians, pharmacists, nurses	4–12 months post	DDD/1000 resident days; no. of prescriptions/1000 resident days; appropriateness (decision to treat; antibiotic selection)	High
Zabarsky, et al 200863	Before-after	USA	1	UTI	Physicians, nurses	1–6 months post, 7–30 months post	Days of therapy/1000 resident days; appropriateness (decision to treat)	High
Zimmerman, et al 201464	Controlled before-after	USA	12	All infections	Physicians, nurses and residents and their families	1–6 months post	No. of prescriptions/1000 resident days	High

*LTCFs include nursing homes, residential aged care facilities, skilled nursing facilities, Veteran’s homes.

†No before measurements.

‡The appropriateness of treatment of nursing home-acquired pneumonia with parenteral antibiotics was the appropriateness outcome.

DDD, defined daily doses; LTCFs, long-term care facilities; NHs, nursing homes; RTI, respiratory tract infection; UTI, urinary tract infection.

Description of interventions

Interventions were aimed at improving antibiotic use for all infection types in eight studies,45 46 51 57 59 61 64 65 UTIs in five studies,49 52 54 56 63 RTIs in three studies50 53 58 and both UTIs and RTIs in three studies48 60 62 (table 1). The health professionals targeted by interventions were most commonly both physicians and nurses (n=14). Four studies52 60 61 64 also targeted facility residents and/or their families (table 1). All studies used interventions involving multiple components (table 2). Education strategies were an intervention component in the majority of studies (n=17),46 48 50–54 56–64 followed by the promotion of clinical practice guidelines (n=15),45 46 48–50 52–54 56–59 62 64 which in a further seven studies were adapted to the local context.45 48 53 56–58 60 Audit and feedback interventions used in five studies involved reporting on antibiotic prescribing to the facility or to physicians.52 57 63–65 Four studies assessed an intervention which included an infectious disease team who provided advice to physicians on appropriate antibiotic use for individual cases of infection.49 51 60 65 The least common intervention component was information and communication technology (ICT; n=2),48 52 which was used in one study to provide physicians with prescribing guidelines,48 and in the other study to change default stop dates for some antibiotics, as well as improved ease of access to guidelines.52

Table 2

Intervention components classified according to the Cochrane Effective Practice and Organisation of Care (EPOC) group’s Taxonomy of Health System Interventions

Author, year	Interventions targeted at healthcare workers							Coordination of care and management of care processes		ICT
	Education strategies	Clinical practice guidelines	Local consensus processes	Audit and feedback	Patient-mediated interventions	Tailored interventions	Continuous quality improvement	Care pathways	Use of infectious disease team	Use of ICT
Colon-Emeric, et al 2009†48	X	X	X							X
Doernberg, et al 2015*†49		X							X
Fleet, et al 2014*46	X	X						X
Hutt, et al 2006*50	X	X				X		X
Jump, et al 2012*†51	X								X
Kassett, et al 2016†52	X	X	X	X	X					X
Linnebur, et al 201153	X	X						X
Loeb, et al 2005 and Lohfeld, et al 200754 55	X	X
McMaughan, et al 201656	X	X	X
Monette, et al 200757	X	X	X	X
Naughton, et al 200158	X	X	X
Pettersson, et al 201159	X	X				X
Rahme, et al 2016†60	X	X	X		X				X
Rummukainen, et al 2009 and 2012*†44 45		X	X
Sloane, et al 2014†61	X				X	X	X	X
Stuart, et al 2015*†65	X			X					X
van Buul, et al 201562	X	X				X
Zabarsky, et al 2008*†63	X			X			X
Zimmerman, et al 2014*64	X	X		X	X	X	X	X
Total for all studies	17	15	7	5	4	5	3	5	4	2

ICT is information and communication technology; ‘X’ indicates study intervention included the component type.

*Studies with statistically significant positive outcomes on indicators of overall antibiotic use or appropriateness of antibiotic use.

Risk of bias in included studies

All but two studies54 57 were rated as having an overall high risk of bias (table 1). Figure 2 shows the risk of bias assessments against each of the criteria for each study. In studies with controls, allocation concealment and random sequence generation were identified to be at a high risk of bias in eight and five studies, respectively. This is largely due to the high proportion of controlled before-after studies which did not undertake a randomisation process.35 The blinding and reliability of measurement of antibiotic appropriateness outcomes were judged to be at a high risk of bias in all studies with controls (n=7).

Figure 2

Risk of bias assessment results for each study. Blank sections in this graph are due to criteria not being applicable to all studies. RT, CBA – criterion applicable to randomised trials and controlled before-and-after studies. ITS, BA – criterion applicable to before-after and interrupted time series studies.

Eight of nine studies without controls did not consider the potential bias of changes in resident characteristics in the before-and-after intervention periods, and thus had a high risk of bias for this criterion (figure 2). Similarly, to controlled studies, the two studies without controls reporting appropriateness of antibiotic use outcomes had a high risk of bias for the blinding and reliability criterion.

Online supplementary file 5 presents the summary of the risk of bias assessment.

Overview of intervention effectiveness

Of the included studies, only eight showed a statistically significant improvement in the outcomes of interest (table 2).45 46 49–51 63–65 These studies used three intervention components more frequently compared with the studies without significant effects on outcomes. These components included: audit and feedback (three63–65 of five52 57 63–65 studies had significant effects), introduction of care pathways (three46 50 64 of five46 50 53 61 64 studies had significant effects) and use of an infectious disease team (three49 51 65 of four49 51 57 65 studies had significant effects). However, five45 49 51 63 65 of the eight studies that showed an improvement in outcomes used study designs without controls and all eight studies were rated as having a high risk of bias (table 2).

Effectiveness of interventions to reduce the overall antibiotic use among LTCF residents

Twelve studies45 49 51 54 59–65 reported the impact of interventions on overall antibiotic use. Three45 46 59 of these reported changes in the percentage of residents on an antibiotic, three46 60 62 changes in the DDD per 1000 resident days, three51 63 65 changes in DOT per 1000 resident days and five49 54 61 62 64 changes in the number of antibiotic prescriptions per 1000 resident days.

Only the three studies45 46 59 reporting changes in the percentage of residents on antibiotics after the intervention contained sufficient information to be included in a meta-analysis. All three studies were rated as having a high risk of bias. One59 of these studies showed a small reduction (RR: 0.74; 95% CI: 0.61 to 0.90) in residents’ use of antibiotics (figure 3). The three studies were found to be homogeneous (heterogeneity between studies: I2=34.2%, p=0.2). Overall, the interventions did not have a significant effect on residents’ antibiotic use (pooled RR: 0.85, 95% CI: 0.61 to 1.18, p=0.2) (figure 3). There was no evidence of publication bias (p=0.9). Sensitivity analyses were conducted using different post-intervention periods from Rummukainen et al (2012).45 There was little difference in the results (post 24 months: pooled RR: 0.85, 95% CI: 0.60 to 1.19, p=0.2; post 36 months: pooled RR: 0.83, 95% CI: 0.52 to 1.33, p=0.2).

Figure 3

Forest plot of studies reporting intervention effect on the percentage of residents on antibiotics. Events are number of residents on antibiotics and total was number of residents. RE is random effects.

One of the three studies measuring intervention effects on DDD per 1000 resident days reported a statistically significant 5% reduction in total antibiotic use post-intervention in comparison to the control group.46 The other two studies did not report statistically significant changes in DDD per 1000 resident days.60 62

The three uncontrolled studies reporting the DOT per 1000 resident days51 63 65 all reported statistically significant reductions in antibiotic use ranging from 20% to 35%. Two studies reporting intervention effects on the number of prescriptions per 1000 resident days reported statistically significant decreases in the order of 5%49 and 28%,64 but three further studies showed no change.54 61 62

Detailed results are shown in online supplementary file 6.

Effect of interventions on appropriateness of the decision to treat with an antibiotic

Five studies46 56 61–63 reported on the impact of interventions on the appropriateness of decisions to treat infections with an antibiotic. Three of these were on decisions to treat any infection,46 61 62 three for UTIs56 62 63 and one for RTI.62 To assess whether antibiotic use was appropriate, four studies46 56 62 63 looked at adherence to various guidelines, and one61 used a panel of experts to assess each antibiotic orders.

The three studies46 61 62 reporting the appropriateness of the decision to treat any infection with antibiotics contained sufficient information to be included in a meta-analysis; none of these studies reported a significant effect of the intervention on the appropriateness of the decision to treat. All three studies were rated as having a high risk of bias. Overall, the intervention did not have a significant effect on the appropriateness of decisions to treat (pooled RR: 1.10, 95% CI: 0.64 to 1.91, p=0.5) (figure 4). The studies were not found to have significant heterogeneity (p=0.07), but had a relatively high value of I2 (63.7%). There was evidence of publication bias (p=0.01). The sensitivity analysis showed little difference in the results when using the partial appropriateness assessment results by Fleet et al (2014)46 (pooled RR: 0.98, 95% CI: 0.78 to 1.24, p=0.8).

Figure 4

Forest plots of studies reporting intervention effect on the appropriateness of antibiotic use. Panel A: appropriateness of decision to treat any infection with antibiotics (episodes are number of episodes with appropriate treatment decision and total was number of orders). Panel B: appropriateness of antibiotic selection for respiratory tract infections (episodes are number of episodes on appropriate antibiotics and total was number of orders). RE is random effects.

Three studies examined the impact of interventions on the decision to treat UTIs or asymptomatic bacteriuria.56 62 63 Only one before-after study showed a statistically significant reduction in inappropriate decisions to treat asymptomatic bacteriuria with antibiotics (from 67.6% to 44.0%, p=0.022), while the two other studies showed no statistically significant improvement in the appropriateness of decisions to treat UTIs or asymptomatic bacteriuria with antibiotics.56 62

One study examined the proportion of RTIs appropriately treated with an antibiotic and reported no changes after the intervention compared with the control facilities (odds ratio: 0.97; 95% CI: 0.42 to 2.27).62 However, 89% of treatment decisions were deemed appropriate at baseline, leaving little room for improvement.

Detailed results are shown in online supplementary file 7.

Effect of interventions on appropriateness of the antibiotic selected

Six studies48 50 53 57 61 62 reported on the impact of interventions on the appropriateness of antibiotic selection. Two57 61 of these reported on antibiotic selection for any infection, two48 62 for UTIs and four48 50 53 62 for RTIs. To assess whether antibiotic selection was appropriate, five studies48 50 53 57 62 used various guidelines, and one61 an expert panel to assess antibiotic orders.

Only three of four studies48 50 52 reporting the effects of interventions on the appropriateness of antibiotic selection for the treatment of RTI contained sufficient information to be included in this meta-analysis; none of these studies reported a significant effect on appropriateness of antibiotic selection (RTI). All three studies were rated as having a high risk of bias. The studies were found to be homogeneous (heterogeneity between studies: I2=0%, p=0.8). Overall, the intervention did not have a significant effect on the appropriateness of antibiotic selection (pooled RR: 1.15, 95% CI: 0.95 to 1.40, p=0.09) (figure 4). There was no evidence of publication bias (p=0.9).

The one study reporting the effect of the intervention on the appropriateness of antibiotic selection for RTI that was not included in the meta-analysis, also did not show a statistically significant improvement in this outcome.53

The two studies reporting intervention effects on the appropriateness of antibiotic selection for any indication showed no improvements post-intervention.57 61 However, they both reported high baseline levels of appropriate antibiotic selection at 84%57 and 90%61 of antibiotic orders. The two studies reporting intervention effects on the appropriateness of antibiotic selection to treat UTIs also did not demonstrate a significant intervention effect.48 62

Detailed results are presented in online supplementary file 7.

Staff perceptions of interventions and levels of intervention uptake as reported in process evaluations

Seven studies conducted process evaluations of their intervention.48 50 56 61 62 One study used a qualitative interview approach to ascertain the perspectives of LTCF staff on the lack of effect of the intervention on the appropriateness of antibiotic prescribing.62 The key issues identified included the high baseline level of appropriate prescribing which was difficult to further improve, a lack of motivation on the part of the physicians to improve prescribing and high physician turnover in the facilities. Another study reported that physicians had positive opinions of the implemented computer algorithms, however there was infrequent use of the algorithms to support decision-making.48 A third study54 55 identified a lack of buy-in from nurses, difficulty in getting physicians to change practices and family expectations of antibiotics as barriers to compliance with the implemented clinical guideline. Continued use of the guidelines after the discontinuation of the study was hampered by difficulties in changing long-standing practices and ensuring adequate training of new staff.54 55

Four other studies assessed the uptake of the interventions either by facilities or individuals targeted by the interventions.46 50 56 61 One of these studies implemented two forms to be completed by nursing staff for all residents newly commenced on an antibiotic.46 They found that on average the two forms were 100% complete in only 31% and 46% cases. Another study also examined the use of implemented forms by staff to communicate acute medical issues including infections.61 They found that uptake was low immediately following implementation of the intervention (less than half of all infections had a completed form), but increased over the 15 month study period to double the initial levels. Another study examined intervention component uptake by facilities and found that not all intervention components were implemented in the intervention facilities and that some components were implemented in the control group (eg, immunisation policy, emergency medication kit containing correct antibiotics).50 Finally, McMaughan et al assessed the use of a decision-making aid for suspected UTIs implemented as part of their intervention.56 They found large differences between facilities in terms of the rates of use, with some facilities rarely using the aid (ie, in less than 25% of UTI cases). Their post-hoc analysis stratified facilities into high and low intervention adherence groups and showed a significant reduction in inappropriate antibiotic prescriptions among the high intervention adherence facilities, which was in contrast to their intention-to-treat analysis which showed no intervention effect.

Discussion

Main findings

This systematic review and meta-analysis showed there is insufficient evidence supporting the effectiveness of interventions to reduce overall antibiotic use and increase the appropriateness of antibiotic use in LTCFs. Meta-analysis showed no statistically significant reductions in overall antibiotic use as measured by the percentage of residents on antibiotics. Most studies showed no reductions in other measures of overall antibiotic use, including DDD per 1000 resident days and the number of prescriptions per 1000 resident days. The two studies46 64 reporting significant changes on these outcomes showed very marginal improvements. Three before-after studies with a high risk of bias showed significant decreases in antibiotic DOT per 1000 resident days.51 63 65 Interventions had little effect on the appropriateness of decisions to treat or on antibiotic selection, with only one study with a high risk of bias showing statistically significant improvements in the decision to treat with antibiotics.63

Implications for practice and policy

LTCFs are a challenging setting for antibiotic interventions due to factors such as doctors being off-site, older populations being more susceptible to infection and having less specific symptoms of infection and other stakeholders’ (eg, family, staff) preferences for treatment with antibiotics.66–68 These factors may explain some of the findings from the process evaluations which reported that adoption of interventions by facilities and staff within the facilities was low, and varied between facilities. Furthermore, contributing factors including high staff turnover and lack of physician motivation to change prescribing behaviours can be difficult to overcome and pose sustainability challenges for interventions. Interventions such as education and implementation of clinical guidelines are unlikely to overcome these challenges, particularly in the relatively short period of time it takes to run a trial. Thus, knowledge of the local culture and context are important factors to consider when selecting and planning interventions. Furthermore, an understanding of the specific antibiotic prescribing patterns and issues in LTCFs will also assist in guiding where intervention efforts should be directed. For example, some studies had higher rates of appropriateness of decisions to treat infections with antibiotics compared with appropriate antibiotic selection,58 62 and others reported high baseline rates of appropriate antibiotic prescribing, leaving little room for improvement. The variability in antibiotic prescribing among LTCF physicians has also been highlighted in the literature.69 It would be useful for LTCFs to understand antibiotic use patterns prior to implementing interventions so that problem areas are targeted.

The interventions implemented in the included studies had varying components, targeted different stakeholders in LTCF resident’s care and different infection types. This makes it difficult to ascertain which intervention component or combination of components are most likely to be effective. However, the studies which showed significant improvements in antibiotic prescribing had some common intervention components. These included audit and feedback, introduction of care pathways and an infectious diseases team being available to advise on prescribing. Despite this, it is not possible to draw any concrete conclusions about these intervention components since the studies in question were mostly uncontrolled studies which limits the ability to make causality links between the intervention and effects. Future studies should explore these components further to ascertain their effectiveness.

Since physicians involved in prescribing antibiotics in LTCFs are based in primary care practices, we can look toward some of the recent innovative approaches to improving antibiotic use in primary care. Recent studies have shown that periodic prescriber feedback with peer comparison is more effective in changing prescribing practices than other interventions, including education strategies.70–72 Peer comparison, which was not used in any of the studies included in this review, is a behavioural economics theory driven intervention that could be explored further in the LTCF setting. These interventions involve providing physicians who are high prescribers of antibiotics, feedback on their antibiotic prescribing levels relative to their peers. Such interventions require timely information on prescribing levels. ICT provides a good opportunity to obtain timely data, as well as a platform for providing feedback on antibiotic use and prescribing levels; however it is currently underexploited for these purposes in LTCFs.

Strengths and limitations

This review has several strengths. We conducted meta-analyses on the effects of antibiotic use interventions in LTCFs on measures of overall and appropriateness of antibiotic use. Furthermore, our results are grouped into key indicators which allows a clear summary of intervention outcomes. Our inclusion of the results of the process evaluations is unique, shedding some light on the challenges faced by LTCFs implementing antibiotic interventions, and aids in explaining some of the results. Another strength was our use of a broad search strategy to maximise capturing relevant studies.

The main limitation of this systematic review and meta-analysis is the small number of included studies. This is further amplified by the lack of consistency in the outcomes and descriptive statistics reported. Additionally, because of the lack of controlled studies, our meta-analyses included only results from the intervention groups of studies. The inclusion of studies other than randomised trials or controlled studies, while allowing us to present more results, also meant that the meta-analyses need to be interpreted keeping the risk of bias in mind. Furthermore, the methods for assessing the appropriateness of antibiotics varied between studies limiting the comparability and generalisability of these results.

Lastly, since we ran our searches in November 2018, we have identified three more articles describing before-after studies of interventions to improve antibiotic prescribing in LTCFs that meet our inclusion criteria. One study evaluated the impact of an educational intervention to improve antibiotic use for UTIs in 10 Canadian LTCFs and reported a 26% decrease (adjusted incidence rate ratio=0.74, 95% CI: 0.47 to 0.74) in the overall number of antibiotic prescriptions per 1000 resident days.10 The second study, in seven Canadian LTCFs, reported a decrease in the use of antibiotics for asymptomatic bacteriuria from 90% pre-intervention to 62.9% post-intervention (p=0.003), following an educational intervention.73 The third study reported that the implementation of new guidelines and physician-pharmacy-nurse quality circles resulted in a 22% reduction in overall antibiotic use over a 6-year period.74 However, it is not possible to attribute the reductions in antibiotic use to the interventions alone due to the absence of a control group, and the inclusion of these studies in our review would not change our conclusions.

Future research

High quality studies are required to test the effectiveness of interventions to improve antibiotic use in LTCFs. Study designs should include controls as otherwise it is very difficult to account for external factors influencing outcomes in health services research. This fact is driven home by two of the included studies which showed improvements in outcomes in the control LTCFs, as well as the intervention LTCFs.59 62 Without comparison to the control groups, drawing conclusions about the effects of the interventions would be different. Planning of outcome measures also requires careful consideration. For example, indicators of overall antibiotic use measure different aspects of use29 75 76 and should be carefully selected based on the intervention planned. An intervention may not reduce the prevalence of antibiotic use, but may reduce the days of treatment, also an important outcome. Lastly, planned interventions should pay close attention that the implementation processes are relevant to the local context to ensure adequate intervention adoption in order to effect change.77 Carefully planned process evaluations can also assist with illuminating where and under what conditions the interventions are successful or otherwise.78 Realistic evaluation is one such approach which considers intervention context, mechanisms and outcome in tandem.79

Conclusions

In this systematic review and meta-analysis, we reported that interventions were no more effective than usual care in improving antibiotic use in LTCFs. In the context of the global challenge of antibiotic resistance and ageing populations, the lack of evidence on interventions to improve antibiotic use in LTCFs is a major gap. The paucity of high quality studies and the heterogeneity in reported measures of antimicrobial use calls for more rigorous study designs, utilising a control group and standardised measures of reporting antibiotic consumption. Process evaluations are an important component of future studies in order to inform further intervention development and implementation in these complex settings.