Provision of social norm feedback to high prescribers of antibiotics in general practice: a pragmatic national randomised controlled trial.

BACKGROUND: Unnecessary antibiotic prescribing contributes to antimicrobial resistance. In this trial, we aimed to reduce unnecessary prescriptions of antibiotics by general practitioners (GPs) in England.
METHODS: In this randomised, 2 × 2 factorial trial, publicly available databases were used to identify GP practices whose prescribing rate for antibiotics was in the top 20% for their National Health Service (NHS) Local Area Team. Eligible practices were randomly assigned (1:1) into two groups by computer-generated allocation sequence, stratified by NHS Local Area Team. Participants, but not investigators, were blinded to group assignment. On Sept 29, 2014, every GP in the feedback intervention group was sent a letter from England's Chief Medical Officer and a leaflet on antibiotics for use with patients. The letter stated that the practice was prescribing antibiotics at a higher rate than 80% of practices in its NHS Local Area Team. GPs in the control group received no communication. The sample was re-randomised into two groups, and in December, 2014, GP practices were either sent patient-focused information that promoted reduced use of antibiotics or received no communication. The primary outcome measure was the rate of antibiotic items dispensed per 1000 weighted population, controlling for past prescribing. Analysis was by intention to treat. This trial is registered with the ISRCTN registry, number ISRCTN32349954, and has been completed.
FINDINGS: Between Sept 8 and Sept 26, 2014, we recruited and assigned 1581 GP practices to feedback intervention (n=791) or control (n=790) groups. Letters were sent to 3227 GPs in the intervention group. Between October, 2014, and March, 2015, the rate of antibiotic items dispensed per 1000 population was 126.98 (95% CI 125.68-128.27) in the feedback intervention group and 131.25 (130.33-132.16) in the control group, a difference of 4.27 (3.3%; incidence rate ratio [IRR] 0.967 [95% CI 0.957-0.977]; p<0.0001), representing an estimated 73,406 fewer antibiotic items dispensed. In December, 2014, GP practices were re-assigned to patient-focused intervention (n=777) or control (n=804) groups. The patient-focused intervention did not significantly affect the primary outcome measure between December, 2014, and March, 2015 (antibiotic items dispensed per 1000 population: 135.00 [95% CI 133.77-136.22] in the patient-focused intervention group and 133.98 [133.06-134.90] in the control group; IRR for difference between groups 1.01, 95% CI 1.00-1.02; p=0.105).
INTERPRETATION: Social norm feedback from a high-profile messenger can substantially reduce antibiotic prescribing at low cost and at national scale; this outcome makes it a worthwhile addition to antimicrobial stewardship programmes. FUNDING: Public Health England.

RCT Entities:   Population Interventions Outcomes

Introduction

The growth of antimicrobial resistance has been recognised as a worldwide public health problem that increases mortality, morbidity, and the cost of health care.1, 2 There are concerns that future resistance will rise to levels that seriously disrupt important medical procedures. An important driver of resistance is the medical use of antibiotics when they are not clinically indicated. Primary care is a focus of antibiotic stewardship efforts for several reasons: it accounts for a large proportion of antibiotic prescriptions; its prescribing practices have been linked to increased antimicrobial resistance; and there is extensive variation between practices, which suggests “significant scope to improve prescribing”.5, 6, 7

Providing feedback has been shown to produce fairly small but important changes in health-care provider behaviour in general. However, studies of feedback specifically focused on antibiotic prescribing have produced mixed results: substantial improvements have come about from more complex, intensive interventions, rather than simple feedback.9, 10, 11, 12 Nevertheless, there are strong theoretical and practical reasons for further examining the effect of feedback on antibiotic prescribing. First, the number of studies on this topic is still small and the quality of evidence they provide is low. Second, there are questions about whether more complex interventions can be scaled successfully and feasibly, since they are often resource intensive. Third, the success of current interventions depends greatly on the way they are designed and implemented. Recently, there has been growing interest in the potential for the behavioural sciences to improve these aspects of antibiotic stewardship. A particularly promising option is social norm feedback: presenting information to show that individuals are outliers in their behaviour leads them to adjust their behaviour towards the social norm. Moreover, since 2012, detailed primary care prescribing datasets have been made publicly available by the Health and Social Care Information Centre (HSCIC). These data are often not actively fed back to prescribers, so there is potential to introduce feedback interventions to improve prescribing for little cost and on a large scale, as recently suggested in an analysis by the Department of Health and Public Health England. Accordingly, we undertook a nationwide randomised trial of a low-cost feedback intervention (clinician-focused letter plus leaflet for use with patients) that incorporated social norms and other findings from the behavioural sciences to reduce antibiotic prescribing in primary care.

Evidence before this study

We reviewed systematic reviews of interventions to reduce antimicrobial stewardship. We searched Google Scholar for terms “antibiotic prescribing”, “antibiotic stewardship”, or “antimicrobial stewardship”, in addition to the term “systematic review”. On review of abstracts, we selected studies published in English between Jan 1, 1980, and Feb 15, 2015, dealing with interventions to reduce prescribing in primary care or outpatient settings. Existing evidence suggests that feedback for antibiotic prescribing has produced mixed results, with the studies that reject the null hypothesis showing small effect sizes. The assessment of patient-focused information-based interventions using printed materials is often weak; high-quality studies show no or small effects. More complex interventions to reduce prescribing have generally produced larger effect sizes, but it is unclear whether these interventions can be implemented at scale.

Added value of this study

Our study shows that a low-cost feedback intervention can reduce the rate at which antibiotic items are dispensed by 3·3% over 6 months. This effect compares favourably with results from the few comparable studies that exist, and suggests that tailored feedback is a worthwhile addition to antimicrobial stewardship programmes. Our trial also adds to other studies by incorporating findings from the behavioural sciences, including social norms, messenger effects, and behavioural instruction. Our results show that a patient-focused information-based intervention may not affect prescribing behaviour. This study was implemented on a larger scale than many previous studies, and took place in a live policy context, giving it good external validity.

Implications of all the available evidence

Antimicrobial stewardship programmes should consider incorporating prescribing feedback into their activities. The effects of this feedback might be enhanced by use of social norms, behavioural instruction, and a high-profile messenger. However, there is still a need for further research to understand how the message content, timing, messenger, medium, and recipient affect the outcomes of such feedback.

The publication of primary care prescribing datasets also offers the much needed opportunity to assess a wider range of antibiotic stewardship interventions, in addition to feedback mechanisms. Campaigns aimed at the public and clinicians have been one of the most prevalent approaches to reduce antibiotic use, and have attracted substantial public funding. However, many of these campaigns have either not been assessed or their evaluations have had substantial methodological weaknesses; the availability of data for prescribing outcomes is a particular problem. Public Health England had scheduled a patient-focused campaign to coincide with European Antibiotics Awareness Day 2014. Therefore, we took a pragmatic approach and amended our trial design to incorporate this patient-focused intervention, to determine whether it reduced prescribing on its own or in combination with the clinician-focused feedback intervention.

Methods

Study design and participants

In this pragmatic, 2 × 2 factorial randomised controlled trial, we recruited general practitioner (GP) practices in England using names and addresses from datasets of GPs and prescribing data from datasets published by the HSCIC. The units of recruitment and measurement were GP practices, because HSCIC prescribing data are available at practice level only. The study design incorporated two interventions. GP practices were first randomised to the intervention or control group for the first intervention, which took place in October, 2014. Every GP in a practice in the feedback intervention group was sent a clinician-focused letter and leaflet from England's Chief Medical Officer. The control group received no communication. Practices were then re-randomised to the intervention or control group for the second intervention, which took place in December, 2014. In the second intervention, practices either received patient-focused posters and leaflets (note that these were not sent to individual GPs, but rather to the practice manager) or no communication. This two-stage process resulted in a 2 × 2 factorial trial design, with four experimental groups by the end of the study period: those who were assigned to both interventions; those who were assigned to the feedback intervention only; those who were assigned to the patient-focused intervention only; and those who were assigned to neither intervention.

The initial population consisted of all GP practices in England who issued prescriptions for any antibiotics (British National Formulary [BNF] 5·1) that were dispensed in May, 2014: this was the latest period of data available immediately before the study's launch, since HSCIC publishes prescribing data with a 3-month lag. Practices that had not been open since at least October, 2013, were excluded because they did not have the historical data necessary to apply a control for past prescribing levels to the main outcome variable. Practices were also excluded if their rate of dispensed antibiotic items per 1000 population, after applying the 2013 specific therapeutic group age-sex related prescribing units (STAR-PU) controls for age and sex for BNF subset 5·1, was classed as an outlier. This exclusion was to prevent inappropriate communication with practices subject to measurement error or who were systematically different from the main population. The outlier threshold was set at the 95th percentile on advice from public health professionals (ie, the very highest prescribers were excluded). We then excluded those practices whose rate of dispensed antibiotic items was not in the top 20% for their National Health Service (NHS) Local Area Team. The 20% figure was selected because we judged that it represented the optimum trade-off between increasing statistical power by widening the sample and diluting the power of the social norms message by increasing the size of the “minority” group. NHS Research Ethics Committee (14/LO/1544) waived participant consent for this trial, since obtaining consent would invalidate the results and create a burden greater than the intervention itself.

Randomisation and masking

A study investigator (MS) randomly assigned GP practices to intervention or control groups, stratified by NHS Local Area Teams, using the random number allocation function in the Stata software package. MS did not implement the interventions. Participants in intervention groups are likely to have been aware of the interventions they were assigned to but unaware that they were involved in a trial. Since the prescribing dataset had to be matched with the dataset of GP names, it was not practical to blind the study team to group assignment.

Procedures

In the feedback intervention, the letter from the Chief Medical Officer was sent to GPs on Sept 29, 2014 (appendix). The letter stated that the practice was prescribing antibiotics at a higher rate than 80% of practices in its NHS Local Area Team, and was accompanied by a copy of the patient-focused “Treating your infection” leaflet developed for the TARGET programme that enables back-up prescribing (appendix). The letter was issued at the end of September to coincide with the seasonal increase in antibiotic prescriptions seen during the winter months. More specifically, it was issued to arrive on the first day of the following month because the data are collected on a monthly basis. The control group did not receive this letter. However, if the intervention had been shown to significantly reduce prescribing levels, we planned to send the intervention to this control group (ie, the practices who had not received the feedback intervention at the end of the 6-month period). On this basis, letters were issued to these remaining practices on March 30, 2015.

The letter used three main concepts from the behavioural sciences. The first was social norm information about how the recipient's practice's prescribing rate compared with other practices in the local area. Social norm comparisons were made within NHS Local Area Teams to increase both the appropriateness of the intervention message (since antibiotic dispensing varies by geographical region), and its salience (since making social norm information more specific to the recipient may increase its effectiveness). Second, the letter was addressed from a high-profile figure, with the assumption that this would increase the credibility of its content. Finally, the letter presented three specific, feasible actions that the recipient could do to reduce unnecessary prescriptions of antibiotics: giving patients advice on self-care, offering a delayed prescription, and talking about the issue with other prescribers in his or her practice. The selection and presentation of these actions drew on the concept of behavioural instruction, which has been linked to increased behaviour change and improved comprehension and recall.21, 22 These recommended actions are all supported by professional prescribing guidance.23, 24 The letter was developed by the study investigators with input from experts on antimicrobial stewardship: two GP academics, a public health practitioner, and officials from Public Health England. The accompanying TARGET leaflet reinforced the message of the letter because it acted as a method to support delayed or reduced prescribing.

The feedback intervention, targeted mainly at prescribers, was followed by a patient-focused intervention in December, 2014. The rationale for the second intervention was that GPs may prescribe antibiotics when not clinically indicated because of (real or perceived) pressure from patients. Therefore, the intervention was intended to promote the attitude among patients that if antibiotics are taken when they are not needed, they are less likely to work when needed. The hypothesis was that adopting this attitude would make patients less likely to exert pressure on the prescriber to give them antibiotics. In turn, this reduced pressure would lead to fewer instances of unnecessary prescribing. The second intervention consisted of two posters, two leaflets, and a letter addressed to the practice manager (appendix). The posters and letter were developed through qualitative research done in October, 2014, with mothers of children from a range of socioeconomic backgrounds (National Readership Survey Grades C2DE and BC1), and with GPs in high-prescribing practices and Prescribing Advisors in high-prescribing Clinical Commissioning Groups. This research suggested that the most effective messages would be those that link unnecessary antibiotic use to future personal consequences arising from antimicrobial resistance, a finding that is reflected elsewhere.16, 17 Between Dec 4 and Dec 11, 2014, the materials were hand-delivered by a health-care communications company to the practices in the intervention group. Photographic evidence was obtained that the posters and leaflets had been set up for display in patient waiting rooms and GP consulting rooms.

See appendix for classification of the intervention materials into behaviour change techniques according to the Behavior Change Technique taxonomy version 1. We note that this is a purely descriptive device, and its use does not imply that certain elements will be effective simply because they are described as behaviour change techniques.

Outcomes

The primary outcome measure was the rate of antibiotic items dispensed per 1000 population, after applying the 2013 STAR-PU controls for age and sex for BNF subset 5·1 and controlling for the previous year's prescribing levels in each practice. For the feedback intervention, the period of October, 2014, to March, 2015, inclusive was used; for the patient-focused intervention, the period of December, 2014, to March, 2015, was used. The secondary outcome measures were only calculated when a statistically significant difference between intervention and control groups was obtained. They were, first, the estimated total difference in the number of antibiotic items dispensed between intervention and control groups for the study period; second, the estimated effect on direct prescription costs.

Statistical analysis

In June, 2014, we used the most recent data available to estimate that our sample would consist of 1490 GP practices. We powered the study to detect a change in the dispensed antibiotic prescribing rate of 0·48% (d=0·0568) with 80% power, once controls for patient demographics, local area prescribing rates, and previous prescribing trends had been applied. We selected a fairly small minimum detectable effect size because of the small effects obtained for similar audit and feedback interventions.9, 10, 11, 12, 27 Additionally, the low cost of the letter meant that even a small effect size could be cost effective.

Our primary analysis used a fixed effects panel regression model, with a difference-in-differences component that used the practice's prescribing level 12 months before each of the months in the study period. These controls were applied to reduce the unexplained variance in the data caused by unobserved, time-inconsistent properties of each GP practice. The model was applied to the primary outcome data for each of the months in the study period, as well as to the pooled data for the entire period. To estimate changes in the intervention's effectiveness over time, we used a fixed effects regression with controls for month and month interacted with treatment. Finally, we interacted the two interventions in the 2 × 2 design to identify whether combining them had a statistically significant additive effect on the outcome measure.

Our secondary analysis was done solely on the pooled primary outcome data for the study period, and consisted of three elements. The first was a post-hoc cost-effectiveness analysis (not featured in our protocol because we were unaware that the necessary data were available). When there was a significant effect, we estimated the total difference in the number of dispensed antibiotic items between intervention and control groups. This calculation was done by using the regression outputs to predict the number of items that would have been dispensed in the intervention group if they been in the control group, and comparing this figure to the actual number dispensed in the intervention group. When there was a significant effect, we estimated the effect on direct prescribing costs for the public sector. To do this, we incorporated the Actual Cost field in the prescribing dataset into the aggregated regression model to create a crude cost estimate. We then assumed that 10·1% of these items incurred an £8·20 prescription charge (in line with the overall exemption rate for 2014), which we deducted from this cost estimate. Finally, we incorporated the £0·90 per item professional fee payable to pharmacy contractors by NHS England, to produce an overall savings figure. We compared this figure to the basic cost of the intervention materials. The second element of the secondary analysis was an analysis of interactions between the main treatment effect and patient demographics, and the number of patients registered to a practice. Sex is included in the interactions, despite also featuring in STAR-PU, because the STAR-PU controls for sex are present as a series of absolute values interacted with age, not linearly. Finally, a subgroup analysis was done to determine the effect on antibiotics usually prescribed for upper respiratory tract infections (penicillins and macrolides), since these are a common source of unnecessary antibiotic prescriptions.

As with any 2 × 2 design, two sets of results are relevant. The first shows the aggregate effect of the two main interventions by, for example, comparing all the practices that received the feedback intervention with all the practices that did not. The second shows the effect of each of the four experimental groups, so that effect of combining interventions (or not) can be discerned. To maximise clarity, we present these two sets of results separately.

The analyses were done on an intention-to-treat basis and attrition was handled by using the prescribing rate for the equivalent month in the previous year for practices that closed during the study period. All analyses were done with Stata IC (version 13). Because all data used in the trial were publicly available, no data monitoring committee was appointed to oversee the study. This trial is registered with the ISRCTN registry, number ISRCTN32349954, and has been completed.

Role of the funding source

As part of its standard responsibilities, the funder of the study had a role in the study design, data interpretation, and writing of the report. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit to publication.

Results

Participants for the feedback intervention were recruited and assigned to treatment or control groups between Sept 8 and Sept 26, 2014. 7993 GP practices were assessed for eligibility because they had issued prescriptions for any antibiotics in May, 2014. Of these, 6412 practices did not meet the inclusion criteria (figure 1), which resulted in a final sample of 1581. These practices were randomly assigned to control (n=790) and feedback intervention (n=791) groups (table 1). After a manual check, two practices were excluded from the feedback intervention group because they specialised in treating the homeless. Matching GP records to practice records resulted in 3227 letters being issued.

Figure 1

Trial profile

In the first intervention, every general practitioner (GP) in a practice in the intervention group was sent a clinician-focused letter from England's Chief Medical Officer and a leaflet for use with patients (feedback intervention); GPs in the control group received no communication. In the second intervention, every practice manager in practices in the intervention group received patient-focused posters and leaflets (patient-focused intervention); practices in the control group received no communication. NHS=National Health Service.

Table 1

Characteristics of GP practices in sample before the feedback intervention (data from May, 2015)

	Control (n=790)	Feedback intervention (n=791)
Number of GPs	4·72 (2·80)	4·75 (3·19)
Female patients	3363·29/6669·52 (50%)	3320·99/6574·80 (51%)
Patient age (years)	40·63 (4·46)	40·31 (4·71)
Antibiotic items dispensed per 1000 population	63·67 (13·94)	62·74 (11·35)
Total number of registered patients	6669·52 (3764·37)	6574·80 (3999·86)

Data are mean (SD) or n/N (%). GP=general practitioner.

By March, 2015, data ceased to be available because of closure or merger for six practices in the control group and four practices in the feedback intervention group. Of these ten practices, one practice in each of the intervention and control groups ceased to prescribe in February, 2015, but did not have prescribing data available for February, 2014 (although they did for March, 2014). This lack of availability meant that attrition could not be dealt with by use of the previous year's prescribing rate. Therefore, these practices were excluded from the analysis for February, 2015, but included for every other month. Public Health England performed random checks of the photographic evidence provided by the company contracted to deliver the materials for the patient-focused intervention; no evidence of non-delivery was recorded. We did not collect data for harms or adverse incidents.

Between October, 2014, and March, 2015, the rate of antibiotic items dispensed per 1000 population was 131·25 (95% CI 130·33–132·16) in the control group and 126·98 (125·68–128·27) in the feedback intervention group, an absolute difference of 4·27 (3·3% relative difference). The incidence rate ratio (IRR) for the difference between groups was 0·967 (95% CI 0·957–0·977; p<0·0001; table 2), which represents a standardised effect size of d=0·19 (a small effect size). The rate at which antibiotics were dispensed in the feedback intervention group was significantly lower than in the control group for every month in the study period (table 2; figure 2). There is no evidence of a trend in the effect size. The difference between the two groups ceased to be significant in April, 2015 (IRR 0·989, 95% CI 0·970–1·009; p=0·281). During this month, the letter was sent to practices that had not already received it; thus, one would expect the difference to be eliminated at this point.

Table 2

Antibiotic items dispensed per 1000 weighted population for the feedback intervention, September, 2014, to April, 2015

	Antibiotic items dispensed per 1000 weighted population*(95% CI)		IRR*(95% CI)	p value
	Control	Feedback intervention
September, 2014 (pre-intervention)	111·65 (109·96–113·34)	111·72 (109·51–113·93)	1·001 (0·981–1·020)	0·9450
October, 2014	123·82 (122·15–125·51)	120·04 (117·93–122·16)	0·969 (0·952–0·987)	0·0005
November, 2014	116·10 (114·43–117·78)	113·44 (111·33–115·56)	0·977 (0·959–0·995)	0·0135
December, 2014	156·70 (155·03–158·37)	150·38 (148·26–152·49)	0·960 (0·946–0·973)	<0·0001
January, 2015	140·07 (138·39–141·75)	135·30 (133·19–137·42)	0·966 (0·951–0·981)	<0·0001
February, 2015	121·02 (119·34–122·69)	116·43 (114·32–118·55)	0·962 (0·945–0·980)	<0·0001
March, 2015	129·76 (128·08–131·43)	126·28 (124·17–128·40)	0·973 (0·957–0·990)	0·0013
April, 2015 (post-intervention)	108·29 (106·64–109·94)	107·14 (105·05–109·23)	0·989 (0·970–1·009)	0·281
October, 2014–March, 2015 (pooled)	131·25 (130·33–132·16)	126·98 (125·68–128·27)	0·967 (0·957–0·977)	<0·0001

Controlling for previous prescribing levels.

Figure 2

Rate of antibiotic items dispensed per 1000 weighted population for the feedback intervention, September, 2014, to April, 2015

Error bars represent 95% CIs.

An estimated 73 406 fewer antibiotic items were dispensed during the study period as a result of the feedback intervention. This estimate is based on 2 178 345 items dispensed by the intervention group and our regression model prediction that this group would have dispensed 2 251 751 antibiotic items in absence of the intervention. The cost of printing and mailing to 3227 GPs in 791 practices was £4335, which equates to £0·06 per prescription prevented during the study period (excluding the time costs of implementing the intervention). We estimate that this intervention created a £92 356 saving in direct prescribing costs for the public sector. There were no significant interactions between the feedback intervention and practice-level patient age or sex profiles. However, the effect size was smaller for practices with more registered patients: for each 1000 increase in practice list size, the effect of the letter was reduced by 0·47 prescribing rate units (95% CI 0·16–0·77; p=0·0022). The marginal treatment effect was highly significant for penicillins and macrolides (−3·916 [95% CI −5·202 to −2·636]; p<0·0001), but was not significant for the other antibiotics (−0·308 [–0·773 to 0·156]; p=0·194).

For the patient-focused intervention, the sample of 1581 practices was re-randomised into control (n=804) and intervention (n=777) groups (table 3). Between December, 2014, and March, 2015, the rate of antibiotic items dispensed per 1000 population was 133·98 (95% CI 133·06–134·90) in the control group and 135·00 (95% CI 133·77–136·22) in the group assigned to the patient-focused intervention (IRR for difference between groups 1·01, 95% CI 1·00–1·02; p=0·105; table 4). The rate at which antibiotics were dispensed was significantly lower in the control group than in the group assigned to the patient-focused intervention for one month in the study period (December, 2014). We did not proceed to the secondary analysis for the patient-focused intervention because we did not disprove the null hypothesis for the primary outcome measure.

Table 3

Characteristics of GP practices in re-randomised sample before the patient-focused intervention (data from July, 2015)

	Control (n=802)	Patient-focused intervention (n=779)
Number of GPs	4·76 (2·98)	4·71 (3·03)
Female patients	3411·75/6766·79 (50%)	3269·91/6574·56 (50%)
Patient age (years)	40·52 (4·45)	40·45 (4·74)
Antibiotic items dispensed per 1000 population	61·01 (12·67)	61·75 (12·75)
Total number of registered patients	6766·79 (4046·03)	6574·56 (3702·78)

Data are mean (SD) or n/N (%). GP=general practitioner.

Table 4

Antibiotic items dispensed per 1000 weighted population for the patient-focused intervention, December, 2014, to March, 2015

	Antibiotic items dispensed per 1000 weighted population*(95% CI)		IRR*(95% CI)	p value
	Control	Patient-focused intervention
December, 2014	151·43 (149·86–153·11)	155·64 (153·63–157·64)	1·027 (1·014–1·040)	0·0001
January, 2015	137·22 (135·60–138·85)	138·14 (136·13–140·15)	1·006 (0·992–1·021)	0·369
February, 2015	119·24 (117·61–120·88)	118·16 (116·15–120·17)	0·990 (0·974–1·007)	0·293
March, 2015	127·98 (126·35–129·61)	128·03 (126·02–130·04)	1·000 (0·984–1·016)	0·957
December, 2014–March, 2015 (pooled)	133·98 (133·06–134·90)	135·00 (133·77–136·22)	1·007 (0·998–1·016)	0·105

Controlling for previous prescribing levels.

In an analysis of all four experimental groups (neither intervention, n=402; feedback intervention only, n=402; patient-focused intervention only, n=388; and both interventions, n=389; figure 1; table 5), the antibiotic prescribing rate was significantly lower in the group assigned to the feedback intervention only compared with the group that received neither intervention, for all months in the study period until March, 2015 (IRR 0·967, 95% CI 0·949–0·982; p<0·0001; table 6, figure 3). The patient-focused intervention produced a significant increase in the prescribing rate in the month it was delivered, but its effect was not significant for the relevant period as a whole (patient-focused intervention only vs neither intervention IRR 1·013, 0·998–1·028; p=0·073). The prescribing rate was also significantly lower in the group assigned to both interventions compared with the group that received neither intervention (IRR 0·975, 0·960–0·990, p=0·002). The interaction between the two interventions was close to the threshold for 0·05 significance level (−1·89 [95% CI −3·95 to 0·61]; p=0·058).

Table 5

Characteristics of GP practices, four experimental groups

	Neither intervention (n=401)	Feedback intervention only (n=402)	Patient-focused intervention only (n=387)	Both interventions (n=389)
Number of GPs	4·70 (2·70)	4·82 (3·23)	4·74 (2·92)	4·63 (3·14)
Female patients	3421·54/6783·00 (50%)	3401·99/6736·10 (51%)	3302·93/6551·38 (50%)	3237·05/6408·11 (51%)
Patient age (years)	40·58 (4·37)	40·47 (4·53)	40·72 (4·57)	40·18 (4·90)
Antibiotic items dispensed per 1000 population	61·35 (13·63)	60·67 (11·63)	62·29 (14·25)	61·25 (11·06)
Total number of registered patients	6783·00 (3925·97)	6736·10 (4169·83)	6551·38 (3590·52)	6408·11 (3814·47)

Data are mean (SD) or n/N (%). GP=general practitioner.

Table 6

Antibiotic items dispensed per 1000 weighted population for all experimental groups, September, 2014, to April, 2015

	Antibiotic items dispensed per 1000 weighted population*(95% CI) in group that received neither intervention	Feedback intervention only		Patient-focused intervention only		Both interventions
		Antibiotic items dispensed per 1000 weighted population* (95% CI)	p value†	Antibiotic items dispensed per 1000 weighted population* (95% CI)	p value†	Antibiotic items dispensed per 1000 weighted population* (95% CI)	p value†
September, 2014 (pre-intervention)	111·39 (109·64–113·14)	112·29 (109·92–114·66)	0·456	111·88 (109·50–114·28)	0·683	111·12 (108·72–113·50)	0·821
October, 2014 (feedback intervention)	123·16 (120·84–125·47)	119·13 (116·20–122·05)	0·006	124·38 (121·44–127·33)	0·453	120·98 (118·03–123·92)	0·128
November, 2014	115·73 (113·54–117·93)	112·57 (109·64–115·86)	0·034	116·46 (113·52–119·38)	0·625	114·33 (111·38–117·26)	0·348
December, 2014 (patient-focused intervention)	154·14 (151·9–156·35)	148·29 (145·36–151·21)	<0·0001	159·33 (156·37–162·28)	0·001	152·51 (149·57–155·45)	0·278
January, 2015	139·29 (137·09–141·48)	134·62 (131·70–137·55)	0·002	140·83 (137·89–143·79)	0·305	136·00 (133·05–138·94)	0·028
February, 2015	120·92 (118·72–123·12)	117·01 (114·08–119·95)	0·009	121·04 (118·09–124·00)	0·934	115·83 (112·87–118·78)	0·001
March, 2015	129·29 (127·09–131·49)	126·15 (123·22–129·07)	0·035	128·38 (125·46–132·69)	0·545	126·42 (123·46–129·36)	0·056
April, 2015 (feedback intervention delivered to all participants)	107·84 (105·63–110·03)	107·59 (104·65–110·51)	0·864	108·76 (104·02–111·25)	0·539	106·71 (103·76–109·65)	0·450
December, 2014–March, 2015 (pooled)	136·22 (134·66–137·77)	131·73 (129·38–133·77)	<0·0001	138·11 (136·03–140·17)	0·073	132·92 (130·85–134·98)	0·002

Controlling for previous prescribing levels.

p value of difference from group that received neither intervention.

Figure 3

Rate of antibiotic items dispensed per 1000 weighted population for all experimental groups, September, 2014, to April, 2015

Error bars represent 95% CIs.

Discussion

To our knowledge, this study is the first national-scale randomised trial of giving feedback on antibiotic prescribing. Our results show that providing a low-cost mail-based intervention incorporating social norm feedback on high antibiotic use can consistently reduce such use over a 6-month period. Importantly, the effect of the feedback intervention disappeared after letters were sent to all practices in the sample at the end of March, 2015. The effect we recorded compares favourably to results from previous studies of antibiotic prescribing feedback.9, 10, 11, 12, 27 To relate the 3·3% relative reduction to policy goals: the UK 5-Year Antimicrobial Resistance Strategy gave an initial target of reducing primary care prescribing by 4%. In the NHS, the Quality Premium currently rewards a 1% reduction in antibiotic items prescribed. We calculate that our feedback intervention would equate to a 0·85% reduction in antibiotic items nationally during the study period (if the control group was also treated). By contrast, we did not note any significant effect of the patient-focused campaign materials. Indeed, in one month the intervention seemed to increase the rate at which antibiotic items were dispensed. This finding strengthens the case for careful assessment of patient-focused information-based interventions, including their effect on prescribing outcomes.

The feedback intervention offered three main advantages compared with the main alternatives for improving antibiotic stewardship, which are often more complex face-to-face interventions. First, a letter-based intervention is likely to be cheaper than a face-to-face one, although previous reviews have noted that cost reporting is usually absent. The material costs of the feedback intervention were roughly £0·06 per antibiotic item not dispensed during the study period. The second advantage of the feedback intervention is scalability, which has been highlighted by previous reviews: it is unclear whether the successes of face-to-face interventions can be achieved on a larger scale, since they may depend on particular local conditions and modes of delivery.10, 12 The third advantage relates to the low barriers to performing the feedback intervention. Since the underlying data are all publicly available, feedback of this kind could be provided by many different interested parties. This situation opens up the possibility that pressure for quality improvement can be exerted in a more dynamic way, by use of distributed networks rather than hierarchical structures. These datasets can also be combined with findings from the behavioural sciences to increase their effect.

This study has several limitations. The effects on health outcomes, and thus harms and benefits, were not measured. More specifically, we cannot measure whether the reductions in antibiotic prescribing occurred only for infections for which antibiotics are ineffective (although the subgroup analyses suggest that this is the case). However, previous reviews have reported no negative effects of antibiotic stewardship interventions on treatment effectiveness or patient satisfaction.10, 11, 12 The sample consisted of the highest prescribers only, so it is not clear how the results would generalise to the rest of the population. Prescribing data were available at the practice level only (rather than practitioner level) and were restricted to prescriptions that were dispensed and submitted to the NHS Business Services Authority in order to claim payment—although we believe this to be the vast majority of prescriptions, in view of this financial incentive to submit. There is also a risk of contamination between experimental groups, since GPs from different practices regularly communicate with each other. It was not possible to measure the rate of delayed prescribing, even though this was one of the actions recommended to practitioners. Because this was a one-off intervention, we cannot assess the effects of repeated messages at this stage. Measuring participant exposure was not straightforward: the study team could not be certain that all GPs received and opened the letters. Our estimate of the savings to the public sector concerns prescribing costs only, and neglects the wider costs that might result from not prescribing antibiotics (eg, multiple visits to the NHS to obtain antibiotics). Finally, having a single intervention group means that it is not possible to disentangle the effects of the different intervention elements—ie, social norm feedback, a high-profile messenger, and behavioural instruction.

The main future research direction suggested by this study is to examine how the effectiveness of antibiotic prescribing feedback can be maximised. Attention should focus on how the outcome varies according to message content, message timing, messenger, medium, and recipient. For example, it is not clear whether emailed feedback would produce a similar effect, or whether a similar message would be effective for those not in the highest 20% of prescribers. The long-term effects should also be considered: do effects continue beyond 6 months, and does repeated exposure to the same message or messenger lead to decay in the effect size? Studies could use a wider range of outcome measures, such as indicators of patient health and antimicrobial resistance. Since the quality of existing evidence on this topic has been rated as low, there is value in testing these questions through randomised field experiments.

Our results suggest that antimicrobial stewardship programmes should consider incorporating primary care prescribing feedback into their activities, if they do not already do so. The effects of this feedback might be enhanced by use of social norms, behavioural instruction, and a credible messenger. Feedback should form part of an overall strategy that also includes more in-depth, complex interventions, because the effect size reported here will not be sufficient to address the policy issue fully. Such feedback could also be expanded to other sectors that use antimicrobial drugs, such as secondary care and veterinary care. Finally, there may be value in introducing feedback where there is substantial variation in other types of prescribing (such as antipsychotic drugs or statins) or other forms of clinical practice (such as diagnostic tests or surgery).