Nationwide audit and feedback on implementation of antibiotic stewardship programmes in Norwegian hospitals.

BACKGROUND: Implementation of antibiotic stewardship programmes (ASPs) in hospitals is challenging and there is a knowledge gap on how to pursue this process efficiently.
OBJECTIVES: To evaluate whether audit and feedback (A&F) is a feasible and useful methodology to assess and support the implementation of ASPs in hospitals.
METHODS: A multidisciplinary team performed document reviews and on-site interviews with professionals involved in the implementation of ASPs. Oral feedback on preliminary findings and areas of improvement were provided on-site, followed by feedback reports summarizing major findings and recommendations. Descriptive statistics were used to present number of hospital trusts, interviewees, professions, disciplines, workload and costs.
RESULTS: All 22 hospital trusts in Norway participated in the A&F conducted October 2017 to April 2019. Altogether, 446 leaders and healthcare workers were interviewed: 110 leaders, 336 health professionals of whom 89 were antimicrobial stewardship team members. Median number of days from audits were performed till reporting were 36 (IQR 30-49). Median workload for auditors per visit was 7 days (6-8). Total costs were €133 952. Main audit findings were that ASP structures were established in most hospital trusts, but leadership commitment and implementation of interventions were often lacking. The hospital trusts received feedback on establishing governance structures, setting local targets, implementing interventions and increased involvement of nurses.
CONCLUSIONS: Nationwide A&F provides a unique and comprehensive insight into the implementation of ASPs in hospitals and is feasible with a reasonable amount of resources. This approach can identify targets for improved implementation of ASPs in hospitals.

Introduction

Many countries have developed national action plans outlining measures needed to combat antimicrobial resistance (AMR)., However, achievements depend on implementation of the measures, which may be evaluated using different types of data, e.g. quality indicators (QIs)., However, a qualitative approach can provide a more in-depth insight into these processes. Audit and feedback (A&F), i.e. any summary of clinical performance of healthcare over a specified period of time, is one suitable method for qualitative data collection when applied within a field with well-defined standards. This methodology has been used in various topics and settings, but as far as we know, it has not been applied to assess and support the implementation of antibiotic stewardship programmes (ASPs) in hospitals.

In 2016, The Norwegian Ministry of Health and Care services launched an action plan against AMR. ASPs became mandatory for hospitals and a target was set for reducing the use of five groups of broad spectrum antibiotics by 30% by the end of 2020 compared with 2012. The Norwegian Advisory Unit for Antibiotic Use in Hospitals (KAS) were to support the Norwegian hospitals in implementing ASPs. As part of this support, KAS together with the Northern Norway Regional Health Authority (RHA) decided to conduct A&F. In this paper, we aim to detail whether A&F is a feasible and useful methodology to assess the implementation of ASPs in hospitals.

Materials and methods

Design

A retrospective evaluation study design was used to investigate whether A&F is a feasible and useful methodology to evaluate the implementation of ASPs in hospitals.

Setting

The study was conducted in Norway, a high-income country with 5.4 million inhabitants. Antibiotic consumption and AMR rates in Norway are below the European average., However, there is large variation in the use of broad-spectrum antibiotics between hospitals, which cannot be accounted for by differences in activity or case mix alone. Across the country, 48 hospitals are organized into 19 hospital trusts that constitute four RHAs: the Northern, Central, Western and South-Eastern RHAs. The Ministry of Health and Care Services finances and owns the RHAs. In addition, three private, non-profit general hospitals are contracted by the RHAs, i.e. there are 22 hospital trusts in Norway.

The audit and feedback

The A&Fs were conducted by two teams. The Northern RHA team consisted of two internal auditors and three experts: an infection control physician/microbiologist (K.G.), a pharmacist (J.U.H.) and an infectious disease (ID) physician from KAS (P.E.A.). The audit team in the three southern RHAs comprised an ID physician (P.E.A.) and a pharmacist (M.I.N.), both from KAS. All audit team members participated in planning the audits, reviewing the documents and writing the feedback reports, whereas only two professionals conducted the audit on site.

Selection of focus areas and review criteria and the development of an interview guide were mainly based on the mandatory content of an ASP as outlined in the Norwegian action plan. In addition, we applied the national guideline on antibiotic use in hospitals, healthcare legislation and assignment agreement documents from the Ministry of Health and Care Services to the RHAs.

We defined four focus areas: (i) the hospital trusts’ formal/written ASP documents, such as policies, mandates, organization, targets and milestones; (ii) the hospital trusts’ strategies for how to achieve 30% reduction in the use of broad-spectrum antibiotics by 2020; (iii) the hospital trusts’ systems for ensuring physicians’ and nurses’ competence and training in antibiotic prescribing; and (iv) the hospital trusts’ antibiotic prescribing practices and the application of the national guideline on antibiotic use. Two to six review criteria were developed for each focus area, defining the standards to which the observations were compared. An interview guide with a total of 40 questions was developed. See Table S1 for the detailed interview guide (available as Supplementary data at JAC-AMR Online).

All 22 hospital trusts in Norway were invited via a letter to the hospital management. Participation was mandatory for the four northern hospital trusts as the Northern RHA had ordered the audit, whereas it was voluntary for the 18 remaining hospital trusts.

Data were collected in two steps. First, relevant documents were obtained from the hospital trusts, either through an online search or by request. Relevant documents were management and milestone plans, an overview of antimicrobial stewardship (AMS) team members, and antibiotic consumption and antibiotic resistance reports.

Secondly, data were collected during on-site visits. We visited one hospital in each hospital trust, primarily the largest hospital, except for the largest hospital trust with two university hospitals, where we visited both. The programme for each on-site visit (duration 7–14 h) included a start-up meeting, semi-structured interviews with various professionals involved in antibiotic prescribing or implementation of the ASP, and a final meeting (Table 1). We primarily targeted departments with high antibiotic consumption and ensured that the interviewed leaders, physicians and nurses were employed at the same units. Notes were taken during the interviews, but they were not recorded.

Table 1.

Example of an on-site visit programme

Time	Activity	Participants/interviewees	Comment
8:30–9:00	Start-up meeting	Leaders and all interviewees	Presentation of purpose/aim, auditors and programme for the day
9:00–9:30	Individual interview	Director of Research and Developmenta
9:45–10:30	Individual or group interview	Department director and/or department managers
10:30–11:15	Group interview	AMS team
11:30–12:00	Group interview	Senior consultants
12:45–13:15	Group interview	Residents
13:30–14:00	Group interview	Nurses (and secretary)
14:00–15:00	Review of results	Auditors
15:00–15:45	Final meeting	Leaders and all interviewees	Presentation of preliminary findings and areas of improvements for all participants

Or other/equivalent Director representing the hospital management.

The hospital trusts were given immediate feedback in the final meetings, meaning preliminary findings and advice for further implementation of the ASPs. Additionally, findings from the audit were presented in a feedback report to each hospital trust (8–15 pages) which provided five to nine points for improvement. Each hospital trust was given the opportunity to comment on the preliminary report before it was finalized and sent to the hospital trust management and published on KAS’ and the Northern RHA’s websites.

Analyses

Descriptive statistical analyses were performed using Microsoft Excel version 16, to present the participating hospital trusts, the interviewees, A&F timeline, auditors’ workload and costs. Data on auditors’ workload were obtained from their working schedules, which included planning, travel, auditing and writing of reports. Costs were calculated by summarizing auditors’ travel expenses and labour costs for all professionals involved in the intervention (auditors, interviewees and administrative staff organizing the visits). Data on the auditors’ travel expenses were obtained from the hospitals’ electronic expense management systems. Since salaries between professions and geographical areas vary, average daily salary expenses for an employee in the Norwegian healthcare system were calculated based on salary expenses for the auditors, including employee costs.

Ethics

The audits were performed as part of the quality improvement of care in healthcare institutions, therefore ethical approval was not required. The participants were informed that the reports would be published on a website, i.e. made publicly available. Data were presented anonymously, with an exception for the Directors of Research and Development who were identifiable in the reports.

Results

Participants

All 22 hospital trusts in Norway agreed to participate and were included in the A&F. In total, 446 leaders and healthcare workers were interviewed. All hospital trusts were represented by a Director of Research and Development, managers, health professionals and AMS team members (Table 2).

Table 2.

Overview of all interviewees (n = 446) in the 22 participating hospital trusts in Norway

Hospital trust	Director of Research and Development	Leaders	Health professionals	AMS team members	Sum
Northern RHA
1	1	4	9	1	15
2	1	4	10	2	17
3	1	4	11	2	18
4	1	3	9	2	15
Central RHA
1	1	3	10	4	18
2	1	6	14	4	25
3	1	2	10	4	17
South-Eastern RHAa
1	1	3	9	6	19
2	1	4	10	5	20
3	1	4	11	5	21
4	1	3	12	4	20
5	1	3	12	5	21
6	1	3	12	3	19
7	1	8	21	6	36
8	1	4	11	6	22
9	1	4	12	5	22
10	1	3	11	6	21
Western RHAb
1	1	6	12	4	23
2	1	3	9	4	17
3	1	4	9	2	16
4	1	4	11	3	19
5	1	6	12	6	25
Total	22	88	247	89	446

Includes two private hospitals.

Includes one private hospital.

Participating health professionals included physicians (senior consultants and residents), nurses and pharmacists. The majority of them worked at departments of medicine, but departments of surgery, intensive care and paediatrics were also represented (Figure 1). The AMS team members were physicians, nurses and pharmacists by profession, though the vast majority were physicians (Figure 2).

Figure 1.

Interviewed health professionals (n = 247) and their affiliation.

Figure 2.

Interviewed members of antimicrobial stewardship teams and their affiliation (n = 89).

Timeline

The audits were performed from October 2017 to April 2019, with one to two visits per month (Figure 3).

Figure 3.

The timeline for the audit and feedback. Letter and number correspond to when the hospital trusts in each regional health trust were visited/audited. Length of box corresponds to time from on-site visit till report/feedback was finalized. N, Northern Regional Health Trust (4 hospital trusts); W, Western Regional Health Trust (5 hospital trusts); C, Central Regional Health Trust (3 hospital trusts); SE, South-Eastern Regional Health Trust (10 hospital trusts).

Overall, the time from the audits being performed to the hospital trusts receiving preliminary reports was a median of 15 days (IQR 9–25 days) and the time to final reports being published was a median of 36 days (IQR 30–49). The median (IQR) time from the audits being performed to preliminary reports being received by the hospital trusts in three southern RHAs and the Northern RHA was 14 days (9–18) and 79 days (61–98), respectively. Time till final reports were published was 33 days (28–38) and 108 days (90–127), respectively. In the Northern RHA, it lasted 137 days (119–156) until the reports were approved by the regional hospital trust board.

Workload and costs

Total workload for all auditors per visit was median 7 days (IQR 6–8).

Total cost of the A&F was €133 952, of which 14% were travel expenses and 86% were salaries for auditors, interviewees and administrative staff (Table 3).

Table 3.

Calculation of audit and feedback costs

Expenses	Sum (€)	Proportion of expenses (%)
Travel	19 405	14
Salary auditors	68 308	51
Salary interviewees	37 091	28
Salary administrative staff	9148	7
Total	133 952	100

Main audit findings and feedback

Our main findings were that ASP structures were put in place in most hospital trusts. Most of them had a formal/written ASP policy document, AMS was identified as a priority objective by the management and a formal organizational multidisciplinary team responsible for AMS was established. All hospital trusts benchmarked their antibiotic consumption in line with the national target of a 30% reduction in use of broad spectrum antibiotics. Additionally, clinicians were familiar with the national guideline on antibiotic use, and they reported that they adhered to the guideline recommendations. However, beyond having these structures put in place there was a general lack of leadership commitment and scarce implementation of AMS interventions. Thus, in general the hospital trusts were given the following feedback: (i) to put in place a governance structure for the programmes; (ii) to set specific targets for improvement of antibiotic use at department and ward level; (iii) to implement evidence-based stewardship interventions, such as A&F of antibiotic use and mandatory review of antibiotic therapy within 48–72 h on the departments and wards; (iv) to define the nurses’ role in the multidisciplinary teams; and (v) to ensure that physicians and nurses had relevant competences and training in antibiotic prescribing and use.

Discussion

When investigating the feasibility and value of applying A&F to evaluate implementation of ASPs in hospital trusts in Norway, we found that with a reasonable use of resources, it was possible to obtain valuable information and provide timely feedback to each hospital addressing explicit targets for further implementation of their ASPs.

To our knowledge, this is the first published evaluation study on A&F used as a systematic approach to assess and support the implementation of ASPs in hospital trusts nationwide. There are studies using surveys to evaluate implementation of ASPs, but these publications do not evaluate the survey methodology’s feasibility and value.,

The audit generated a substantial amount of data, which identified several targets for further implementation of ASPs in Norwegian hospitals. A major finding was the lack of governance structures. This urgently needs to be addressed as hospital leadership commitment is considered to be an important core element for an ASP, providing legitimacy and resources to the programmes., Furthermore, specific targets were rarely defined on the department and ward levels and evidence-based stewardship measures were only sporadically implemented, which is a prerequisite for implementation and impact of an ASP. The limited involvement of nurses in AMS activities represents an opportunity to strengthen the AMS task force, as nurses constitute the majority of healthcare staff and are central in caring for patients with infectious diseases. A reassuring finding from the audit was the reported adherence to the national guideline on antibiotic use, which is associated with favourable patient outcomes and reduced consumption of broad-spectrum antibiotics.

According to a Cochrane review, A&F is most effective when conducted by a supervisor or a colleague. Therefore, the on-site audit team consisted of at least one AMS expert. Furthermore, the impact of an A&F may increase when feedback is delivered both orally and in writing, providing the target audience with well-defined targets and an action plan for improvement. In this approach, the hospital trusts were provided several specific points for improvement, both orally and through the feedback reports, but each hospital trust was responsible for development of an action plan. The literature indicates that timeliness also may influence the effectiveness of an A&F. Ideally, the time period between the audits and the distribution of the feedback reports could have been shorter. However, oral feedback was provided immediately after completion of the on-site audit.

This study indicates that A&F was a feasible and sound methodology to assess and support nationwide implementation of ASPs in Norwegian hospital trusts. Norway has a small population and a mainly publicly funded healthcare system, so our findings may be most valid for similar countries, such as the Nordic countries. Still, regions with healthcare networks and an acknowledged and financed authority within AMS may also benefit from a similar approach. This is highly warranted in order to validate and supplement our findings.

Furthermore, there is a need for studies evaluating the efficacy of the A&F approach to support the implementation of ASPs. From January 2021 onward, KAS has initiated a follow-up of all the Norwegian hospital trusts to get an update on the implementation of ASPs in the hospitals and evaluate the impact of the A&F. The follow-up meetings are standardized by using a semi-structured interview guide when interviewing AMS team members and Research and Development Directors. In a future publication, the more specific A&F findings will be analysed in light of the findings from the follow-up meetings, to identify facilitators and barriers for implementation of ASPs, thereby addressing an important knowledge gap in AMS.

A major strength of this study is that all hospital trusts in Norway participated in the A&F with a substantial number of professionals with diverse backgrounds, providing an extensive amount of data. Another strength is the study’s uniqueness in evaluating A&F as a method to assess nationwide implementation of ASPs in hospitals. A limitation is that the A&F was designed as a quality improvement measure, and not a scientific study, implying retrospective data collection. Furthermore, some of the professionals in the team evaluating the A&F were responsible for the A&F, potentially interpreting data more favourably. This was tentatively handled by including several other professionals in the evaluation team.

Conclusions

In this evaluation study, we found that A&F is a feasible and valuable methodology to assess nationwide implementation of ASPs in hospitals. Furthermore, it can identify relevant targets for improved implementation of the programmes. This approach can be applied in other healthcare settings and may generate new knowledge on the implementation of ASPs.

Click here for additional data file.