Metrics to assess the quantity of antibiotic use in the outpatient setting: a systematic review followed by an international multidisciplinary consensus procedure.

Background: The international Innovative Medicines Initiative (IMI) project DRIVE-AB (Driving Reinvestment in Research and Development and Responsible Antibiotic Use) aims to develop a global definition of 'responsible' antibiotic use.
Objectives: To identify consensually validated quantity metrics for antibiotic use in the outpatient setting.
Methods: First, outpatient quantity metrics (OQMs) were identified by a systematic search of literature and web sites published until 12 December 2014. Identified OQMs were evaluated by a multidisciplinary, international stakeholder panel using a RAND-modified Delphi procedure. Two online questionnaires and a face-to-face meeting between them were conducted to assess OQM relevance for measuring the quantity of antibiotic use on a nine-point Likert scale, to add comments or to propose new metrics.
Results: A total of 597 articles were screened, 177 studies met criteria for full-text screening and 138 were finally included. Twenty different OQMs were identified and appraised by 23 stakeholders. During the first survey, 14 OQMs were excluded and 6 qualified for discussion. During the face-to-face meeting, 10 stakeholders retained five OQMs and suggestions were made considering context and combination of metrics. The final set of metrics included defined daily doses, treatments/courses and prescriptions per defined population, treatments/courses and prescriptions per defined number of physician contacts and seasonal variation of total antibiotic use. Conclusions: A small set of consensually validated metrics to assess the quantity of antibiotic use in the outpatient setting was obtained, enabling (inter)national comparisons. The OQMs will help build a global conceptual framework for responsible antibiotic use.

Introduction

Antibiotics are the cornerstone of the treatment of infectious diseases, but bacterial resistance is a rapidly increasing global threat. An important driver of resistance is antibiotic use. Accepted antibiotic use metrics or units are needed for benchmarking and to evaluate trends over time., Measuring the quantity of antibiotic use is also crucial for designing antibiotic stewardship programmes and target setting by policy makers. In addition, it is a long-standing goal to predict rates of resistance evolution and design optimal treatment strategies accordingly.

Data on outpatient antibiotic use from the EU and countries belonging to the European Economic Area/European Free Trade Association are currently expressed as the number of defined daily doses (DDDs) per 1000 inhabitants and per day; and the number of packages per 1000 inhabitants and per day (PID), depending on the availability of data on packages from the national surveillance networks. In addition, many more different types of quantity metrics have been used worldwide. The Innovative Medicines Initiative (IMI) funded the international project DRIVE-AB (Driving Reinvestment in Research and Development and Responsible Antibiotic Use), which aimed to develop a global framework of ‘responsible’ antibiotic use (http://drive-ab.eu/). This study aimed to identify consensually validated quantity metrics for antibiotic use in the outpatient setting.

Materials and methods

We used a systematic literature and web site review followed by a RAND-modified Delphi consensus procedure to develop and select in a multistep approach a set of consensually validated outpatient quantity metrics (OQMs) as described below. The systematic literature review is reported according to the PRISMA statement.

Systematic literature and web site search and extraction of quantity metrics for antibiotic use

Search strategy

We first performed a systematic literature search using the MEDLINE database to review the international literature for information about quantity metrics for antibiotic use (articles published until 12 December 2014). The search strategy combined three concepts: antibiotics; quantity metrics (split into quantity and metric); and outpatient setting. An additional search included articles that were neither included in the outpatient concept nor in the inpatient concept search on quantity metrics. These articles were found when using the combination of the outpatient and inpatient search on quantity metrics (additional search; see Table S1 available as Supplementary data at JAC Online). Finally, we performed a complementary search based on the combined three concepts and outpatient setting on English web sites of relevant organizations and institutions active within the field of antibiotic stewardship (Table S2, available as Supplementary data at JAC Online).

Inclusion and exclusion criteria

The definition of a quantity metric we used was: ‘a measurable element (generic metrics) of practice performance of the volume or cost of antibiotic use whereby the outcome only gains value in its comparison’. An outpatient was defined as a patient who is not hospitalized and visits a physician in the ambulatory care setting. The definition included also ambulatory patients visiting a physician based in a hospital, which is commonly seen in developing countries. We included studies on antibacterials for systemic use (ATC J01) written in English. Papers without abstracts were included for full-text screening if the title seemed relevant to our search. We excluded publications about topical or vaginal use of antibiotics, antituberculosis drugs, antimalarial antibiotics, antiviral drugs including anti-HIV drugs, anti-helminthics, antifungals, anti-leprosy drugs and anti-Helicobacter drugs, as well as interventional studies and studies concerning very limited patient populations, e.g. cystic fibrosis patients. Finally, we excluded papers whose full text could not be retrieved from any of the libraries of the centres participating in the study (eight different catalogues) or from Google Scholar®.

Screening process, data collection and analysis

We searched for eligible papers, using the literature review management software DistillerSR® (Evidence Partners, Ottawa, Ontario, Canada). Two researchers (A. V. and N. A.) independently performed the extraction of quantity metrics from the selected literature and subsequently independently examined the title and abstract of the publications to include any publication potentially describing quantity metrics for antibiotic use in the outpatient setting, for full-text screening. Articles were systematically included if there was no abstract available, or if the abstract was insufficiently detailed to allow a proper assessment of the eligibility criteria. We also searched the reference list of each article for additional suitable studies. Discrepancies between the results were discussed until consensus was reached.

During the ‘full screening’, we were specifically interested in papers that included measurable elements (generic metrics) of the volume or cost of antibiotic use, papers comparing different quantity metrics for antibiotic use or that reported a correlation between quantity metric(s) and the selection or propagation of bacterial resistance as well as potentially unknown quantity metrics for antibiotic use. Meaningful units were made with a numerator and a denominator, where the numerator measures the amount of antibiotic used and the denominator controls for the size of the population studied. Use of antibiotics was defined as prescribing, buying, dispensing, reimbursing and/or consuming of antibiotics.

The articles were categorized by publication year, whether it concerned national, regional or international data, by socioeconomic setting (low- to middle-income or high-income countries), by study type (prospective or retrospective, cross-sectional or longitudinal), by data source (wholesale, pharmacy, reimbursement, medical records or other), by type of numerator (measurable unit) as well as denominator (population) used, and whether the standardized WHO/ATC method of classification of antibacterials was used.

All quantity metrics were discussed with the senior authors of this study. Duplicate OQMs were deleted and those measuring similar outcomes were grouped, removing overlaps.

RAND-modified Delphi procedure

The list of OQMs extracted from the literature was presented to a multidisciplinary stakeholder panel following the RAND-modified Delphi consensus procedure., It consisted of two online surveys (first and second round) with a face-to-face meeting between them to achieve consensus on a set of key quantity metrics. Stakeholders having the following backgrounds were selected: medical community (medical specialists and clinical pharmacists); public health and patients; antibiotic research and development (R&D); and payers, policy makers, governments and regulators (Table S3). The selection process of the stakeholders is described elsewhere. All the stakeholders consented to participate in the study and were aware that their responses would be used for research purposes.

First questionnaire round

The literature-derived list of OQMs was presented to the stakeholder panel using an internet-based SurveyMonkey® (Palo Alto, CA, USA) questionnaire (Figure S1). The first round of the survey took place between August and September 2015.

All stakeholders assessed each OQM on its relevance on a nine-point Likert scale (1 = clearly not relevant, 9 = clearly relevant, plus a ‘cannot assess’ option). Relevance was graded by the stakeholders in response to the following question: ‘To what extent do you consider this OQM as a relevant metric to measure quantity of antibiotic use in the outpatient setting?’. Stakeholders could comment on proposed OQMs and could also add OQMs.

Relevance scores were calculated for each item. If the OQM had a median ≥7 and ≥70% of the stakeholders scored in the upper tertile, then the OQM was marked as ‘accepted’. If the OQM had a median <7 then the OQM was marked as ‘not accepted’ and was excluded. If the OQM had a median ≥7 and <70% of the stakeholders scored in the upper tertile, then the OQM was marked as ‘to be discussed’.

Consensus meeting

Stakeholders who participated in the first questionnaire round were invited to a face-to-face consensus meeting on 30 September 2015. The results of the analysis of the questionnaires were sent to the stakeholders in advance of the consensus meeting. During the meeting, the stakeholders could comment or rephrase OQMs with the label ‘to be discussed’ and discuss the newly proposed OQMs.

Second questionnaire round

After the consensus meeting, all discussed, reformulated and added potential metrics were included in a second questionnaire. All stakeholders who participated in the first round were asked whether they agreed (yes, no or cannot assess) with the proposed OQMs and their definitions. For each OQM, comments could be added.

Results

Literature search and extraction of quantity metrics for antibiotic use

Out of 597 screened articles based on title and abstract, 138 studies met the criteria for full-text screening. We added another 52 articles through reference checking. Finally, we included 138 articles for detailed review (Figure 1), of which 13 (9.4%) originated from low- to middle-income countries, 42 (30.4%) concerned international data and 49 (35.5%) national data. Thirty-five articles (25.4%) reported the use of two or more metrics, 29 (21.0%) made a link between antibiotic use and resistance and 5 (3.6%) reported the relevance of OQMs for policy making. Figure 2 provides an overview of the obtained OQMs based on the metric in the numerator. The most frequently reported metric was based on the DDDs in the numerator. Other reported metrics included metrics based on cost calculations (tons or number of packages sold), proportions of consultations or encounters for certain diagnoses [e.g. respiratory tract infections, drug utilization 90%, or populations (e.g. children)]. Denominators were grouped into: (i) defined population; (ii) 100 person-years; (iii) physician contacts; and (iv) km2 defined as the area equal to a square of 1 km on each side. All reported metrics were finally grouped into 20 different numerator–denominator combinations for assessment during the first questionnaire round (Table 1). These 20 OQMs were discussed with all researchers involved in this study to make the definition more generic with respect to its definition.

Figure 1.

Flowchart of included and excluded publications. *Additional search included articles that were included neither in the outpatient concept nor in the inpatient concept search on quantity metrics.

Figure 2.

Results on OQMs of antibiotic use by RAND-modified Delphi procedure.

Table 1.

Proposed set of 20 outpatient quantity metrics presented to stakeholders for assessment during the first questionnaire round

Metric number	Proposed metric
1	Defined daily doses per defined population
2	Defined daily doses per 100 person-years
3	Defined daily doses per defined number of physician contacts
4	Defined daily doses per km2
5	Treatments/courses per defined population
6	Treatments/courses per defined number of physician contacts
7	Standard units per defined population
8	Packages per defined population
9	Packages per defined number of physician contacts
10	Prescriptions per defined population
11	Prescriptions per 100 person-years
12	Prescriptions per defined number of physician contacts
13	Individuals treated with antibiotics per defined population
14	Individuals treated with antibiotics per 100 person-years
15	Individuals treated with antibiotics per defined number of physician contacts
16	Kilograms per defined population
17	Antibiotic cost per defined population
18	Average daily quantities per defined population
19	Percentage of antibiotics per total drug use
20	Number of types of antibiotic prescribed in each patient visit

Out of 43 invited stakeholders, 23 (53%) completed the online survey, of whom 15 were based in Europe, 5 in North America, 2 in Asia and 1 in Australia. Detailed information on the stakeholders’ backgrounds is described elsewhere.

The first survey resulted in 0 ‘accepted’ OQMs and 14 OQMs were excluded. Six OQMs qualified for discussion, meaning that the stakeholders did not reach consensus on the relevance or non-relevance of the items; and three new OQMs were suggested (Figure 2). These nine OQMs were further discussed during the consensus meeting.

Ten stakeholders participated in the consensus meeting to discuss the nine OQMs (morning session). Five OQMs were found relevant without modification. The metric ‘individuals treated with antibiotics per defined population’ was not retained due to its limited added value with respect to DDDs, courses and prescriptions. Also, the three suggested OQMs during the first round were not retained. On the other hand, two supplementary metrics on seasonal variation were added to the list of OQMs (Figure 2). Consequently, seven OQMs were found relevant for assessment during the second questionnaire round. During the discussion, suggestions were made considering the context and combination of metrics, especially that the DDD should be complemented with another metric such as treatment, courses or prescriptions in the numerator. When using the DDD, one should also only focus on adults, excluding children because the DDD is based on the average adult dose used for the main indication and refers to the body weight of an adult of 70 kg. Finally, the choice of denominator was discussed.

The list of seven OQMs was sent to the complete stakeholder panel (n = 23). Twenty stakeholders returned the second questionnaire (response rate 87%). Six OQMs were selected by >70% of the panel members, including a new one that was suggested during the consensus meeting (Figure 2).

The final set of outpatient quantity metrics is presented in Table 2. Details about the definitions used are presented in Table S4.

Table 2.

Final set of quantity metrics for the outpatient setting (OQMs), their advantages and disadvantages of use

Outpatient quantity metric		Advantages	Disadvantages
OQM 1	DDDs per defined population	Standardized tool for drug utilization research allowing presentation, comparison and benchmarking of drug consumption statistics at international and other levels.	The DDD provides a fixed unit of measurement independent of price and dosage form and is not suitable for guiding decisions about reimbursement, pricing and therapeutic substitution.
			The DDD is a unit of measurement and does not necessarily reflect the recommended or prescribed daily dose.
			The DDD gives a rough estimate of consumption and not an exact picture of actual use.
			Not suitable for measuring consumption among children and neonates because the DDD reflects the assumed average maintenance dose per day for a drug used for its main indication in adults.
			DDDs are only assigned for drugs that already have an ATC code; a local DDD may be used or its use might be omitted.
			The WHO Collaborating Centre for Drug Statistics Methodology occasionally updates the DDD value for a drug, which might introduce a bias when comparing between years, unless the DDDs are retrospectively corrected.
			Recommended to be used in combination with OQMs 2–5
OQM 2	Treatments/courses per defined population	Counts number of treatments independent of the prescribed doses.	Does not take into account prescribed doses (daily dose and duration of the antibiotic course).
		Good proxy for packages.	If obtained from medical charts, this OQM might overestimate antibiotic use depending on the number of patients not collecting their antibiotic at the pharmacy.
OQM 3	Treatments/courses per defined number of physician contacts	Counts number of treatments independently of the prescribed doses.	Does not take into account prescribed doses (daily dose and duration of the antibiotic course).
		Good proxy for packages.	The denominator allows comparison or benchmarking ‘within countries only’.
			If obtained from medical charts, this OQM might overestimate antibiotic use depending on the number of patients not collecting their antibiotic at the pharmacy.
OQM 4	Prescriptions per defined population	Counts number of treatments independently of the prescribed doses.	Does not take into account prescribed doses (daily dose and duration of the antibiotic course).
		Good proxy for packages.	If obtained from pharmacy data and counting only the prescriptions, this OQM may underestimate antibiotic use depending on the prevalence rate of OTC antibiotic use.
OQM 5	Prescriptions per defined number of physician contacts	Counts number of treatments independently of the prescribed doses.	Does not take into account prescribed doses (daily dose and duration of the antibiotic course).
		Good proxy for packages.	The denominator allows comparison or benchmarking ‘within countries only’.
			If obtained from pharmacy data and counting only the prescriptions, this OQM may underestimate antibiotic use depending on the prevalence rate of OTC antibiotic use.
OQM 6	Seasonal variation of total antibiotic use	Especially useful to measure seasonal variation within a certain time period (e.g. a calendar year) using the numerators as described under OQMs 1–5. Often expressed as a proportion or ratio.	Takes into account each OQM disadvantage as described for OQMs 1–5 (depending on the numerator and denominator).

Discussion

In this study, we selected a set of six key generic quantity metrics for measuring antibiotic use in the outpatient setting, based on scientific literature and a web site search, while applying a systematic RAND-modified Delphi procedure.

The first online survey round reached a response rate of 53%, which is satisfactory because there are many challenges to conducting surveys among physicians (and/or researchers), whose response rates are in general lower than those of the general population. Our response rate was similar to the mean response rate of 54% resulting from a review of 178 manuscripts published in medical journals. The response rate of 87% for the second online survey was very satisfactory. The number of stakeholders (10) participating in the consensus meeting was within the range of 7–15 participants as recommended, large enough to permit diversity of representation while still being small enough to allow everyone to be involved in the group discussion of the proposed OQMs. Diversity of expert panel members leads to consideration of different perspectives and a wider range of alternatives. At the meeting, important comments were made.

Most stakeholders emphasized the need to combine different quantity metrics to optimize interpretation of the volumes of antibiotic use as each of the single metrics had some pitfalls in interpretation. In particular, quantity metrics based on the Anatomical Therapeutic Chemical (ATC) classification and the DDD measurement unit were a point of discussion. Although the DDD is the most commonly used numerator, the feasibility and usefulness of this standardized metric strongly favours its complementation with another metric such as treatments/courses or prescriptions per defined population. We hypothesize that none of the metrics from the review was accepted after the first round for this reason.

When using a combination of metrics, divergent trends in time of antibiotic use may occur, depending on the metric used. This was, for example, observed when comparing outpatient antibiotic use expressed as the number of DDDs with the number of packages, treatments or insured individuals per 1000 inhabitants per day. An increased trend of antibiotic use was observed when using the metric based on the DDD in the numerator, while a decreased trend in time was observed for the other metrics. It is very important to clarify and understand the definition of the metric as contradictory results may confuse policy makers, researchers and the general public. Very often, the concept of a DDD is not well understood. The DDD aims to capture the dosing regimen prescribed to an adult 70 kg patient (single-unit dose, times a day and the duration of the course), but this does not always reflect the actual prescribed doses in some countries. Bruyndonckx et al. further showed that the average content of an antibiotic original package in European countries has significantly increased over time, with substantial differences between countries and antibiotic subgroups, except for the quinolones. This finding had implications in understanding associations between antibiotic use and resistance, which also depended on the metric used. Inconsistent associations and predictions of resistance were observed depending on whether antibiotic use was expressed as ‘defined daily doses per 1000 inhabitants per day (DID)’ or ‘packages per 1000 inhabitants per day (PID)’. The authors concluded that for this reason both metrics, DID and PID, should be used for a better understanding and interpretation of outpatient antibiotic use and its relation to resistance.

Although the two metrics DID and PID are jointly reported by ESAC-Net,, the metric PID, based on packages, was rejected during the first questionnaire round. Instead, prescriptions and treatments (also defined as an antibiotic course with undefined length of intake) in the numerator were put forward, which can be considered as a good proxy for packages. While DDD and packages in general originate from wholesale data, the source of prescription data is pharmacies, whereas treatment/courses data originate from medical charts. For monitoring antibiotic use, countries will need to choose the appropriate metric depending on the availability of data, but also on national legal provisions for the prescription. It is, for example, inappropriate to use a metric based on packages if antibiotics are dispensed by individual blister, single-pill items or reconstituted packs. This is the case in the Netherlands, the UK and Ireland. Table 2 further describes the advantages and disadvantages of each OQM. It may help researchers to correctly decide which OQM to use and should be adapted to their country or local setting of interest.

The definition of the denominator was also discussed during the consensus meeting. ‘Defined number of physician contacts’ as denominator was considered as an unstable denominator that can change over time and might differ a lot between countries depending on local policies and structures. It was concluded by the stakeholders that the two metrics based on this denominator (treatment/courses and prescriptions per defined number of physician contacts) are only useful to compare rates within countries and not to compare or benchmark between countries. Very different kinds of data sources providing denominator data exist. Besides the commonly used WHO mid-year population, covering the whole population, reimbursement data might provide less coverage depending on the proportion of citizens covered by the insurance. The choice of the denominator has implications for the estimation of overall antibiotic use, which might be underestimated for countries using reimbursement data where there is a high prevalence of over-the-counter (OTC) use of antibiotics. Wholesale data, on the other hand, might include export of antibiotics to other countries and this could overestimate overall antibiotic use.

Finally, the metric ‘seasonal variation of total antibiotic use’ (defined as the use of antibiotics in certain periods in a year compared with other periods in a year, marked by specific weather conditions, temperatures or length of day) was added during the consensus meeting and was accepted during the second questionnaire round. Use of antibiotics is herewith expressed as a proportion, most of the time a percentage (Table S4), independently of the kind of numerator measuring the amount of antibiotics used. During the discussion, the issue was raised whether this metric could be considered as a ‘quality indicator’. In this study we defined a quantity metric as a measure that reflects the volume or the costs of antibiotic use, whereas a quality indicator reflects the degree to which an antibiotic prescription is correct or appropriate. Despite their different definition, some overlap or interpretation seemed possible. Consequently, seasonal variation was considered as a quantity metric. Despite lack of evidence, this metric might be a surrogate marker for quality of prescription, since over-prescription in the winter might possibly be the result of unnecessary antibiotic use for viral infections.

The strength of our study is found in our panel of 23 multidisciplinary members from around the world having ample experience in antibiotic stewardship, ensuring optimal measurement of the list of OQMs (face validity). The global background of the panel may encourage transferability to all continents. The stakeholders brought expertise from various fields, such as infectious disease control, policy making, public health administration, contingency planning and public health nursing, and had been involved in regional and national meetings on the subject. Our key OQMs were selected to combine evidence- and consensus-based OQMs, with a strong evidence-based development method as recommended.

A limitation of our study is that we only searched the MEDLINE database for our literature review. We also screened relevant web sites and the reference lists of all included articles. However, this did not result in the inclusion of supplementary references. Importantly, the list of OQMs is merely based on practice and expert opinion. Expert opinion is considered to be the least valuable form of evidence. Our set of OQMs, however, could be the starting point for methodologically sound empirical studies.

Conclusions

It is crucial to use a robust quantity metric enabling (inter)national benchmarking and stability over time to ensure stable longitudinal trends of antibiotic use. We present a small set of consensually validated quantity metrics assessing the quantity of antibiotic use in the outpatient setting. The OQMs will help build an international conceptual framework on responsible antibiotic use.

Click here for additional data file.