How to measure the impacts of antibiotic resistance and antibiotic development on empiric therapy: new composite indices.

OBJECTIVES: We aimed to construct widely useable summary measures of the net impact of antibiotic resistance on empiric therapy. Summary measures are needed to communicate the importance of resistance, plan and evaluate interventions, and direct policy and investment. DESIGN, SETTING AND PARTICIPANTS: As an example, we retrospectively summarised the 2011 cumulative antibiogram from a Toronto academic intensive care unit. OUTCOME MEASURES: We developed two complementary indices to summarise the clinical impact of antibiotic resistance and drug availability on empiric therapy. The Empiric Coverage Index (ECI) measures susceptibility of common bacterial infections to available empiric antibiotics as a percentage. The Empiric Options Index (EOI) varies from 0 to 'the number of treatment options available', and measures the empiric value of the current stock of antibiotics as a depletable resource. The indices account for drug availability and the relative clinical importance of pathogens. We demonstrate meaning and use by examining the potential impact of new drugs and threatening bacterial strains.
CONCLUSIONS: In our intensive care unit coverage of device-associated infections measured by the ECI remains high (98%), but 37-44% of treatment potential measured by the EOI has been lost. Without reserved drugs, the ECI is 86-88%. New cephalosporin/β-lactamase inhibitor combinations could increase the EOI, but no single drug can compensate for losses. Increasing methicillin-resistant Staphylococcus aureus (MRSA) prevalence would have little overall impact (ECI=98%, EOI=4.8-5.2) because many Gram-positives are already resistant to β-lactams. Aminoglycoside resistance, however, could have substantial clinical impact because they are among the few drugs that provide coverage of Gram-negative infections (ECI=97%, EOI=3.8-4.5). Our proposed indices summarise the local impact of antibiotic resistance on empiric coverage (ECI) and available empiric treatment options (EOI) using readily available data. Policymakers and drug developers can use the indices to help evaluate and prioritise initiatives in the effort against antimicrobial resistance. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://www.bmj.com/company/products-services/rights-and-licensing/.

Provides novel and clinically meaningful summary measures of our current ability to provide empiric coverage, and of the available stock of antibiotics.

Uses widely available cumulative antibiogram data.

Accounts for variation in resistance, the relative clinical importance of pathogens, and the availability of drugs among infection types and contexts.

Does not account for variation in toxicity, cost, or in vivo efficacy among drugs that are considered appropriate for empiric therapy.

Does not account for variation in the propensity of pathogens to acquire resistance.

Introduction

Efforts to understand the epidemiology and burden of antibiotic resistance have largely focused on a few sentinel multidrug-resistant organisms,1–3 but this strategy is increasingly unfeasible given the increasing prevalence and diversity of resistance. There is a need to summarise and understand the net clinical impact of resistance—accounting for variation in strain prevalence and drug availability—to focus attention on the most pressing problems.

Strategies for minimising the impact of antibiotic resistance on patients include introducing new drugs, preventing the spread of threatening strains, and reducing unnecessary antibiotic use.3–9 A central challenge is that strategies may be effective and appropriate in one context, but not in another, depending on the incidence and virulence of resistant strains, the type of infections, the consequences of failed treatment, and the availability of treatment options. In order to better understand variation in the effectiveness of stewardship strategies, we need to better understand variation in the impact of resistance. We propose two indices10–12 that can be used to summarise the local impact of resistance on empiric treatment of common bacterial infections, and to assess threats and interventions in that context. The indices account for resistance, drug availability and the relative importance of pathogens, and can be easily calculated from readily available cumulative antibiogram data.13

We concentrate our attention on the failure of empiric therapy because it is the most common problem caused by resistance in many contexts.14 15 Empiric therapy may fail microbiologically (ie, related to bug-drug mismatch) because resistance is prevalent, because clinicians prescribe antibiotics contrary to available evidence, or because patients do not follow prescriptions. Although poor treatment decisions, patient adherence and resistance each deserve attention, the problems require different solutions, so we do not include poor treatment or adherence in our indices. Our approach differs in this respect from the drug resistance index (DRI) proposed by Laxminarayan and Klugman.10 The DRI can be interpreted as the probability of inadequate treatment given observed drug use.11 In contrast, we estimate potential empiric coverage by assuming that the empiric therapy that is most likely to succeed is selected, informed by cumulative antibiograms, Gram stains, and knowledge of the likely causative organism(s) for each clinical syndrome.16–18 This allows investigation of new drugs, new resistant strains and other cases in which drug-use data are not available.

We propose two complementary indices: an Empiric Coverage Index (ECI) to measure available empiric coverage of common infections, and an Empiric Options Index (EOI) to measure the empiric value of the current stock of antibiotics on the understanding that antibiotic resources are inevitably diminished by use.19 Together, the ECI and EOI measure current difficulties and vulnerability to increasing resistance, providing a clinically meaningful summary of antibiotic resistance that can be used to assess the real and potential impacts of threats and interventions. As an example, we consider device-associated infections in North American intensive care units (ICUs), but the indices can be adapted for any population for which cumulative antibiogram data are available.13

Materials and methods

Common infections and their causes

The impact of resistance depends on the incidence of infections and their causes, but these data are not readily available at most institutions. Instead, we have a general understanding of the causes of infections. Economists solve an analogous problem using classic fixed-basket consumer price indices.20 These indices measure the average price of a standard basket of consumer goods weighted by the relative importance of each good, ignoring variation in the set of goods consumed in order to focus on price variation. Following a similar logic, we construct a standard basket of infections in order to examine variation in resistance. Our basket consists of common infections, with weights indicating infection frequency.

Proceeding with a critical care example, we construct a basket of device-associated infections consisting of central-line associated bloodstream infections (CLABSI), catheter-associated urinary tract infections (CAUTI), and ventilator-associated pneumonia (VAP). These represent a substantial portion of serious bacterial infections in ICUs,21 22 and relatively good surveillance and aetiology data are available worldwide.2 23–25 Infections in the basket are weighted using data from the US National Healthcare Safety Network (figure 1).23 Surveillance of CLABSIs suggests that aetiology is similar in Canada and the US.23 24 Indices can be easily constructed for other contexts by defining an appropriate basket of infections, and the basket can be updated as new surveillance data becomes available. Local aetiology data can also be used where available.

Figure 1

A basket of device-associated bacterial infections. The height of each bar shows the relative frequency of common species associated with CLABSIs, VAPs, and CAUTIs in US acute care hospitals.23 The width of each bar indicates the relative frequency of each infection type in adult critical-care units.23 The area of each bar therefore shows the proportion of device-associated infections that are of a particular type, caused by a particular species or species group. CoNS are coagulase-negative Staphylococci. CAUTIs, catheter-associated urinary tract infections; CLABSIs, central-line associated bloodstream infections; VAPs, ventilator-associated pneumonia.

Resistance to available drugs

As an example, we used the 2011 cumulative antibiogram from the ICUs of a large Toronto teaching hospital, constructed according to guidelines of the Clinical and Laboratory Standards Institute (CLSI).13 26 The antibiogram includes the first clinical isolate from each patient, as recommended by CLSI to guide initial empiric therapy.13 Blood, respiratory, urine and skin and soft-tissue infections were included. Resistance data are widely available in this format.13

To determine the set of drugs available for empiric treatment of device-associated infections, we excluded unsuitable and unavailable options from Magiorakos et al27 (table 1). Although aminoglycosides are avoided in most circumstances due to toxicity and poor efficacy, they have been included because they are used in empiric therapy for severe infections.28 Aminoglycosides were considered individually because many strains are resistant to only a subset of drugs in the class.29 We also include drugs that may be reserved for treatment of difficult infections, including carbapenems, linezolid, daptomycin and antipseudomonal cephalosporins,30 31 and then consider a scenario omitting these drugs. Cross-resistance within other classes is common but not universal, so the selection of class representatives is somewhat subjective. We selected class representatives from the antibiogram26 to construct indices using available data (table 1).

Table 1

The set of available drugs for empiric treatment of device-associated infections

Drug or class	GN	GP	Notes
Amikacin	y	x
Gentamicin	y	x
Tobramycin	y	x
Rifampin	x	x	Not for monotherapy.
Fluoroquinolones	y	y	Represented by ciprofloxacin.
Tmp-smx	y	y	Trimethoprim-sulfamethoxazole.
Glycopeptides	x	y	Represented by vancomycin.
Tigecycline	tox	tox
Clindamycin	x	*	*Not for CAUTI.
Daptomycin	x	*	*Not for CAUTI or VAP. Only approved for S. aureus.
Erythromycin	x	*	*Not for CLABSI or CAUTI.
Nitrofurantoin	*	*	*Not for CLABSI or VAP.
Linezolid	x	y
Chloramphenicol	tox	tox
Fosfomycin	*	*	*Not for CLABSI or VAP.
Polymixins	tox	x	Represented by colistin.
Quin-dalf	x	x	Quinupristin-dalfopristin.
Tetracyclines	*	*	Represented by doxycycline. Not for CLABSI or VAP.
Oxacillin/clox	x	x	Represented by cloxacillin. Only for Staphylococcus.
Ceftaroline	x	x	Unclear for VAP.36
Penicillins	y	y	Represented by ampicillin. Penicillinase-labile penicillins.
CephN	y	y	Cephalosporins I+II and cephamycins represented by cefazolin.
Cefixime	x	x
CephE	y	CephN	Cephalosporins III+IV represented by ceftazidime.
ComboN	y	y	Penicillins+β-lactamase inhibitors represented by amox-clav.
ComboE	y	y	Antipseudomonal penicillins+β-lactamase inhibitors represented by pip-tazo.
Carbapenems	y	y	Represented by meropenem.

A ‘y’ indicates a drug or class that is suitable for treatment of susceptible Gram-negative (GN) or Gram-positive (GP) central-line associated bloodstream infections (CLABSIs), ventilator-associated pneumonia (VAPs) and catheter-associated urinary tract infections (CAUTIs).

*Indicates a drug or class that is not suitable for treatment of some infection types due to poor tissue penetration or other considerations—see notes. Unsuitable x and toxic drugs are excluded from the indices. In addition, netilmicin, fusidic acid, teicoplanin are excluded due to lack of FDA approval, and aztreonam is excluded because it is not available in Canada.

Antibiotic susceptibility testing was not done when resistance was intrinsic (table 2), implied by other test results (table 3), or known to be rare or common.26 These gaps in the data were filled using general knowledge (see online supplementary appendix A).26 27 32–35

Table 2

Intrinsic resistance among bacterial causes of device-associated infections

Drug	Class	Escherichia coli	Enterobacter	Klebsiella	Proteus	Serratia	Acinetobacter baumannii	Pseudomonas aeruginosa	CoNS	Enterococcus faecalis	Enterococcus faecium	Staphylococcus aureus
Amikacin		y	y	y	y	y	y	y	y	x	x	y
Gentamicin		y	y	y	y	y	y	y	y	y	y	y
Netilmicin		y	y	y	y	y	y	y	y	x	x	y
Tobramycin		y	y	y	y	y	y	y	y	x	x	y
Rifampin		x	x	x	x	x	x	x	y	y	x	y
Ciprofloxacin	Fluoroquinolones	y	y	y	y	y	y	y	y	y	x	y
Levofloxacin	Fluoroquinolones	y	y	y	y	y	y	y	y	y	x	y
Moxifloxacin	Fluoroquinolones	y	y	y	y	y	y	y	y	y	y	y
Tmp-smx		y	y	y	y	y	y	x	y	x	x	y
Fusidic acid		x	x	x	x	x	x	x	y	x	x	y
Teicoplanin	Glycopeptides	x	x	x	x	x	x	x	y	y	y	y
Telavancin	Glycopeptides	x	x	x	x	x	x	x	y	y	y	y
Vancomycin	Glycopeptides	x	x	x	x	x	x	x	y	y	y	y
Tigecycline		y	y	y	x	y	y	x	y	y	y	y
Clindamycin		x	x	x	x	x	x	x	y	x	x	y
Daptomycin		x	x	x	x	x	x	x	y	y	y	y
Erythromycin		x	x	x	x	x	x	x	y	y	y	y
Nitrofurantoin		y	y	y	x	x	x	x	y	y	y	y
Linezolid		x	x	x	x	x	x	x	y	y	y	y
Chloramphenicol		y	y	y	y	y	x	x	y	y	y	y
Fosfomycin		y	y	y	y	y	x	y	y	y	x	y
Colistin	Polymixins	y	y	y	x	x	y	y	x	x	x	x
Polymixin B	Polymixins	y	y	y	x	x	y	y	x	x	x	x
Quin-dalf		x	x	x	x	x	x	x	y	x	y	y
Doxycycline	Tetracyclines	y	y	y	y	y	y	x	y	y	y	y
Minocycline	Tetracyclines	y	y	y	y	y	y	x	y	y	y	y
Tetracycline	Tetracyclines	y	y	y	x	y	y	x	y	y	y	y
Aztreonam		y	y	y	y	y	x	y	x	x	x	x
Cloxacillin	Oxacillin/clox	x	x	x	x	x	x	x	y	x	x	y
Oxacillin	Oxacillin/clox	x	x	x	x	x	x	x	y	x	x	y
Ceftaroline		y	y	y	y	y	y	x	y	y	x	y
Ampicillin	Penicillins	y	x	x	y	x	x	x	y	y	y	y
Penicillin	Penicillins	x	x	x	x	x	x	x	y	y	y	y
Cefazolin	CephN	y	x	y	y	x	x	x	y	x	x	y
Cefuroxime	CephN	y	x	y	y	x	x	x	y	x	x	y
Cefotaxime	CephE	y	y	y	y	y	y	x	y	x	x	y
Ceftazidime	CephE	y	y	y	y	y	y	y	y	x	x	y
Ceftriaxone	CephE	y	y	y	y	y	y	x	y	x	x	y
Cefepime	CephE	y	y	y	y	y	y	y	y	x	x	y
Cefotetan	CephN	y	x	y	y	y	x	x	y	x	x	y
Cefoxitin	CephN	y	x	y	y	y	x	x	y	x	x	y
Amox-clav	ComboN	y	x	y	y	x	x	x	y	y	y	y
Amp-sul	ComboN	y	x	y	y	x	y	x	y	y	y	y
Pip-tazo	ComboE	y	y	y	y	y	y	y	y	y	y	y
Tic-clav	ComboE	y	y	y	y	y	y	y	y	y	y	y
Caz-avi		y	y	y	y	y	y	y	y	x	x	y
Cef-tazo		y	y	y	y	y	y	y	y	x	x	y
Doripenem	Carbapenems	y	y	y	y	y	y	y	y	y	x	y
Ertapenem	Carbapenems	y	y	y	y	y	y	x	y	y	x	y
Imipenem	Carbapenems	y	y	y	y	y	y	y	y	y	x	y
Meropenem	Carbapenems	y	y	y	y	y	y	y	y	y	x	y

‘y’ indicates a potentially effective drug, and ‘x’ indicates intrinsic resistance. See online supplementary appendix A for references and discussion. ‘CephN’, ‘CephE’, ‘ComboN’ and ‘ComboE’ classes are defined in table 1.

Table 3

Cases in which resistance to one drug implies resistance to others

Species/strain	Rule
All	If S to tetracycline then S to doxycycline, and minocycline (ref. 26, p.36).
All (except Staphylococcus)	If S/R to penicillin then S/R to penicillinase-stable penicillins (FDA label).
Enterics	If S/R to ampicillin then S/R to amoxicillin (ref. 26, p.45).
Enterics	If S/R to cephalothin then S/R to cephalexin (ref. 26, p.45).
Enterics	If S/R to ceftriaxone then S/R to cefotaxime (ref. 26, p.34).
Enterics	Ceftriaxone represents cephalosporins III.23
KPC*	Ertapenem represents carbapenems.33
KPC*	If R to ertapenem then R to chloramphenicol, and ceftriaxone.33
KPC	If R to ertapenem then R to ceftolozane.tazobactam.37
ESBL/AmpC/KPC*	If R to ceftriaxone then R to penicillins, penicillinase-stable penicillins, cephalosporins I, cephalosporins II, cephalosporins III, β-lactams+β-lactamase inhibitors (ref. 33, 38 p.15, 39).
ESBL/AmpC/KPC*	If R to ceftriaxone then R to fluoroquinolones, and trimethoprim.sulfamethoxazole.39
Staphylococcus	Penicillin represents penicillins (ref. 26, p.37).
MRSA/CoNS	Cefoxitin is a surrogate for oxacillin (ref. 26, p.37).
MRSA/CoNS	Oxacillin represents penicillinase-stable penicillins (ref. 26, p.37).
MRSA/CoNS	If R to oxacillin then R to penicillins (ref. 26, p.37).
MRSA/CoNS	If S/R to oxacillin then S/R to β-lactams+β-lactamase inhibitors, cephalosporins+β-lactamase inhibitors, carbapenems, cephalosporins I, cephalosporins II, cephalosporins III, cephalosporins IV, cephalosporins oral, and cephamycins (ref. 26, p.37).
Staphylococcus*	If S/R to vancomycin then S/R to telavancin.40
MRSA*	If R to oxacillin then R to aminoglycosides, fluoroquinolones, daptomycin, and chloramphenicol.33
Enterococcus	If S to penicillin then S to ampicillin (ref. 26, p.91).
Enterococcus	If S/R to ampicillin then S/R to amoxicillin, piperacillin, β-lactams+β-lactamase inhibitors (ref. 26, p.91).
VRE*	If R to vancomycin then R to rifampin, fluoroquinolones, nitrofurantoin, chloramphenicol, fosfomycin, quin-dalf, β-lactams+β-lactamase inhibitors, carbapenems, penicillins, and penicillinase-stable penicillins.33
VRE*	If S/R to vancomycin then S/R to glycopeptides.33
VRE*	If S/R to vancomycin then S/R to telavancin.40
Enterococcus faecalis	If S/R to ampicillin then S/R to imipenem (ref. 26, p.91).

*Indicates a pattern that is common but not ubiquitous so exceptions may be found. ‘R’ indicates resistant and ‘S’ indicates susceptible. Rules were applied iteratively until no more changes occurred, except in cases of intrinsic resistance (table 2). VRE is vancomycin-resistant Enterococcus, CREs are carbapenem-resistant Enterobacteriaceae that produce the KPC enzyme, ESBLs are Enterobacteriaceae that produce extended-spectrum β-lactamase, and MRSA is methicillin-resistant Staphylococcus aureus.

Index construction—empiric coverage and loss of options

Index construction begins by calculating coverage, defined as the proportion of infections of each type in the basket (figure 1) that are susceptible to each available drug (figure 2, online supplementary appendix B). Empiric therapy of severe device-associated infections often includes two drugs, but combination antibiograms15 41 42 are not readily available at most institutions. To get a rough, conservative measure of the coverage provided by drug combinations, we assume that treatment is informed by Gram stains.18 We also assume that each infection is caused by a single species.

Figure 2

Coverage of the basket of device associated infections in the ICUs of a large Toronto hospital. Drugs are identified by the first five letters of each name (table 2). Light bars show current coverage, medium grey shows uncertainty, and dark shows loss of coverage due to acquired resistance.

We then define the ECI for common device-associated infections as the percentage of infections in the basket (figure 1) that are covered by empiric therapy, provided that each infection is treated with the drug that is most likely to provide coverage (see online supplementary appendix B). In cases where coverage is uncertain due to missing resistance data (table 4), we estimate a range, assuming susceptibility in the best case and resistance in the worst case. Coverage in a ‘utopian’ scenario is obtained by assuming that all species are 100% susceptible, except in cases of intrinsic resistance. This scenario shows what has been lost due to acquired resistance.

Table 4

Resistance in the ICUs of a large Toronto hospital in 2011

	Escherichia coli n=85	Klebsiella n=56	Enterobacter n=42	Serratia n=15	Proteus n=7	Acinetobacter baumannii n=5	Pseudomonas aeruginosa n=89	CoNS n=127	Staphylococcus aureus n=102	Enterococcus faecalis n=12	Enterococcus faecium n=21
Amikacin	0	0	0	0	0	20	11	.	.	.	.
Tobramycin	16	6	2	0	0	0	11	.	.	.	.
Gentamicin	15	6	2	0	0	0	27	.	.	.	.
Ciprofloxacin	37	6	5	0	0	0	33	m	≥15	100	.
Tmp-smx	26	14	21	0	0	0	.	47	1	.	.
Vancomycin	.	.	.	.	.	.	.	0	0	0	48
Clindamycin	.	.	.	.	.	.	.	67	45	.	.
Daptomycin	.	.	.	.	.	.	.	0	0	0	0
Erythromycin	.	.	.	.	.	.	.	75	48	100	100
Nitrofurantoin	3	60	83	.	.	.	.	0	0	m	100
Linezolid	.	.	.	.	.	.	.	1	0	0	0
Fosfomycin	m	m	m	m	m	.	100	m	m	m	.
Doxycycline	41	20	33	100	100	m	.	26	3	100	50
Ampicillin	67	.	.	.	14	.	.	100	100	0	100
Cefazolin	38	≥9	.	.	m	.	.	81	15	.	.
Ceftazidime	22	9	41	0	0	100	24	.	.	.	.
Amox-clav	36	14	.	.	14	.	.	81	15	0	100
Pip-tazo	25	0	41	0	m	0	18	81	15	0	100
Caz-avi	0	0	0	0	0	74	4	.	.	.	.
Cef-tazo	0	9	5	0	0	60	7	.	.	.	.
Meropenem	0	0	0	0	0	0	23	81	15	m	.

Numbers are the percentage of resistant isolates. ‘m’ indicates a missing value, italics indicate an inferred value (see online supplementary appendix A, table 3), and ‘.’ indicates information that is not required for calculation of the indices.

Antibiotics are a resource that is depleted by use.1 3 19 34 Our EOI measures the value of available treatment options from this perspective. The EOI is constructed on the assumption that repeated use of drugs leads to resistance, so there are a finite number of patients that can be successfully treated before a drug becomes completely ineffective. The EOI is then defined as the number of patients that can be covered by a set of drugs, relative to the number that can be covered by a single fully effective drug for each syndrome (figure 3, online supplementary appendix B). For simplicity, we assume that a linear relationship between drug use and resistance applies to all drugs, that the set of drugs that provide highest coverage are used in equal proportions, and that the total number of infections remains constant over time. Our EOI formulation allows antibiotic mixing to slow the spread of resistance (see online supplementary appendix B),43 but in this analysis, we assume the only advantage of using multiple drugs is that we use less of each.

Figure 3

The EOI is based on this very simple model of the change in coverage over time (equation A.3). In this example, the best available coverage of Gram-negative CLABSIs (first panel of figure 2) declines rapidly at first, then more slowly as more drugs are used. The amount of time until all drugs become ineffective depends on model parameters β and n, but the shape of the curve depends only on the initial coverage provided by each available drug (first panel of figure 2). CLABSIs, central-line associated bloodstream infections; EOI, Empiric Options Index.

The Empiric Therapy Indices (ETI) R package includes methods for calculating the indices. Find instructions for installation and use at https://github.com/josie-hughes/EmpiricTherapyIndices/blob/master/Example.R.

Results

The ECI measures our ability to provide empiric coverage of common bacterial infections within a population. An ECI of 100% indicates that at least one drug provides full coverage of each infection type in the basket, and an ECI of 0% indicates no empiric coverage. The current ECI at our site remains high (98%) because susceptibility to vancomycin, linezolid, meropenem and the aminoglycosides remains high (figure 4). However, the calculation includes drugs that are often reserved for difficult infections or those associated with a high risk of complications.30 31 If we chose to restrict the use of carbapenems, linezolid, daptomycin, antipseudomonal cephalosporins and aminoglycosides for empiric therapy, the ECI would be 86–88% (figure 4). Uninformed by a Gram-stain, treating with a single non-reserved drug further reduces the ECI to between 69% and 77%.

Figure 4

Variation in indices among scenarios. ‘utopian’ shows the best possible case with no acquired resistance and ‘current’ shows the situation in the ICUs of a large Toronto hospital in 2011. Other scenarios show the potential impact of resistant strains, new drugs, and formulary restriction in the current context. ‘Reserve drugs’ in this example include carbapenems, linezolid, daptomycin, antipseudomonal cephalosporins, and aminoglycosides.

The index accounts for the importance of resistant strains as causes of infection, so is most threatened by common pathogens that are resistant to the few drugs that still provide good empiric coverage; for example, plasmid-mediated carbapenem resistance among Enterobacteriaceae poses a major threat to the ECI (100% KPC in figure 4). Pan-drug-resistant A. baumannii are similarly resistant to remaining empiric drugs, but would have less impact (100% PDR Acinetobacter in figure 4) because A. baumannii causes fewer infections (figure 1). The ECI is not altered by increasing resistance to drugs that provide less coverage; for example increasing MRSA prevalence would not alter the ECI or the empiric effectiveness of vancomycin and linezolid (100% MRSA in figure 4).

The EOI measures the value of the set of available antibiotics, including drugs that provide less coverage. The EOI reaches a maximum value equal to the number of available treatment options if all drugs provide full coverage, and can therefore be interpreted as the equivalent number of fully effective treatment options. We assume that more treatment options are better because drug use leads to resistance. We also assume that partially effective drugs have some value, but that a single drug providing 100% coverage is better than two drugs providing 50% coverage. Thus, a higher EOI is assigned when the best available coverage is high, all else being equal (compare rows in figure 5). Essentially, a low ECI indicates an urgent problem with failing empiric therapy, while a low EOI indicates treatment vulnerability. Without acquired resistance, the EOI in our example ICU would have been 9.4 at our site, but now it is 5.3–5.9 (utopian vs current scenarios in figure 4). We can still provide coverage, but 37–44% of treatment potential measured by the EOI has been lost (1−5.3/9.4=0.44) as resistance has developed to nearly all available drugs (figure 2). This analysis assumes the only advantage of using multiple drugs is using less of each. If mixing also slows the spread of resistance,43 then redundancy is more valuable and our losses are even more concerning (see online supplementary appendix C figure C.1).

Figure 5

The effect of the distribution of resistance on the EOI. In each case, there are Dy=3 treatment options. When all drugs provide full coverage EOIy=Dy=3 (right column). Reducing the average coverage has the largest effect when all drugs provide the same coverage (top row), and the smallest effect when drugs provide full coverage (bottom row). EOI, Empiric Options Index.

New antibiotics can reduce vulnerability and increase the EOI, but not all new antibiotics improve our ability to empirically cover common infections. For example, dalbavancin, oritavancin and tedizolid are closely related to existing drugs and do not provide additional coverage,3 4 so would have no impact on the EOI. Ceftolozane-tazobactam and ceftazidime-avibactam provide additional coverage of multidrug-resistant Gram-negatives.37 44 45 Provided they prove useful for CLABSIs and VAPs,3 46 both drugs could increase the EOI by providing good coverage of Gram-negative infections, but neither drug would alter the ECI at our site because amikacin and meropenem remain effective (figure 4). However, in a context where carbapenemase (KPC)-producing Enterobacteriaceae are common, ceftazidime-avibactam could mitigate their impact on the ECI (100% KPC with and without ceftazidime-avibactam in figure 4).

Empiric therapy and the EOI are most threatened by common pathogens that are resistant to the few drugs that still provide good empiric therapy. For example, S. aureus is a common cause of disease (figure 1), but MRSA and resistant CoNS have already rendered many drugs relatively ineffective for empiric therapy of Gram-positive infections (figure 2), so further increasing the prevalence of MSRA would not have a large impact (100% MRSA in figure 4 and online supplementary supplementary appendix C figure C.2), even though S. aureus causes more disease (figure 1), because VRE threatens the empiric effectiveness of vancomycin. Increasing aminoglycoside resistance among Gram-negative strains could substantially reduce the EOI (Gram-negatives 100% resistant to aminoglycosides in figures 4 and online supplementary appendix C figure C.2) because aminoglycosides are among the few drugs that provide good empiric coverage of Gram-negative infections.28

Discussion

Efforts to understand the epidemiology and burden of resistance have largely focused on a few sentinel multidrug-resistant organisms.1–3 However, the selection of focal strains does not always reflect clinical significance.47 As an alternative, we propose indices that summarise the impact of resistance on empiric therapy. These indices can draw attention to strains such as aminoglycoside-resistant Gram-negatives which threaten empiric therapy28 but do not receive much attention. We do not know the collective impact of delayed therapy if aminoglycosides lose effectiveness but we suspect it may be comparable to other more widely recognised threats.1 14

We examined the potential impacts of new drugs and resistant strains on our indices in the intensive care units (ICUs) of a Toronto hospital. Results confirm the indices are reasonable. Empiric therapy is most threatened by common pathogens that are resistant to the few drugs that currently provide good empiric coverage. These considerations are not new,1 3 5 6 8 16 47 but our indices make the assessment process explicit, transparent, quantitative and easily repeatable.

The major advantage of our approach over alternative index-based approaches10–12 is investigation of scenarios for which drug use data are not available. Our indices can thus provide insight into the potential impact of threats and interventions in a local context. For example, in the context of the ICUs of a large Toronto hospital in 2011, increasing prevalence of MRSA does not pose a major threat to empiric therapy because MRSA is already common. However, this conclusion is context-dependent; before MRSA became common,48 it posed a major threat to the empiric effectiveness of several important drug classes. The value of the new drug ceftazidime-avibactam is also context dependent; the drug would not substantially alter our ECI or our current ability to provide empiric coverage, but could be invaluable where carbapenemase (KPC)-producing Enterobacteriaceae have reduced the empiric effectiveness of carbapenems and aminoglycosides.3

Identifying threats is useful, but the pressing question is what to do about them. Broadly, index values can be improved by adding new drugs or reducing resistance to available drugs. Rapid identification of the infecting species18 could also improve index values by converting syndrome-based infection baskets (eg, VAP) to organism-based baskets (eg, Pseudomonas VAP), but if rapid tests preclude phenotypic resistance testing, the change could also reduce our ability to predict empiric coverage. Unfortunately, we do not know enough about the effectiveness of many stewardship and infection control strategies,5–9 47 49 in part because it is unclear what outcomes are most relevant, and in part because strategies may be effective in one context, but not in another. We hope our indices prove to be relevant outcome measures that help resolve apparent differences by explicitly acknowledging context-dependence.

Our example indices are based on a basket of device-associated infections that we believe to be representative in our context.23 However, other infections are important, and the causes of infections can vary substantially over space and time.21–23 50–54 We encourage users to choose an appropriate basket of infections for their context using local data where available, and caution against using our coverage estimates to guide clinical decisions. The weighted-incidence syndromic combination antibiogram (WISCA)15 41 42 is a more appropriate measure of coverage in a clinical context because it accounts for local variation in disease aetiology and reports the coverage provided by drug combinations. In vivo efficacy, side effects, previous drug exposure and other patient risk factors, previous culture results, rapid diagnostic results, administration route, costs and ecological impacts should also be considered.6 16 18 55–57

We also encourage users to keep assumptions in mind when interpreting results. For example, variation in the toxicity, effectiveness and ecological impacts of drug are important when considering the potential value of new drugs3 4 for empiric therapy, but our indices do not include this information. Neither ceftazidime-avibactam nor ceftolozane-tazobactam would substantially alter the ECI in our ICUs because other drugs still provide good empiric coverage in this context. However, ceftolozane-tazobactam may be preferable to aminoglycosides due to considerations of toxicity and efficacy28 and may provide an alternative to carbapenems that helps reduce selection for carbapenem resistant strains. Ceftazidime-avibactam should be reserved for treatment of carbapenem-resistant infections.

Despite simplifying assumptions, index-based approaches can yield useful information about the burden of resistance; Ciccolini et al11 successfully predicted the appropriateness of empiric therapy of complicated urinary tract infections using a drug resistance index (DRI).10 Poor treatment decisions and consideration of allergies, side effects and ecological impacts all reduce empiric coverage, so we expect the actual percentage of appropriate empiric therapy to be lower than our ECI.14 15 DRIs take these other considerations into account,10–12 providing complementary information to our ECI. One could distinguish inappropriate treatment attributable to resistance from inappropriate treatment attributable to poor drug choices and other considerations by comparing the ECI and DRI.

In our example of ICUs, the ECI remains high. This does not mean we are free of resistance problems. Remaining drugs are often more expensive,3 less effective28 or more toxic28 than alternatives, and some should perhaps be reserved for treatment of difficult infections.30 31 Administering multiple drugs to cover Gram-negative and Gram-positive bacteria increases costs and risks. Restricting the use of reserved drugs would reduce the ECI by more than 10%, implying that we now rely on reserved drugs for empiric therapy and that drug restriction would reduce the empiric coverage. Treating with a single non-reserved drug would further reduce the ECI, implying that dual therapy is also now necessary. Resistance testing, surveillance, infection control, prevention and stewardship interventions are also costly.3 5–9 13 Empiric and definitive therapies are often inappropriate,14 15 and difficult multidrug-resistant infections do occur.1 Our resistance rates are conservative because they use microbiology data from first isolates only, and therefore under-report resistance acquired during ICU stay which may or may not be due to antimicrobial therapy.13 The ECI only measures the impact of resistance on our ability to empirically cover initial infections.14

History suggests that the development of resistance to widely used drugs is inevitable;1 3 34 48 antibiotics are a resource that continue to be diminished by use.19 Although we can still provide empiric coverage in our ICUs, the losses are concerning, and measuring them is important. However, the value of our current stock of antibiotics depends on future development of resistance, which in turn depends on many factors,58 making measurement difficult. In order to construct the EOI, we assumed a very simple treatment model and a very simple linear relationship between drug use and resistance that applies equally to all drugs. The EOI formulation allows mixing to slow the spread of resistance,43 but we ignored the possibility in this analysis because parameter values are unknown. Increasing the value of redundancy would make our losses even more concerning. This simple linear model also likely undervalues drugs that provide full coverage because the effects of selection are generally smaller when the frequency of resistance is low.59 The use of one drug can select for resistance to others, but it is difficult to predict the precise relationships. Our simple model assumes resistance to each drug class develops independently. It would be interesting to investigate more realistic models58 but they would necessarily be less general. Our assumptions yield an EOI that depends only on coverage and is easy to calculate, while still capturing the understanding that more treatment options are better because drug use leads to resistance.

The indices we present here apply only to ICUs in North America, but our approach can easily be generalised to other contexts by defining an appropriate basket of infections and set of available drugs. Unfortunately, we do not expect a high ECI in all contexts. Across the globe, resistance rates vary widely, and the prevalence of carbapenem-resistant K. pneumoniae already exceeds 50% in some places3 which would substantially reduce the ECI. Drug availability also varies because the high cost of newer drugs can be prohibitive.3 We generally also expect a lower ECI for surgical prophylaxis60 and uncomplicated outpatient infections because none of the drugs that provide good empiric coverage in our ICUs are used for prophylaxis or empiric primary care. Options for treatment of severe hospital infections in North America are dwindling, but we can still provide empiric coverage, provided we use at least two drugs, aminoglycosides, and expensive last-resort drugs. Outside of ICUs and in many other parts of the world, empiric therapy is no longer adequate.