The Use of Bloodstream Infection Mortality to Measure the Impact of Antimicrobial Stewardship Interventions: Assessing the Evidence.

This review sets out to evaluate the current evidence on the impact of inappropriate therapy on bloodstream infections (BSI) and associated mortality. Based on the premise that better prescribing practices should result in better patient outcomes, BSI mortality may be a useful metric to evaluate antimicrobial stewardship (AMS) interventions. A systematic search was performed in key medical databases to identify papers published in English between 2005 and 2015 that examined the association between inappropriate prescribing and BSI mortality in adult patients. Only studies that included BSIs caused by ESKAPE (Enterococcus faecium/faecalis, Staphylococcus aureus, Klebsiella pneumoniae, Acinetobacter baumannii, Pseudomonas aeruginosa and Enterobacter species) organisms were included. Study quality was assessed using the GRADE criteria and results combined using a narrative synthesis. We included 46 studies. Inappropriate prescribing was associated with an overall increase in mortality in BSI. In BSI caused by resistant gram positive organisms, such as methicillin resistant S. aureus, inappropriate therapy resulted in up to a 3-fold increase in mortality. In BSI caused by gram negative (GN) resistant organisms a much greater impact ranging from 3 to 25 fold increase in the risk of mortality was observed. While the overall quality of the studies is limited by design and the variation in the definition of appropriate prescribing, there appears to be some evidence to suggest that inappropriate prescribing leads to increased mortality in patients due to GN BSI. The highest impact of inappropriate prescribing was seen in patients with GN BSI, which may be a useful metric to monitor the impact of AMS interventions.

Competing interest statement

Conflict of interest: the authors declare no potential conflict of interest.

Introduction

One of the main goals of an antimicrobial stewardship (AMS) intervention is to ensure patients with infections receive the most appropriate antimicrobial agent at the optimal dose at the earliest time.[1] As AMS programs can take many forms there is a need to ensure that the intervention(s) selected can maximize the outcomes of the program. However, there is a lack of clarity around the outcome measures that provide the best indicators of a successful AMS program with most studies focusing on changes in antimicrobial utilization rates.[2] An update by Akpan et al. (2016) on current metrics to measure the impact of AMS programs in a recent review reported that only a handful of studies included patient outcomes.[3] The authors reported that only 13 of the 63 studies that met their inclusion criteria reported on mortality, length of stay and unplanned admissions related to post-AMS infection as an outcome measure. Okumura et al. (2015) focused on six studies that examined mortality and the non-significant impact was highlighted with only one study reporting an absolute risk reduction in 30-day mortality.[4]

A recent study by Cairns (2016) found that active review of patients with bloodstream infections (BSI) by their AMS team improved the timeliness of appropriate therapy.[5] BSI mortality has been proposed as a useful indicator to evaluate AMS programs,[6,7] based on the premise that better prescribing practices should improve patient outcomes. While this premise may make sense, the evidence to support this link is not clear.

Bloodstream infections are serious infections and factors such as the choice of antimicrobial, duration of therapy, dosage, and route of administration can impact patient outcomes. While there are many infectious agents causing serious infections in a hospital environment, a group of organisms referred to as the ESKAPE bacteria (Enterococcus faecium/faecalis, Staphylococcus aureus, Klebsiella pneumoniae, Acinetobacter baumannii, Pseudomonas aeruginosa and Enterobacter species) are particularly problematic.[8] Patients with BSI due to ESKAPE organisms who are not receiving appropriate therapy, and patients with infections caused by resistant organisms, have worse outcomes than those caused by susceptible organisms.[9] It follows that if BSI are managed more effectively, morbidity, mortality and length of stay in this subset of patients can be reduced.[10]

This systematic review aims to synthesize findings from epidemiological studies to determine whether there is an association between inappropriate (i.e., inadequate, incorrect or delayed) prescribing and an increased risk of mortality in adult hospitalized patients with BSI caused by ESKAPE organisms. In addition, the impact of factors such as organism group (gram positive, GP, or gram negative, GN) and resistance status will be assessed.

Search strategy

A systematic literature search was performed in November 2015 in key medical databases (PubMed, Embase and Cochrane) to identify all papers published in English between January 2005 and November 2015 that assessed the association between inappropriate prescribing and BSI mortality. The review protocol was not registered but was aligned with the PRISMA Statement.[11]

The search strategy used is given in Supplementary Figure S1. All retrieved studies were scanned using title and abstract to determine whether the inclusion and exclusion criteria were met for the review. Where a decision could not be made based on the title or abstract, the paper was subject to full review.

Only primary studies in English that met the following inclusion criteria were analyzed: adult inpatients in hospital settings; studies performed in member countries of the Organization for Economic Cooperation and Development (OECD); the sample size was greater than 99; the risk of mortality related to BSI was expressed as a relative risk (odds ratio, relative risk or hazard ratio); and, included only organisms belonging to the ESKAPE group. Exclusions included: immunocompromized populations (transplant and oncology patients); neonatal and pediatric populations; and non-hospital inpatients.

Data on the study design, context, organisms, definition of inappropriate prescribing, main study objective and main outcome measure was extracted from all included studies using specifically designed data extraction forms that were piloted prior to use. The overall quality of the evidence to support an association between inappropriate prescribing and BSI mortality was assessed using the GRADE criteria.[12] The main summary measure was the relative risk of mortality. Results were combined using a narrative synthesis due to anticipated heterogeneity between studies. Subgroup analyses were conducted to look at differences in outcomes by organism group (GP or GN) and resistance status (sensitive or resistant) of BSIs.

Results

Of 9046 studies screened for inclusion, forty-six met the inclusion and exclusion criteria. Reasons for exclusion are shown in Figure 1. Of the 46 studies two were large multi-center studies: one including data from USA, Canada and Saudi Arabia;[13] and the second including data from nine European countries.[14] Thirteen studies were from the USA,[12,15-26] eight from Spain,[27-34] five from Italy,[35-39] four from Korea,[40-43] three each from Turkey[44-46] and Israel,[47-49] two each from the Netherlands[50,51] and the UK,[52,53] and single studies from Denmark,[54] Germany,[55] Norway,[56] and Australia.[57] Of the 46 studies 34 were retrospective in design and only three of these studies were case controlled.[16,36,49] Whilst a few studies focused on specific patient groups, most included all adult patients.

Figure 1

Search strategy to identify studies of bloodstream infections (BSI) and mortality or quality of life measures, from the Organization for Economic Cooperation and Development (OECD) countries.

The majority of studies reviewed used a retrospective study design but used different analytical methods and adjusted for different confounders, making results difficult to directly compare. The definition for inappropriate prescribing varied significantly between countries although most studies defined appropriate prescribing as the correct antimicrobial for the pathogen according to local guidelines. The definition for inadequate therapy was no different from the one used for inappropriate therapy, in most cases. The definition of inappropriate, inadequate and delayed therapy used in each of the studies is described in Supplementary Tables S1 and S2. The time to antimicrobial therapy varied with a range from 6 hrs in septic shock and severe sepsis to up to 72 hrs being deemed acceptable. This variation of sickness severity in some patient cohorts explains the range in time to therapy, as a shorter time would be more appropriate for more severely unwell patients.

Impact of delayed therapy

The impact of timing of antimicrobial therapy on the risk of mortality in patients with BSI was reported in 12 of the 46 studies (Supplementary Table S3). The definition of delayed therapy varied from ≥1hr to >72hrs in the studies making it difficult to compare the impact on mortality in these patients. The impact of delayed therapy on BSI-associated mortality combined with the analysis of GP and GN organisms were reported in five of the 12 studies and reported a two-fold increase.[15,50,52,56,57] Only two of the twelve studies were case controlled and reported on the impact of Methicillin Resistant Staphylococcus aureus (MRSA) with a delay in therapy of two days, resulting in an odds of mortality of 1.85 (95%CI: 0.094-3.64, P=0.074)[16] and patients with BSI due to ESBL organisms with a delay in therapy of 48 hours resulting in an OR of 25.1 (95%CI: 10.5-60.2, P≤0.001).[49] Overall a greater impact was seen in GN resistant infections ranging from 3 to 25-fold increases in the risk of BSI-associated mortality.

Impact of inappropriate or inadequate therapy

The definitions for inadequate and inappropriate therapy used in the studies overlap and as such these studies will be presented collectively. Six of 34 studies that described the impact of inappropriate therapy on BSI-associated mortality combined the analysis of GP and GN organisms, seven only reported GP BSI and 21 reported GN BSI. In the six studies that pooled GP and GN BSI data, the odds of mortality ranged from no significant impact to a nine-fold increase in death; however, this higher estimate was observed in a subset of patients with severe sepsis. The risks associated with inappropriate prescribing were higher for studies looking at MRSA than studies reporting on Methicillin Susceptible Staphylococcus aureus (MSSA) infections.

The highest reported risks were due to infections with resistant GN infections. Studies that looked at the impact of mortality in all GN BSI, including Enterobacteriaceae, Pseudomonas aeruginosa and Acinetobacter species, reported an increase in the odds of mortality from two to five-fold due to inappropriate therapy. The impact increased significantly when inappropriate prescribing involved BSI due to resistant GN gram negative organisms ranging from two to nine-fold increases in the risk of death. The only case controlled study in this group reported data from multi drug resistant P mirabilis BSI and observed a nine-fold (9.85, 95%CI: 2.67-36.25, P<0.001) increase in mortality due to inappropriate prescribing.[36]

The overall quality of the papers included in this study were of medium to low quality only reaching an average score of Grade C according to the GRADE criteria. The evidence on GN as well as GP BSIs were of the same quality.

Discussion

Inappropriate (i.e., inadequate, incorrect or delayed) therapy is associated with a higher risk of mortality in BSI caused by resistant GP organisms (e.g., MRSA) than MSSA organisms. However, the impact of inappropriate prescribing on mortality is greatest in BSI caused by GN organisms, with the risk of death ranging from 3 to 25-fold depending on the resistance status of the pathogen.

While the quality of studies is tempered due to study design, and the variation in the definition of appropriate prescribing, there is some evidence to suggest that inappropriate prescribing leads to unfavorable outcomes in patients with BSI, particularly in those BSI caused by resistant organisms. Only two studies investigating outcomes of BSI mortality in GN organisms reported no impact;[22,23] all other studies reported a higher risk of death if therapy was inappropriate. However, the exact magnitude of this association is somewhat unclear. While most studies adjusted for co-morbidities using multivariate logistic regression there was variety in the confounders considered and no studies used methods that accounted for time-dependent bias. Earlier work has demonstrated that adjustment for time-dependent bias can lead to a dramatic reduction in estimates of the attributable mortality and length of stay associated with infection in hospitalized patients.[58]

While short term outcomes such as 30-day mortality in susceptible GP infections, e.g., those caused by S. aureus, does not seem to be associated with inadequate prescribing these infections can cause significant complications such as osteomyelitis and endocarditis in the longer term if not treated effectively.[59] The long term effects of poor prescribing are more difficult to capture and there is a need for prospective studies assessing longer term mortality outcomes and an assessment of morbidity impact in this area of research.

While it is perhaps intuitive that the dose as well as the timely delivery of antimicrobials to a patient with a BSI would be considered important, it remains not well studied in the current published evidence. To be considered appropriate therapy, the correct dose according to the local guidelines needs to be delivered. Roberts et al. (2008) advocated the delivery of the highest tolerable dose to achieve the best clearance of infection.[60] However in this review, most studies focused solely on timely delivery of the correct antibiotic, with no information as to what percentage of patients with BSI received an inadequate dose of the correct antimicrobial. The dose of the antimicrobial is important for a number of reasons. If therapeutic exposures of antimicrobials are not achieved at the site of infection, then the infection will not be controlled. Additionally, sub-optimal doses have been associated with the emergence of resistance.

To achieve improved prescribing practices in hospitals, the most informative data on the causative pathogen needs to be made available to the treating clinician in the most efficient manner. The delivery of rapid and accurate information on the identification and susceptibility of the pathogen causing the infection could lead to better prescribing and improved outcomes for patients. There has been published evidence to support the claim that rapid technology in conjunction with an AMS intervention improved survival in resistant GN BSI.[61] More data needs to be collected on the long term effects to measure the impact of AMS interventions and the use of rapid technology over longer periods of time. This data may indicate whether the improvements in prescribing achieved by AMS programs translate into improved patient outcomes and provide some insight as to the sustainability of the outcomes of these interventions.

Our review has a number of limitations due to the quality and the heterogeneity of the studies included therefore a meta-analysis of the data was not able to be performed. The various study designs, definitions and variation in the way the risk of death was quantified in these studies made it difficult to compare outcomes. A standardized metric of measuring the risk of mortality would be of benefit when measuring these outcomes so that it would be easier to compare studies in the future. Some studies may have been missed as the strict inclusion criteria meant other potentially important groups, such as the immunocompromized, were excluded due to their higher risks of mortality and morbidity. Studies that had a sample size of less than 99 were also excluded to minimize the influence of chance findings on our summary; this exclusion may have excluded small but potentially important studies.

The nature of the subject matter is not suitable for RCT methodology as it would be unethical to randomize any patient or patient group to inappropriate therapy. Hence it is unlikely that GRADE A evidence will be available in this space. The Cochrane handbook for systematic reviews endorses the use of high quality observational evidence where the quality of studies is of a high to moderate quality.[62] In our case what is thought of as lower level evidence in the absence of RCT studies might be the best available evidence on this topic.

The ESKAPE organisms are the main pathogens in hospital settings. BSI-associated mortality in patients caused by these organisms resulted in an increased risk of mortality; this finding suggests a link between inappropriate prescribing and an increased risk of death in these BSI. Since the largest impact of inappropriate prescribing was seen in resistant GN BSI, these may be a suitable metric to describe the impact of an AMS intervention on patient outcomes. However, little is known on the longer term impacts of BSI that are treated with inappropriate antimicrobials and future longitudinal studies would provide better information on morbidity and quality of life impacts on patients receiving inappropriate therapy for BSI.

Conclusions

More effort into better study design and more consistent definitions of appropriate versus inappropriate therapy would be advantageous. The current review of evidence suggests that BSI mortality in GN may be associated with the adequacy of prescribing and thus may be a useful metric for evaluating the impact of AMS programs that focus on improving prescribing practices. Further evidence is needed to make a more conclusive recommendation.