Minimizing Time to Optimal Antimicrobial Therapy for Enterobacteriaceae Bloodstream Infections: A Retrospective, Hypothetical Application of Predictive Scoring Tools vs Rapid Diagnostics Tests.

BACKGROUND: Bloodstream infections (BSIs) due to ceftriaxone (CRO)-resistant Enterobacteriaceae are associated with delays in time to appropriate therapy and worse outcomes compared with infections due to susceptible isolates. However, treating all at-risk patients with empiric carbapenem therapy risks overexposure. Strategies are needed to appropriately balance these competing interests. The purpose of this study was to compare 4 methods for achieving this balance.
METHODS: This was a retrospective hypothetical observational study of patients at the Detroit Medical Center with monomicrobial BSIs due to E. coli, K. oxytoca, K. pneumoniae, or P. mirabilis. This study compared the effectiveness of 4 methods to predict CRO resistance at the time of organism isolation. Three methods were based on applying published extended-spectrum beta-lactamase (ESBL) scoring tools. The fourth method was based on the presence or absence of the CTX-M marker from Verigene.
RESULTS: Four hundred fifty-one Enterobacteriaceae BSIs were included, 73 (16%) of which were CRO-resistant. Verigene accurately predicted ceftriaxone susceptibility for 97% of isolates, compared with 70%-81% using the scoring tools (P < .001). Verigene was associated with fewer cases of treatment with CRO when the isolate was CRO-resistant (15% vs 63%-71% with scoring tools) and fewer cases of overtreatment with a carbapenem for CRO-susceptible strains (0.3% vs 10%-12%).
CONCLUSIONS: Verigene significantly outperformed published ESBL scoring tools for identifying CRO-resistant Enterobacteriaceae BSI. Institutions should validate scoring tools before implementation. Stewardship programs should consider adoption of rapid diagnostic tests to optimize early therapy.

Bloodstream infections (BSIs) are the seventh leading cause of death in the United States [1]. With increasing rates of antimicrobial resistance, gram-negative BSIs are of particular concern. One of the most challenging resistance threats is the increasing frequency of ceftriaxone-resistant Enterobacteriaceae [2]. Given the commonality of Enterobacteriaceae as a cause of BSIs and the increasing frequency of resistance to first-line therapies (ie, ceftriaxone), impact on patients is substantial. Data have shown that BSIs due to extended-spectrum beta-lactamase (ESBL)–producing Enterobacteriaceae, the most common mechanism of resistance to third-generation cephalosporins in Enterobacteriaceae, are associated with increased mortality, length of hospitalization, and health care costs, with a primary reason being significant delays in administration of appropriate antibiotics [3, 4].

In many institutions, resistance to third-generation cephalosporins is not identified until final antimicrobial susceptibility tests (ASTs) return 48–96 hours after collection of blood cultures [5]. This leads to significant delays in time to therapy modification and increases risk of negative outcomes. Conversely, widespread use of carbapenems as empiric therapy to ensure coverage of these resistant organisms is of concern, as carbapenem usage is a known risk factor for the emergence of carbapenem-resistant organisms [4, 6]. Therefore, strategies to more rapidly and accurately identify patients with third-generation cephalosporin-resistant (3GCR) Enterobacteriaceae are urgently needed in order to walk the tight rope between earlier appropriate therapy and overuse of carbapenems.

To aide in predicting patients with 3GCR Enterobacteriaceae, clinicians can utilize risk factors reported for ESBL including the presence of invasive devices, increased age, intensive care unit (ICU) stay, nursing home residence, and prior antimicrobial exposures [3]. These risk factors can be utilized to create and ultimately implement a bedside prediction tool. If derived properly, bedside prediction tools could be of great value to the end user, allowing high probability of detecting an ESBL with only a few clinical variables. Ideally, variables included in the prediction tool would be easily retrievable from the electronic medical record (EMR) at the time of admission or suspicion of infection.

At the time of this study, 2 prediction scores had been published to assist providers in identifying those at risk for ESBLs. Augustine and colleagues developed a risk score that included 3 risk factors: recent outpatient genitourinary/gastrointestinal procedure, prior beta-lactam or fluoroquinolone exposure, and prior infection/colonization with ESBL Enterobacteriaceae within the previous year. This scoring tool created weight-based scores based on magnitude of risk with each factor, and after analysis of various scores, the authors recommended that patients with high risk of an ESBL BSI (prediction score of ≥3) or critically ill moderate-risk patients (score of 1–2) should receive an empiric carbapenem. This cutoff threshold of 3 demonstrated a negative predictive value (NPV) of 97%, and thus there was confidence that it would not lead to undertreatment in many patients [7].

Similarly, Lee and colleagues created a scoring tool for patients at risk for community-onset ESBL BSI [7]. The authors demonstrated that nursing home residents, frequent and recent emergency department (ED) visits, recent antimicrobial exposure, and recent invasive procedures were all independent predictors. Each independent factor was equally assigned 1 point, and the authors recommended starting empiric carbapenem therapy with a score ≥2. This score had an NPV of 99%, and thus the authors had confidence that it would not lead to undertreatment. An important limitation of both scoring tools is that they were derived in populations with low ESBL rates (5%–6%). Given the infrequency of the event, the NPV would be expected to be high regardless of the performance of the test. Importantly, the positive predictive value (PPV) of the test was poor in both analyses, ranging from 33% to 40%, suggesting a high likelihood of overtreatment with carbapenems with their application [6, 7]. Furthermore, the performance of these tests at an institution like ours that has ESBL rates approaching 20% remains unclear [8].

An alternative approach to scoring tools to more rapidly identify patients with resistant pathogens is using rapid diagnostic tests (RDTs). When paired with antimicrobial stewardship (ASP), RDTs have demonstrated reductions in mortality for patients with gram-negative bacteremias. While multiple RDT platforms can rapidly determine organism identification to allow application of the aforementioned scores, Verigene has the capability to identify the predominant ESBL resistance gene CTX-M. Verigene’s performance to accurately predict ceftriaxone susceptibility in Enterobacteriaceae was recently assessed at both the Detroit Medical Center (DMC) and the University of Maryland Medical Center [8]. Although the platform only detects 1 resistance gene for ESBL (blaCTX-M), the sensitivity, specificity, PPV, and NPV were high in this analysis, suggesting that other mechanisms contribute only a minor proportion to 3GCR in these pathogens. Notably, the PPV in this study was much higher (~95%), with similar NPV to the scoring tools. These findings suggest that a treatment algorithm based on CTX-M status could appropriately identify patients who warrant carbapenem usage while limiting unnecessary overexposure.

At the time of this study, no analyses have either validated the scoring tools or a Verigene-based algorithm or compared these methods in a head-to-head analysis. Therefore, the purpose of this study was to compare 4 different treatment algorithms for the management of Enterobacteriaceae BSI in a single cohort of patients.

METHODS

This was a retrospective observational study of adult patients within the DMC, an 8-hospital health system. Patients with gram-negative bacteremia were identified from a microbiological database. Patients were eligible if they had a monomicrobial Enterobacteriaceae BSI from July 1, 2016, to July 31, 2017. For this analysis, only Escherichia coli, Klebsiella oxytoca, Klebsiella pneumoniae, or Proteus mirabilis BSIs were eligible as those are the Enterobacteriaceae species detected by Verigene’s Gram Negative-Blood Culture (GN-BC) panel [9]. Although Enterobacter and Citrobacter spp. are also identified by Verigene, it is standard practice at the DMC to avoid 3GC for the treatment of BSIs due to these pathogens regardless of susceptibility, and therefore these pathogens were excluded. Patients were also excluded if carbapenemase genes were detected, as the purpose of this analysis was not to assess the application of these pathways for carbapenemase-producing organisms. Additionally, only the first patient episode over the study period was eligible.

CLSI considers any E. coli, K. pneumoniae, K. oxytoca, or P. mirabilis with a minimum inhibitory concentration (MIC) ≥2 mcg/mL to ceftriaxone (CRO), ceftazidime, or aztreonam to be an ESBL [10]. The DMC utilized BD Phoenix testing (Becton, Dickinson and Co., Sparks, MD, USA) for susceptibility testing and at the time of the study had adopted these CLSI breakpoints [11]. Therefore, for the purposes of this study, CRO resistance or 3GCR and ESBL are used interchangeably.

This study assessed the appropriateness of 4 methods for determining whether to start either a carbapenem or ceftriaxone at the time of organism identification, defined as 24 hours after the blood culture was drawn, for treatment of a gram-negative bacteremia. For the purposes of the study, we assumed all blood cultures turned positive at 24 hours. Each patient was hypothetically managed and had treatment decisions based on the aforementioned algorithms by each of the 4 approaches: (1) Verigene, based on presence or absence of the CTX-M resistance marker; (2 & 3) 2 scores applied by the Augustine scoring tool’s different approaches; and (4) a score applied by the Lee scoring tool.

For consistency in treatment, all patients were hypothetically started on ceftriaxone for gram-negative coverage. For study purposes, time 0 represents the day blood cultures were drawn and initial ceftriaxone was started. The first decision point occurred at 24 hours, assuming that at this time point either traditional microbiology would have identified a gram-negative rod in the blood or Verigene would have determined both organism identification and presence/absence of CTX-M. The decision to either continue ceftriaxone or escalate to a carbapenem occurred at this point in each of the 4 pathways (Figure 1). If CTX-M was identified with Verigene, patients were hypothetically changed from ceftriaxone to a carbapenem. For the Augustine scoring tool, patients were hypothetically changed to a carbapenem if the ESBL prediction score was ≥3 or if a patient was critically ill with an ESBL score of 1–2 (defined as Augustine Approach 1). Additionally, this study also applied the Augustine scoring tool regardless of severity of illness by hypothetically giving a carbapenem only if the ESBL prediction score was ≥3, as this score was associated with the highest performance in their analysis (Augustine Approach 2). Similarly, for the Lee approach, in accordance with the authors’ recommendations, patients were treated with a carbapenem if the ESBL prediction score was ≥2. If any of these results were negative (CTX-M-negative or scoring systems below threshold values), patients were “maintained” on ceftriaxone. Final antibiotic decisions were made at 72 hours when phenotypic antibiotic susceptibility from Phoenix BD became available and were based on actual patient isolates.

Figure 1.

Hypothetical treatment pathways. Threshold for giving carbapenem at time 24 include the following: Verigene = CTX-M gene detected; Augustine 1 = score of ≥3 or score ≥1 in critically ill; Augustine 2 = score ≥3; Lee = score ≥2. Abbreviations: CARB, carbapenem; CRO, ceftriaxone; R, resistant; S, susceptible.

Study Outcomes

The sensitivity, specificity, PPV, and NPV for predicting ceftriaxone susceptibility were assessed with the 4 approaches (where the “test” was the treatment approach and the “condition” was presence/absence of ceftriaxone resistance). Additionally, the number of patients who would have been inappropriately maintained on ceftriaxone therapy in the setting of ceftriaxone resistance and the number of patients unnecessarily escalated to a carbapenem in the setting of ceftriaxone susceptibility were assessed.

In order to gauge the impact that following the different treatment pathways would have on total carbapenem usage, the following procedure was used. For each patient, the actual inpatient antimicrobial days of therapy for the BSI was determined. Then each decision tree as described above was applied to that patient to assess the number of carbapenem patient days per total treatment days that would have occurred had the algorithms been followed. For example, if a patient met the scoring criteria for escalation to a carbapenem but ultimately would have been de-escalated back to ceftriaxone based on absence of ceftriaxone resistance and completed a 7-day course of intravenous antibiotics in the hospital, they would have been considered to have 2 carbapenem days (days 2 and 3 of therapy) out of 7 total BSI treatment days. This process was then completed for every patient with each treatment algorithm, and the results for each individual method were summed. The resultant number of carbapenem days was then normalized to 1000 inpatient BSI treatment days in order to compare total carbapenem exposures between application of the different pathways.

Covariates Collected

Covariates collected from the electronic health record included demographics; comorbid conditions, presence of indwelling devices, microbiological and antimicrobial histories, components of the Pitt bacteremia score; ICU admission; physician diagnosis and source of infection; and other variables from previously published scoring tools [12].

Statistical Analysis

Chi-square tests were utilized to compare the accuracy of the various treatment pathways to predict ceftriaxone susceptibility. The performance rates of ESBL risk scores and Verigene were compared with phenotypic testing using receiver operating characteristics (ROC) curve analysis. These analyses were performed in R, version 3.5.0 (R Foundation for Statistical Computing, Vienna, Austria), using the pROC package.

RESULTS

Description of the Cohort

During the study period, 451 patients with Enterobacteriaceae BSI were included. Overall, the mean age was 65 ± 17 years, 242 (54%) were female, and 327 (73%) were African American. Common comorbidities included diabetes (191, 40%), chronic kidney disease (108, 24%), and chronic obstructive pulmonary disease (79, 18%). Eighty-four patients (17%) presented from a nursing home, and 186 patients (41%) were hospitalized in the previous 90 days. Forty-eight (11%) patients had indwelling urinary catheters, and 47 patients (10%) had central venous catheters (Table 1).

Table 1.

Baseline Characteristics

Characteristics	n = 451
Age, y	65 ± 17
Female	242 (54)
Race
African American	327 (72.5)
White	81 (18)
Other	43 (9.5)
Admission source
Home	334 (74)
Nursing home	86 (19)
Rehabilitation center	12 (3)
Outside hospital	4 (1)
Other	17 (4)
Comorbidities
Hospitalization in last 90 d	186 (41)
Indwelling urinary catheter	48 (10.6)
Indwelling central venous device	47 (10.4)
GI feeding tube	33 (7.3)
Congestive heart failure	85 (18.8)
Dementia	57 (12.6)
COPD	79 (17.5)
Chronic kidney disease	108 (23.9)
Solid tumor	85 (18.8)
Cerebrovascular disease	58 (12.9)
Liver disease	54 (12)
Diabetes mellitus	181 (40.1)
Charlson Comorbidity Index	2 ± 2
Other relevant variables for ESBL scoring tools
Recent GI/GU procedure within 30 d	42 (9.3)
Invasive procedure in previous 4 wk	40 (8.9)
No. of prior beta-lactam and/or fluoroquinolone courses in previous 90 d
1	84 (18.6)
≥ 2	40 (8.9)
Any antibiotic exposure in previous 4 wk	86 (19.1)
Infection or colonization with ESBL in previous year	33 (7.3)
≥3 ED visits within the previous year	74 (16.4)
Pitt bacteremia score ≥4	59 (13.1)

Data are presented as No. (%) or mean ± SD.

Abbreviations: COPD, chronic obstructive pulmonary disease; ED, emergency department; ESBL, extended-spectrum beta-lactamase; GI/GU, gastrointestinal/genitourinary tract.

Antibiotic exposure in the previous 4 weeks was observed in 86 (19%) patients, with 86 (19%) receiving at least 1 and 40 (9%) receiving ≥2 beta-lactam or fluoroquinolone courses within the previous 90 days. Forty-two (9%) patients had a recent gastrointestinal/genitourinary tract procedure, and 40 (9%) had an invasive procedure within the previous 4 weeks. Additionally, 74 (16%) had ≥3 ED visits within the previous year, and 33 (7%) had infection or colonization with an ESBL-producing organism within the previous year. The distribution of scores via the Augustine and Lee methods is displayed in Table 2.

Table 2.

ESBL Score Distribution for Augustine and Lee Scoring Tools

ESBL Score Distribution (n = 451)	Augustine et al., No. (%)	Lee et al. No. (%)
0	291 (64.5)	240 (53.2)
1	86 (19.1)	151 (33.5)
2	13 (2.9)	47 (10.4)
≥3	61 (13.5)	13 (2.8)

Threshold to treat with a carbapenem awaiting final susceptibilities: score ≥3 or if a patient was critically ill with an ESBL score of 1–2 (Augustine Approach 1); score ≥3 (Augustine Approach 2); score ≥2 (Lee).

Abbreviation: ESBL, extended-spectrum beta-lactamase.

The predominant organism was E. coli (287, 64%), followed by K. pneumoniae (112, 25%) and P. mirabilis (48, 11%). A total of 73 isolates (16%) were resistant to third-generation cephalosporins. Rates of resistance to third-generation cephalosporins were 17% for E. coli, 17% for K. pneumoniae, 10% for Proteus mirabilis, and 25% for K. oxytoca.

Hypothetical Escalation at 24 hours Based on the 4 Treatment Pathways

CTX-M was detected in 63 isolates (14%), leading to escalation in those patients, while the remaining 388 patients (86%) remained on ceftriaxone. Using the scoring tool from Augustine, approach 1 led to escalation in 74 patients (16%), whereas approach 2 led to escalation in 61 (14%) patients. Similarly, applying the Lee scoring tool led to escalation in 60 (14%) patients.

Accuracy of Approaches

With regards to appropriately predicting CRO susceptibility, Verigene correctly predicted ceftriaxone susceptibility results in 439 isolates (97%), and this was significantly higher than all score-based approaches: Augustine Approach 1 (358, 79%), Augustine Approach 2 (359, 80%), and Lee (364, 81%; P < .001 for all comparisons) (Table 3). In the 378 ceftriaxone-susceptible isolates, unnecessary escalation to a carbapenem would have occurred in 1 patient utilizing the Verigene approach, compared with 47 (12%) and 40 (11%) utilizing the 2 Augustine approaches and 37 (10%) utilizing the Lee method. Additionally, failure to appropriately escalate in patients with ceftriaxone-resistant isolates would have occurred in 11 of 73 (15%) resistant isolates with the Verigene-based algorithm. The rates of failure to escalate were higher with application of both the Augustine (n = 46, 63%, approach 1; n = 52, 71%, approach 2) and Lee (n = 50, 69%) methods.

Table 3.

Comparative Accuracy of the Various Methods

n = 451	Cutoff	Appropriately Predicted Ceftriaxone Susceptibility, No. (%)	Overtreatment (n = 378), No. (%)	Undertreatment (n = 73), No. (%)	Sensitivity, %	Specificity, %	Positive Predictive Value, %	Negative Predictive Value, %
Verigene	CTX-M	439 (97.3)	1 (0.3)	11 (15.1)	85	99.7	98	97
Lee	2	364 (80.7)	37 (9.8)	50 (68.5)	32	90	38	87
Augustine Approach 1	3 OR 1–2 and critically ill	358 (79.4)	47 (12.4)	46 (63)	37	88	36	88
Augustine Approach 2	3	359 (79.6)	40 (10.6)	52 (71.2)	29	89	34	87

Overtreatment = algorithm escalated patient to an empiric carbapenem on day 1 (final susceptibility = CRO susceptible); undertreatment = algorithm continued ceftriaxone on day 1 (final susceptibility = CRO resistant).

The sensitivities, specificities, PPVs, and NPVs of the various approaches are presented in Table 3, while the areas under the ROC curves are displayed in Figure 2. Verigene performed significantly better than any scoring tool with regards to each of these measures.

Figure 2.

ROC curves. Abbreviations: AUC, area under the curve; ROC, receiver operating characteristics.

Total Carbapenem Consumption by Treatment Approach

The number of carbapenem days per 1000 BSI patient days for Verigene, Lee, and Augustine Approach 1, and Augustine Approach 2 were similar at 136 days, 134 days, 142 days, and 134 days, respectively.

DISCUSSION

The most important finding of this analysis is that while Verigene performed very well in predicting ceftriaxone susceptibility in these isolates, both published ESBL scoring tools performed poorly. While the PPVs of the scoring tools were expectedly low (34%–38% in this study, which was similar to the 33%–40% values in other published analyses), the sensitivity and subsequent NPVs were unacceptably low. Roughly two-thirds of ceftriaxone-resistant isolates would have been missed, suggesting limited utility of these scores at our institution. These findings highlight the importance of site-specific validation of published scoring tools before implementation.

There are multiple possible explanations for the failure of these scoring tools in this analysis. The most important is local epidemiological differences between institutions. ESBL rates in target pathogens at the DMC are ~3-fold higher than either of the sites where these scores were developed. However, this increased rate would more be expected to impact the specificity than sensitivity, and thus the failure to detect roughly two-thirds of ceftriaxone-resistant isolates was alarming. This is despite relatively similar patient populations between the current study and those from which the scores were derived. This was particularly true for the study by Augustine, where demographics, health care exposures, and presence of indwelling catheter rates were similar. Conversely, all variables that were high predictors of ESBL in their model occurred more frequently in our patient population (recent gastrointestinal/genitourinary tract procedure 9.3% vs 5.4%; ≥2 prior courses of beta-lactams or fluoroquinolones 8.9% vs 4.3%; and prior infection/colonization with ESBL-producing organisms 7.3% vs 1.6%). The high reliance on these infrequent events likely overstated their importance in predicting 3GC resistance (leading to the low specificity) and potentially “hid” the impact of other important factors (leading to the low sensitivity) in this population. The low incidence of ESBL at their institution likely exacerbated these issues.

With regards to the Lee study, there were significant differences between the study populations with respect to multiple important components of the scoring tool, in addition to the low ESBL rate in their analysis. In the current analysis, there were significantly higher numbers of nursing home patients (17% vs 5%), recent invasive procedures (8.9% vs 2.5%), and significantly fewer patients with “frequent ED visits” (16% vs 30%). Therefore, the failure of this scoring tool to predict ESBLs in our population is not overly surprising. Similarly, a high reliance on infrequent events could limit the reliability of a scoring tool in an external population.

Another interesting finding was that even though the Verigene-based algorithm would have led to 35–41 patients being placed on appropriate carbapenem therapy 2 days earlier, it would not have led to an increase in overall carbapenem use in the population (136 carbapenem days/1000 BSI patient days compared with 134–142 carbapenem days/1000 BSI patient days). This is because the early appropriate usage of carbapenems in the Verigene algorithm was offset by the early inappropriate usage directed by the scoring tools. Thus, the Verigene-based algorithm was successful in optimizing empiric activity while limiting unnecessary use of carbapenems and demonstrates the importance of rapid diagnostics for optimal stewardship practices.

There are limitations to this analysis that warrant comment. First, we excluded patients who had BSI due to Enterobacter spp., Citrobacter spp., and other “off-panel” Enterobacteriaceae, which can be important causes of BSI. However, this was done purposefully, as we were assessing strategies to determine if ceftriaxone was appropriate for the management of infections based on various treatment algorithms, and at many institutions, including the DMC, ceftriaxone is not considered for the treatment of these pathogens. Additionally, the scoring tools assessed were developed on the same 4 pathogens included in this study, and the distributions of these pathogens were similar to the present analysis. Nonetheless, these findings would not be applicable to any of these other pathogens or nonfermenting gram-negative bacilli, where resistance mechanisms can be quite diverse.

Second, this was a hypothetical pathway implementation, and therefore we are unable to assess either success with interventions based on these algorithms or their impact on outcomes. That said, these findings should help better implement these strategies in the future. Additionally, although we were unable to validate the scoring tools assessed in this study, they may still benefit institutions with similar ESBL rates and patient characteristics as the study populations from the Augustine and Lee studies and warrant further exploration in that setting. Finally, this study was performed at an institution where CTX-M was known to predict ceftriaxone resistance with high degrees of accuracy [8]. While a similar finding was demonstrated at the University of Maryland Medical Center, it would be important to validate these findings before implementing a Verigene-based pathway at another institution. While CTX-M is the predominant cause of 3GC resistance in these study pathogens in the United States, resistance can be seen due to other mechanisms, highlighting the importance of incorporating local epidemiology into any treatment paradigm [13].

In closing, this analysis demonstrated that Verigene was highly accurate in predicting ceftriaxone susceptibility in Enterobacteriaceae BSI and that published ESBL scoring tools performed poorly. Antimicrobial stewardship programs should incorporate rapid diagnostic tests into the management of Enterobacteriaceae BSIs wherever possible; however, internal validation of pathways is critical.