Impact of bacteremia prediction rule in CAP: Before and after study.

OBJECTIVE: In cases of community acquired pneumonia (CAP), it has been known that blood cultures have low yields and rarely affect clinical outcomes. Despite many studies predicting the likelihood of bacteremia in CAP patients, those results have been rarely implemented in clinical practice, and use of blood culture in CAP is still increasing. This study evaluated impact of implementing a previously derived and validated bacteremia prediction rule.
METHODS: In this registry-based before and after study, we used piecewise regression analysis to compare the blood culture rate before and after implementation of the prediction rule. We also compared 30-day mortality, emergency department (ED) length of stay, time-interval to initial antibiotics after ED arrival, and any changes to the antibiotics regimen as results of the blood cultures. In subgroup analysis, we compared two groups (with or without the use of the prediction rule) after implementation period, using propensity score matching.
RESULTS: Following the implementation, the blood culture rate declined from 85.5% to 78.1% (P=0.003) without significant changes in 30-day mortality and antibiotics regimen. The interval to initial antibiotics (231min vs. 221min, P=0.362) and length of stay (1019min vs. 954min, P=0.354) were not significantly changed. In subgroup analysis, the group that use the prediction rule showed 25min faster antibiotics initiation (P=0.002) and 48min shorter length of stay (P=0.007) than the group that did not use the rule.
CONCLUSION: Implementation of the bacteremia prediction rule in CAP patients reduced the blood culture rate without affecting the 30-day mortality and antibiotics regimen.

Introduction

Blood culture before antibiotics treatment has been a routine diagnostic test in patients presumed to be infectious [1]. However, in patients with community acquired pneumonia (CAP), it has been known that the rate of true positive blood cultures is < 10% [2], [3]. In emergency departments (ED), the true positive results from blood cultures rarely affected patient management [4], [5]. Additionally, ED overcrowding may increase the risk of blood culture contamination [6], [7].

As part of an effort to reduce unnecessary blood culture and improve the yield of blood culture in CAP, there have been many studies describing bacteremia prediction rules [8], [9], [10]. Despite those studies, in a recent study, it has been reported that the blood culture rate in CAP patients is still increasing [11]. Therefore, whether those decision rules clinically impacted the reduction of blood cultures is questionable.

For clinical decision rules to have an impact on standard treatment, the clinical decision rules should pass three stages: derivation, validation, and implementation [12], [13]. In our previous studies, we derived the bacteremia prediction model in CAP patients and tested the generalizability of the rule by a multicenter external validation [10], [14].

Here, we evaluated the impact of implementing the bacteremia prediction model on the rate of blood cultures. We hypothesized that implementation may reduce the blood culture rate without changing the mortality rate. Also without blood culture, the time interval from patient presentation in the ED to the first intra-venous antibiotics administration could be shortened as well as the length of ED stay.

Materials and methods

Study design and setting

This study is an uncontrolled before and after study based upon a retrospective review of a patient registry database. Before implementation of the bacteremia prediction rule, a prospective registry of patients diagnosed with pneumonia in the ED had been in place since 2008. The registry includes the patient's baseline characteristics and co-morbidities. Time variables such as ED arrival time and ED discharge time were included in the registry as well as the initial antibiotics administration time. The registry also provided the patient's initial vital signs and laboratory findings including the blood culture results if a blood culture had been performed. Finally, the patient's dispositions from the ED including admission to the ward or ICU as well as the 30-day mortality were included in the registry. This study was performed at a 950-bed tertiary academic hospital with an annual ED census of 90,000 patients and was approved by the institutional review board of the hospital.

Study population and implementation of intervention

The implementation of the bacteremia prediction rule was initiated in January of 2015. We retrospectively analyzed the patient registry data from January 2013 to January 2017. Exclusion criteria were the same as in previous studies [10], [14]. In short, we excluded patients who were diagnosed with hospital-acquired pneumonia (HAP), health care-associated pneumonia (HCAP), and ventilator-associated pneumonia (VAP). We also excluded patients who were transferred from our hospital because of difficulty in follow-up and limited information on the changes of antibiotics.

Education and feedback after implementation

For ease of use of the bacteremia prediction rule, we created an Excel file in which the recommendations for blood culture appeared with risk-stratified scores, calculated automatically by entry of variables (Supplementary Fig. 1). Because the resident physicians in the ED changed monthly according to a rotation system with other hospitals, the instructions for the Excel file and the explanations of the prediction rule were introduced on the first Wednesday of every month. All emergency physicians were encouraged to record the score calculated with the Excel file on electronic medical record (EMR) of patients suspected of pneumonia regardless of the performance of blood cultures. The prediction rule use records for each emergency physician were reported as a feedback three times a week.

Outcome measures

The primary outcome was the blood culture rate before and after implementation of the bacteremia prediction rule. The secondary outcomes included the 30-day mortality rate, the ED length of stay, the time-interval to initial antibiotics after ED arrival, and any changes to the antibiotics regimen as a result of the blood culture findings. For the cost analysis of reduced blood culture usage due to this intervention, we compared the mean cost for blood cultures per patient before and after implementation of the prediction rule.

Blood culture results and contaminations

For any positive blood culture results, we thoroughly reviewed the results, including preliminary reporting and final reporting times. In preliminary results, only gram staining results of the species growing in culture bottle were reported. In the final report, the exact names of the identified pathogens were reported with the antibiotics susceptibility test results. For changes to the antibiotics regimen following the blood culture results, we reviewed the daily antibiotics administration with the exact times until 24 h after the final result report time. For adverse events due to contaminations, we defined the unnecessary administration of vancomycin as the start of administration after the preliminary reports and its discontinuation following the identification of contaminants in the final reports. Additionally, unnecessary follow-up blood cultures were defined as blood cultures performed after preliminary reports that were confirmed as contaminants in the final reports and the follow-up results were negative. The following isolates were identified as contaminants: coagulase-negative Staphylococcus, Bacillus species, Micrococcus species, Corynebacterium species, and Propionibacterium species.

Statistical analysis

Continuous variables were presented as the means and standard deviations. Binomial variables were presented as the frequency of occurrence. Wilcoxon's rank-sum test, the chi-square test, or Fisher's exact test was performed, as appropriate, for comparisons before and after intervention. All P values were 2-tailed, and P values < 0.05 were considered statistically significant.

We adopted a piecewise regression discontinuity approach using generalized linear models (using odds for binary results) to evaluate the impact of the intervention. For adjusted intervention effects, we constructed multivariable linear regression models using variables with statistical significance (P value < 0.10) from comparisons before and after the intervention. Additionally, we performed Box-Pierce tests to identify statistically significant autocorrelations in time series models we used (P value < 0.05).

Because irrespective of prediction rule, the blood culture rates could be reduced only by education introducing the low yield of blood culture in CAP, we performed subgroup analysis only using the patients after intervention. For the subgroup analysis, we used propensity score matched analysis to compare the groups that use the prediction rule and did not. We compared the blood culture rate, time-interval to initial antibiotics, and ED length of stay of both groups. All analyses were performed using STATA (version 13; StataCorp, College Station, TX) and any patients with missing variables were not included in analyses.

Results

Patient characteristics

Of the 5178 registered patients during study period, a total of 4130 patients were included in the study after 1048 patients were excluded (Fig. 1 ). Among the 4130 eligible patients, 2480 and 1650 patients were assigned to the before and after intervention groups, respectively. Among the 1650 patients after intervention, 602 patients (36.5%) had the bacteremia score record on their EMR. Table 1 summarizes the baseline characteristics of the two groups.

Fig. 1

Flow chart.

Table 1

Patient characteristics before and after intervention.

Category, parameter	Before intervention (n = 2480)	After intervention (n = 1650)	P value
Epidemiological data
Mean age ± SD, years	67.5 ± 17.3	64.1 ± 19.8	< 0.001
Male sex (%)	1533 (61.8)	940 (57.0)	< 0.001
Diabetes mellitus (%)	535 (21.6)	316 (19.2)	0.060
Hypertension (%)	1005 (40.5)	593 (35.9)	0.003
Heart failure (%)	50 (2.0)	10 (0.6)	< 0.001
Cerebrovascular disease (%)	569 (22.9)	230 (13.9)	< 0.001
Renal disease (%)	191 (7.7)	102 (6.2)	0.084
Liver disease (%)	104 (4.2)	52 (3.2)	0.086
COPD (%)	272 (11.0)	182 (11.0)	0.946
Known neoplasm (%)	439 (17.7)	213 (12.9)	< 0.001
Vital signs, mean ± SD
Systolic blood pressure, mm Hg	130 ± 27	131 ± 24	0.016
Heart rate, beats/min	99 ± 21	99 ± 20	0.701
Respiratory rate, cycles/min	21 ± 6	20 ± 5	< 0.001
Body temperature, °C	37.4 ± 1.0	37.5 ± 1.0	< 0.001
Laboratory findings, mean ± SD
WBC count, × 103/mm3	11.6 ± 6.1	10.9 ± 5.4	< 0.001
Hematocrit, %	36.6 ± 5.6	38.3 ± 5.7	< 0.001
Platelet count, × 103/mm3	223.7 ± 99.7	229.5 ± 93.9	0.060
Glucose, mg/dL	141.0 ± 66.9	137.5 ± 64.8	0.095
Albumin, mg/dL	3.6 ± 0.6	3.6 ± 0.5	0.001
BUN, mg/dL	21.4 ± 16.3	19.6 ± 17.6	0.001
Creatinine, mg/dL	1.1 ± 1.1	1.1 ± 0.8	0.215
Sodium, mmol/dL	136.2 ± 5.9	136.0 ± 4.7	0.233
C-reactive protein, mg/dL	10.3 ± 8.3	10.4 ± 8.0	0.546
PSI	92.0 ± 40.8	79.7 ± 39.4	< 0.001
Score implementation rate (%)		602 (36.5)
Blood culture rate (%)	2049 (82.6)	1157 (70.1)	< 0.001
True bacteremia (%)a	86 (4.2)	34 (2.9)	0.071
Admission rate (%)	1333 (53.7)	826 (50.1)	0.021
30-day mortality (%)	209 (8.4)	81 (4.9)	< 0.001

Abbreviations: SD, standard deviation; COPD, chronic obstructive pulmonary disease; WBC, white blood cell; BUN, blood urea nitrogen; PSI, pneumonia severity index.

True bacteremia rate was calculated in patients in which blood culture performed.

Primary outcome

After implementation of the bacteremia prediction rule, the blood culture rate decreased from 85.5% to 78.1%, representing 7.4% reduction (95% confidence interval [CI] = 2.6 to 12.2, P  = 0.003) (Table 2 ). Additionally, the monthly trend of the blood culture rate changed from increasing to decreasing (Fig. 2 ). After model adjustment, the blood culture rate decreased from 82.1% to 77.6%, which represents 4.5% reduction (95% CI = 0.1 to 8.9, P  = 0.044) reduction. The monthly trend of the blood culture rate still appeared to decrease after implementation of the intervention (Table 2).

Table 2

Primary outcome before and after implementation of bacteremia prediction model with or without adjustment.a

Outcome	Before	After	Intervention effect (95% CI)	P value	Box-Pierce P value
Blood culture rate (%)	85.5	78.1	− 7.4 (− 2.6 to − 12.2)	0.003	0.332
Monthly trend (% per month)	0.2	− 0.7	− 0.9 (− 0.6 to − 1.2)	< 0.001
Blood culture rate (%)⁎	64.6	59.9	− 4.6 (− 0.5 to − 8.7)	0.029	0.605
Monthly trend (% per month)⁎	− 0.1	− 0.5	− 0.4 (− 0.2 to − 0.6)	0.007

Adjusted by variables with statistical significance (P value < 0.1): age, sex, history of diabetes mellitus, hypertension, heart failure, cerebrovascular disease, renal disease, liver disease, and known neoplasm, systolic blood pressure, respiratory rate, pulse rate, body temperature, white blood cell count, hematocrit, platelet count, glucose, albumin, BUN, PSI, admission rate.

The numbers of patients with any missing variables used in analysis was 116 and those patients were not included in analyses.

Fig. 2

Monthly trends of blood culture in the emergency department (ED) before and after intervention.

Secondary outcomes

After the implementation, the 30-day mortality rate decreased from 12.6% (95% CI = 10.6 to 14.6) to 7.3% (95% CI = 5.1 to 9.5, P  < 0.001). However, after model adjustment, this difference was not statistically significant (P  = 0.182) (Table 3 ). The time-interval to initial antibiotics after ED arrival decreased from 231 min to 221 min but did not reach statistical significance (P  = 0.362). After model adjustment, no statistically significant difference was found between the two groups (Table 3). After implementation, the ED length of stay also decreased from 1112 min to 954 min but the difference between the two groups was not statistically significant (Table 3).

Table 3

Secondary outcomes before and after implementation of bacteremia prediction model.

Outcome	Before	After	Intervention effect (95% CI)	P value	Adjusted P value⁎	Box-Pierce P value⁎
Antibiotics time (min)	231	221	10.2 (− 11.8 to 32.3)	0.362	0.346	0.074
ED length of stay (min)	1019	954	65.4 (− 72.8 to 203.7)	0.354	0.791	0.028
Mortality rate (%)	12.6	7.3	5.3 (2.4 to 8.3)	< 0.001	0.162	0.378

Adjusted by variables with statistical significance (P value < 0.10): age, sex, history of diabetes mellitus, hypertension, heart failure, cerebrovascular disease, renal disease, liver disease, and known neoplasm, systolic blood pressure, respiratory rate, pulse rate, body temperature, white blood cell count, hematocrit, platelet count, glucose, albumin, BUN, PSI, admission rate.

Table 4 shows the blood culture results before and after intervention. The overall blood culture contamination rate was 1.5% (48/3026). After the implementation, the blood culture contamination rate was not changed (P  = 0.689). Antibiotics regimen changes based on positive blood culture results were not significantly different between the two groups (41.5%; 49/118 before implementation vs. 38.0%; 19/50 after implementation, P  = 0.181). Among the 48 patients with contaminations, 47.9% (23/48) had to undergo unnecessary phlebotomy for follow-up blood cultures and 12.5% (6/48) were administered unnecessary vancomycin treatment. After score implementation, the total adverse event rates decreased from 68.7% (22/32) to 50.0% (8/16). However, this difference did not reach statistical significance (P  = 0.297).

Table 4

Blood culture results with contamination and antibiotics regimen changes before and after implementation of bacteremia prediction model.

Category, n (%)	Before intervention (n = 2049)	After intervention (n = 1157)	P value
False positive result	32 (1.5)	16 (1.4)	0.689
True bacteremia	86 (4.2)	34 (2.9)	0.071
Positive culture result	118 (5.8)	50 (4.3)	0.079
Antibiotics step upa	17 (0.8)	9 (0.8)	0.875
Adverse eventb	22 (1.1)	8 (0.7)	0.280
Low risk group (%)c	1725 (82.4)	951 (67.7)	< 0.001
True bacteremia	42 (2.4)	19 (2.0)	0.469
False positive result	26 (1.5)	14 (1.5)	0.901
Antibiotics step upa	8 (0.5)	5 (0.5)	0.780
Adverse eventb	17 (1.0)	7 (0.7)	0.669
Moderate risk group (%)c	287 (97.3)	185 (97.9)	0.771
True bacteremia	33 (11.5)	11 (6.0)	0.043
False positive result	6 (2.1)	2 (1.1)	0.490
Antibiotics step upa	7 (2.4)	3 (1.6)	0.747
Adverse eventb	5 (1.7)	1 (0.6)	0.411
High risk group (%)c	27 (93.1)	15 (100)	0.540
True bacteremia	11 (40.7)	4 (26.7)	0.362
False positive result	0	0
Antibiotics step upa	2 (7.4)	1 (6.7)	1.000
Adverse eventb	0	0

Antibiotics step up including adding or changing to vancomycin, carbapenem, or antifungal agents according to isolated pathogens.

Adverse event including unnecessary follow up blood culture or adding vancomycin due to contaminant pathogens.

Percentages of patients to whom blood culture was performed from the total patients stratified each risk groups.

Subgroup analysis

In the propensity score matched subgroup analysis, we compared two groups that used the prediction rule and did not, during intervention period (Supplementary Table 1). The baseline characteristics of the two groups were similar but, the group with the bacteremia score had the lower blood culture rate than the group that did not use the bacteremia score (54.6% vs. 72.5%, P  < 0.001). Additionally, the group that use the prediction rule had shorter time-interval to initial antibiotics (198 min vs. 223 min, P  = 0.002) and reduced ED length of stay (371 min vs. 419 min, P  = 0.007).

Discussion

In our time series analysis study, we reported that the implementation of the bacteremia prediction rule successfully reduced the blood culture rate without any significant 30-day mortality and antibiotics regimen changes.

Blood culture technique is time-consuming and resource-consuming considering the need for an aseptic technique and venous access to more than two sites [15], [16]. Especially in the ED, it is well known that blood culture contamination is common, and many EDs have higher contamination rates than Clinical and Laboratory Standards Institute (CLSI) recommendation of 3% [17], [18], [19]. False positive results from blood cultures are associated with increased medical cost and hospital length of stay [20], [21]. Clinically, blood culture contaminations result in unnecessary antibiotics administration [15]. In one previous study, a 20% of additional unnecessary glycopeptide administration in pseudobacteremia cases was observed, which is consistent with our results (Table 4) [22].

As multidrug resistance pathogens in pneumonia have been increasing, concerns exist about treatment failure without initial blood cultures [23]. However, in a recent study, multidrug resistance pathogens were uncommon in CAP patients, which accounted for 1.9% [24]. According to our bacteremia score system, blood cultures could be omitted in patients in the low-risk group, of which the true bacteremia rate was < 3% [10]. In this study, the true bacteremia in the low-risk group was 2.3% (51/2676) and antibiotics step up due to multidrug resistance organisms isolated from blood cultures was 0.5% (13/2676). Concerning the adverse event rate due to false positive blood cultures of 0.9% (24/2676), it seems reasonable to omit blood culture in the low-risk group (Table 4).

Contrary to our hypothesis, the time to initial antibiotics and ED length of stay tended to decrease, but no statistically significant differences were measured between the before and after intervention groups (Table 3). According to the sepsis guideline, initial antibiotics should be administered within 1 h after recognition of sepsis [25]. For patients with severe initial presentation, ED physicians may urge the initiation of IV antibiotics and blood cultures without adherence to the bacteremia score. Calculation of the bacteremia score takes > 1.5 h because the scoring system includes albumin and C-reactive protein results. This fact may be why the bacteremia score adherence group had mild presentations and lower Pneumonia Severity Index (PSI) scores (Supplementary Table 1). For patients with mild symptoms and stable initial vital signs, ED physicians may wait for the laboratory tests before the initiation of IV antibiotics. Those patients may be stratified into the low-risk group for bacteremia and could have more chances to avoid unnecessary blood cultures of low yield. Therefore, by the time a decision had to be made regarding initiating IV antibiotics without blood culture, > 90 min had already passed, and the time-saving effect of omitting blood cultures may have been diminished.

This study has several limitations. First because this study was a time series before and after study based on a retrospective review of a registry. Missing not at random (MNAR) which occupied a small fraction (116/4130; 2.8%) but not included in analysis, may have created a bias. Also there could be many confounders influencing the results besides the intervention. Second, the baseline characteristics of the two groups before and after intervention were significantly different. The Middle East Respiratory Syndrome coronavirus (MERS-CoV) outbreak in 2015 may have increased the sensitivity of the ED visit population to respiratory symptoms, which may be one of the reasons accounting for the difference [26]. Third, the adherence rate for the bacteremia score was 36.5% which may be too low to expect clinical and statistical significance. Generally, evidence-based changes in the clinical practice of physicians take a substantial amount of time and effort [27], [28]. In addition to those periods of adaptation, the time needed for calculation of the score make a decrease of adherence inevitable in severe patients for whom blood cultures and IV antibiotics were urgent. Additionally, the calculation of the score with multiplication and addition is difficult and the use of Excel file could be another cumbersome step to emergency physicians in a crowded ED. Finally the target population only included patients with CAP and this would limit the broad application of the rule in the clinical field. Despite those limitations, we could observed a statistically significant reduction in blood cultures after intervention without affecting 30-day mortality as well as decreasing trends in the initial antibiotics administration time and ED length of stay. Considering the low adherence rate to the score system, a more thorough education and feedback is needed to achieve statistically significant clinical impacts on the time of initial antibiotics administration and ED length of stay.

In conclusion, the implementation of the bacteremia prediction rule in CAP patients reduced the blood culture rate without affecting the 30-day mortality and antibiotics regimen. And bacteremia prediction score was independently associated with a reduction in initial antibiotics administration time and ED length of stay. Further prospective multicenter studies should be performed to evaluate the impact of wide implementation of the clinical prediction rule.

Source of support

None.