Criteria for Identifying Patients With Staphylococcus aureus Bacteremia Who Are at Low Risk of Endocarditis: A Systematic Review.

This systematic review examines the methods and results of recent studies reporting clinical criteria able to identify patients with Staphylococcus aureus bacteremia who are at very low risk of endocarditis. We searched PubMed, EMBASE, and the Cochrane Collaboration CENTRAL database for articles published after March 1994 using a combination of MeSH and free text search terms for S. aureus AND bacteremia AND endocarditis. Studies were included if they presented a combination of clinical and microbiological criteria with a negative likelihood ratio of ≤0.20 for endocarditis. We found 8 studies employing various criteria and reference standards whose criteria were associated with negative likelihood ratios between 0.00 and 0.19 (corresponding to 0%-5% risk of endocarditis at 20% background prevalence). The benefit of echocardiography for patients fulfilling these criteria is uncertain.

Up to 25% of cases of Staphylococcus aureus bacteremia (SAB) are complicated by infective endocarditis [1, 2], resulting in significant excess mortality compared with uncomplicated cases of SAB [3, 4]. Echocardiography has become the mainstay of diagnosis for this condition, both because of modern diagnostic definitions [5] and because many cases are not clinically evident prior to echocardiographic examination [1]. As a result, current guidelines [6-8] recommend echocardiography in all cases of SAB for the purpose of diagnosing endocarditis. Implicit in this recommendation is an assumption that either the risks and costs of echocardiography are outweighed by the benefits of identifying clinically occult endocarditis for all patients with SAB (which include preventing relapse through an appropriate antibiotic duration or identifying complicated disease requiring cardiac surgery) or that the subgroup who may not benefit are not identifiable by other means.

Until recently, the lowest identifiable risk of endocarditis was thought to be amongst patients with intravascular catheter-associated nosocomial SAB. These patients have approximately a 5%–10% risk of infective endocarditis [9, 10], a level of risk that was thought to justify routine transoesophageal echocardiography (TEE) [9]. More recently, a number of studies have identified criteria able to identify patients at an even lower risk of endocarditis for whom echocardiography might be unnecessary. We performed a systematic search of the literature to identify and compare these criteria and to explore their implications for the management of SAB.

PATIENTS AND METHODS

This review was conducted in accordance with the PRISMA statement [11], where relevant for nonrandomized studies not subjected to meta-analysis.

Criteria for Inclusion

We aimed to collect all studies of adult patients with SAB published in the era of the Duke criteria (after March 1994) [5] reporting combinations of clinical and microbiological criteria able to confer a low risk of endocarditis prior to the performance of echocardiography. We used a negative likelihood ratio of ≤0.20 to define low risk to account for variations in the underlying prevalence and distribution of endocarditis between series. Patients meeting criteria with a negative likelihood ratio of ≤0.20 would have a probability of endocarditis of <5% even at a background prevalence of 20%. Studies were included if they (i) examined a consecutive series of patients with SAB, (ii) reported the prevalence of endocarditis among patients meeting combinations of clinical and microbiological criteria, and (iii) provided sufficient data to construct a 2 × 2 contingency table for each combination to allow for the calculation of likelihood ratios. Series containing only patients with S. aureus endocarditis and those only reporting data from within a selected low-risk subgroup (eg, nosocomial SAB) were excluded, as likelihood ratios able to be applied to an unselected patient with SAB would not be calculable. Studies examining serological assays no longer in use (eg, antiteichoic acid serology) were not included. Where studies presented a multivariate regression equation (including intercept), a scoring system, or multiple criteria sets, we analyzed the most inclusive dichotomization still maintaining a negative likelihood ratio of ≤0.20.

Search Method

PubMed, EMBASE, and the Cochrane Collaboration CENTRAL database were searched on February 16, 2017, for articles published after March 1994 using a combination of MeSH and free text search terms for S. aureus AND bacteremia AND endocarditis. Search terms for S. aureus were “Staphylococcus aureus” OR “S* aureus”; for bacteremia, they were “bacteremia,” “bacteraemia,” “bloodstream infection,” “septicemia,” “sepsis,” OR “positive blood culture”; and for endocarditis, we searched “endocarditis.” We did not use search terms directed toward the measurements of interest (sensitivity, specificity, etc.) as they were unreliably included in lists of MeSH terms or in published titles and abstracts. Two authors (G.H., K.C.) performed the database search and reviewed all titles and abstracts for relevance. Full-text articles were retrieved for review for all studies meeting the inclusion criteria, and for those where there was uncertainty after review of the title and abstract. The reference lists of included articles were hand-searched for further articles not identified by the search strategy.

Data Extraction

Two authors (G.H., K.C.) performed the data extraction for all included studies. As well as measures of diagnostic performance, we extracted demographic and clinical details of the study population, the criteria used to define the low-risk group, the method of ascertainment of cases of endocarditis, and rates of echocardiography performance in the whole series and within the identified low-risk group.

Assessment of Bias of Included Studies

The risk of bias in included studies was assessed using the QUADAS-2 tool [12]. Two authors (G.H., K.C.) independently assessed the risk of bias for each included study, and differences were resolved by discussion.

Statistical Analysis

Due to the anticipated heterogeneity of the criteria sets, we did not plan to perform meta-analysis. For each study, we calculated the sensitivity, specificity, and negative predictive value of the criteria set accompanied by 95% confidence intervals based on the Wilson score. Negative likelihood ratios were calculated for each study and were accompanied by 95% confidence intervals using either the method of Simel et al. [13] or the bootstrapping approach described by Marill et al. [14] when sensitivity was equal to 1.00.

RESULTS

The database searches returned 1182 unique records. Screening of titles and abstracts excluded 1124 records, leaving 58 for full-text review, of which 8 fulfilled the inclusion criteria [2, 15–21] (Figure 1). Among the excluded studies, 5 reported criteria applied only to a selected low-risk subset of patients with SAB, whose performance for unselected patients was not calculable, and 1 presented criteria with a negative likelihood ratio of >0.20. These 6 studies are discussed separately below. We also excluded 18 studies reporting data for individual (but not combinations of) risk factors for endocarditis in SAB; 2 [22, 23] presented odds ratios from multivariate analyses but did not provide complete regression equations for the construction of contingency tables for combinations of criteria.

Figure 1.

Literature search flow chart.

Characteristics of Included Studies

The included studies were published between 2011 and 2016 and included between 177 and 2008 episodes of SAB. Three studies were carried out in Europe [2, 15, 21], 3 in North America [16, 19, 20], and 2 in Australasia [17, 18]. All studies were based in tertiary referral centers. Two studies were prospective multicenter studies [2, 21], and the remainder were single-center retrospective studies [15, 17, 19, 20] or retrospective assessments of previous prospective series [16, 18]. The apparent prevalence of endocarditis in the included series ranged from 6% [18] to 24% [17] and was strongly associated with the chosen reference standard (Table 3).

Table 3.

Performance of Low-risk Criteria for Endocarditis in Included Studies, Ordered by Proportion of Patients Undergoing Transesophageal Echocardiography

First Author	Reference Standard	TEE, %	Apparent Endocarditis Prevalence, %	Sensitivitya (95% CI)	Specificitya (95% CI)	NPV (in Original Series)a (95% CI)	NLR (95% CI)
Heriot et al. [17]	TEE	100	24	1.00 (0.93–1.00)	0.15 (0.10–0.21)	1.00 (0.86–1.00)	0.00 (0.00–0.42)b
Khatib et al. [16]	TEE	100	24	0.98 (0.88–1.00)	0.22 (0.16–0.30)	0.97 (0.84–0.99)	0.11 (0.02–0.76)c
Palrajd et al. [19]	Duke criteria	72	13	0.94 (0.87–0.97)	0.41 (0.37–0.45)	0.98 (0.95–0.99)	0.15 (0.06–0.35)c
Rasmussen et al. [2]	Duke criteriae	62	22	0.89 (0.77–0.95)	0.60 (0.53–0.66)	0.95 (0.90–0.98)	0.19 (0.09–0.41)a
Buitron de la Vega et al. [20]	Duke criteria	32	11	1.00 (0.92–1.00)	0.19 (0.15–0.24)	1.00 (0.95–1.00)	0.00 (0.00–0.33)b
Tubianaf et al. [21]	Duke criteria	30	11	0.96 (0.92–0.98)	0.44 (0.42–0.46)	0.99 (0.98–0.99)	0.09 (0.05–0.18)c
Joseph et al. [15]	TTE or TEE	27	10	1.00 (0.89–1.00)	0.38 (0.33–0.44)	1.00 (0.96–1.00)	0.00 (0.00–0.23)b
Gow et al. [18]	Duke criteria	16	6	1.00 (0.90–1.00)	0.15 (0.12–0.18)	1.00 (0.95–1.00)	0.00 (0.00–0.57)b

Abbreviations: NLR: negative likelihood ratio; NPV: negative predictive value; TEE: transoesophageal echocardiography; TTE: transthoracic echocardiography.

Proportions accompanied by Wilson score 95% confidence intervals.

Bootstrapped 95% confidence interval.

Simel 95% confidence interval.

Data presented for PREDICT day 5 score ≥2.

All patients underwent either TTE or TEE.

Data presented for VIRSTA score ≥3.

All of the included studies started with a consecutive series of patients with ≥1 blood cultures positive for S. aureus, as identified by microbiology laboratory records. One study [20] included only patients with methicillin-resistant S. aureus, and 2 excluded patients where the significance of the positive culture was questioned [16, 21]. Four studies excluded patients admitted for fewer than 48 hours after collection of the initial blood culture [2, 16–18], and 2 excluded patients who died prior to completing the intended diagnostic work-up [2, 19]. One study also excluded patients who were neutropenic at the time of the first blood culture [2], and another excluded patients who suffered a relapse of SAB not thought to be due to endocarditis by investigators within 100 days of the initial episode [18].

Four studies applied prespecified criteria [2, 16, 17, 20]. The other 4 assessed the performance of criteria generated by univariate [15] or multivariate logistic regression [18, 19, 21] of the same cohort. None of these last 4 studies examined an independent validation cohort.

Four studies were restricted to patients with echocardiography: 2 of these examined patients undergoing either TTE or TEE [2, 15], and 2 were restricted to patients undergoing TEE [16, 17]. As a result of this selection, these studies only examined 18% [16] to 57% [2] of the original consecutive series of SAB episodes at their sites. Three of these studies [15-17] used findings of endocarditis on echocardiography performed at a median of 7–10 days after the first positive culture as their reference standard; the fourth [2] used the modified Duke criteria (including echocardiography results) at 30 days after discharge.

The other 4 studies reported the performance of their criteria for all included episodes of SAB, using the modified Duke criteria applied at 5–12 weeks after the initial culture as the reference standard, even where no echocardiography had been performed. In these 4 studies, the proportion of all included episodes examined with echocardiography ranged between 60% [20] and 72% [19], and the proportion examined with TEE varied between 16% [18] and 72% [19]. In the 3 studies where data were available, echocardiography rates among patients meeting the low-risk criteria were lower than in patients reported to be at higher risk of endocarditis [18, 20, 21]. Only 1 study reported the median duration of antibiotic therapy for patients not considered to have endocarditis (28 days) [19].

Low-risk Criteria

Table 1 presents the criteria used to identify patients at low risk of endocarditis in each included study. No 2 studies examined identical criteria sets. The number of separate specified criteria varied between 2 [15] and 10 [2], and criteria ranged from the fixed and objective to the potentially changeable and subjective (such as the clinical identification of a source of bacteremia [2, 15, 18]). The most commonly used criteria were the presence of an intracardiac prosthetic device (all studies in some form), and prolonged bacteremia (6 studies with varying definitions). Two studies presented scoring systems offering a number of potential partitions [19, 21]—we selected a PREDICT score of ≥2 in the study by Palraj et al. [19], and a VIRSTA score of ≥3 in the study by Tubiana et al. [21] for inclusion in the analysis as these were the most inclusive dichotomizations that maintained a negative likelihood ratio of ≤0.20. Interestingly, these partitions were the same as those suggested for implementation by the original authors, who did not present negative likelihood ratios or discuss the desirable postcriteria probability of endocarditis.

Table 1.

Criteria Used to Identify Patients at Low Risk of Endocarditis

	Rasmussen et al. [2]	Joseph et al. [15]	Khatib et al. [16]	Heriot et al. [17]	Gow et al. [18]	Palraj et al. [19]a	Buitron de la Vega et al. [20]	Tubiana et al. [21]b
Source/acquisition
Nosocomial bacteremiac				+
Health care–associated (including nosocomial) bacteremiae						+		+d
Central line–associated bacteremia		+			+
Known source of bacteremia	+
Presence of an implantable central venous catheter							+
Duration of bacteremia, h
<12							+
<48								+
<72			+	+		+
<96					+
Preexisting risk factors
No prosthetic heart valve	+	+	+	+	+		+	+
No cardiac rhythm management device	+	+	+	+	+	+	+	+
No dialysis dependency							+
No intravenous drug use	+						+
No preexisting cardiac abnormality	+f						+g	+f
Clinical signs of endocarditis
No embolic eventsg	+		+					+
No murmur	+
No heart failure	+
No immunological phenomenah	+
No severe sepsis with C-reactive protein >190 mg/L								+d
Other foci of infection
No vertebral osteomyelitis or epidural abscess			+				+	+d
No appendicular osteomyelitis			+				+
No meningitis			+					+
No secondary focus or relapse apparent within 100 d			+

Except where indicated, low risk cases were required to fulfill all listed criteria.

Criteria for PREDICT day 5 score <2.

Criteria for VIRSTA score <3.

First positive blood culture collected more than 48 hours after hospital admission.

VIRSTA score of <3 requires no more than 1 of these 3 criteria.

As defined by Friedman et al. [26].

Known native heart valve disease or previous infective endocarditis.

“Cardio-structural abnormality” not further defined in text.

As per modified Duke criteria.

Risk of Bias in Included Studies

The results of the QUADAS-2 assessment of the included studies are presented in Table 2. For the purpose of this review, the ideal study would examine a complete consecutive series of patients with SAB, all of whom would undergo an appropriately timed TEE as part of a complete assessment of the presence of endocarditis according to the modified Duke criteria (or similar reference criteria) [5]. As noted above, all 8 studies in this review either used an insensitive reference standard [18-21] or had uncertain applicability due to selection on the basis of echocardiography performance [2, 15–17]. Additional potential sources of bias related to (i) the index test in 3 studies including a clinical assessment of the source of bacteremia [2, 15, 18]; (ii) the reference standard in 1 study that accepted TTE findings without TEE [15] despite the inadequate diagnostic performance of this test [25]; and (iii) study flow and timing in 1 study due to losses to follow-up where follow-up data were required to complete the index test [16]. This latter study was also considered to have problems with applicability to the review question due to the inclusion of 100-day follow-up data in criteria to be used to select patients for echocardiography early after the onset of bacteremia. One study [20] was restricted to patients with methicillin-resistant S. aureus, but it was deemed not to have major applicability concerns given the absence of consistent evidence suggesting different rates of endocarditis in these patients compared with those with methicillin-susceptible isolates [22].

Table 2.

QUADAS-2 Assessment of Risk of Bias for the 8 Included Studies

Study	Risk of Bias				Applicability
	Patient Selection	Index Test	Reference Standard	Flow and Timing	Patient Selection	Index Test	Reference Standard
Rasmussen et al. [2]	✓	✗	✓	✓	✗	✓	✓
Joseph et al. [15]	✓	✗	✗	✓	✗	✓	✓
Khatib et al. [16]	✓	✓	✓	✗	✗	✗	✓
Heriot et al. [17]	✓	✓	✓	✓	✗	✓	✓
Gow et al. [18]	✓	✗	✗	✓	✓	✓	✓
Palraj [19]	✓	✓	✗	✓	✓	✓	✓
Buitron de la Vega et al. [20]	✓	✓	✗	✓	✓	✓	✓
Tubiana et al. [21]	✓	✓	✗	✓	✓	✓	✓

For this review, “Patient Selection” refers to the process by which episodes of sab were identified and selected for inclusion in the reported diagnostic performance statistics; “Index Test” refers to the criteria used to identify patients at very low risk of endocarditis; the “Reference Standard” was the means by which patients received an ultimate diagnosis of endocarditis; and “Flow and Timing” refers to the temporal relationship between the onset of bacteremia, the application of the criteria, and the reference standard.

✓ low risk of bias; ✗ high risk of bias.

Diagnostic Performance of Criteria Sets

For the sake of consistency, the diagnostic performance of included studies was calculated by defining episodes of SAB not meeting the low-risk criteria as “test positive” and those meeting the low-risk criteria (or with a subthreshold score) as “test negative.” As such, sensitivity is the proportion of cases of endocarditis failing the low-risk criteria, and specificity is the proportion of episodes without endocarditis that satisfy the low-risk criteria. Table 3 presents the diagnostic performance of the various criteria sets alongside the reference standards and transoesophageal echocardiography rates.

Excluded Studies

Five studies were excluded from the review because they examined a low-risk selection of SAB episodes where a negative likelihood ratio was not calculable for the original unselected series. Two examined episodes of health care–associated SAB only [26, 27] and found low rates of endocarditis for patients without prolonged bacteremia or predisposing factors such as the presence of an intracardiac prosthetic device. Another [28] included only episodes of nosocomial bacteremia and reported that the absence of prolonged bacteremia, an intracardiac prosthetic device, dialysis dependency, spinal infection, and osteomyelitis at any site conferred a negative likelihood ratio of 0.06 for endocarditis among these patients. A fourth study [29] examined a cohort of non-neutropenic oncology patients and reported that the documentation of a wound or pulmonary source of SAB gave a negative likelihood ratio of 0.25. The final study [30] examined patients without intracardiac prosthetic devices and reported a negative likelihood ratio of 0.24 for patients who had no clinical evidence of peripheral emboli. Four of these studies used the modified Duke criteria as the reference standard and reported echocardiography rates between 50% and 83% (TEE in 28%–50%); the fifth study [30] was restricted to patients investigated with TEE.

One other study [1] reported a combination of clinical criteria that failed to achieve a negative likelihood ratio of ≤0.20. In this study of 144 episodes of SAB undergoing TEE (63% of a single-institution consecutive series), the absence of a clinical suspicion of endocarditis (defined as more than 1 minor Duke criterion in the context of SAB), community onset of bacteremia, and a “preexisting valve lesion” (a prosthetic valve or native valve disease resulting in “significant regurgitation or turbulence of blood flow”) conferred a likelihood ratio of 0.51 (0.29–0.85). This criteria set is most similar to that of Tubiana et al. [21], but used universal TEE as the reference standard and did not include the requirement for a brief duration of bacteremia.

Discussion

While significant differences exist in the criteria, reference standards, and precision of the studies presented in this review, there does appear to be a group of patients with SAB who can be reliably and fairly easily identified as having a very low risk of endocarditis prior to the performance of echocardiography. In general terms, this group is comprised of patients with health care–associated SAB (including nosocomial- and central line–associated bacteremia) [26] who lack intracardiac prosthetic devices (prosthetic valves and rhythm management devicces) and clinical signs of endocarditis, although a documented brief duration of bacteremia (<48–72 hours) may be a prudent additional requirement [1]. These criteria can be assessed with information collected as part of routine care within 5 days of the onset of bacteremia, around the time when decisions regarding screening echocardiography need to be made [6, 7].

Although this is a promising result, there are 2 issues that are likely to prevent the immediate adoption of a risk-stratification approach to echocardiography in SAB based on 1 or more of these criteria sets: (i) the validity of their reported diagnostic performance and (ii) the appropriateness of their use as a triage tool for echocardiography.

Ironically, the major challenge to the validity of the diagnostic performance of the criteria presented in this review stems from the fact that clinicians managing SAB already seem to be selecting patients for echocardiography based on endocarditis risk. The proportion of SAB episodes investigated with echocardiography in the included studies ranged from 43% [16] to 73% [2], and all but 2 [2, 19] reported rates of TEE performance of less than 40%. These results are similar to other published series of SAB, even for patients receiving infectious diseases consultation [31-35], suggesting deliberate selection that appears to be largely based on an assessment of endocarditis risk [36].

As a result, studies in this review either prioritized the sensitivity of their reference standard or the applicability and precision of their results. Three of the 8 studies only included patients with a sensitive reference standard, either TEE (>90% sensitive for endocarditis compared with surgical findings or autopsy) [37] or the modified Duke criteria including echocardiographic assessment (~95% sensitive compared with valve histopathology in the setting of positive blood cultures) [38], limiting their generalizability to patients not selected for echocardiography in these and other settings. In contrast, the 4 most recent studies included all episodes of SAB but used the modified Duke criteria as the reference standard regardless of the performance of echocardiography. While the authors of these studies felt that few patients with occult endocarditis would remain unrecognized throughout the period of follow-up, there are few data to support this assumption, particularly in the setting of prolonged antibiotic therapy. Furthermore, where reported, echocardiography rates (particularly TEE) in these studies were lower in patients reported to be at low risk of endocarditis compared with those at higher risk, thereby exaggerating the reported negative likelihood ratios.

None of the studies included in this review have been directly replicated either deliberately or accidentally due to comparable criteria sets. Tubiana et al. [21] performed bootstrap aggregation on their multisite data set to minimize overfitting, but did not examine a separate validation data set. Gow et al. [18] reported a 4% prevalence of endocarditis for patients meeting the criteria reported in the study by Heriot et al. [17] in their series, although details were not provided. After the completion of this review, we retrieved all articles citing 1 or more of the included studies looking for other examples of replication—none were found.

Putting aside issues of validity, it remains unclear whether the likelihood of endocarditis among patients fulfilling these low-risk criteria is low enough that the harms of echocardiography (either from the procedure itself or from erroneous results) might outweigh its benefits (guiding antibiotic duration and identifying intracardiac complications of endocarditis). The only previous decision analysis of TEE in SAB [9] suggested that TEE was cost-effective only above a 2% risk of endocarditis. In the base-case example used in this study (pretest probability of endocarditis, 6.1%), TEE offered only an additional 16 days of quality-adjusted life expectancy over short-course antibiotics alone. In order for the studies included in this review to be used as clinical decision aids, the probability of endocarditis that they confer must be interpreted in the context of the harms and benefits of testing and treatment for endocarditis.

Since the completion of this review, we have become aware of a similar review currently in press elsewhere [39]. Our review differs from the one by Bai et al. in a number of important ways: (i) our more recent database search included 2 recent studies [20, 21] that do not appear among the clinical prediction rules discussed by Bai et al., including the largest study published to date [21]; (ii) we have been more restrictive in our criteria for inclusion to ensure that the criteria sets we discuss all apply to all patients with SAB prior to echocardiography; (iii) we consider the incomplete echocardiography coverage in studies using the modified Duke criteria as a reference standard to be a major validity concern, as discussed above, whereas Bai et al. do not consider this issue; and (iv) we have avoided the altered negative likelihood ratios, artefactual distinction between studies with 100% sensitivity (which have a true point estimate negative likelihood ratio of 0) [40], and inappropriate 95% confidence intervals [14] all generated by adding a continuity correction of ε = 0.5 to empty cells by instead employing a bootstrapping approach to generate empiric 95% confidence intervals around the likelihood ratio estimates.

Our review has some limitations. Although we tried to be inclusive in our search strategy, the variability of abstracts and MeSH terms for observational studies may have resulted in the omission of relevant studies with criteria reported only in the body of the manuscript. Second, we chose not to attempt meta-analysis or pooled analysis due to the significant heterogeneity in the criteria and definitions used in the different studies.

Conclusions

Despite various methodological limitations, the studies presented in this review suggest that there is an identifiable group of patients with SAB who have a very low risk of endocarditis. The benefit of TEE for the purpose of diagnosing endocarditis in these patients is questionable, although work is needed to better define the probability of endocarditis below which its risks might outweigh its benefits.