Prediction Rules for Ruling Out Endocarditis in Patients With Staphylococcus aureus Bacteremia.

BACKGROUND: Staphylococcus aureus bacteremia (SAB) is in 10% to 20% of cases complicated by infective endocarditis. Clinical prediction scores may select patients with SAB at highest risk for endocarditis, improving the diagnostic process of endocarditis. We compared the accuracy of the Prediction Of Staphylococcus aureus Infective endocarditiseTime to positivity, Iv drug use, Vascular phenomena, preExisting heart condition (POSITIVE), Predicting Risk of Endocarditis Using a Clinical Tool (PREDICT), and VIRSTA scores for classifying the likelihood of endocarditis in patients with SAB.
METHODS: Between August 2017 and September 2019, we enrolled consecutive adult patients with SAB in a prospective cohort study in 7 hospitals in the Netherlands. Using the modified Duke Criteria for definite endocarditis as reference standard, sensitivity, specificity, negative predictive (NPV), and positive predictive values were determined for the POSITIVE, PREDICT, and VIRSTA scores. An NPV of at least 98% was considered safe for excluding endocarditis.
RESULTS: Of 477 SAB patients enrolled, 33% had community-acquired SAB, 8% had a prosthetic valve, and 11% a cardiac implantable electronic device. Echocardiography was performed in 87% of patients, and 42% received transesophageal echocardiography (TEE). Eighty-seven (18.2%) had definite endocarditis. Sensitivity was 77.6% (65.8%-86.9%), 85.1% (75.8%-91.8%), and 98.9% (95.7%-100%) for the POSITIVE (n = 362), PREDICT, and VIRSTA scores, respectively. NPVs were 92.5% (87.9%-95.8%), 94.5% (90.7%-97.0%), and 99.3% (94.9%-100%). For the POSITIVE, PREDICT, and VIRSTA scores, 44.5%, 50.7%, and 70.9% of patients with SAB, respectively, were classified as at high risk for endocarditis.
CONCLUSIONS: Only the VIRSTA score had an NPV of at least 98%, but at the expense of a high number of patients classified as high risk and thus requiring TEE. CLINICAL TRIALS REGISTRATION: Netherlands Trial Register code 6669.

Staphylococcus aureus is a common cause of bacteremia [1] and is associated with myriad clinical syndromes, ranging from an uncomplicated central line-associated bacteremia to fulminant endocarditis. In prospective cohort studies, 10%–20% of S aureus bacteremia (SAB) episodes is complicated by infective endocarditis (IE) [2, 3]. Recognition of IE can be difficult, as up to 40% of S aureus infective endocarditis (SA-IE) occurs in patients without known predisposing risk factors for IE [4], and typical signs of endocarditis (eg, heart murmur, embolic event) may be absent on presentation [2-4].

Echocardiography is the preferred imaging modality screen for patients with a clinical suspicion of IE, with transesophageal echocardiography (TEE) having a higher sensitivity than transthoracic echocardiography [5]. However, TEE is an invasive and expensive procedure and not feasible for all patients with SAB in most healthcare settings [6, 7].

There are a number of well-recognized risk factors for IE, and several sets of clinical criteria have been developed to help clinicians decide which patients should undergo TEE [6]. In the past 5 years, 3 multivariate prediction rules have been proposed to classify patients with SAB as being at either high or low risk for SA-IE. The Predicting Risk of Endocarditis Using a Clinical Tool (PREDICT) score, based on a cohort of 678 patients from a single center in the United States, provides risk scores for day 1 and day 5 of SAB [8]. The VIRSTA score, developed using a multicenter prospective cohort of 2008 patients with SAB in France, provides a risk score for day 4 of SAB [9]. The Prediction Of Staphylococcus aureus Infective endocarditiseTime to positivity, Iv drug use, Vascular phenomena, preExisting heart condition POSITIVE score is based on a cohort of 465 patients with SAB from Sweden [10]. All scores use different combinations of known risk factors and PREDICT and VIRSTA also use results of follow-up blood cultures, whereas POSITVE ignores follow-up cultures but uses time to positivity (TTP) of blood cultures as a predictor (Table 1).

Table 1.

Overview of POSITIVE, PREDICT, and VIRSTA Scores

POSITIVE Cutoff: >4		PREDICT Cutoff: ≥2 (for Day 5 Score)		VIRSTA Cutoff: ≥3
Item	Points Assigned	Item	Points Assigned	Item	Points Assigned
TTP <9 h	5	ICD	2	Cerebral or peripheral emboli	5
TTP 9–11 h	3	Permanent pacemaker	3	Meningitis	5
TTP 11–13 h	2	Community acquisition	2	Permanent intracardiac device or previous IE	4
IV drug use	3	Healthcare acquisition	1	Preexisting native valve diseasea	3
Vascular phenomenab	6	Positive culture after 72 h	2	IV drug use	4
Predisposing heart diseasec	5			Positive culture after 48 h	3
				Community or healthcare-associated bacteremia	2
				Severe sepsis or septic shock	1
				C-reactive protein >190 mg/L	1

Abbreviations: ICD, implantable cardioverter defibrillator; IV, intravenous; TTP, time to positivity.

aAny condition classified as medium or high risk by Dajani et al [25].

bDefined as arterial embolus, septic pulmonary embolus, mycotic aneurysm, intracranial bleeding, conjunctival hemorrhage, or Janeway lesions.

cPrevious endocarditis, prosthetic heart valve, or any condition classified as medium or high risk [25].

VIRSTA and PREDICT reported excellent negative predictive values (NPV) for endocarditis in the development cohorts (98.5% for PREDICT, 98.8% for VIRSTA), and NPVs were 99.5% and 95.1%, for VIRSTA and PREDICT, respectively, in a retrospective cohort study [11]. The derivation study for the POSITIVE score did not report an NPV, but reported a sensitivity of 93%. This implies all 3 scores may improve the diagnostic process in patients with SAB, leading to more accurate diagnosis of SA-IE and more appropriate use of TEE.

External validation and comparison of these prediction rules in different clinical settings is necessary for future implementation in clinical practice [12]. In this study, we evaluate the diagnostic performance of the POSITIVE, PREDICT, and VIRSTA scores in a cohort of 477 patients with community-, hospital-, and healthcare-associated SAB. We aimed to determine whether these scores can be used to accurately classify a subset of patients with SAB as low risk for endocarditis, in whom TEE could be safely withheld without missing a diagnosis of SA-IE.

METHODS

Study Design

From August 2017 to September 2019, we conducted the Improved Diagnostic Strategies in Staphylococcus aureus study, a prospective, multicenter cohort study among hospitalized patients with SAB in 2 university hospitals and 5 teaching hospitals in the Netherlands. We follow the Standards for Reporting of Diagnostic Accuracy Studies guidelines for reporting the results [13].

Participants

All consecutive patients aged 18 years and older with 1 or more blood cultures positive for S aureus were eligible for inclusion. In patients with multiple episodes of SAB, we included only the first episode. For the current analysis, patients who died within 48 hours after blood culture collection were excluded because risk stratification is meaningless for patients who die before the prediction rule can be applied.

Patients were identified through each hospital’s microbiology service. A dedicated member of the study team (T.v.d.V.) then approached the patient or his or her legal representatives for written informed consent. Patients who died before informed consent could be obtained were included in the study as appropriate under Dutch law. The Medical Ethics Committee of the Academic Medical Centre Amsterdam approved this study (METC2017_094). This study is registered in the Netherlands Trial Register under trial code 6669. The study was originally intended to evaluate VIRSTA and PREDICT; POSITIVE was added to the analysis after its publication [10].

Procedures

Information needed to calculate the POSITIVE, PREDICT, and VIRSTA scores; signs and symptoms of SAB and SA-IE; and laboratory, microbiology, and imaging data were extracted from the electronic health record (EHR). We used the results from echocardiography, radiology, and nuclear medicine studies performed for regular care and as reported in the EHR. No additional imaging or microbiology studies were required for the study. Patients were followed for up to 90 days after bacteremia. Follow-up was conducted by telephone interview after day 90. For patients who could not be reached by telephone, we determined postdischarge health status through contact with the patient’s primary care physician, the hospital EHR, and the municipal death records. Data extraction was performed by the study research physician and trained junior researchers. The study research physician checked all data entered by the junior researchers. Data were stored in an electronic case record file (Research Online).

Definitions

Time of first positive blood culture was defined as the time the first blood culture that grew S aureus had been collected. Blood cultures were collected in BactTec (Becton Dickinson) and BacT/ALERT (Biomerieux) bottles and incubated in the appropriate automated culture systems. TTP was the interval between placement of the culture vial in the culture system and signal of positivity by the incubator. If more than 1 positive blood culture had been drawn on the first day of bacteremia, we used the shortest TTP from that day for the calculation of the POSITIVE score. Severe sepsis was defined according to the 2001 International Sepsis definition criteria [14]. Comorbidities were scored using the Charlson Comorbidity index and the McCabe score [15, 16].

Analysis

The prespecified cutoffs suggested by the authors of the POSITIVE, PREDICT, and VIRSTA scores were used to classify patients as either being at low or high risk of SA-IE [8-10]. There were no missing data needed to calculate the risk scores, except for values of C-reactive protein on the day of blood culture (required for VIRSTA). These missing C-reactive protein values were imputed using multiple imputation using the mice R-package. For follow-up blood culture results (required for PREDICT and VIRSTA), we assumed that if no blood cultures were drawn at the required timepoint, but later blood cultures were positive, the missed earlier blood cultures would have been positive (ie, if culture results at 48 hours were missing, but the blood culture at 72 hours was positive, we assumed the culture at 48 hours to be positive). If no further blood cultures had been collected, we assumed the follow-up cultures to be negative.

The reference standard was the presence of definite infective endocarditis according to the modified Duke Criteria [17]. The presence of endocarditis was used to calculate the diagnostic accuracy measures for the risk scores. The Duke Criteria were applied blinded to the results of the risk scores and vice versa.

The primary endpoint was the diagnostic accuracy of the POSITIVE, PREDICT, and VIRSTA scores for the diagnosis SA-IE. Sensitivity, specificity, NPV, and positive predictive value (PPV) were calculated, and 95% confidence intervals (CIs) were calculated using exact binary confidence intervals. We constructed receiver-operating curve (ROC) plots and calibration plots. All statistical analysis was done in R version 4.0.0 (R Core Team, 2019. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/).

We conducted 2 sensitivity analyses, 1 restricted to patients who received echocardiography, and the second to patients that had at least 1 follow-up culture drawn in the first 96 hours.

The potential impact of implementing the scores was determined by comparing the number of TEEs actually performed to the number of TEEs recommended by each risk score. In the impact analysis, patients who were unable to undergo TEE because of a contraindication or who had died before a TEE would have been possible were excluded.

We prespecified that a risk score would need a 98% NPV to be of clinical use. With a 1-sided difference of at most 2.5%, a power of 80%, an alpha of 5%, and an SA-IE prevalence of 15%, 419 patients with SAB had to be included in the study. To account for potential loss to follow-up and lower endocarditis prevalence than expected, we aimed to include 450 patients in this study.

RESULTS

Patients

Between August 2017 and September 2019, 637 patients with SAB were screened and informed consent was obtained for 491 (77%). Main reasons for noninclusion were refusal to provide informed consent (n = 29), discharged home before consent was possible (n = 46), and incapacitated patients without a legal representative (n = 22) (supplementary Table S1). For the analysis of the risk scores, 14 patients who died within 48 hours of blood culture collection were excluded, leaving a total of 477 patients. Detailed postdischarge information was available for 99% of patients (supplementary materials).

Demographics and Clinical Characteristics of Patients

The demographic and clinical characteristics of included patients are shown in Table 2. Predisposing conditions for IE were found in 30%, and 33% had community-acquired bacteremia. Echocardiography was performed in 87% of patients, and 91% of patients had at least 1 follow-up blood culture drawn after the index culture.

Table 2.

Demographic and Clinical Characteristics

	All Patients	No Endocarditis	Definite Endocarditis
	N = 477	N = 390	N = 87
Demographics
Sex
Female	157 (33%)	124 (32%)	32 (37%)
Male	320 (67%)	265 (68%)	55 (63%)
Age, y	68 (57–76)	68 (57–76)	68 (57–75)
Diabetes mellitus	153 (32%)	132 (34%)	21 (24%)
Immunosuppressant use	86 (18%)	76 (19%)	10 (11%)
HIV-AIDS	3 (1%)	3 (1%)	0 (0%)
Chronic renal failure	136 (28%)	114 (29%)	18 (21%)
Hemodialysis	28 (6%)	26 (7%)	2 (2%)
MRSA bacteremia	10 (2%)	9 (2%)	1 (1%)
Intravenous drug use	5 (1%)	1 (0%)	4 (5%)
Charlson Comorbidity Index	3 (2–5)	3 (2- 5)	2 (1–4)
McCabe score
Nonfatal	202 (43%)	163 (41%)	39 (45%)
Ultimately fatal	216 (45%)	170 (44%)	46 (53%)
Rapidly fatal	59 (12%)	57 (15%)	2 (2%)
Any predisposing cardiac condition	143 (30%)	94 (24%)	49 (56%)
History of endocarditis	13 (3%)	6 (2%)	7 (8%)
Native valve disease	86 (18%)	62 (16%)	24 (28%)
Prosthetic valve	39 (8%)	19 (5%)	20 (23%)
CRT-D	12 (3%)	5 (1%)	7 (8%)
ICD	10 (2%)	8 (2%)	2 (2%)
Pacemaker	28 (6%)	14 (4%)	14 (16%)
Place of acquisition
Community-acquired	165 (35%)	114 (29%)	51 (59%)
Healthcare-associated	155 (32%)	133 (34%)	22 (25%)
Hospital-acquired	157 (33%)	143 (37%)	14 (16%)
Signs and symptoms present within 72 h of first positive blood culture
Severe sepsis	186 (39%)	132 (34%)	54 (62%)
Septic shock	43 (9%)	29 (7%)	14 (16%)
C-reactive protein >190 mg/La	167 (40%)	115 (35%)	52 (63%)
Meningitis	5 (1%)	1 (0%)	4 (5%)
Vertebral osteomyelitis	24 (5%)	16 (4%)	8 (9%)
Intensive care as admitting specialty	38 (8%)	26 (7%)	12 (14%)
Positive follow-up culture at 48 h	142 (30%)	83 (22%)	59 (67%)
Positive follow-up culture at 72 h	103 (22%)	57 (15%)	46 (52%)
Time to positivityb	12.6 (8.8–16.9)	13.3 (9.7–17.9)	9.00 (6.6–11.9)
Management and outcomes of SAB
Follow-up culture taken within 96 h	434 (91%)	355 (91%)	79 (91%)
TTE performed	404 (85%)	320 (82%)	84 (97%)
TEE performed	201 (42%)	128 (33%)	73 (84%)
18-FDG PET/CT performed	179 (38%)	128 (33%)	51 (59%)
Infectious diseases consultation performed	383 (80%)	308 (79%)	75 (86%)
30-day mortality	101 (21%)	74 (19%)	27 (31%)
90-day mortality	146 (31%)	109 (28%)	37 (43%)
90-day relapse rate	13 (3%)	10 (3%)	3 (3%)

Data are n (%) or median + interquartile range unless otherwise indicated.

Abbreviations: CRT-D, Cardiac resynchronization device; HIV, human immunodeficiency virus; ICD, implantable cardioverter-defibrillator; MRSA, methicillin resistant Staphylococcus aureus; TEE, transesophageal echocardiography; TTE, transthoracic echocardiography; 18-FDG PET/CT, 18-fluorodeoxyglucose positron emission tomography/computed tomography.

aC-reactive protein at day of blood culture was missing for 63 patients: 4 patients with SA-IE and 59 patients without SA-IE.

bTime to positivity was available for 362/477 patients.

Prevalence of Endocarditis

Definite SA-IE was diagnosed in 87 patients (18.2%), in 63 (72.4%) patients based on 2 major criteria, and in 24 (27.6%) on 1 major with 3 or more minor criteria. Endocarditis was surgically or pathology proven in 17 (19.5%) of these patients, 14 of whom met the criteria for definite endocarditis before surgery or death. SA-IE involved native valves in 53 (60.9%) patients, prosthetic valves in 20 (23.0%), and a cardiac implantable electronic device (CIED) in 14 (16.1%).

Diagnostic Accuracy of POSITIVE, PREDICT, and VIRSTA Scores

TTP data were available for 6 of 7 hospitals (362 patients); diagnostic accuracy estimates of the POSITIVE score are reported for these patients. The VIRSTA score met the prespecified NPV threshold of 98% with an NPV of 99.3 (95% CI, 94.9–100) and also had the highest discriminatory ability (area under the ROC curve: 88.9 [95% CI, 85.3–92.5] (Table 3). Figure 1 shows the ROC curves of the risk scores. Visual inspection of the calibration curves (Figure 2) shows that VIRSTA systematically overestimates the risk of endocarditis, whereas PREDICT and POSITVE underestimate the risk. Findings from primary analyses were robust in both the echocardiography and the follow-up blood culture sensitivity analyses (supplementary Tables S2 and S3).

Table 3.

Diagnostic Accuracies of POSITVE, PREDICT, and VIRSTA Scores

Score	Sensitivity (% + 95% CI)	Specificity (% + 95% CI)	Negative Predictive Value (% + 95% CI)	Positive Predictive Value (% + 95% CI)	AUC
POSITIVEa	77.6 (65.8–86.9)	63.1 (57.3–68.6)	92.5 (87.9–95.8)	32.3 (25.1–40.1)	77.8 (71.9–83.7)
PREDICT day 1	22.9 (14.6–33.5)	97.4 (95.3–98.8)	85.0 (81.4–88.2)	66.7 (47.2–82.7)	71.4 (65.2–77.5)
PREDICT day 5	85.1 (75.8–91.8)	56.9 (51.8–61.9)	94.5 (90.7–97.0)	30.5 (24.7–36.8)	79.7 (73.9–85.4)
VIRSTA	98.9 (95.7–100)	35.7 (30.8–40.6)	99.3 (94.9–100)	25.5 (20.7–30.3)	88.9 (85.3–92.5)

Abbreviations: AUC, area under the receiver operating characteristic curve; CI, confidence interval.

aPOSITIVE score was calculated on the TTP cohort of 362 patients.

Figure 1.

Receiver operating characteristic curves of POSITIVE, PREDICT, and VIRSTA scores.

Figure 2.

Calibration curves of the POSITIVE, PREDICT, and VIRSTA scores. The points show the relation between predicted probability of Staphylococcus aureus infective endocarditis for each decile of predicted probability of endocarditis as calculated using the respective scores. The solid line is the Loess fit through the points. The dashed line represents perfect calibration.

Hypothetical Effect on Patient Care

A high-risk classification was given to 44.5% (161/362) by the POSITIVE score, 50.7% (242/477) by the PREDICT day 5 score, and 70.9% (338/477) by the VIRSTA score. In our cohort, 42.1% (201/477) of patients actually received a TEE. When excluding patients with a contraindication to TEE and patients who died before a TEE was possible, the TEE rate in our cohort increased to 47.6% (201/422). Had the risk scores been applied to this cohort, POSITIVE would have required fewer TEEs, PREDICT approximately the same, and implementing VIRSTA would mean an increase of 45.3% in TEEs necessary (Table 4). Of note, many TEEs were performed in patients classified as low risk by the risk scores.

Table 4.

Effect of Applying Risk Scores on TEE Usage

	n (%)	Change Compared With Baseline
Actually done (full cohorta)	201/422 (47.6)	Reference
PREDICT d 5: high risk	210/422 (49.8)	+4.5%
VIRSTA: high risk	292/422 (69.2)	+45.3%
Separate Cohort for POSITIVE Score
Actually done (TTP cohortb)	165/316 (52.2)	Reference
POSITIVE: high risk	137/316 (43.4)	–16.9%

Abbreviations: TEE, transesophageal echocardiography; TTP, time to positivity.

aExcluded were 55 patients who could not undergo TEE because of death before TEE was possible or had a contraindication for TEE.

bExcluded were 45 patients who could not undergo TEE because of death before TEE was possible or had a contraindication for TEE.

Of the patients who underwent a TEE, 52% (85/165) were classified as low risk by POSITIVE, 38% (76/201) were low risk according to day 5 PREDICT, and 20% (40/201) were low risk according to VIRSTA.

Patients Misclassified as Low Risk

Supplementary Table S4 gives an overview of the patients with SA-IE misclassified as low risk by the scores and their respective risk factors. Nearly all patients misclassified as low risk for SA-IE by POSITIVE had either community-acquired SAB, a CIED, or persistent SAB, whereas patients misclassified as low risk by PREDICT often had either preexisting native valve disease or a prosthetic valve. One patient with native aortic valve SA-IE was incorrectly classified as low risk by all 3 scores.

Discussion

In this multicenter cohort study, we found that only the VIRSTA score met our predefined threshold of a NPV for the diagnosis SA-IE above 98%. Although both the POSITIVE and PREDICT (day 5) scores had acceptable sensitivity and NPV, both had a NPV below the preset threshold (92.5% and 94.5%, respectively).

The primary goal of these risk scores is to classify patients with SAB as low or high risk for SA-IE to avoid TEE in those with low risk. All 3 scores had a high NPV, but the VIRSTA score had the highest discriminatory ability (area under ROC curve 88.9%). PPV of the scores was relatively low (with the exception of the day 1 PREDICT score), with PPVs ranging from 25.5% for VIRSTA to 32.3% for POSITIVE. Yet, it is equally important to consider the effects of implementing the risk score on clinical practice. Although VIRSTA had the highest sensitivity and NPV, it also classified 70% of SAB patients as high risk, and thus having an indication for TEE. In most studies on SAB, TEE rates are between 30% and 50% [9, 10, 18, 19]. Therefore, VIRSTA would allow a clinician to safely exclude SA-IE, but would also increase the number of patients undergoing TEE. Naturally, patients with high risk scores may not undergo TEE for other reasons (such as clinical instability, refusal, or an alternative diagnosis). Therefore, the clinical consequences of implementation of any of these scores remains to be determined. These risk scores only provide additional information to the clinician and should not replace clinical judgment in individual patients.

The PREDICT day 1 score was designed for early recognition of the patients with the highest risk of SA-IE, who should undergo immediate TEE [8]. Day 1 PREDICT performs well in this regard, with high specificity (97%) and PPV (67%).

Both POSITIVE and PREDICT use fewer parameters than VIRSTA, and both lack some traditional risk factors that are associated with the risk of SA-IE. POSITIVE does not take into account the setting where the SAB was acquired, the presence of a CIED, or the results of follow-up blood cultures, whereas PREDICT does not include preexisting native valve disease or a prosthetic valve as a risk factor. This probably results from the statistical approach that was taken to avoid overfitting while developing the risk scores [20]. Although a methodologically sound method to develop a diagnostic model, lack of these traditional risk factors in POSITIVE and PREDICT results in decreased sensitivity. Fourteen of the 15 patients incorrectly classified as low risk by POSITIVE had either persistent SAB, a CIED, or had community-acquired SAB, and 10 of 12 patients incorrectly classified by PREDICT had a prosthetic valve or preexisting native valve disease.

PREDICT was previously validated in 199 patients from the same center that developed the score, and this validation study yielded 100% sensitivity and NPV [18]. These results contrasted those from another cohort with 205 patients with SAB in which day 5 PREDICT had 87% sensitivity and a 93% NPV, which is comparable to our results [21]. Peinado-Acevedo et al validated both PREDICT and VIRSTA in a cohort of 922 SAB patients from Colombia. In this study, PREDICT had lower sensitivity and NPV than VIRSTA (51.6% and 95.1% vs 99.6% and 99.5%) [11]. Because of the low prevalence of traditional risk factors for IE and the high proportion of catheter-related SAB, their results may not be generalizable to other settings.

The sensitivity of POSITIVE is lower in our cohort (77.6%) than in the development cohort and the NPV (92.5%) did not meet our predefined threshold of 98%. POSITIVE is the only score that uses TTP as a predictor of endocarditis. In the development cohort, all patients with SA-IE had a TTP shorter than 15 hours. This is in contrast to other studies that reported patients with SA-IE and TTP well above 15 hours [22, 23]. The strong weight assigned to TTP in the POSITIVE score may have led to overreliance of the statistical model on this predictor, resulting in elimination of other important predictors from the model, explaining the lower accuracy in our cohort. Adjustment of the cutoff times for TTP would likely improve the POSITIVE score, but this adjustment would require additional validation.

Strengths of our study include the prospective enrollment with nearly complete data, the multicenter design allowing for an unselected cohort from both general and university hospitals, and the high percentages of TTE, TEE, and follow-up blood cultures.

Study limitations include that not all patients with SAB were enrolled because of the need for informed consent. That patients who refused or were discharged before consent was possible may have resulted in selection bias; in particular, the latter group may comprise the least ill patients, with a lower likelihood of SA-IE. Yet, because most would likely have been classified as low risk by the prediction scores, this would not influence sensitivity and could further increase NPV. Furthermore, our study population is comparable in demographics, rates of echocardiography, prevalence of endocarditis, and mortality to other cohort studies that included all patients without the need for informed consent [3, 4, 8, 19, 24]. As in nearly all studies on SAB, blood culture collection was not standardized, leading to some uncertainty on duration of bacteremia. Despite this, at least 1 follow-up blood culture was taken in 91% of patients. The diagnostic accuracy results were robust in the sensitivity analysis including only patients that had at least 1 follow-up blood culture. Finally, because of a recent change in laboratory software, TTP data were missing from 1 hospital. This smaller population for validation of POSITIVE may have resulted in wider CIs, but is unlikely to bias the diagnostic accuracy point estimates.

In summary, we found that of the 3 proposed prediction sets for risk of IE in patients with SAB, VIRSTA has the highest NPV but at the expense of a high number of patients classified as high risk and thus requiring TEE. Risk scores should therefore not only be evaluated with regard to diagnostic accuracy, but also on their clinical implications. The POSITIVE and PREDICT scores need adjustment before they can be used to rule out endocarditis. Until further adjustment of the POSITVE and PREDICT scores, clinicians can use the VIRSTA score to safely rule out SA-IE without performing a TEE.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Click here for additional data file.