Benefit of Echocardiography in Patients With Staphylococcus aureus Bacteremia at Low Risk of Endocarditis.

BACKGROUND: The risk of endocarditis among patients with Staphylococcus aureus bacteremia is not uniform, and a number of different scores have been developed to identify patients whose risk is less than 5%. The optimal echocardiography strategy for these patients is uncertain.
METHODS: We used decision analysis and Monte Carlo simulation using input parameters taken from the existing literature. The model examined patients with S aureus bacteremia whose risk of endocarditis is less than 5%, generally those with nosocomial or healthcare-acquired bacteremia, no intracardiac prosthetic devices, and a brief duration of bacteremia. We examined 6 echocardiography strategies, including the use of transesophageal echocardiography, transthoracic echocardiography, both modalities, and neither. The outcome of the model was 90-day survival.
RESULTS: The optimal echocardiography strategy varied with the risk of endocarditis and the procedural mortality associated with transesophageal echocardiography. No echocardiography strategy offered an absolute benefit in 90-day survival of more than 0.5% compared with the strategy of not performing echocardiography and treating with short-course therapy. Strategies using transesophageal echocardiography were never preferred if the mortality of this procedure was greater than 0.5%.
CONCLUSIONS: In patients identified to be at low risk of endocarditis, the choice of echocardiography strategy appears to exert a very small influence on 90-day survival. This finding may render test-treatment trials unfeasible and should prompt clinicians to focus on other, more important, management considerations in these patients.

Staphylococcus aureus bacteremia (SAB) is complicated by endocarditis in up to 18%–25% of unselected cases [1, 2], although the prevalence of endocarditis among these patients is not uniform and can be estimated using a variety of clinical and microbiological predictors [3]. Several studies have now reported criteria able to identify patients at very low risk (less than 5%) of endocarditis [4]. Although these studies vary in size, methodology, and the exact criteria used, it appears that patients at very low risk of endocarditis can be identified by day 3–5 after the onset of bacteremia, before any echocardiographic assessment, using clinical and microbiological criteria. In general, these patients have (1) nosocomial, healthcare-associated nonnosocomial, or central line-associated bacteremia, (2) a brief duration of bacteremia (less than 48–72 hours), and (3) no intracardiac prosthetic device (prosthetic heart valves or cardiac rhythm management devices). A detailed description of this literature and its limitations can be found in our previous systematic review [4]. In particular, we refer readers to Table 1 in the previous review (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5751065/table/T1) for a summary of the various low-risk criteria sets.

Although current guidelines suggest that transesophageal echocardiography (TEE) is required for most, if not all, patients with SAB [5, 6], it remains uncertain whether this recommendation applies to very low-risk patients. Our previous work examining the benefit of TEE in patients with SAB suggested that TEE may be warranted in all patients with a risk of endocarditis above 1% [7], although this estimate did not examine the magnitude of the benefit of TEE or the role of transthoracic echocardiography (TTE).

The aim of this study was (1) to extend our previous modeling work to examine echocardiography strategies involving TTE and (2) to estimate the absolute 90-day survival resulting from various echocardiography strategies to inform the design of future trials in this area.

METHODS

We used decision analysis [8] with Monte Carlo simulation [9] to examine the effect of various echocardiography strategies on the 90-day survival of patients with SAB assessed as having less than 5% probability of endocarditis. The binary outcome of 90-day survival was chosen to reflect the recognized outcome measures of bacteremia trials [10].

Model Structure

Each patient in the model had SAB without clinical features of endocarditis (eg, new murmur, heart failure, emboli, or immunological phenomena) and an estimated probability of endocarditis of less than 5% based on the criteria outlined above. Such patients were considered to have 1 of 3 mutually exclusive static disease states: uncomplicated SAB, clinically occult S aureus left-sided native valve endocarditis (NVIE) without perivalvular abscess, and NVIE with perivalvular abscess formation.

We examined 6 echocardiography strategies: TEE as the only study (abbreviated “TEE”); TTE as the only study (“TTE”); TTE first followed by TEE only if the TTE is negative for endocarditis (“TEE|TTE-”); TTE first followed by TEE only if the TTE is positive for endocarditis (“TEE|TTE+”); short-course antibiotic treatment (2 weeks) without echocardiography (“NE2”); and long-course antibiotic treatment (6 weeks) without echocardiography (“NE6”). We did not include the strategy of performing both TTE and TEE, because under the assumptions described below this strategy would have identical results to the TEE strategy.

Echocardiography results (for the 4 strategies in which echocardiography was performed) conformed to these same 3 states, with the correlation between the actual disease state and the echocardiographic result being dependent on the diagnostic performance of each echocardiography strategy. Transthoracic echocardiography and TEE were not assumed to be independent tests given the similarities between the 2 procedures. The degree of statistical covariance between these tests was described as the percentage of the maximal covariance allowed by the observed diagnostic performance statistics [11].

Patient treatment was based on the results of echocardiography alone. Patients thought not to have endocarditis received 2 weeks of parenteral anti-staphylococcal therapy; those thought to have uncomplicated NVIE received 6 weeks of parenteral anti-staphylococcal therapy; and those thought to have NVIE complicated by perivalvular abscess received 6 weeks of parenteral anti-staphylococcal therapy and cardiac surgery (assumed to be valve replacement). If a patient underwent both TTE and TEE, treatment was based on the results of the TEE. The 2 strategies without echocardiography resulted in all patients receiving antibiotic therapy for the duration specified without surgery for perivalvular abscess.

A decision tree summarizing the model is presented in Figure 1. Patients were assumed not to manifest clinical signs of endocarditis during treatment, such that interventions for endocarditis (antibiotic prolongation or surgery) were based solely on the echocardiography results. Likewise, no patient underwent surgery for indications other than clinically occult perivalvular abscess. Patients were also assumed not to have any other sites of staphylococcal infection warranting prolonged treatment.

Figure 1.

Summary decision tree for the model. TEE, transesophageal echocardiography; TTE, transthoracic echocardiography.

Survival outcomes in the model were additive. The expected survival resulting from any given strategy was the sum of the products of each possible permutation’s probability and its respective expected survival.

Model Inputs

Parameter estimates and error distributions were identified from the medical literature published before November 16, 2017 using a stepwise search strategy, as described in a previous publication [7]. Additional inputs required for this investigation included the diagnostic performance of TTE compared with TEE for the detection of clinically occult S aureus NVIE and the statistical covariance (non-independence) of TTE and TEE.

Monte Carlo Simulation

For each specific input scenario, we conducted a 10000-patient Monte Carlo simulation to estimate the 90-day survival resulting from each of the 6 strategies. Two-variable strategy analyses were built from multiple 1000 patient simulations. The decision model and the Monte Carlo simulations were constructed using Microsoft Excel (Microsoft Corporation, Redmond, WA).

Statistical Analysis

Due to the dominant effect of the prevalence of endocarditis in the model and the previously noted broad uncertainty regarding the procedural mortality of TEE [7], we planned to present the result of the decision model primarily as a strategy analyses based on these 2 variables. We also examined the absolute benefit of each of the strategies above NE2 (as the simplest and least burdensome option) in 2 informative scenarios. The first examined patients with a VIRSTA score [3] of less than 3 points (risk of endocarditis 1.8%, 95% confidence interval 1.0%–3.6%, assuming a prevalence of 18% among unselected patients with SAB [2]) and a TEE-associated mortality of 1.0%. This scenario represents the lowest expected survival benefit of any echocardiography strategy because it is not currently possible to reliably identify patients at any lower risk of endocarditis [4]. The second scenario examined patients with a 5% risk of endocarditis and a TEE-associated mortality of 0.01% (the largest expected benefit within the model envelope).

Literature Review and Model Inputs

The majority of model inputs were sourced from a previous review [7]. Estimates of the diagnostic performance of TTE compared with TEE were drawn from a recent systematic review specifically addressing this issue [12]. No studies were identified that presented data allowing an estimate of the statistical covariance of TTE and TEE. The inputs and their associated sampling distributions are presented in Table 1.

Table 1.

Parameter Estimates, Error Distributions, and Sources

Definition	Estimate	Range or 95% Confidence Interval	Sampling Distribution	References
Probability of Disease
Prevalence of clinically occult endocarditis in patients with apparently uncomplicated SAB and no intracardiac prosthetic device	Variable	0%–5%		[4, 13]
Prevalence of perivalvular abscess among patients with clinically occult NVIE	15%	10%–20%	Triangular	[14–18]
Probability of relapse if clinically occult left-sided NVIE treated with 2 weeks of parenteral antibiotic therapy	60%	40%–80%	Triangular	[19–27]
Diagnostic Performance of TEE
Sensitivity of TEE for clinically occult NVIE	96%	92%–99%	Triangular	[5, 29–31]
Specificity of TEE for clinically occult NVIE	90%	90%–95%	Triangular	[5, 29–31]
Sensitivity of TEE for perivalvular abscess in NVIE	75%	40%–90%	Triangular	[17, 18, 32–34]
Specificity of TEE for perivalvular abscess in NVIE	98%	96%–100%	Triangular	[16, 18, 32–34]
Diagnostic Performance of TTE
Apparent sensitivity of TTE for occult NVIE (compared with TEE)	58%	53%–62%	Beta	[12]
Apparent specificity of TTE for occult NVIE	92%	91%–93%	Beta	[12, 26, 27]
Apparent sensitivity of TTE for perivalvular abscess	48%	29%–67%	Beta	[12]
Apparent specificity of TTE for perivalvular abscess	100%	99%–100%	Beta	[12]
%Maximum statistical covariance between TTE and TEE	25%	0%–50%	Triangular	No data
Survival Estimates for Various Diseases States
90-day survival of patients with apparently uncomplicated SAB	80%	76%–83%	Beta	[35]
Excess mortality associated with diagnosed and treated NVIE	15%	10%–25%	Triangular	[36–38]
Additional mortality associated with relapsed partially treated NIVE	15%	12%–17%	Triangular	[39, 40]
Excess mortality associated with diagnosed and treated perivalvular abscess in NVIE	15%	0%–30%	Triangular	[14, 16, 1–43]
Additional mortality if perivalvular abscess initially goes unrecognized	15%	0%–30%	Triangular	[14, 16, 41–43]
Risks Associated With Testing and Treatment
Excess mortality due to the procedures required for TEE	Variable	0.01%–1%		[44–47]
Excess mortality due to the procedures required for TTE	0%			Assumed
Excess mortality due to adverse drug events in weeks 2–6 of therapy	0.2%	0%–0.7%	Beta	[48, 49]
Excess mortality due to line infection arising in weeks 2–6 of therapy	0.3%	0.1%–0.8%	Beta	[50–61]
Excess mortality due to cardiac surgery for perivalvular abscess	5%	3%–9%	Triangular	[62, 63]

Abbreviations: NVIE, native valve endocarditis; SAB, Staphylococcus aureus bacteremia; TEE, transesophageal echocardiography; TTE, transthoracic echocardiography.

RESULTS

In general, multiple independent sources in general agreement with each other were found for each of these parameters, with 3 exceptions: (1) the excess mortality associated with the procedure of TEE; (2) the likelihood of relapse if unrecognized clinically occult S aureus NVIE were to be treated with short-course antibiotic therapy; and (3) the statistical covariance (non-independence) of TTE and TEE. Our rationale for the ranges examined in the sensitivity analyses is presented in the Supplementary Materials.

Strategy Analysis

The preferred echocardiography strategy for 90-day survival after apparently uncomplicated SAB varied depending on the probability of occult endocarditis and the procedural risk of TEE (Figure 2). Over the ranges tested, only 2 of the 6 strategies were not preferred at some combination of these inputs. In general, at low TEE procedural risk (<0.1% mortality), the choice appeared to be between TEE and TEE|TTE+. At high TEE procedural risk (>0.5% mortality), strategies utilizing TEE were never preferred.

Figure 2.

Preferred echocardiography strategy for patients with Staphylococcus aureus bacteremia at low risk of endocarditis. Each box represents 1000 simulations with a flat sampling distribution across both the x and y intervals. Large symbols denote where a single strategy was preferred in ≥90% of simulations; small symbols denote where a strategy was preferred in 75%–90% of simulations. Empty regions denote where no single strategy was preferred in at least 75% of simulations. TEE, transesophageal echocardiography; TTE, transthoracic echocardiography.

Given the uncertainty in the literature regarding the relapse risk of undertreated clinically occult endocarditis and the statistical covariance of TTE and TEE, we conducted additional sensitivity analyses examining the effect of altering our chosen values for these uncertain parameters (see Supplementary Figures S1–S4). Assuming universal relapse of occult endocarditis treated for only 2 weeks resulted in a lower threshold at which TEE was preferred over TEE|TTE+. A more complex effect was seen when the errors of TTE and TEE were assumed to be highly correlated—the consequent reduction in the corrected diagnostic performance of TTE and the lower informational value of combination strategies led to the TTE-based strategies being preferred across a very small range of the model envelope.

Absolute Benefit of Echocardiography Strategies in Specific Scenarios

The median 90-day survival benefit of any echocardiography strategy compared with NE2 remained less than 0.5%, even at an endocarditis risk of 5%. As seen in Figure 3A, the maximal expected survival benefit for the TEE strategy compared with NE2 was +0.43% (95% confidence interval, +0.28%–0.58%) at 90 days, corresponding to a number-needed-to-test of 233. The maximal harm for any strategy within the clinically observable model envelope was an excess 0.87% mortality (95% confidence interval, 0.69%–1.06%) seen for the TEE strategy when the mortality from TEE was set to 1% and the risk of endocarditis wast set to 1.8% (Figure 3B). Although it is only preferred over the other echocardiography strategies at high TEE procedural mortality, the TTE strategy appeared to retain a net benefit (albeit marginal) across the modeled range of endocarditis probabilities.

Figure 3.

Absolute benefit of various echocardiography strategies for 90-day survival compared with no echocardiography and treatment with short-course therapy (NE2). Boxes denote the 25th, 50th, and 75th percentile values, and the whiskers contain the 95% confidence interval. (A) considers patients with a 5% risk of endocarditis and a transesophageal echocardiography (TEE)-associated mortality risk of 0.01%. (B) considers patients with a VIRSTA score of <3 points (risk of endocarditis 1.8%) and a 1% TEE-associated mortality risk. These points respectively represent the highest and lowest expected survival benefits of echocardiography within the clinically observable portion of the strategy analysis presented in Figure 2. TTE, transthoracic echocardiography.

When considering patients with a VIRSTA score of less than 3 points (risk of endocarditis, 1.8%; 95% confidence interval, 1.0%–3.6%) at the traditional estimate for TEE mortality (0.01%), the TEE strategy still offered the best 90-day survival, but the absolute benefit compared with NE2 was only 0.12% (95% confidence interval, −0.02%–0.30%), corresponding to a number-needed-to-test of more than 800 patients (Figure 4).

Figure 4.

Absolute benefit of various echocardiography strategies for 90-day survival compared with no echocardiography and treatment with short course therapy (NE2) for a patient with a VIRSTA score [ of less than 3 points (risk of endocarditis 1.8%; 95% confidence interval, 1.0–3.6%) at a transesophageal echocardiography (TEE)-associated mortality risk of 0.01%. Boxes denote the 25th, 50th, and 75th percentile values, and the whiskers contain the 95% onfidence interval. TTE, transthoracic echocardiography.

DISCUSSION

The preferred echocardiography strategy for patients with SAB who are at low risk of endocarditis varies depending on the absolute endocarditis risk and the procedural risks of TEE, but the magnitude of the effect of any strategy on 90-day survival is small. We believe this result has implications for clinicians making decisions regarding the management of these patients as well as for the design of future trials examining the optimal management of patients with SAB.

The small absolute benefit of echocardiography suggested by the results of our modeled analysis is not unexpected. Although it is conceivable that incidental noninfectious echocardiographic findings may be of help to clinicians managing these patients, the benefit of echocardiography in this setting is largely limited to the detection of occult endocarditis. As such, regardless of the degree to which patients with endocarditis might benefit from its identification and treatment, these benefits are restricted to the subgroup of patients with the disease. In other words, the absolute benefit of disease-specific testing is necessarily less than the prevalence of disease. In this study, the low prevalence of endocarditis among modeled patients translates to an even smaller benefit from echocardiography.

The results of our analysis are generally consistent with 2 previous modeling studies examining the role of echocardiography in patients at risk of endocarditis. Rosen et al [22] examined the cost-effectiveness of the TEE strategy compared with no echocardiography (NE2 and NE6) in patients with a 6.1% prevalence of endocarditis (the authors’ estimate of the prevalence of endocarditis among patients with central-line associated SAB). Even with assumptions favorable to TEE (very low TEE-related procedural mortality, high TEE specificity, and uncomplicated local phlebitis as the only adverse effect of unnecessary prolonged antibiotics), the authors estimated that performing TEE would obtain, on average, an additional 16 quality-adjusted life days over short-course antibiotic therapy alone in their base case scenario. Due to its low modeled cost, TEE was found to be cost effective (at a threshold of USD 50000/QALY in 1997) for patients with a risk of endocarditis above 1.7%.

Heidenreich et al [64] performed a cost-effectiveness study of echocardiography (TEE, TTE, or TEE|TTE-) in patients suspected of endocarditis due to any organism. Using assumptions that are not necessarily applicable to patients with SAB (most notably a low incidence of non-endocarditis mortality), the authors estimated a benefit in overall nonadjusted survival of 41 days from the use of TEE over short-course treatment in patients at a 20% risk of endocarditis. Although a utility-only threshold was not reported, none of the echocardiography strategies were reported to be cost effective for patients whose risk of endocarditis was less than 2%.

As has been noted previously [65], the ability of a diagnostic testing strategy to affect the outcomes of only a small segment of the population to which it is applied has major implications for the feasibility of conducting randomized test-treatment trials. The structure and results of our model can be used to examine the sample sizes required in a trial comparing echocardiography strategies in patients with SAB at low risk of endocarditis in which the outcome is 90-day survival. For example, for uncomplicated patients at a 1.8% risk of endocarditis (the point of apparent clinical equipoise [66]), a trial designed to demonstrate noninferiority for the binary outcome of 90-day survival for the NE2 strategy compared with TEE with a delta of 1.8% and standard power (α 0.05 2-sided, β 0.90) would require between 7948 and 19708 participants, depending on the estimated procedural mortality of TEE—more than 10 times the size of the largest S aureus trial conducted to date [67]. Using a wider noninferiority margin would certainly reduce the required sample size, but it would render the trial redundant because the failure to diagnose endocarditis by echocardiography cannot contribute excess mortality in patients without the disease. Alternative approaches to assessing the value of echocardiography in low-risk patients, such as cost-effectiveness modeling or clinical quality registries, may be more feasible than comparative test-treatment trials.

From a practical perspective, the results of our study should help clinicians understand where decisions regarding echocardiographic screening of patients at low risk of endocarditis should fit among the numerous other management priorities in SAB. In the first place, it should be evident that accurate stratification of patients by risk of endocarditis using any of a number of published criteria [4] is much more important than the choice of echocardiography strategy in those predicted to be at low risk. Likewise, in patients at low risk of endocarditis, more attention and effort should be directed towards optimizing interventions expected to be broadly beneficial (eg, appropriate antimicrobial therapy and expeditious source control) than to those with the potential to benefit only a small fraction of patients (echocardiography). For patients at the lowest risk of endocarditis, the survival benefit of any echocardiography strategy compared with empiric short-course therapy is marginal at best. Although many clinicians may not be ready to remove echocardiography entirely from management algorithms, our results suggest that the mortality benefit of pursuing TEE for patients at low risk of endocarditis is not of clinical significance compared with TTE-based strategies. This finding may have particular relevance in settings where TEE is not easily accessible.

Limitations

Our analysis has a number of unavoidable limitations that arise from the structure of the model and the uncertainty of its inputs. We have made several assumptions within the structure of the model that generally favor the performance of echocardiography, including that occult NVIE never manifests clinically other than by relapse after the cessation of parenteral antimicrobial therapy (crossovers to endocarditis treatment would be expected to reduce any differences in survival) and that no patient with apparently uncomplicated SAB receives more than 14 days of parenteral therapy for any indication other than echocardiographically documented endocarditis (en passant treatment of unrecognized endocarditis would reduce the expected incidence of relapse). We also assumed that echocardiography results have no prognostic impact (ie, that patients with endocarditis have the same prognosis regardless of the echocardiographic demonstration of endocardial involvement), although recent evidence would suggest that this may not be the case [68]. These assumptions are clinically unrealistic but have been used in the model to maximize the apparent benefit of echocardiography over clinical observation alone; there are few available data to guide how often these assumptions may be violated in clinical practice.

Our model also only examined the binary outcome of 90-day survival and did not include nonfatal morbidity arising from endocarditis or its treatment. Due to the low proportion of patients with endocarditis within the model envelope, the addition of morbidity might be expected to be more detrimental to strategies involving more intervention (TEE or extended antibiotic therapy) because the additional outcomes would apply to a larger fraction of patients than are at risk of morbidity from unrecognized endocarditis.

The results of any decision analysis are highly dependent on the accuracy of the input parameters. We needed to identify estimates for 22 separate inputs from existing literature that was at times very diffuse and oblique. We were able to find multiple sources in rough agreement for all but the 3 parameters noted above, all of which have been subjected to sensitivity analyses across a wide range of values. Although the preferred echocardiography strategy did appear sensitive to varying inputs (see the unresolved sections of Figure 1 and Supplementary Figures S1 and S3), the infrequency of endocarditis within the model envelope minimized the effect of parameter uncertainty on all estimates of absolute 90-day survival.

CONCLUSIONS

In conclusion, a variety of echocardiography strategies may offer a survival advantage to patients with SAB at a low risk of endocarditis, but none results in an absolute risk reduction in 90-day mortality of more than 0.5%. This small benefit may render test-treatment trials unfeasible, and it should prompt clinicians to focus on other, more important, management considerations in these patients.

Supplementary Data

Supplementary materials are available at Open Forum Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Click here for additional data file.