Early differentiation between uncomplicated and complicated Staphylococcus aureus bacteraemia: Potential value and limitations of a clinical risk score.

OBJECTIVE: A cornerstone in the management of Staphylococcus aureus bacteraemia (SAB) is the differentiation between a complicated and an uncomplicated SAB course. The ability to early and accurately identify patients with - and without - complicated bacteraemia may optimise the utility of diagnostics and prevent unnecessary prolonged antibiotic therapy.
METHODS: Development and validation of a prediction score in SAB using demographic, clinical, and laboratory data from two independent Dutch cohorts; estimating the risk of complicated disease at the time of the first positive blood culture. Models were developed using logistic regression and evaluated by c-statistics, ie area under the ROC-curve, and negative predictive values (NPV).
RESULTS: The development- and validation cohorts included 150 and 183 patients, respectively. The most optimal prediction model included: mean arterial pressure, signs of metastatic infection on physical examination, leucocyte count, urea level and time to positivity of blood cultures (c-statistic 0.82, 95% CI 0.74-0.89). In the validation cohort, the c-statistic of the prediction score was 0,77 (95% CI 0.69-0.84). The NPV for complicated disease for patients with a score of ≤2 was 0.83 (95% CI 0.68-0.92), with a negative likelihood ratio of 0.14 (95% CI 0.06-0.31).
CONCLUSION: The early SAB risk score helps to identify patients with high probability of uncomplicated SAB. However, the risk score's lacked absolute discriminative power to guide decisions on the management of all patients with SAB on its own. The heterogenicity of the disease and inconsistency in definitions of complicated SAB are important challenges in the development of clinical rules to guide the management of SAB.

What’s known

Complications in Staphylococcus aureus bacteraemia, especially metastatic infection and infective endocarditis, are associated with high morbidity and mortality rates.

In Staphylococcus aureus bacteraemia (SAB) the differentiation between a complicated and an uncomplicated SAB is essential to guide both diagnostics and treatment.

Current risk scores to assess the risk of complicated disease lack discriminative power and/or are unvalidated.

What’s new

This article adds a validated risk score that supports discriminating patients with low and high risk of complicated SAB in daily clinical practice.

INTRODUCTION

Staphylococcus aureus is the second most common pathogen identified as the cause of bloodstream infection (BSI). The complications of Staphylococcus aureus bacteraemia (SAB), such as endocarditis and metastatic infection are associated with severe morbidity and high mortality rates. , The identification of patients with complicated SAB at an early stage is notoriously difficult, but has important implications. For complicated SAB, consensus guidelines recommend higher dosages of antibiotics and prolonged duration of intravenous therapy. Moreover, in this setting infectious complications often need specific additional treatment, eg surgical drainage of skin and soft tissue abscesses or valve replacement in case of endocarditis. Patients with unrecognised complications of SAB may have higher relapse rates and an increased morbidity and mortality risk. ,

However, misclassification of uncomplicated bacteraemia as complicated bacteraemia may result in unnecessary diagnostic procedures, overconsumption of antibiotics and increased treatment related side effects. , Current recommendations for the duration of antibiotic therapy in SAB are based on low quality scientific evidence. Guidelines recommend prolonged therapy (4‐6 weeks) in case of implanted prostheses; positive follow‐up blood cultures; persisting fever and evidence of infective endocarditis (IE) or metastatic sites. It is the identification of IE and metastatic infection that is challenging in clinical practice. An echocardiogram is recommended in all patients, but adherence to this guideline is limited and the sensitivity of transthoracic echocardiography for endocarditis is low. , The likelihood of metastatic sites is traditionally assessed based on clinical and laboratory clues. By these alone, asymptomatic metastatic infection may be difficult to detect. Positron emission tomography (PET) scan is valuable for the detection of metastatic foci, that were not detected by clinical examination. , , However, as SAB is very common, performing a PET in all patients with SAB is time‐ and resource consuming.

An efficient SAB‐risk score to timely stratify the risk of complicated disease would therefore be of great additional value to efficiently direct additional testing. In this study, we report the development and validation of an early clinical risk score for complicated disease and illustrate the challenges of risk scores in SAB.

METHODS

Setting and study population of the development cohort

In the retrospective development cohort all consecutive adult patients (age ≥ 18 years) presenting at the Leiden University Medical Center (LUMC), the Netherlands, with SAB between January 2013 and December 2015 were eligible for inclusion. SAB was defined by ≥1 blood culture positive for S aureus. Patients were excluded if: (a) S aureus was detected simultaneously with other pathogens or with contaminants (polymicrobial culture), (b) The patient died within 24 hours after blood culture collection. In patients with multiple episodes of SAB only the first episode was included.

Study definitions

Uncomplicated SAB was defined as an episode of bacteraemia with ≥1 blood culture with Staphylococcus aureus, without evidence of endocarditis/metastatic infection and without positive cultures after 48 hours of adequate therapy and that was treated for a maximum of two weeks and no relapse occurred and the patient survived >72 hours after presentation. Adequate therapy was defined as treatment with a least one effective antimicrobial agent, based on in vitro sensitivity testing of the microorganism detected in the blood culture. Relapse was defined as a positive culture of S aureus from any sterile body site within 3 months after sterilisation of blood cultures. All cases that did not meet the criteria for uncomplicated SAB were considered complicated SAB. Confirmed complicated SAB was defined as S aureus bacteraemia with endovascular infection (ie endocarditis), and/or other metastatic foci and/or positive blood cultures after 48 hours of adequate antimicrobial therapy. Infective endocarditis (IE) was defined by modified Duke's criteria. Metastatic infection was defined as a radiographical examination and/or culture concordant with vertebral osteomyelitis, epidural abscess, deep tissue abscess (eg psoas‐) septic pulmonary or cerebral emboli, arthritis or meningitis.

Data collection

In the study centre, all patients with SAB are evaluated by the infectious diseases team through bedside consultation and findings are reported in the electronic patient files. The clinical data were collected through review of the electronic medical charts by two reviewers separately. The following data were obtained: demographic characteristics, medical history, antibiotic therapy at the time of presentation, duration and type of symptoms, clinical parameters, endocarditis stigmata and signs of metastatic infection on physical examination, laboratory test results, radiography results and outcome parameters: duration of hospital admission, relapse, admission to the intensive care unit, 30 day mortality. In addition, time to positivity of blood cultures (TTP) was collected as previous studies indicated TTP to be prognostic of hematogenous spread in SAB. , , Time to positivity was defined as the time between venepuncture and the positive alert signal of the blood culture monitoring system. If multiple blood cultures were obtained within a time frame of two hours, the shortest TTP was included in the analysis. Blood samples were inoculated in both anaerobic and aerobic bottles and incubated in the BACTEC FX continuous monitoring system (Becton Dickinson BV, Breda, The Netherlands). The time of blood culture sampling was automatically recorded. All samples were placed in the BACTEC, within one hour after arrival at the microbiology department.

Setting and study population of the validation cohort

In the validation cohort, patients with SAB were included in three Dutch hospitals. Patients were included consecutively between Jan 1st 2016 and August 1st 2017. For each of these patients the demographic variables, the variables needed for calculation of the risk score and outcome variables were collected through review of the electronic patient files. Definitions of (un)complicated SAB were identical for the development and validation cohort.

Statistical analyses

Descriptive statistics were performed in both the developmental and validation cohort. Data are presented as rates (percentages) for categorical variables and as medians (interquartile range/IQR) for continuous variables.

Risk score development

In the developmental cohort, patients with complicated SAB were compared with patients with uncomplicated SAB using Student's t test and Mann‐Whitney test for continuous variables and Fisher's exact test for nominal variables. A logistic regression model was applied with complicated SAB as the dependent (outcome) variable. All possible clinical and laboratory variables with P < .2 in the univariate analysis were included in the multivariable regression analysis. Continuous variables were categorised if the model's predictive value was not negatively affected by categorisation. Points for individual predictors were based on the co‐efficient from the multivariable model rounded to the nearest 0.5 or 0.0. The values of the independent predictive values were summed, resulting in the early SAB risk score. These SAB risk‐scores were compared to the observed proportion of patients with complicated SAB. The negative and positive predictive value of the SAB‐risk score was calculated for several cut‐offs. A clinically applicable cut‐off was selected based on the negative predictive value (NPV), as the primary goal of the risk score is to exclude complicated SAB. The area under the receiver operating characteristic (c‐statistic, AUC‐ROC) curve was reported as a measure of the discriminative value of the model.

Risk score validation

The performance of the model was tested in an independent validation cohort and the c‐statistic was determined. The NPV and negative likelihood ratio (NLR) of the SAB‐risk score for complicated SAB were reported. The NLR is defined as the probability that a patient with complicated SAB has a low SAB‐risk score (false negative) divided by the probability that a person with uncomplicated SAB tested has a low SAB‐risk score (true negative). The NLR represents how the probability of complicated disease shifts when the SAB risk score is low.

Missing data in the variables of the risk score were imputed in the validation cohort, using multiple imputation. All analyses were performed with SPSS (IBM statistics, version 25) software for Windows.

Ethical approval

Ethical approval was granted by Leiden University Medical Center institutional ethical review committee, the Haga Teaching Hospital and the Alrijne hospital.

RESULTS

A total of 150 patients were included in the development cohort. The patient characteristics are summarised in Table 1. Borderline oxacillin‐resistant S aureus and methicillin‐resistant S aureus (MRSA) were both isolated in one episode. In 58 (38.7%) patients complicated bacteraemia was confirmed. Endovascular infection (endocarditis, or infected thrombi) and metastatic infection were diagnosed in 12 (8.0%) and 22 (14.7%) patients, respectively. In 23 (15.3%) patients, complicated bacteraemia was not confirmed by diagnostics, but the patient was treated for complicated disease, with prolonged intravenous therapy. In the development cohort, 69 (46.0%) patients fulfilled the definition for uncomplicated SAB. Missing data fields were <2%.

TABLE 1

Characteristics of the developmental (n = 150) and validation cohort (n = 183)

	Development cohort N = 150	Validation cohort N = 183
Male gender	108 (72)	113 (61.4)
Age	62 (51.0‐75.3)	71 (61‐81)
Comorbidities
Neutropenia	5 (3.3)	8 (4.4)
Organ transplantation	14 (9.3)	6 (3.3)
Diabetes	35 (23.3)	52 (28.3)
Receiving dialysis	7 (4.7)	7 (3.8)
Intravascular catheter	33 (22.0)	19 (3.3)
Location
Emergency department or outpatient clinic	93 (62.0)	137 (75.3)
General ward	57 (38)	42 (22.8)
Intensive care department	11 (7.3)	4 (2.2)
Clinical parameters
Mean arterial pressure	88.5 (79.6‐100.0)	90 (78‐102)
Newly diagnosed hearth murmur	14 (9.3)	27 (14.8)
Time to positivity (h)	18.1 (14.8‐22.6)	16.3 (13.5‐16.3)
Diagnosis
Uncomplicated SAB	69 (46.0)	73 (39.9)
Complicated SAB	81 (54.0)	110 (60.1)
Confirmed complicated SAB	58 (38.7)	80 (43.7)
Endocarditis	8 (5.3)	28 (15.2)
Metastatic disease	22 (14.7)	53 (28.8)
Persistent positive blood cultures	39 (26.0)	45 (24.5)
Outcome
Intensive care admission	36 (24.0)	30 (16.3)
30‐d mortality	31 (20.7)	35 (19.1)

Values are numbers (%) for continuous variables and median ± IQR for continuous variables. Uncomplicated SAB was defined as an episode of bacteraemia with ≥1 blood culture with Staphylococcus aureus, without evidence of endocarditis/metastatic infection and without positive cultures after 48 h of adequate therapy and that was treated for a maximum of two weeks and no relapse occurred and the patient survived >72 h after presentation. All cases that did not meet the criteria for uncomplicated SAB were considered complicated SAB.

Abbreviation: TTP, time to positivity.

Derivation of the early SAB risk‐score

The univariate analyses for complicated bacteraemia in the development cohort are shown in Supplement A. Community acquired infection was associated with complicated SAB (OR 4.6, 95% CI 2.2‐9.2, P < .01). Urea levels (P < .01) and leukocyte count (P < .01) were associated with complicated SAB. A TTP below 16 hr was associated with complicated disease (OR 3.3, 95% CI 1.6‐6.9, P < .01). Sensitivity, specificity and predictive values for different TTP cut‐offs are shown in Supplement B.

In the multivariable logistic regression analyses, independent predictive variables for complicated diseases were mean arterial pressure, signs of metastatic infection on physical examination, neutropenia, urea level, leukocyte count and time to positivity (P < .01). For the resulting model (Table 2), the fraction of explained variation (Nagelkerke R 2) was 0.39. The range of the constructed prediction score was 0 to 9, with a higher score indicating a higher probability of complicated SAB (Table 2). When using a cut‐off of 2 points, the negative predictive value was 91.9% (78.5‐97.2). The discriminative ability, c‐statistic was 0.82 (95% CI 0.74‐0.89).

TABLE 2

Independent predictive variables for development of complicated S aureus bacteraemia and attributed points in the prediction score

Variable	B	OR (95% CI)	P‐value	Points a
Clinical parameters
Signs of metastatic infection b	1.4	4,2 (1.6‐10.9)	<.01	1.5
Mean arterial pressure <90 mmHg	1.1	2.9 (1.3‐6.8)	.01	1
Laboratory parameters
Leucocyte count > 15 × 109/L	1.2	3.2 (1.3‐7.7)	.01	1
Neutropenia < 0.5 109/L	3.1	20.4 (1.4‐307.4)	.03	3
Urea > 13 mmol/L	1.2	3.3 (1.4‐7.8)	.01	1
Time to positivity
0‐16 h	2.3	8.7 (2.6‐29.0)	<.01	2.5
16‐24 h	1.0	2.7 (0.9‐8.3)	.09	1
>24 h	0	—	—	0

Abbreviations: B, regression coefficients; OR, odds ratio.

Points were attributed based on the regression co‐efficient.

Signs of metastatic infection’ was defined as: newly diagnosed diastolic hearth murmur, endocarditis stigmata and/or signs of metastatic infection on physical examination.

Validation of the risk‐score

In the validation cohort, 183 patients were included (Table 1), 73 (39.9%) patients fulfilled the criteria for uncomplicated SAB. In 80 (43.5%) patients a complicated disease was confirmed. Missing data were <2%. The risk scores for patients with uncomplicated SAB compared to the patients with complicated SAB (confirmed or unconfirmed) are presented in Figure 1. In patients with uncomplicated disease the median prediction score was 2.5 (IQR 1.5‐3.5), for complicated disease the prediction score was 4 (IQR 3‐5). The AUC‐ROC value was 0.77 (95% CI 0.69‐0.84). The performance of the SAB‐risk for different cut‐off values is presented in Table 3. The negative predictive value for the cut‐off 2 was 0.83 (95% CI 0.68‐0.92), with a negative likelihood ratio of 0.14 (95% CI 0.06‐0.31).

FIGURE 1

Prediction scores for patients with S aureus bacteraemia in the validation cohort. Uncomplicated SAB was defined as an episode of bacteraemia with ≥1 blood culture with Staphylococcus aureus, without evidence of endocarditis/metastatic infection and without positive cultures after 48 h of adequate therapy and that was treated for a maximum of two weeks and no relapse occurred and the patient survived >72 h after presentation. Complicated SAB: All cases that did not meet the criteria for uncomplicated SAB. The red line indicates the 2 points cut‐off

TABLE 3

Performance of the Staphylococcus aureus bacteraemia (SAB) risk‐score, in the validation cohort (n = 183)

Score	Uncomplicated disease N (%)	Complicated disease N (%)	Endocarditis N (%)	Metastatic infection N (%)
0‐2	29 (82.9)	6 (17.1)	3 (8.6)	2 (5.7)
2.5‐4.5	36 (35.0)	67 (65.0)	15 (12.3)	29 (28.2)
≥5	8 (17.8)	37 (82.2)	10 (23.8)	22 (48.9)

Values are the number (%) of patients with a score in the corresponding range. Complicated SAB = evidence of endocarditis/metastatic infection and/or with positive cultures after 48 h of adequate and/or that was treated with prolonged antibiotic therapy (>2 wk), and/or relapse occurred and/or the patient diseased <72 h after presentation. All other cases were considered uncomplicated. Endocarditis was defined by the modified Duke criteria. Metastatic infection = radiographical examination and/or culture concordant with vertebral osteomyelitis, epidural abscess, deep tissue abscess (eg psoas) septic pulmonary or cerebral emboli, arthritis or meningitis.

DISCUSSION

The SAB risk‐score, developed and validated in this study, facilitates to discriminate patients with low probability of complicated SAB from patients with high probability of complicated SAB, using readily available parameters. However, the rule lacked negative predictive power to accurately guide decisions on the management of patients with SAB on its own. This is exemplified by the observation that with a low‐score, the probability of complicated disease was 17.1%, which is not acceptable, considering the morbidity and mortality associated with unrecognised sequelae and relapse.

A prognostic model for SAB should primarily aim to reliably exclude complicated disease, with a high negative predictive power. However, prevalence of complicated disease depends on the setting and patient population and negative predictive values are prevalence dependant. Therefore, reported NPVs may not be applicable to other settings. Unlike NPV, the negative likelihood ratio (NLR) does not vary with prevalence and is a relevant marker in SAB risk scores.

Previous clinical risk scores

Multiple attempts have been made to assess the risk of complicated SAB in the past. Table 4 provides an overview of prior published prediction rules in SAB. Most of these prediction rules focus on infective endocarditis alone, discarding other foci of metastatic infection that may be relevant for the management of the infection. Furthermore these studies are limited by low rates of TEE and therefore lack a sensitive reference standard for endocarditis. The rules that do focus on all aspects of complicated SAB most often go unvalidated. The prediction score by Fowler et al was derived from a large, prospective cohort study, and proposed a comprehensive prognostic model of four clinical factors to estimate the likelihood of complications. However, even with a score of 0, approximately 16% of patients had complicated disease. This result is comparable to the current study. The model by Fowler et al was not validated externally.

TABLE 4

Clinical risk scores for complications in S aureus bacteraemia

Study	N	End‐point	NPV (95% CI)	NLR (95% CI)	External validation
Joseph 2013 27	306	IE (TTE or TEE)	1.00 (0.96‐1.00)	0.00 a	No
Gow 2015 28	574	IE (Duke)	1.00 (0.99‐1.00)	0.00 a	No
Rasmussen 2011 29	244	IE	0.95 (0.90‐0.98)	0.19 (0.09‐0.41)	No
Palraj 2015 30	678	IE (Duke)	0.98 (0.95‐0.99)	0.09 (0.04‐0.20)	No
Buitron de la Vega 2016 31	398	IE (Duke)	1.00 (0.99‐1.00)	0.00 a	No
Kaasch 2011 32	304	IE (Duke)	1.00 (0.94‐1.00)	0.00 a	Yes b
				0.08 (0.02‐0.59)
			0.99 (0.95‐1.00)
	432
Kaasch criteria in Khatib 19	177	IE (TEE)	0.80 (0.66‐0.90)	0.72 (0.40‐1.28)	—
Khatib 2013 19	177	IE (TOE)	0.98 (0.86‐1.00)	0.20 (0.01‐0.78)	No
Tubiana 2014 9	2091	IE (Duke)	0.99 (0.98‐0.99)	—	No
Heriot 2015 33	532	IE (TEE)	1.00 (0.86‐1.00)	0.00 a	No
Showler 2015 34	268	IE (Duke)	0.99 (0.95‐1.00)	0.05 (0.01‐0.35)	No
Incani 2013 35	144	IE (Duke)	0.84 (0.72‐0.92)	0.51 (0.30‐0.88)	No
Mölkänen 2016 36	430	Metastatic infection	0.36 (0.30‐0.44)	0.41 (0.32‐0.53)	No
Gliddon 2015 37	259	Metastatic infection	1.00 (—)	0.00 a	No
Lesens 2004 38	104	Metastatic infection	0.83 (0.73‐0.90)	0.34 (0.19‐0.62)	No
Fowler 2003 3	724	Complicated SAB	0.84 (—)	—	No
Lambregts (this study)	150	Complicated SAB	0.83 (0.68‐0.92)	0.14 (0.06‐0.31)	Yes

The negative predictive value (NPV) and negative likelihood ratio (NLR) are provided in this table as they represent the performance of the score in excluding complicated SAB/endocarditis. If a score performs well, the NPV will be high and the NLR will be low.

Abbreviations: IE, infective endocarditis; TEE, transesophageal echocardiography; TTE, transthoracic echocardiography.

Confidence interval calculations could not be performed because of zero events of endocarditis in the low‐risk group.

The criteria by Kaasch were applied to two separate cohorts. The risk score was later applied in the study by Khatib et al to a selected population of patients assessed with TEE.

Lesens 2004

Unfortunately, external validation in SAB risk scores has often been omitted. The importance of validation was illustrated with the disappointing performance of the Kaasch criteria for endocarditis in a cohort of patient assessed with TEE. The diversity in patient population, reflected in the differences in prevalence of complicated SAB in the various studies stresses the need for external validation. , ,

Recognition of SAB in clinical practice

Despite the lack of solid validated risk scores, a recent study randomised patients to algorithm based therapy vs standard of care. Therapy failure among patients that were treated for uncomplicated SAB using the algorithm was relatively high, 29.4%. High rates of relapse and therapy change due to unsatisfactory clinical response, suggest that these patients may have been misclassified using the algorithm.

Failure to identify patients with complicated SAB at an early timepoint may be explained by the heterogeneity of disease associated with SAB. Both host and pathogen virulence factors determine the clinical presentation as well as the course of the disease. , It may simply not be feasible to develop a comprehensive risk score with an acceptable negative predictive value for this clinical entity. Another challenge in the development of clinical rules is the definition of complicated SAB and the translation of this definition to observational studies. In daily practice, a relevant proportion of patients is treated with prolonged courses of antibiotic treatment based on clinical clues, without additional tests to confirm complications. This ‘grey zone’ of patients who receive prolonged treatment without confirmed complications impairs the development and validation of risk scores.

Strengths and limitations

In this study a broad definition of complicated SAB was applied, to limit misclassification as uncomplicated bacteraemia. This may have negatively impacted NPV, as patients may have been misclassified as complicated disease.

A second limitation of the study is that one of the predictors (neutropenia) was estimated imprecisely, because of the low prevalence of neutropenia in the study cohort.

An innovative feature of the current study is the use of TTP as an important element of the risk‐score. TTP may vary between institutions and is dependent on hospital logistics. Despite this limitation, use of TTP is biologically plausible and promising with regard to the assessment of SAB. The association between TTP and metastatic infection has been described previously and hence was confirmed in this study.

SUMMARY AND CONCLUSIONS

Despite the high incidence of SAB globally, contemporary strategies for differentiating uncomplicated and complicated bacteraemia in real life clinical practice, are based upon low or moderate quality evidence. This study provides a validated risk score for discriminating patients with low and high risk of complicated SAB. More studies, incorporating both clinical and laboratory variables, with thorough work‐up including nuclear imaging to define the clinical end‐point, are needed to optimise the clinical rule, aiming at further improvement of the negative predictive power. ,

DISCLOSURE

The authors declared no conflicts of interest. No funding was received for this work.

AUTHOR CONTRIBUTIONS

ML(A): Conception and design, data collection developmental cohort, statistical analyses and interpretation of the data, first draft of the manuscript and revisions. EM/MS: data collection validation cohort, manuscript revisions. ES/ND: supervision data collection validation cohort, manuscript revision. ML(E)/AB: TTP analyses, manuscript revisions. LG: Analysis and interpretation of the data, revision of the manuscript. OD: Conception and design, statistical analyses, revisions. MB: Conception and design, analyses and interpretation of the data, revision of the manuscript.

Supplementary Material

Click here for additional data file.