Predictive performance of comorbidity for 30-day and 1-year mortality in patients with bloodstream infection visiting the emergency department: a retrospective cohort study.

OBJECTIVES: To investigate whether the Charlson Comorbidity Index (CCI) predicted short-term and long-term mortality in patients with a bloodstream infection visiting the emergency department (ED) and compare it to the often-validated National Early Warning Score (NEWS).
DESIGN: A retrospective cohort study.
SETTING: A tertiary hospital in the Netherlands. PARTICIPANTS: Adult patients attending the ED with a blood culture-proven infection between 2012 and 2017 were included. We collected the comorbidities from the CCI and the vital signs from the NEWS. MAIN OUTCOMES: Short-term mortality (30-day) and long-term mortality (1 year). We assessed the predictive performance by discrimination, expressed as the area under the curve (AUC).
RESULTS: We included 1039 patients with a blood culture-proven infection. Mortality was 10.4% within 30 days and 27.8% within 1 year. On average patients had two comorbidities (ranging from 0 to 6). Highly prevalent comorbidities were malignancy (30.2%) and diabetes mellitus (20.5%). The predictive performance of the CCI was highest for 1-year mortality (AUC 0.696 (95%CI) (0.660 to 0.732)) and better compared with the NEWS (AUC (95% CI) 0.594 (0.555 to 0.632)). For prediction of 30-day mortality, the NEWS was superior (AUC (95% CI) 0.706 (0.656 to 0.756)) to the comorbidities of the CCI (AUC (95% CI) 0.568 (0.507 to 0.628)).
CONCLUSIONS: We found that presenting comorbidity (ie, the CCI) is most useful to prognosticate long-term outcome in patients with bloodstream infection in the ED. Short-term mortality is more accurately predicted by deviating vital signs (ie, the NEWS). © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

We used retrospectively collected data making our study prone to bias.

However, the quality of available data was high as all data used was essential for daily clinical practice.

All patients with a blood culture-proven infection in the emergency department were selected, thus missing culture negative infections

Our study was performed in a tertiary care centre with potentially higher prevalence of underlying comorbidity and higher mortality rates.

This would, however, only increase the chance of finding associations between comorbidity and mortality.

Introduction

Bloodstream infections are serious conditions with a profound global burden.1 Patients with infection often present in an acute care setting, such as the emergency department (ED). Early estimation of mortality risk is crucial to decide which patients need prompt treatment or might have self-limiting disease. Current triage systems and early warning scores in the ED mainly focus on deviating vital signs and less on underlying disease or comorbidity.2 3 Comorbidity can increase the risk to acquire an infection especially if altering the immune function (eg, in case of diabetes mellitus, malignancy, chronic renal failure, chronic liver disease, chronic obstructive pulmonary disease or HIV).4 However, less is known about whether presenting comorbidity in the ED also affects outcome due to infection and, if so, on which term.

The Charlson Comorbidity Index (CCI) is a chart review instrument that assigns weights to seventeen comorbidities and age in order to estimate mortality risk.5 6 The CCI was developed in 1987 to predict 1-year mortality and was validated during a 10-year follow-up. Weights were updated in 2011 based on relative risk of in-hospital mortality, resulting in a reduced index with twelve comorbidities.6 The CCI was previously proposed as an accurate tool to estimate mortality risk in various patient groups, for example, with heart disease,7 8 lung disease9 and malignancy.10 11 In patients with serious infection in the ED the CCI is already used in research setting to account for comorbidity and prevent confounding,12 13 however, its use was not often validated for both short-term and long-term outcome.14–16

The National Early Warning Score (NEWS) is based on deviating vital signs and has already shown accurate performance in predicting short-term mortality in patients with infection.17–20 Gaining more insight in the impact of underlying comorbidity versus vital signs may help to estimate outcome in patients with bloodstream infection in the ED. The aim of this study is to examine the predictive performance of the CCI for short-term mortality (30 days) and long-term mortality (1 year) among patients with a serious infection in the ED (ie, with a blood culture-proven infection). Subsequently, we compared the CCI to the often-validated NEWS.

Method

Study design and setting

We conducted a retrospective cohort study at the Erasmus University Medical Center Rotterdam (Erasmus MC), which is the largest tertiary referral centre in the Netherlands with an open access ED. We manually collected data from patient charts for all patients admitted to the ED with blood culture-proven infection between 1 July 2012 and 31 December 2017.

Patient and public involvement

Our research questions were developed by clinical expertise from acute physicians. Patients or public were not involved, only previous patient data were anonymously used with exempt from the Medical Ethics Committee of the Erasmus MC. We aimed to use patient data to be able to improve future clinical practice.

Selection of participants

Patients were eligible for inclusion if they were at least 18 years of age and had a blood culture-proven infection in the ED. Blood culture-proven infection was defined as presence of a known pathogen (eg, Escherichia coli) in one blood culture or a common commensal (eg, Staphylococcus epidermidis) in at least two blood cultures collected on separate occasions within 2 days from ED admission.21 22 Only the first episode of blood culture-proven infection was included to prevent domination of results by individuals that frequently visited the ED.

Data collection and processing

Data were derived from the ED by chart review and combined with a database with all collected blood cultures. Data are publicly available online.23 The ED database included demographics (ie, sex, age), first recorded vital signs (ie, systolic blood pressure, body temperature, respiratory rate, peripheral oxygen saturation, consciousness,24 and whether there was need for any supplemental oxygen in order to calculate the NEWS.3 We collected the CCI, which are underlying diseases that were already known during the ED visit (eg, diabetes mellitus, liver disease, malignancy, table 1). Mortality data were updated from municipal death registration records. Outcome was short-term mortality (30 days) and long-term mortality (1 year).

Table 1

Comorbidities of the Charlson Comorbidity Index (CCI) in patients with a blood culture-proven infection in the ED (n=1039)

Comorbidity	Prevalence (%)	Weights
		CCIoriginal	CCIupdated
Age*		1	1
Diabetes mellitus†	20.5
Uncomplicated	19.2	1	0
End-organ damage	1.3	2	1
Liver disease‡	14.3
Mild	13.5	1	2
Moderate to severe	0.9	3	4
Malignancy§	30.2
Leukaemia, lymphoma, solid tumour	17.4	2	2
Metastatic solid tumour	12.8	6	6
AIDS¶	0.3	6	4
Chronic kidney disease**	16.3	2	1
Congestive heart failure ††	12.8	1	2
Myocardial infarction‡‡	13.4	1	0
Chronic pulmonary disease§§	12.9	1	1
Peripheral vascular disease¶¶	11.6	1	0
CVA or TIA***	13.6	1	0
Dementia†††	3.5	1	2
Hemiplegia‡‡‡	0.3	2	2
Connective tissue disease§§§	7.4	1	1
Peptic ulcer disease¶¶¶	2.4	1	0

*Age: 1 point for each decade from 50 to 90 years of age.

†Diabetes mellitus: uncomplicated (= diabetes with medication), end-organ damage (= diabetes with retinopathy, neuropathy, nephropathy or Brittle diabetes).

‡Liver disease: mild (= cirrhosis without portal hypertension, chronic hepatitis), moderate to severe (= cirrhosis with portal hypertension, variceal bleeding).

§Malignancy: leukaemia, lymphoma or solid tumour. All initially treated in the last 5 years, excluding non-melanomatous skin cancers and in situ cervical carcinoma.

¶AIDS: AIDS (not just HIV positive).

**Chronic kidney disease: on dialysis, status post kidney transplant, uraemia, creatinine > 265 umol/L (not acute).

††Congestive heart failure: exertional or paroxysmal nocturnal dyspnea and has responded to digitalis, diuretics or afterload reducing agents.

‡‡Myocardial infarction: history of definite or probable myocardial infarction (ECG changes and/or enzyme changes).

§§Chronic pulmonary disease: symptomatic dyspnoea due to chronic respiratory conditions (including asthma).

¶¶Peripheral vascular disease: intermittent claudication, peripheral arterial bypass for insufficiency, gangrene, acute arterial insufficiency, untreated aneurysm (≥ 6cm).

***CVA or TIA: history of CVA (without hemiplegia) or TIA.

†††Dementia: chronic cognitive deficit.

‡‡‡Hemiplegia: hemiplegia or paraplegia.

§§§Connective tissue disease: systemic lupus erythematosus, polymyositis, mixed connective tissue disease, polymyalgia rheumatic, moderate to severe rheumatoid arthritis.

¶¶¶Peptic ulcer disease: history of treatment for ulcer disease or history of ulcer bleeding.

CVA, cerebrovascular accident; ED, emergency department; TIA, transient ischaemic attack.

The Charlson Comorbidity Index

The original CCI is calculated from age and seventeen comorbidities, that is,: diabetes mellitus (uncomplicated or with end-organ damage), liver disease (mild or moderate to severe), malignancy (leukaemia, lymphoma, solid tumour or metastatic solid tumour), AIDS, chronic kidney disease, congestive heart failure, myocardial infarction (MI), chronic pulmonary disease, peripheral vascular disease, cerebrovascular accident (CVA) or transient ischaemic attack (TIA), dementia, hemiplegia, connective tissue disease, and peptic ulcer disease.5 The updated CCI is reduced to twelve comorbidities (ie, following comorbidities were excluded: uncomplicated diabetes mellitus, MI, peripheral vascular disease, CVA or TIA, and peptic ulcer disease). See table 1 for a detailed description on scoring the CCI.

Each comorbidity has an associated weight ranging from 1 to 6. We investigated both the original and updated weights. The sum of all the weights results in a single comorbidity score for a patient. A score of 0 indicates absence of comorbidity and the CCI increases with presence of more comorbidities. The CCI also includes age, 1 point is added for each decade after an age of 50 years (ie, 1 point for 50–59 years, 2 points for 60–69 years).

Data analysis

We visualised the distribution of the original and updated CCI with use of histograms. Also, we examined the prevalence of all separate comorbidities. Data were presented as absolute numbers (%). We had complete data on demographics, comorbidity, and outcome. Incomplete data on vital signs were imputed as normal.

We investigated the association between comorbidity and mortality (30 days and 1 year) both univariably (each comorbidity individually included) and multivariably (all comorbidities included in the model) with logistic regression. Results were presented as ORs with 95% CIs. We performed a Bonferroni correction to prevent type 1 error.

Additionally, we assessed mortality rates (30 days and 1 year) for the number of comorbidities (ie, 0–6) and each level of the CCI (ie, 0–15) and categorised these levels to a corresponding mortality rate. Also, we assessed mortality rates for each age decade from 50 years.5

Discriminative ability of the CCI was assessed with area under the curve (AUC) for 30-day and 1-year mortality. We assessed the predictive performance of the CCI (consisting of age and comorbidities), age and comorbidities. Also, we compared the original to the updated CCI. Additionally, we compared the AUC of the CCI to the NEWS, which has previously shown accurate performance in predicting short-term mortality in patients with infection.17–20 Subsequently, we combined the CCI, NEWS, and age. Finally, we assessed the predictive performance of CCI and NEWS for mortality over time by constructing a time-dependent AUC.

Statistical analyses were performed using R V.3.6.3.

Results

Patient characteristics

We identified 1286 adult patients with a blood culture-proven infection in the ED between 1 July 2012 and 31 December 2017. We excluded 247 patients with a recurrent infection, resulting in 1039 unique patients. Patient characteristics are shown in table 2. Table 1 shows the prevalence of each comorbidity of the CCI and the original and updated weights that were assigned to each comorbidity. In our study, the original CCI ranged from 0 to 15 (median 4, IQR 4). We observed no major differences in distribution of the original and updated CCI (figures 1 and 2).

Table 2

Characteristics of patients with a blood culture-proven infection in the ED

Characteristic	Missing	Total population (n=1039)
Sex, male	0	626 (60.3)
Age, mean (SD)	0	61 (15.6)
Arrival, by ambulance	0	249 (24.0)
Triage by MTS, acute/highly urgent	49 (4.7)	238 (22.9)
Direct intensive care unit admittance	0	22 (6.8)
Comorbidity
CCIoriginal, mean (SD)*	0	4 (2.9)
CCIupdated, mean (SD)*	0	4 (2.8)
Vital signs, mean (SD)
Temperature, °C	9 (0.8)	38.3 (1.2)
Heart rate, /min	24 (2.3)	106 (22.9)
Respiratory rate, /min	369 (35.5)	23 (8.2)
Systolic blood pressure, mm Hg	20 (1.9)	125 (27.4)
Oxygen saturation, %	43 (4.1)	96 (5.1)
Any supplemental oxygen	0	401 (38.6)
Consciousness, not alert	174 (16.7)	112 (10.8)
NEWS, mean (SD)†	0	5 (3.7)
Isolated bacteria
Escherichia coli	0	341 (32.8)
Staphylococcus aureus	0	105 (10.1)
Streptococcus pneumoniae	0	87 (8.4)
Mortality
30 days	0	108 (10.4)
1 year	0	289 (27.8)

Data are presented as number (percentage) of patients unless otherwise indicated.

*For the prevalence of all comorbidities, see table 1.

†NEWS imputed as normal.

CCI, Charlson Comorbidity Index; ED, emergency department; MTS, Manchester Triage System; NEWS, National Early Warning Score.;

Histogram of the original CCI. Frequency for each score of the CCI. CCI, Charlson Comorbidity Index.

Histogram of the updated CCI. Frequency for each score of the CCI. CCI, Charlson Comorbidity Index.

Comorbidities

Comorbidity was common among patients with a blood culture-proven infection. On average patients had two comorbidities, ranging from 0 to 6. Of all patients, 209 (20.1%) had no comorbidity (figures 1 and 2, online supplemental appendix A). Highly prevalent comorbidities were malignancy (30.2%) and diabetes mellitus (20.5%, table 1). Also prevalent were chronic kidney disease (16.3%), liver disease (14.3%), CVA or TIA (13.6%), MI (13.4%), chronic pulmonary disease (12.9%), congestive heart failure (12.8%) and peripheral vascular disease (11.6%, table 1).

Comorbidity and mortality

In our population of patients with blood culture-proven infection in the ED we found 10.4% mortality within 30 days (table 2). After Bonferroni correction, no prevalent comorbidities were independently associated with 30-day mortality (table 3). Mortality within 30 days was comparable for patients with 0–4 comorbidities, with an average mortality rate of approximately 10.0%. For patients with 5–6 comorbidities the average mortality rate was higher (32.1%, online supplemental appendix A).

Table 3

Association between comorbidity (and age) and 30-day mortality

Comorbidity	N	30-day mortality	OR (95% CI) 30-day mortality		1-year mortality	OR (95% CI) 1 year mortality
			Univariable	Multivariable		Univariable	Multivariable
Diabetes mellitus:
Uncomplicated	200 (19.2)	24 (12.0)	1.23(0.74 to 1.96)	1.09(0.65 to 1.79)	53 (26.5)	0.92(0.64 to 1.30)	0.88(0.60 to 1.28)
End-organ damage	13 (1.3)	6 (46.2)	7.76(2.46 to 23.8)*	9.28(2.40 to 3.73)*	11 (84.6)	14.8(3.92 to 96.0)*	14.0(3.42 to 95.0)*
Liver disease:
Mild	140 (13.5)	7 (5.0)	0.42(0.18 to 0.85)	0.46(0.19 to 0.96)	40 (28.6)	1.04(0.70 to 1.54)	0.95(0.61 to 1.46)
Moderate to severe	9 (0.9)	1 (11.1)	1.08(0.06 to 5.96)	0.39(0.01 to 4.45)	4 (44.4)	2.09(0.51 to 7.96)	1.47(0.19 to 8.10)
Malignancy:
Leukaemia, lymphoma, solid	181 (17.4)	14 (7.7)	0.68(0.36 to 1.19)	0.71(0.36 to 1.28)	57 (31.5)	1.24(0.87 to 1.75)	1.64(1.11 to 2.40)
Metastatic solid tumour	133 (12.8)	24 (18.0)	2.15(1.29 to 3.49)*	2.06(1.18 to 3.54)	85 (63.9)	6.09(4.16 to 9.02)*	6.93(4.56 to 10.6)*
AIDS	3 (0.3)	0 (0.0)	∞	∞	2 (66.7)	∞	∞
Chronic kidney disease	169 (16.3)	11 (6.5)	0.55(0.28 to 1.02)	0.47(0.22 to 0.93)	37 (21.9)	0.69(0.46 to 1.01)	0.75(0.48 to 1.16)
Congestive heart failure	133 (12.8)	17 (12.8)	1.31(0.73 to 2.23)	1.08(0.57 to 1.94)	35 (26.3)	0.92(0.60 to 1.37)	0.92(0.57 to 1.46)
Myocardial infarction	139 (13.4)	17 (12.2)	1.24(0.69 to 2.10)	1.02(0.54 to 1.84)	40 (28.8)	1.06(0.71 to 1.56)	1.06(0.67 to 1.66)
Chronic pulmonary disease	134 (12.9)	22 (16.4)	1.87(1.10 to 3.06)	1.87(1.07 to 3.18)	41 (30.6)	1.17(0.78 to 1.72)	1.31(0.85 to 2.00)
Peripheral vascular disease	121 (11.6)	12 (9.9)	0.94(0.48 to 1.71)	0.63(0.30 to 1.21)	35 (28.9)	1.06(0.69 to 1.60)	0.90(0.55 to 1.43)
CVA or TIA	141 (13.6)	23 (16.3)	1.86(1.11 to 3.03)	1.44(0.82 to 2.45)	50 (35.5)	1.52(1.04 to 2.20)	1.51(0.99 to 2.29)
Dementia	36 (3.5)	8 (22.2)	2.58(1.07 to 5.57)	1.32(0.52 3.01)	12 (33.3)	1.31(0.63 to 2.61)	0.93(0.41 to 1.99)
Hemiplegia	3 (0.3)	0 (0.0)	∞	∞	0 (0.0)	∞	∞
Connective tissue disease	77 (7.4)	9 (11.7)	1.15(0.52 to 2.27)	1.07(0.48 2.22)	18 (23.4)	0.78(0.44 to 1.32)	0.99(0.54 to 1.74)
Peptic ulcer disease	25 (2.4)	3 (12.0)	1.18(0.28 to 3.48)	0.89(0.18 to 3.06)	9 (36.0)	1.47(0.62 to 3.31)	1.35(0.50 to 3.35)
Age, per decade			10.5(10.3 to 10.7)*	10.4(10.2 to 10.6)*		10.3(10.2 to 10.4)*	10.3(10.2 to 10.4)*

Data are presented as number (percentage) of patients and as ORs with 95% CIs.

*Statistically significant after Bonferroni correction (p<0.003).

CVA, cerebrovascular accident; TIA, transient ischaemic attack.

One-year mortality was 27.8% (table 2). After Bonferroni correction, we found that only an underlying metastatic solid tumour was independently associated with 1-year mortality (online supplemental appendix A). Also, diabetes mellitus with end-organ damage was associated with mortality, however, prevalence of this comorbidity was very low in our population (only 1.3%). One-year mortality was lower for patients with no comorbidity (17.7%) compared with patients with 1–4 comorbidities (29.8%) and 5–6 comorbidities (46.4%, online supplemental appendix A).

Age and mortality

Age (per decade from 50 years) was independently associated with both 30-day and 1-year mortality in patients with blood culture-proven infection (OR per decade increase (95% CI): 10.4 (10.2 to 10.6) for 30-day mortality and 10.3 (10.2 to 10.4) for 1-year mortality, table 3 and online supplemental appendix B).

Predictive performance of the CCI, NEWS and age

The predictive performance of the CCI was highest for 1 year mortality with an AUC of 0.696 that increased to 0.703 when excluding short-term deaths. The CCI had a better predictive performance for 1 year mortality (AUC (95% CI) 0.696 (0.660 to 0.732)) compared with the NEWS (AUC (95% CI) 0.594 (0.555 to 0.632)). Combining the CCI with the NEWS did not improve the predictive ability of the CCI (AUC (95% CI) 0.696 (0.662 to 0.730)).

For prediction of 30-day mortality, the NEWS was superior (AUC (95% CI) 0.706 (0.656 to 0.756)) to the comorbidities of the CCI (AUC (95% CI) 0.568 (0.507 to 0.628)). Combining the NEWS with the CCI increased the AUC of the NEWS from 0.706 to 0.743, however, this increasing trend was largely explained by adding age to the NEWS (AUC 0.740) and not much by adding comorbidities to NEWS (AUC 0.719, table 4). Also, using time-dependent AUC’s showed more accurate prediction of longer-term mortality for CCI, whereas short-term mortality was more accurately predicted by the NEWS (online supplemental appendix C).

Table 4

Validation of the CCI and comparison with the news

	AUC (95% CI)
	30-day mortality	1-year mortality
CCIoriginal*	0.643(0.589;0.697)	0.696(0.660;0.732)†
Age‡	0.661(0.609 to 0.712)	0.616(0.581 to 0.652)
Comorbidities	0.568(0.507 to 0.628)	0.663(0.625 to 0.701)
NEWS	0.706(0.656 to 0.756)	0.594(0.555 to 0.632)
NEWS and CCIoriginal	0.743(0.697 to 0.789)	0.696(0.662 to 0.730)
NEWS and age‡	0.740(0.695 to 0.785)	0.623(0.587 to 0.660)
NEWS and comorbidities	0.719(0.669 to 0.769)	0.681(0.645 to 0.716)

*The AUC of the original CCI was comparable to the AUC of the updated CCI, that is, 0.642 (0.588 to 0.696) for 30-day mortality and 0.695 (0.659 to 0.732) for 1-year mortality.

†When excluding short-term deaths, the AUC of the CCI increased to 0.703.

‡Age, per decade from 50 years.

AUC, area under receiver operator curve; CCI, Charlson Comorbidity Index; NEWS, National Early Warning Score.

The updated CCI performed similar to the original CCI (table 4). See online supplemental appendix D for the specific mortality rates for each CCI level.

Discussion

Our research shows that patients with a serious infection (ie, blood culture-proven infection) in the ED have high mortality and comorbidity is common, specifically underlying malignancy and diabetes mellitus. None of the prevalent comorbidities from the CCI were independent predictors of mortality, except from having a metastatic solid tumour. The CCI seems most useful to prognosticate long-term outcome (1 year), while short-term mortality (30 days) is more accurately predicted by the NEWS.

The CCI had its highest predictive performance for 1 year mortality, which is in line with previous research.15 Compared with the CCI, the predictive performance of the NEWS was worse for 1-year mortality. Combining both scores did not improve the prediction of long-term outcome. We found equal predictive performance for both the original and updated CCI, which is a simplified version. The updated CCI was designed to predict in-hospital mortality6 and validated by multiple studies.15 16 25 However, in our study, performance of the CCI was not convincing for short-term mortality. Presence of up to four comorbidities yielded the same risk of 30-day mortality. Also, none of the prevalent comorbidities were independently associated with short-term mortality.

The NEWS is based on vital signs and has previously shown accurate performance in predicting short-term mortality risk in patients attending the ED with infection, which we confirmed.17–20 Compared with the NEWS, the predictive performance of the comorbidities from the CCI was worse for 30-day mortality. Combining the NEWS with the CCI yielded the highest predictive performance for short-term mortality. However, this increasing trend was largely explained by adding age to the NEWS, and not much by adding comorbidities to the NEWS. An explanation for the improving prediction by age can be that elderly are less resilient to cope with stressors such as a serious infection, for example, due to immunosenescence26 or sarcopenia.27 This hypothesis corresponds to previous research about tolerance to surgery in elderly28 and was also observed during the COVID-19 pandemic.29

Our study has limitations. We used retrospectively collected data making our study prone to bias. However, the quality of available data was high as all data used was essential for daily clinical practice. We had no data on measures for frailty (ie, weight loss, mobility, muscle weakness), which can be useful to further characterise the effect of age on short-term mortality. Also, we chose to select all patients with a blood culture-proven infection in the ED to represent a patient group with true serious infection, thus missing culture negative infections. Finally, our study was performed in a tertiary care centre and, therefore, the prevalence of (complex) underlying comorbidity and the risk of adverse outcome is likely higher compared with lower level care centres. This would, however, only increase the chance of finding associations between comorbidity and mortality.

Concluding, in patients with bloodstream infection in the ED we found that presenting comorbidity (ie, the CCI) is most useful to prognosticate long-term outcome in patients with bloodstream infection in the ED. Short-term mortality is more accurately predicted by deviating vital signs (ie, the NEWS), and less by comorbidity. Our finding indicates that comorbidity adjustment is more important when studying long-term outcomes than for research of short-term mortality.