Prediction of median survival time in sepsis patients by the SOFA score combined with different predictors.

BACKGROUND: Sepsis is the leading cause of intensive care unit (ICU) admission. The purpose of this study was to explore the prognostic value of the Sequential Organ Failure Assessment (SOFA) score, the Acute Physiological and Chronic Health Evaluation II (APACHE II) score, and procalcitonin (PCT), albumin (ALB), and lactate (LAC) levels in patients with sepsis.
METHODS: Consecutive adult patients with suspected or documented sepsis at ICU admission were recruited. Their basic vital signs and related auxiliary examinations to determine their PCT and ALB levels and APACHE II score were recorded at ICU admission, and their LAC levels and SOFA scores were recorded for one week after admission. The influence of these variables on hospital mortality was evaluated. Logistic regression was used to derive the Sepsis Hospital Mortality Score (SHMS), a prediction equation describing the relationship between predictors and hospital mortality. The median survival time was calculated by the Kaplan-Meier method. In the validation group, the kappa value was calculated to evaluate the stability of the derived formula.
RESULTS: This study included 894 sepsis patients admitted to 18 ICUs in 16 tertiary hospitals. Patients were randomly assigned to an experimental group (626 cases) and validation group (258 cases). In addition, a nonsurvival group (248 patients) of the experimental group was established according to the outcome at the time of discharge. The hospital mortality rate in the experimental group was 39.6% (248/626). Univariate and multivariate regression analyses revealed that the APACHE II score (odds ratio [OR] = 1.178), △SOFA (OR = 1.186), △LAC (OR = 1.157), and SOFA mean score (OR = 1.086) were independently associated with hospital mortality. The SHMS was calculated as logit(p) = 4.715 - (0.164 × APACHE II) - (0.171 × △SOFA) - (0.145 × △LAC) - (0.082 × SOFA mean). A receiver operating characteristic curve was constructed to further investigate the accuracy of the SHMS, with an area under the curve of 0.851 (95% confidence interval [CI] 0.821-0.882; p < 0.001) for hospital mortality. In the low-risk group and high-risk groups, the corresponding median survival times were 15 days and 11 days, respectively.
CONCLUSION: The APACHE II score, △SOFA, △LAC and SOFA mean score were independently associated with hospital mortality in sepsis patients and accurately predicted the hospital mortality rate and median survival time. Data on the median survival time in sepsis patients could be provided to clinicians to assist in the rational use of limited medical resources by facilitating prudent resource allocation. TRIAL REGISTRATION: ChiCTR-ECH-13003934, retrospectively registered on August 03, 2013.

This article is a multicenter study which was performed from January 2014 to August 2015 in 18 ICUs in 16 tertiary hospitals in China.

The APACHE II score, ΔSOFA, ΔLAC and SOFA mean score were independently associated with hospital mortality in sepsis patients and accurately predicted the hospital mortality rate and median survival time.

Data on the median survival time in sepsis patients could be provided to clinicians to assist in the rational use of limited medical resources by facilitating prudent resource allocation.

Background

Sepsis is the leading cause of intensive care unit (ICU) admission [1]. In addition, sepsis is associated with concurrent multiple organ dysfunction syndrome, which is the main cause of death in ICU patients [2]. The social and economic impact of sepsis consumes a considerable proportion of healthcare resources [3, 4]. Despite the decline in mortality in the past decade due to increased sepsis awareness and management, the short-term mortality rate has remained at 20% or higher as the population ages, the number of invasive medical procedures increases, and the incidence of cancer in the elderly population increases [5, 6]. Early stratification and identification of patients with a high risk of death is essential [7]. The early warning score is an alternative tool for risk stratification. This will help guide clinicians in developing different treatment plans in a timely manner for the individual patient with different risk levels.

The latest definition of sepsis emphasizes organ failure [8], referring to two or more changes in the Sequential Organ Failure (SOFA) score. One of the scoring systems used for predicting mortality in septic patients is the initial SOFA score. Harm-Jan de Grooth and colleagues [9] pointed out that the effects of treatment on the change in the SOFA score appear to be reliably and consistently associated with mortality in random controlled trials (RCTs). Fixed-day SOFA was the most frequently reported outcome among the reviewed RCTs, but it was not significantly associated with mortality.

One of the characteristics of a clinical early warning scoring system is that it needs to apply commonly used and easily available predictive indicators. The Acute Physiological and Chronic Health Evaluation II (APACHE II) score, SOFA score, lactate (LAC) and procalcitonin (PCT) levels are all meet this requirement. However, the establishment of the diagnostic criteria was based on a retrospective analysis of the database, and significant data loss inevitably affected the final results. Their limited performance prevents these markers from being applied to individual risk stratifications [10, 11].

Combining several predictors into a single classification rule should help to improve their accuracy and practicability. We investigated the dynamic changes in the SOFA score and LAC levels in patients with sepsis during the first week after admission. The purpose of the present study was to derive a prediction equation using a combination of the APACHE II score, SOFA score, and LAC and PCT levels to predict hospital mortality in sepsis patients; this combined score is called the sepsis hospital mortality score (SHMS). Furthermore, according to the formula, the severity of the conditions of patients was evaluated, and stratification was performed to calculate the median survival time corresponding to each category of patients. Data on the median survival time in sepsis patients could be provided to clinicians to assist in the rational use of limited medical resources by facilitating prudent resource allocation and appropriate classification for research or administrative purposes.

Methods

Patient selection

This prospective observational study was performed from January 2014 to August 2015 in 18 ICUs in 16 tertiary hospitals in China.

The sepsis 1.0 standard was used to establish the database initially [12]. We conducted a retrospective study with information from this prospective cohort database, so database patients were diagnosed according to the new definitions proposed by the Third International Consensus Definitions for Sepsis and Septic Shock [8].

Consecutive adult patients with suspected or documented sepsis at ICU admission were recruited. We excluded patients who met any one of the following conditions: (1) withdrawal of life-sustaining treatment; (2) postoperative cardiopulmonary resuscitation (CPR); or (3) incomplete data.

The protocol used in this study was approved by the local ethics committee (Fuxing Hospital, Capital Medical University, 2013FXHEC-KY018). Eight hundred and ninety-four patients were recruited. García-Gallo et al. reported the grouping ratio as 7:3 [13], approximately 70% of the patients were randomly assigned to the experimental group (626 sepsis cases). It is generally believed that the sample size should be at least 10–20 times higher than the number of independent variables [14]; the relatively stricter requirement is that the number of samples in each category in the two-classification results is at least 10 times higher than the number of independent variables.

Basic vital signs and patient characteristics, such as age, sex, admission category, auxiliary examinations, comorbidities, primary sites of infection, PCT level, albumin (ALB) level, and APACHE II score, were collected at ICU presentation, while the LAC levels and SOFA scores were recorded for one week after admission. The values immediately preceding missing values were used when data values were missing [15].

Definition of predictive indicators

To create a predictive model, biomarker levels were presented in absolute values and measured at different time points; the relative change in biomarker values was analyzed according to each indictor, which was described as follows:

SOFA max = maximum SOFA score over 7 consecutive days, SOFA min = minimum SOFA score over 7 consecutive days, SOFA initial = SOFA score on the first day after admission, SOFA mean = average SOFA score for 7 consecutive days.

△SOFA1 = SOFA max – SOFA min, △SOFA2 = SOFA max – SOFA initial.

LAC max = maximum LAC over 7 consecutive days, LAC min = minimum LAC over 7 consecutive days, LAC initial = LAC on the first day after admission, LAC mean = average LAC for 7 consecutive days.

△LAC1 = LAC max – LAC min, △LAC2 = LAC max – LAC initial.

Statistical analyses

The baseline patient characteristics were summarized in percentages for categorical variables, and data were presented as the mean ± standard deviation for normally distributed variables or median (25th–75th percentile) for nonnormally distributed variables. The Shapiro–Wilk test was used to assess the normality of the data.

Univariate comparisons of variables between the two groups were analyzed using the independent t-test for normally distributed variables and the Mann–Whitney U test for nonnormally distributed variables. Categorical variables were presented as frequencies (percentages) and were compared using a chi-squared test; p < 0.05 was considered statistically significant.

Our sepsis mortality score represents the predicted probability of hospital mortality. To derive this score, we used logistic regression by including all predictors with a univariate significance of p < 0.1 as covariates and hospital mortality as the dependent variable, employing the forward elimination method. In multiplicative models, such as logistic regression [16], the product term is used to indicate the presence or absence of an interaction (p < 0.05). The generated coefficients for each predictor in the final step of the logistic regression were used to create the equation to predict a logit transformation of the probability of hospital mortality: Logit(p) = β0(intercept) + β1 (APACHE II) + β2 (△SOFA2) + β3(△LAC2) + β4(SOFA mean).

The model calibration was evaluated using the Hosmer–Lemeshow goodness-of-fit test. The area under the receiver operating characteristic (AUROC) curve [95% confidence interval (CI)] for the sepsis mortality score was calculated to predict hospital mortality.

Patients were then grouped according to the optimal cut-off value of the sepsis mortality score. A log-rank test was performed to compare the survival curves of the groups. The median survival time was calculated by the Kaplan–Meier method. In the validation group, the kappa value was calculated to evaluate the stability of the derived equation.

All statistical analyses were performed using SPSS version 22.0 (IBM, Armonk, NY, USA).

Results

Patient characteristics and prevalence of sepsis

The patients eligible for inclusion and exclusion are outlined in Fig. 1. A total of 894 patients diagnosed with sepsis were included in this study. According to the ratio of 7:3 [13], approximately 70% of the patients were randomly assigned to the experimental group (626 cases). In addition, the experimental group was further divided into a survivor group (378 cases) and a nonsurvivor group (248 cases) according to the outcome at the time of discharge.

Figure 1.

Inclusion of patients for analysis of the association between different predictors and mortality at hospital discharge between January 2014 to August 2015. CPR Cardiopulmonary Resuscitation

Table 1 shows the demographics and clinical characteristics of all patients. In the experimental group, the hospital mortality was 39.6%. The mean age was 68 years (range 55–80 years). Occurrences of mechanical ventilation and renal replacement therapy correlated with death during the hospital day. ICU admission sources were from medical wards (65.8%) and surgical wards (34.2%). Age, sex, and comorbidity influenced the diagnostic or predictive accuracy of the predictors. However, there was no significant difference in the above factors between the two groups, suggesting good homogeneity. In terms of infection focus, only pulmonary infection, bacteremia and abdominal infection showed significant differences.

Table 1

Baseline demographics and clinical characteristics in survivors and nonsurvivors at ICU discharge of two groups

Variables	Experimental group				Validation group
	All	Survivors	Nonsurvivors	P	All	Survivors	Nonsurvivors	P
	n = 626	n = 378	n = 248		n = 268	n = 190	n = 78
Demographic
Age (years)	68.0 (55.0, 80.0)	68.5 (55.0, 80.0)	68.0 (55.0, 79.5)	0.728	68.0 (58.0, 80.0)	64.0 (54.0, 77.0)	75.0 (65.0, 82.0)	0.000
Sex (male), n (%)	394 (62.9)	229 (60.6)	163 (65.7)	0.193	169 (63.1)	125 (65.8)	44 (56.4)	0.148
BMI	22.9 (21.0, 24.9)	23.0 (21.1, 24.7)	22.9 (20.8, 25.0)	0.354	22.7 (20.5, 24.5)	22.8 (20.8, 24.5)	22.3 (19.6, 24.2)	0.134
Clinical
Admission Category, n (%)
Medical	412 (65.8)	237 (62.7)	175 (70.6)	0.011	225 (84.0)	152 (80.0)	73 (93.6)	0.010
Surgical	214 (34.2)	141 (37.3)	73 (29.4)	0.011	43 (16.0)	38 (20.0)	5 (6.4)	0.010
Severity of illness, n (%)
MV	469 (74.9)	246 (65.1)	223 (89.9)	0.000	185 (69.0)	116 (61.1)	69 (88.5)	0.000
CRRT	111 17.7)	36 (9.5)	75 (30.2)	0.000	38 (14.2)	13 (6.8)	25 (32.1)	0.000
Sepsis	325 (51.9)	227 (60.0)	98 (39.5)	0.000	128 (47.8)	105 (55.3)	23 (29.5)	0.000
Septic shock	301 (48.1)	151 (39.9)	150 (60.4)	0.000	140 (52.2)	85 (44.7)	55 (70.5)	0.000
CCI	1 (0, 2)	1 (0, 2)	2 (1, 2)	0.001	1 (0, 2)	1 (0, 2)	2 (1, 3)	0.000
Primary sites of infection, n (%)
Lungs	333 (53.2)	180 (47.6)	153 (61.7)	0.001	144 (53.7)	94 (49.5)	50 (64.1)	0.029
Thoracic cavity	11 (1.8)	7 (1.9)	4 (1.6)	1.000	8 (3.0)	7 (3.7)	1 (1.3)	0.513
Abdomen	162 (25.9)	110 (29.1)	52 (21.0)	0.023	65 (24.3)	52 (27.4)	13 (16.7)	0.063
Urinary tract	30 (4.8)	19 (5.0)	11 (4.4)	0.157	7 (2.6)	7 (3.7)	0 (0.0)	0.195
Bacteremia	38 (6.1)	15 (4.0)	23 (9.3)	0.007	17 (6.3)	11 (5.8)	6 (7.7)	0.562
Catheter	3 (0.5)	1 (0.3)	2 (0.8)	0.712	2 (0.7)	1 (0.5)	1 (1.3)	1.000
Soft tissue	15 (2.4)	9 (2.4)	6 (2.4)	0.975	10 (3.7)	9 (4.7)	1 (1.3)	0.317
Nervous system	6 (1.0)	4 (1.1)	2 (0.8)	1.000	2 (0.7)	0 (0.0)	2 (2.6)	0.152
Hospital days	16 (9, 27)	17 (10, 28)	15 (6, 24)	0.003	19 (11, 30)	20 (11, 31)	15 (9, 25)	0.007

BMI body mass index, CCI Charlson Comorbidity Index, CRRT continuous renal replacement therapy, ICU intensive care unit, MV mechanical ventilation. Continuous variables are presented as mean ± standard deviation or median and interquartile range when not normally distributed. Categorical variables are presented as number (%)

Predictor profiles

The medians and interquartile ranges are shown for each predictor for the population as a whole and stratified by the outcome of hospital mortality (Table 2). The APACHE II score, SOFA score, ALB and LAC levels differed significantly among the groups. The median SOFA score was higher in the nonsurvival group than in the survival group (p < 0.001). In contrast, ALB was lower in the nonsurvival group than in the survival group (25.5 vs. 27.0 mmol/L, respectively, p < 0.001). Factors other than PCT were independent predictors of mortality in patients with sepsis. The clinical value of the different predictors composed of original laboratory parameters in the diagnosis of sepsis was evaluated with a receiver operating characteristic (ROC) curve. We determined the AUROC and the cut-off values for each predictor to classify patients with in-hospital mortality as a summary measure of the predictive accuracy (Table 2).

Table 2

APACHE II, SOFA, LAC, and PCT in survivors and nonsurvivors and their predictive for hospital mortality

Biomarkers	All (n = 626)	Survivors (n = 378)	Nonsurvivors (n = 248)	P value	AUROC (95%CI)	Cut-off
APACHE II	18 (14, 25)	16 (11, 20)	25 (19, 30)	0.000	0.823 (0.789–0.856)	21.5
PCT	1.5 (0.2, 9.9)	1.3 (0.2, 10.0)	2.1 (0.2, 10.9)	0.290	0.525 (0.479–0.572)	0.8
ALB	26 (22.7, 30.0)	27.0 (23.0, 31.2)	25.5 (22.0, 8.5)	0.001	0.419 (0.373–0.464)	27.7
Temperature	37.6 (36.7, 38.7)	37.5 (36.6, 38.5)	37.6 (36.8, 38.9)	0.022	0.554 (0.508–0.600)	38.9
SOFA initial	7.0 (4.0, 10.0)	6.0 (4.0, 8.0)	9.0 (6.0, 13.0)	0.000	0.715 (0.674–0.756)	6.7
SOFA mean	6.3 (3.9, 9.7)	5.0 (3.1, 7.3)	9.3 (6.3, 12.7)	0.000	0.774 (0.737–0.811)	5.8
SOFA max	8.0 (5.0, 12.0)	6.7 (4.0, 10.0)	12.0 (8.0, 15.0)	0.000	0.767 (0.730–0.804)	7.3
△SOFA1	3.0 (2.0, 5.0)	3.0 (1.0, 5.0)	3.0 (2.0, 5.0)	0.004	0.567 (0.521–0.613)	2.5
△SOFA2	0.0 (0.0, 2.0)	0.0 (0.0, 1.0)	1.0 (0.0, 3.8)	0.000	0.634 (0.589–0.680)	0.5
LAC initial	1.7 (1.1, 2.7)	1.6 (1.1, 2.4)	2.0 (1.2, 3.3)	0.000	0.609 (0.564–0.655)	1.9
LAC mean	1.6 (1.1, 2.4)	1.4 (1.0, 2.0)	2.0 (1.4, 3.4)	0.000	0.697 (0.656–0.739)	2.2
LAC max	2.3 (1.6, 3.5)	2.0 (1.3, 2.8)	2.8 (2.0, 6.4)	0.000	0.699 (0.657–0.740)	3.3
△LAC1	1.1 (0.6, 2.1)	0.9 (0.5, 1.6)	1.5 (0.8, 3.4)	0.000	0.650 (0.606–0.695)	2.3
△LAC2	0.2 (0.0, 0.9)	0.0 (0.0, 0.6)	0.5 (0.0, 2.2)	0.000	0.630 (0.584–0.676)	1.5

AUROC area under the receiver operating characteristic curve, APACHE II Acute Physiology and Chronic Health Evaluation II, ALB albumin, CI confidence interval, PCT procalcitonin, SOFA Sequential Organ Failure Assessment, LAC lactate, LAC initial lactate on the first day after admission, LAC mean average lactate for 7 consecutive days, LAC max maximum lactate for 7 consecutive days, LAC min minimum lactate for 7 consecutive days, △LAC1 LAC max – LAC min, △LAC2 LAC max – LAC initial, SOFA initial SOFA score on the first day after admission, SOFA mean average SOFA score for 7 consecutive days, SOFA max maximum SOFA score for 7 consecutive days, SOFA min minimum SOFA score for 7 consecutive days, △SOFA1 SOFA max – SOFA min, △SOFA2 SOFA max – SOFA initial. Data are expressed as medians (interquartile range). Comparison of the two groups was conducted using the Mann–Whitney test

Multivariable logistic regression model derivation and development of the SHMS

We used multivariate logistic regression to model the ability of the predictors to identify patients who have a high risk of in-hospital mortality. Binary logistic regression applied to predict mortality in patients with sepsis showed that among the APACHE II score, temperature, △SOFA2, △LAC 2, and SOFA mean score were independent predictors of sepsis mortality (Table 3). The regression model was constructed employing the forward elimination method.

Table 3

Univariate and multivariate analysis of the risk factors for hospital mortality

Variables	Univariate		Multivariate
	OR (95%CI)	P value	OR (95%CI)	P value
APACHE II	1.216 (1.177–1.257)	0.000	1.188 (1.144–1.235)	0.000*
ALB	0.962 (0.938–0.986)	0.002	NI
Temperature	1.139 (1.010–1.285)	0.034	0.847 (0.724–0.991)	0.038*
SOFA initial	1.220 (1.167–1.276)	0.000	NI
SOFA mean	1.312 (1.247–1.381)	0.000	1.089 (1.021–1.162)	0.010*
SOFA max	1.271 (1.215–1.329)	0.000	NI
△SOFA1	1.097 (1.030–1.168)	0.004	NI
△SOFA2	1.271 (1.173–1.377)	0.000	1.193 (1.081–1.316)	0.000*
LAC initial	1.236 (1.135–1.345)	0.000	NI
LAC mean	1.743 (1.492–2.035)	0.000	NI
LAC max	1.285 (1.200–1.375)	0.000	NI
△LAC1	1.267 (1.176–1.364)	0.000	NI
△LAC2	1.423 (1.263–1.604)	0.000	1.156 (1.027–1.301)	0.017*

NI not included in multivariate survival analysis. Risk is presented as hazard ratio and 95% confidence interval (CI) with an increment of 1 unit of the biomarker concentration. The following covariates, deemed important clinical variables, were considered in the multivariable models: APACHE II, temperature, SOFA mean, △SOFA2, △LAC2. *P < 0.05. APACHE II Acute Physiology and Chronic Health Evaluation II, ALB albumin, SOFA Sequential Organ Failure Assessment, LAC lactate, OR odds ratio

Regarding the multivariable analysis, four covariates remained in the final model (Table 4). A scoring system was developed to discriminate between patient survival and nonsurvival upon hospital discharge; this scoring system represented the final “SHMS”. The product term of △SOFA2 and SOFA mean score was 1.161, which suggested that interaction was not an issue. The score was derived as follows: xβ = −4.715 + (0.164 × APACHE II) + (0.171 × △SOFA2) + (0.145 × △LAC2) + (0.082 × SOFA mean). The calibration was also demonstrated to be accurate by the Hosmer–Lemeshow goodness-of-fit test (p = 0.59, 8° of freedom).

Table 4

Multivariate analysis of the risk factors for development of model

Variables	β	OR (95%CI)	P value
APACHE II	0.164	1.178 (1.136–1.223)	0.000*
△SOFA2	0.171	1.186 (1.076–1.307)	0.001*
△LAC2	0.145	1.157 (1.029–1.299)	0.014*
SOFA mean	0.082	1.086 (1.019–1.157)	0.012*
Constant	−4.715	0.378 (−)	0.000*

* P<0.05. APACHE II Acute Physiology and Chronic Health Evaluation II, CI confidence interval, SOFA Sequential Organ Failure Assessment, LAC lactate, OR odds ratio

Trend of the SOFA score as an independent risk factor in two groups

The data of each group did not satisfy the spherical symmetry of the covariance matrix. Greenhouse–Geisser correction showed an interaction between processing factors and time factors (F = 14.557, p < 0.001), indicating a trend in SOFA score changes with time; it can be considered that the SOFA score changes linearly with time. As shown in Table 5, the overall mean D1, D3, and D7 SOFA scores of the patients were different, with p values < 0.001, and the SOFA scores of the survival group were lower than those of the nonsurvival group.

Table 5

The Sequential Organ Failure Assessment (SOFA) scores of survivors and nonsurvivors group in the D1, D3, and D7

Time	(I) Group	(J) Group	Mean difference (I – J)	OR	P value	95% CI
						Lower	Upper
D1	Survivors	Nonsurvivors	−1.871	0.398	0.000	−2.65	−1.09
	Nonsurvivors	Survivors	1.871	0.398	0.000	1.088	2.654
D3	Survivors	Nonsurvivors	−2.338	0.388	0.000	−3.1	−1.58
	Nonsurvivors	Survivors	2.338	0.388	0.000	1.575	3.101
D7	Survivors	Nonsurvivors	−3.523	0.409	0.000	−4.33	−2.72
	Nonsurvivors	Survivors	3.523	0.409	0.000	2.717	4.328

CI confidence interval, OR odds ratio

As seen in Table 6, the overall mean D1, D3, and D7 SOFA scores in the survival group were not equal (p < 0.001), and the nonsurvival group showed the opposite trend. The overall mean values of the D1, D3, and D7 SOFA scores were not statistically significant. The patients in the nonsurvival group showed no significant changes in the severity of disease within 7 days after admission.

Table 6

The Sequential Organ Failure Assessment (SOFA) scores of the D1, D3, and D7 in the survivors and nonsurvivors

Group	(I) Time	(J) Time	Mean difference (I – J)	OR	P value	95% CI
						Lower	Upper
Survivors	D1	D3	0.643	0.148	0.000	0.288	0.999
		D7	1.952	0.244	0.000	1.365	2.539
	D3	D1	−0.643	0.148	0.000	−1.287	−0.288
		D7	1.309	0.195	0.000	0.841	1.777
	D7	D1	−1.952	0.244	0.000	−2.539	−1.365
		D3	−1.309	0.195	0.000	−1.777	−0.841
Nonsurvivors	D1	D3	0.176	0.181	0.993	−0.259	0.612
		D7	0.301	0.299	0.948	−0.419	1.021
	D3	D1	−0.176	0.181	0.993	−0.612	0.259
		D7	0.124	0.239	1.000	−0.449	0.698
	D7	D1	−0.301	0.299	0.948	−1.021	0.419
		D3	−0.124	0.239	1.000	−0.698	0.449

CI confidence interval, OR odds ratio

The SOFA scores were markedly variable during sepsis episodes (Fig. 2). There were statistically significant differences between the downward trends for the predictors and hospital survival (p < 0.001 for D1–D3 trend and p < 0.001 for D3–D7 trend).

Figure 2.

Changes in Sequential Organ Failure Assessment (SOFA) scores on the first, third and seventh day of admission in the nonsurvival group and the survival group. D1 day 1, D2 day 2, D3 day 3

Differences in clinical outcome among three distinct situations of the same △SOFA score

The △SOFA score accounted for three distinct situations. The statistical results of D1–D3 and D1–D7 were the same. Comparison between the three groups showed that there were significant differences between the outcomes of the decreasing group and the increasing group, as well as between the increasing group and stable group, but there was no significant difference between the decreasing group and the stable group (Table 7).

Table 7

Differences in clinical outcome among three distinct situations of the same delta Sequential Organ Failure Assessment (SOFA) score

Same △SOFA	D1–D3		D1–D7
	Survivors	Nonsurvivors	Survivors	Nonsurvivors
Decreasing group	201	58	179	45
Stable group	113	48	44	19
Increasing group	98	84	41	55

D1–D3: compared with decreasing group and stable group, decreasing group and increasing group, stable group and increasing group, p1 = 0.000 × 3, p2 = 0.009 × 3, p3 = 0.001 × 3, respectively; D1–D7:compared with decreasing group and stable group, decreasing group and increasing group, stable group and increasing group, p4 = 0.000 × 3, p5 = 0.089 × 3, p6 = 0.002 × 3, respectively

Predictive performance of the SHMS and cut-off values to discriminate nonsurvivors with sepsis

There was a prognostic value of the new risk factor, SHMS (based on APACHE II, △SOFA2, △LAC2, and SOFA mean), for hospital mortality. The AUROC of the sepsis mortality score and each of its constituent individual predictors of hospital mortality are shown in Table 8.

Table 8

The receiver operating characteristic (ROC)analysis of APACHE II, △SOFA2, △LAC2, and SOFA mean score for prediction prognosis

Variables	AUC	P value	95% CI		Youden	Cut-off	Sensitivity	Specificity	NPV	PPV
			Lower	Upper
APACHE II	0.823	0.000	0.789	0.856	0.489	21.50	0.661	0.828	0.716	0.788
△SOFA2	0.634	0.000	0.589	0.680	0.218	0.50	0.617	0.601	0.503	0.705
△LAC2	0.630	0.000	0.584	0.676	0.233	1.45	0.315	0.918	0.716	0.671
SOFA mean	0.774	0.000	0.737	0.811	0.463	5.83	0.831	0.632	0.597	0.851
SHMS	0.851	0.000	0.821	0.882	0.550	0.33	0.778	0.772	0.692	0.841

AUROC area under the curve, APACHE II Acute Physiology and Chronic Health Evaluation II, SOFA Sequential Organ Failure Assessment, LAC lactate, CI confidence interval, NPV negative predictive value, PPV positive predictive value, SHMS Sepsis Hospital Mortality Score, SHMS = 0.164APACHEII + 0.171△SOFA2 + 0.145△LAC2 + 0.082SOFA mean

The distribution of scores calculated by the predictive formula in the experimental group and the validation group were shown in Fig. 3a, b. The AUC of the APACHE II score, △SOFA2, △LAC2, and SOFA mean for hospital mortality was 0.823, 0.634, 0.630, and 0.774, respectively, and the AUC was 0.851, which was improved when all 4 factors were combined (95% CI (0.821–0.882), standard error 0.033), suggesting very good model discrimination (Fig. 3c). The optimal cut-off value was 0.332 ≈ 0.33 (sensitivity: 78%, specificity: 77%, Youden’s index: 0.55). In all patients, considering a score threshold of ≤ 0.33, the positive predictive value was 84.1%, and the negative predictive value was 69.2%.

Figure 3.

Prognostic value of the new risk factor, Sepsis Hospital Mortality Score (SHMS), for hospital mortality. (a, b) The value of SHMS in the survival and nonsurvival groups; (c, d) comparison of receiver operating characteristic (ROC) analysis of the Acute Physiology and Chronic Health Evaluation II (APACHE II) score, △LAC2, △SOFA2, mean Sequential Organ Failure Assessment (SOFA) score and SHMS for the prediction of hospital mortality of patients with sepsis. LAC lactate

Two subgroups stratified by the severity of illness and their corresponding median survival times

The predicted value fluctuated between 0.001 and 0.996, and the cut-off value was 0.332, as described in Fig. 3a, b. The experimental group was divided into two subgroups according to the cut-off value, those with 0.01 < ŷ ≤ 0.332 and those with 0.332 < ŷ ≤ 0.996. As shown in Fig. 4, the log-rank values of the different cases were counted, and the following conclusions were drawn. Using a score threshold of < 0.322, 56 (16.1%) of the 348 nonsurvivors were correctly identified. In the Kaplan–Meier analysis, the median survival time in the high-risk group was significantly shorter than the low-risk group (11 vs. 15 days, p < 0.001) (Fig. 4a). If we reclassify the severity of the disease into low-, moderate-, and high-risk groups using another demarcation point, the curve indicates that the median survival time in the moderate- and low-risk groups coincided after 18 days (Fig. 4b).

Figure 4.

Kaplan–Meier plot showing survival according to increased sepsis hospital mortality score in the experimental group. (a) Kaplan–Meier plot showing the sepsis-related hospital mortality in the low-risk group: 0.001–0.332 and high-risk group: 0.332–0.996. The median survival times of the two groups were 15 days and 11 days, respectively (log-rank = 0.000). (b) Kaplan–Meier plot showing the sepsis-related hospital mortality in the low-risk group: 0.001–0.332, moderate-risk group: 0.332–0.800, and high-risk group: 0.332–0.996. The median survival times of the three groups was 15 days, 13 days, and 5 days, respectively (log-rank = 0.000)

Consistency of the diagnostic tests

Consistency of the diagnostic tests refers to the degree of consistency between the observations from the diagnostic tests and those from the validation group. The kappa statistic is usually used as an index to evaluate the degree of consistency. The value of the kappa statistic is usually in the range of 0–1. The larger the value, the higher the consistency level. In the validation group, 78 patients were predicted to die, and there were 60 true deaths. The kappa statistic of the deduced formula was 0.614, and the standard error was 0.060, showing good consistency. Furthermore, in the validation group (268 patients), the AUC was 0.899 when all four factors were combined (95% CI 0.801–0.897, standard error 0.025) (Fig. 3d).

Discussion

Early detection and the use of sepsis care bundles are associated with reduced mortality [8]. In this study, we assembled a cohort of 626 patients with sepsis and studied the effect of different predictors on their mortality with the overall goal of creating a prediction equation: the “SHMS”. The statistical results show that the APACHE II score, △SOFA2, △LAC2, and SOFA mean score were independently associated with hospital mortality in sepsis patients admitted to the ICU, and the new formula showed very good results in predicting hospital mortality (AUROC 0.851).

In our study, the SOFA scores were recorded for 7 days after admission. We wanted to explore an improved prediction method by calculating the diagnostic efficiency of the changes in the SOFA score, SOFA mean, SOFA initial, and SOFA max. Research [17] showed that the median SOFA value of in-hospital surviving patients (SOFA = 7) had a sensitivity of 95.3% and a specificity of 49.4% for hospital mortality. Our results are consistent with previous studies [8,18,19], which have shown that the mean SOFA score provides the best discrimination to predict hospital mortality (cut-off = 5.8, AUROC = 0.774, p = 0.000) compared with other scoring tools. We consider that SOFA means can reflect the severity of a patient’s condition for a period of time; it is more representative than the score at any time and more reflective of the patient’s response to treatment.

According to a study by Ferreira et al. [20], independent of the initial score, an increase in the SOFA score during the first 48 hours in the ICU predicts a mortality rate of at least 50% and differences in mortality were better predicted in the first 48 hours than in the subsequent 48 hours, which differ from our finding that the statistical results of D1–D3 and D1–D7 were the same. In our study, the patients in the nonsurvival group showed no significant changes in the severity of disease within 7 days of admission. Furthermore, the overall mean D1, D3, and D7 SOFA scores of the patients were different, and the SOFA scores of the surviving patients were lower than those in the nonsurvival group.

We believe that the △SOFA score accounts for three distinct situations: increasing scores, decreasing scores, and stable scores. Comparison between the three score groups showed that there were significant differences in the outcomes of the increasing group and the decreasing group and between the decreasing and stable group, but there was no significant difference between the increasing group and the stable group. The AUROC curve of the △SOFA score was less than 0.7. We consider the six systems in which SOFA scores are included, and the scores of the six systems varied from one system to another within 7 days, with the total score unchanged. Therefore, △SOFA should be combined with disease severity to better predict the outcomes of patients.

Aublanc and Richard [21] stressed that the basic requirement for a sepsis diagnosis is organ dysfunction, which may delay the early identification and treatment of sepsis, thus affecting the prognosis of patients. A combination of predictors may present a more reliable and objective guide for mortality prediction in sepsis patients. Recent studies also demonstrated that two prediction models that combined predictors performed better than routinely used clinical scores in predicting sepsis-related mortality [22, 23].

In clinical practice, many of the family members of the patient will consult the doctor for answers; questions usually do not involve the patient’s overall outcome, but revolve around how long the patient will live. Similar to the different median survival times for different types of tumors or tumor patients at different stages, our goal is to develop a prediction formula to classify sepsis patients according to different risk levels and then calculate the corresponding median survival time. This formula could provide theoretical support for the clinician to allocate limited medical resources and answer the questions of the patient’s family. For example, palliative therapy can be used for patients with a short median life expectancy.

In the validation group, the AUROC curve of the joint prediction index was 0.899, and the kappa statistic was 0.614, indicating good validation of the clinical utility of sepsis in the prediction of mortality in sepsis. The latest standard of sepsis is also applicable to the Chinese patient population to some extent and has a moderate effect in predicting patient outcomes. Our study had several limitations. Patient outcomes depended on patient management, which may have varied among different institutions. Although we attempted to control the confounding of other clinical variables by establishing a SHMS model with a logistic regression model, we were not to account for other unmeasured confounding or collinear effects. In addition, we did not record the lowest daily LAC value, which may explain the limited predictive power of LAC, and we did not obtain information on ICU-acquired sepsis. In addition, patients were followed only during hospitalization. We stratified the risk severity of the patient’s condition only into low-risk and high-risk. When risk stratification was performed with low-, moderate-, and high-risk groups, there was no significant difference in the corresponding median survival time between the low-risk group and the moderate-risk group. Therefore, we need more deliberate prospective cohort studies to verify the clinical significance of this formula.

Conclusions

The APACHE II score, △SOFA, △LAC, and SOFA mean score were independently associated with hospital mortality in sepsis patients and accurately predicted the hospital mortality rate and median survival time. Data on the median survival time in sepsis patients could be provided to clinicians to assist in the rational use of limited medical resources by facilitating prudent resource allocation.

Funding

This work was supported by the National Science and Technology Support Program (2012BAI11B05).

Availability of data and materials

The data sets are available from the corresponding author on request with justification.

Authors’ contributions

WL was involved in the acquisition of data, data processing, study design, statistical analysis and manuscript writing and drafting; MPW was involved in data processing and performed the statistical analysis; BZ was involved in the study design and data processing; YBZ and XMX were involved in the study design and the final revision of the manuscript; and all authors read and approved the final manuscript.

Ethics approval and consent to participate

The protocol used in this study was approved by the local ethics committee (2013FXHEC-KY018).

Consent for publication

Not applicable.

Conflicts of interest

On behalf of all authors, the corresponding author states that there are no conflicts of interest.