The Added Value of Lactate and Lactate Clearance in Prediction of In-Hospital Mortality in Critically Ill Patients With Sepsis.

We investigated the added predictive value of lactate and lactate clearance to the Acute Physiology and Chronic Health Evaluation IV model for predicting in-hospital mortality in critically ill patients with sepsis.
DESIGN: Retrospective observational cohort study.
SETTING: Mixed ICU of Leiden University Medical Center, The Netherlands. PATIENTS: Critically ill patients adult patients with sepsis who have been admitted to the ICU of Leiden University Medical Center, The Netherlands, from 2006 to January 2018.
INTERVENTIONS: None.
MEASUREMENTS AND MAIN RESULTS: We fitted a baseline model with the Acute Physiology and Chronic Health Evaluation IV predictors and added 13 prespecified combinations of lactate and lactate clearance at 0, 6 and 24 hours after admission to create a set of extended models to compare with the baseline Acute Physiology and Chronic Health Evaluation IV model. Among 603 ICU admissions, 451 patients met the inclusion criteria. A total of 160 patients died in-hospital, of which 106 died in the ICU. Their lactate and lactate clearance measurements were higher at all time points than those of survivors. The Akaike Information Criterion score improved in 10 of 13 prespecified extended models, with best performance for models that included lactate at 24 hours, alone or in combination with lactate at admission or lactate clearance at 24 hours. We compared the observed and predicted probabilities of in-hospital mortality of the baseline Acute Physiology and Chronic Health Evaluation IV model with the best model in our data, lactate at 24 hours added to the Acute Physiology and Chronic Health Evaluation IV model. This resulted in an increase in specificity of 29.9% (95% CI, 18.9-40.9%).
CONCLUSIONS: Lactate measurements at 24 hours after admission add predictive value to the prediction of mortality with Acute Physiology and Chronic Health Evaluation IV among ICU patients with sepsis. External validation is needed to develop extended prediction models.

Sepsis is a life-threatening organ dysfunction caused by a dysregulated host response to infection (1). It is one of the leading causes of mortality and prolonged disability among critically ill patients (2, 3). Lactate and lactate clearance (LC) are cornerstones in the management of critically ill patients with sepsis (4), after multiple studies showing the association between mortality and elevated lactate levels (>4 mmol/L) or LC at 6 or 24 hours (5–9). Early lactate-guided resuscitation in critically ill patients with sepsis showed evidence of mortality reduction in different randomized control studies (10, 11), which was confirmed in later meta-analyses (12, 13).

However, lactate and LC are not implemented yet in prediction models in critically ill patients. One of the most widely used prediction models in the ICU is the Acute Physiology and Chronic Health Evaluation (APACHE) for predicting in-hospital mortality (14, 15). The APACHE IV is the latest version, and it was based on 142 variables collected in the first 24 hours after admission to the ICU, of which none included lactate or LC (14).

Our aim was therefore to exploratively investigate the added value of lactate and LC at 0, 6, and 24 hours after ICU admission to the original APACHE IV model for mortality predictions in critically ill patients with sepsis.

MATERIALS AND METHODS

Study Design

We used a retrospective observational cohort of patients admitted to the mixed ICU of Leiden University Medical Center (LUMC) with sepsis between January 2006 and January 2018. Adult patients with sepsis admitted to the ICU were identified by their APACHE IV admission diagnosis. Patients under 18 years old, patients without any lactate measurement during their ICU admission, and patients admitted for less than 24 hours in the ICU were excluded since the APACHE IV predicts from 24 hours after ICU admission. Only patients’ first ICU admission in the same hospital admission was analyzed. Patient discharged to another ICU or admitted from another ICU were also excluded from the analysis (14). A waiver for informed consents was granted by the Medical Ethics Committee of LUMC (reference G17.094). The Transparent Reporting of a multivariable prediction model for individual Prognosis Or Diagnosis checklist for prediction model development was used for the reporting of this study (16, 17).

Data Collection

Demographic, physiologic, diagnostic, and outcome data of each admitted ICU patient are collected and registered in the electronic medical records of LUMC. All patients with APACHE IV admission diagnosis “sepsis” were extracted from these records. The data included age, sex, chronic health conditions, weight, length, source of sepsis, admission source, length of stay in days (ICU and hospital), mortality (ICU and in-hospital), APACHE II and IV score, and all arterial lactate measurements during the first 3 days of admission. Physiologic data were also extracted and consisted of pulse rate, mean arterial blood pressure (MAP), temperature, respiratory rate, Pao2, Fio2, hematocrit, WBC count, serum creatinine, urine output, blood urea nitrogen, sodium, albumin, bilirubin, glucose, arterial carbon dioxide tension, pH, and the total Glasgow Coma Scale score. The need for mechanical ventilation and renal replacement therapy on the day of admission and the next day was also extracted. Missing data were handled using multiple imputation. More details regarding the handling of missing can be found in the supplemental material (Supplemental Digital Content 1, http://links.lww.com/CCX/A143), as well as the sample size calculation.

Outcome and Predictors

The primary outcome was in-hospital mortality, which was defined as mortality in the ICU or in another ward during the ongoing hospital admission. The same predictors as in the original APACHE IV model were used in this study. Continuous predictors were age, pre-ICU length of stay, Glasgow Coma Scale, and the Acute Physiology Score (APS). Categorical predictors were admission type, chronic comorbidities, and source of sepsis. The pre-ICU length of stay was calculated as the difference in days between ICU admission time and hospital admission time. For the Glasgow Coma Scale, we took the worst measurement during ICU day 1. The APS was calculated as the sum of weights of the worst values during ICU day 1 for pulse rate, MAP, temperature, respiratory rate, Pao2 conditional on mechanical ventilation and Fio2, hematocrit, WBC count, serum creatinine conditional on urine output and kidney functionality, urine output, blood urea nitrogen, sodium, albumin, bilirubin, glucose, and acid-base abnormalities.

The following predictors were added to the original APACHE IV model in various compositions (Supplemental Table 1, Supplemental Digital Content 1, http://links.lww.com/CCX/A143): absolute lactate values (in mmol/L) at ICU admission (baseline), 6 and 24 hours after ICU admission, and LC in % at 6 (9) and 24 (18) hours after admission to the ICU, for which we used formula 1:

A positive value denoted a decrease in lactate over time, whereas a negative value denoted an increase in lactate. Baseline lactate was defined as the first arterial lactate value within 2 hours before or after admission to the ICU.

Statistical Analysis

Descriptive statistics were used to describe the characteristics of the study population. We fitted a logistic regression model with the APACHE IV set of predictors to predict in-hospital mortality. As in the original model, we allowed age, pre-ICU length of stay, and APS to have a nonlinear relationship with the outcome using restricted cubic splines with respectively five, four, and five knots. Next, to the baseline model, we added 13 prespecified combinations of lactate and LC at 0, 6, and 24 hours to create a set of extended models to compare with the baseline APACHE IV model (Supplemental Table 1, Supplemental Digital Content 1, http://links.lww.com/CCX/A143).

The Akaike Information Criterion (AIC) was used to compare the models to the baseline model and selecting the best models, while taking the number of variables in each model into account, as depicted in formula 2 (19, 20):

The AIC estimates the relative information loss by each given model. Therefore, a smaller AIC indicated better fit (19). The difference AICbaseline – AICnew model (ΔAIC) was used to assess the improvement from the baseline APACHE IV model. A larger ΔAIC denotes a better predictive model. We selected the three best predicting models, according to their ΔAIC for further analysis.

Internal validation was performed for each model using the bootstrap procedure with 200 bootstrapped samples (21). The procedure was repeated in each imputed dataset, and the average estimates for the C-statistic and the Nagelkerke R2 (R2n) were extracted to assess discrimination and overall fit respectively (22). To have more insight into the discrimination benefit, we also calculated net reclassification improvement (NRI) for the best fitted model selected by the highest ΔAIC. The NRI allowed us to assess how patients were classified differently according to the predicted probabilities of the two models. In line with recent recommendations (23), we reported the NRI for events (in-hospital mortality) and the NRI for non-events separately, thereby quantifying the effect of an added predictor as it depends more on the effect of the predictor rather than on the strength of the baseline APACHE IV model (24). The NRI for events can be interpreted as the change in sensitivity, while the NRI for non-events can be interpreted as the change in specificity (25). The fraction of our cohort being correctly reclassified with the best fitted model will be calculated using formula 3:

The odds ratios (ORs) of the added lactate and LC predictors were estimated in the three best performing models and adjusted for other covariates in each model. To improve comparability of the continuous predictors, effects of the predictors at the 75th percentile were compared with the effects at the 25th percentile. The OR therefore represented the odds of in-hospital mortality for patients at the 75th percentile of the distribution of the predictor versus patients at the 25th percentile in each model (26, 27).

All analyses were performed in R (R foundation for Statistical Computing, Vienna, Austria) (28).

RESULTS

Characteristics of the Study Population

A total of 603 septic patients were admitted to the ICU of LUMC. As shown in Supplemental Figure 1 (Supplemental Digital Content 1, http://links.lww.com/CCX/A143), 451 patients were eventually included in the analysis after assessment of eligibility, in which 152 patients were excluded (19 due to no available lactate measurements, 127 due to admittance <24 hr, and 6 due to readmittance). Patient characteristics are summarized in Table . The median age was 66 years (interquartile range [IQR], 56–74 yr), and most patients were male (64.5%). The most common source of sepsis was gastrointestinal (n = 94; 20.8%). The median APACHE IV score was 87 (IQR, 67–109, missing = 15). Median length of ICU stay was 3.4 days (IQR, 1.9–8.9 d). Eventually, 160 patients (35.5%) died during their hospital admission, of which 106 (66.3%) died in the ICU and 54 (33.8%) in-hospital but outside ICU (Table 1).

TABLE 1.

Characteristics of Patients With Sepsis Admitted to the ICU at Leiden University Medical Center (2006–2018) of the Overall Cohort and Stratified by In-Hospital Mortality

Lactate and LC

Lactate at admission and LC values were all worse for patients who died compared with patients who did not die (Table ). Lactate values for patients who died were 2.6 mmol/L (IQR, 1.7–4.7 mmol/L) at admission, 2.5 mmol/L (IQR, 1.7–5.2 mmol/L) after 6 hours, 2.5 mmol/L (IQR, 1.7–4.2 mmol/L) after 24 hours. Whereas, lactate values for patients who survived were 2.2 mmol/L (IQR, 1.5–3.6 mmol/L) at admission, 2.0 mmol/L (IQR, 1.4–3.0 mmol/L) after 6 hours, and 1.6 mmol/L (IQR, 1.2–2.2 mmol/L) after 24 hours. LCs were –1% (IQR, –25% to 21%) and 11% (IQR, –11% to 31%) after 6 hours and 8% (IQR, –35% to 35%) and 22% (IQR, –7% to 53%) after 24 hours, in nonsurvivors and survivors, respectively.

Internal Validation

Model performances after internal validation showed optimism in every model with a range from 0.12 to 0.14 for R2n and of 0.05 for the C-statistic. The baseline APACHE IV model had an original R2n of 0.40 and a corrected R2n of 0.28, whereas the model with the addition of lactate at 24 hours had a R2n of 0.42 and 0.30, respectively, reflecting a slight improvement in goodness-of-fit. The C-statistic for the baseline APACHE IV was 0.83 and increased to 0.84 with the addition of lactate at 24 hours, showing a small increase in model discrimination. After correction for optimism, the C-statistics were 0.78 and 0.79, respectively, for the baseline APACHE IV and the model with lactate at 24 hours (Supplemental Table 3, Supplemental Digital Content 1, http://links.lww.com/CCX/A143).

Comparison of the Prediction Models

A better fit than the baseline APACHE IV model was reported by the ΔAIC in 10 of 13 models (Fig. ). The three largest ΔAIC were seen in the models with added lactate at 24 hours (ΔAIC= 7.7), with added lactate at 24 hours in combination with baseline lactate (ΔAIC= 7.5), and with added lactate at 24 hours in combination with LC at 24 hours (ΔAIC= 7.3). Better predictive models were seen with addition of lactate at 24 hours to the baseline model alone or in combination with lactate and LCs at different time points (Supplemental Table 4, Supplemental Digital Content 1, http://links.lww.com/CCX/A143).

Absolute improvement of the Akaike Information Criterion (AIC) between each model and the baseline model.

Adjusted OR of 1.43 (1.13–1.82) was reported for the model with only added lactate at 24 hours. For the model with lactate at baseline and at 24 hours added to the APACHE IV model, adjusted OR of 0.86 (0.67–1.11) and 1.52 (1.17–1.97) were found respectively. For the model with lactate and LC at 24 hours added to the APACHE IV model, adjusted OR of 1.36 (1.06–1.75) and 0.87 (0.70–1.08) were found respectively (Table ).

NRI

We compared the observed and predicted probabilities of in-hospital mortality of the baseline APACHE IV model with the best model in our data, where we added lactate at 24 hours to the APACHE IV model (Fig. ). Among the patients with in-hospital mortality (n = 160), the proportion of patients who were correctly reclassified was 46.3%, whereas 53.7% were incorrectly reclassified. This resulted in an event NRI of 7.5% (95% CI, –8.0% to 23.0%), reflecting a small increase in sensitivity. Among the patients without in-hospital mortality, the percentage of patients who were correctly reclassified was 64.9%, whereas 35.1% were incorrectly reclassified. The nonevent NRI was therefore 29.9% (95% CI, 18.9%–40.9%), reflecting a marked improvement in specificity. The fraction of our cohort of patients that was correctly reclassified could then be calculated using formula 3 as 22%.

Reclassification plots for the continuous predictor lactate at 24 hr. Box A shows the net reclassification for the events, whereas box B shows the net reclassification for nonevents.

DISCUSSION

Lactate is a complex surrogate marker for microcirculation (4), which is not only dependent on its production, but also on its metabolism and clearance, which are all impaired by sepsis (8). Nevertheless, it has been associated with mortality, regardless of the complex mechanism of elevation. An overall improvement in predictive performance was reported by the ΔAIC for most combinations of lactate and LC at different time points. Lactate at 24 hours had the highest added predictive value to predict in-hospital mortality when added to the baseline APACHE IV model alone or in combination with lactate at admission and LC at 24 hours. The strength of the association of lactate with in-hospital mortality was also seen in the adjusted OR.

Interpretation

The increase in lactate values corresponded with an increase in the probability of dying during the ongoing hospital admission. Previous studies (9, 18, 29) put more emphasis on lactate and LC at certain time points. Our study suggests that, in line with findings reported in recent articles (5, 18), a strong association is seen with in-hospital mortality whenever 24 hour lactate alone or in combination with LC at 24 hours is used. This might be explained because measurements at 24 hours are closer to the predicted outcome, hence more informative. On the other side, being the result of intrinsic severity of illness of the patient and the therapy initiated, 24 hours might be the best point to evaluate the patient and have a relative trustworthy prediction of the further course.

Improvements for C-statistics and Nagelkerke R2 were seen in most of the models. The C-statistic should be interpreted as an equivalent of the area under the curve since we fitted a model with a binary outcome (30). The small improvement in the C-statistic was expected since the baseline model, and the new models were developed and tested in the same dataset (31). The NRI for events and nonevents provided more insight into the discriminative power of the model with addition of 24 hour lactate, which suggested that the added predictor was especially beneficial for the specificity of the model, hence for prediction of patients who eventually will survive.

The adjusted OR for lactate at different time points confirmed the strong association of lactate with mortality and suggested that for prediction of mortality after 24 hours, it is more important to know the trend of lactate over time rather than the absolute value at admission. The adjusted OR of lactate at 24 hours was greater than 1 in each model, reflecting the association with mortality. However, the adjusted OR of lactate at admission needs careful interpretation, since the protective effect of high lactate at admission on mortality is contradictory from a biological point of view. An explanation might be that patients with high lactate level at admission to the ICU are more able to large changes in lactate level in the first 24 hours after admission, than patients with a low lactate level at admission. This underlines the importance of lactate trend over the first 24 hours rather than only having the absolute lactate value at admission. Furthermore, the optimistic OR could be the result of survival bias due to the exclusion of patients who died within 24 hours after admission to the ICU. The adjusted OR for LC at 24 hours was also less than 1, suggesting a protective effect of positive LC toward mortality.

Limitations

Our relatively small sample of 451 patients and 160 events did not allow to compensate for all the degrees of freedom spent to develop the models due to the complexity of the original APACHE IV model. As a consequence, optimism was seen in our models. Due to the observational nature of our study, measurement errors and missing data were expected to some extent (32). We handled missing data using multiple imputation which might have introduced more uncertainty in our estimates. Furthermore, the data were collected over a large time window (12 yr) in which definition and management of sepsis and septic shock has changed (1, 4), which was not possible to take into account in the analysis. We plan to perform an external validation study to gain more insight in the optimism and generalizability of our results.

Implications

Many ICUs worldwide rely on the APACHE IV predictions of in-hospital mortality and for benchmarking using case mix adjustments, which is an essential prerequisite for meaningful comparisons of different ICUs. Although the predictive performance of the original APACHE IV model was already solid, we showed that the APACHE IV might be further improved with the addition of lactate and or LC at 24 hours. Measurement time at 24 hours is proposed as most informative. However, these findings need to be confirmed in an external validation study, which will also assess the generalizability of the results to all critically ill patients with sepsis. The clinical application of lactate and LC after 24 hours is difficult, since the sepsis management has already past the golden hours. The effect of lactate and LC guided resuscitation on length of stay and mortality has been a topic of interest with various results (10, 33, 34). It is however advised in the Surviving Sepsis guideline for management of sepsis, but more research is recommended (4).

CONCLUSIONS

Lactate added predictive value to the APACHE IV model in prediction of in-hospital mortality in critically ill patients with sepsis. Lactate at 24 hours and LC at 24 hours stood out as new possible predictors. This study might provide an input to develop a new reliable model for sepsis patients, after external validation.

ACKNOWLEDGMENTS

We thank the business intelligence unit of Leiden University Medical Center and Aad Pors with the data management of this study. Furthermore, we would like to thank the ICU department of Leiden University Medical Center for their help. We acknowledge the helpful comments from the Center for Clinical Transfusion Research Integrity Program’s Scientific committee on the statistical analysis plan and final article.

TABLE 2.

Lactate and Lactate Clearance Measurements at Different Time Points for the Overall Cohort and Stratified by In-Hospital Mortality

TABLE 3.

Adjusted Odds Ratios for Lactate Measurements in the Three Most Improved Models