Utility of a prediction model for delirium in intensive care unit patients (PRE-DELIRIC) in mechanically ventilated patients with sepsis.

AIM: Delirium frequently develops in patients with sepsis during their intensive care unit (ICU) stay, which is associated with increased morbidity and mortality. A prediction model for delirium in patients in ICU, PRE-DELIRIC, has been utilized in overall ICU patients, but its utility is uncertain among patients with sepsis. This study aims to examine the utility of PRE-DELIRIC to predict delirium in mechanically ventilated patients with sepsis.
METHODS: This is a post hoc analysis of a randomized clinical trial in eight Japanese ICUs, which aimed to evaluate the sedative strategy with/without dexmedetomidine in adult mechanically ventilated patients with sepsis. The Confusion Assessment Method for the ICU was used every day to assess for delirium throughout their ICU stay. We excluded patients who were delirious on the first day of ICU, those who were under sustained coma throughout their ICU stay, and those who stayed in the ICU less than 24 h. The discriminative ability of PRE-DELIRIC was evaluated by measuring the area under the receiver operating characteristic curve (AUROC).
RESULTS: Of the 201 patients enrolled in the trial, we analyzed 158 patients. The mean age was 69.4 ± 14.0 years, and 99 patients (63%) were men. Delirium occurred at least once during the ICU stay of 63 patients (40%). The AUROC of PRE-DELIRIC was 0.60 (95% confidence interval, 0.50-0.69). Subgroup analyses indicated that PRE-DELIRIC was useful in those with Sequential Organ Failure Assessment score >8 with AUROC of 0.65 (95% confidence interval, 0.51-0.77).
CONCLUSIONS: The PRE-DELIRIC model could not predict delirium in mechanically ventilated patients with sepsis.

Introduction

Delirium is acute disturbance in attention, awareness, and cognition, which tends to fluctuate during the course of the day. It is known as an independent risk factor of worse outcomes, affecting length of hospital stay, mortality, and long‐term cognitive dysfunction. , Effective preventive strategies for delirium could be facilitated by accurate prediction of delirium in intensive care units (ICU).

As a delirium prediction model for adult general ICU patients, PRE‐DELIRIC was developed, which included 10 variables that could be obtained within 24 h after ICU admission. Since its development, PRE‐DELIRIC has been recalibrated and validated in multicenter observational studies, which reported good calibration and discrimination. ,

The PRE‐DELIRIC model has been validated in overall ICU populations, but it has not yet been validated in subpopulations, such as emergently admitted patients or those with sepsis. For implementation of the prediction model in clinical settings, the applicability of the model in each clinical context should be understood. We undertook this study to validate PRE‐DELIRIC in the specific ICU population of mechanically ventilated patients with sepsis.

Methods

This study is a post hoc analysis of the Dexmedetomidine for Sepsis in Intensive Care Unit Randomized Evaluation (DESIRE) trial. The DESIRE trial was a multicenter randomized controlled trial undertaken in eight ICUs in Japan, which enrolled 201 mechanically ventilated adult patients with sepsis. It compared the sedation strategy with and without dexmedetomidine (dexmedetomidine group and control group). The sedatives were titrated towards the target of Richmond Agitation Sedation Scale (RASS) score 0 (calm, daytime) and −2 (lightly sedated, night‐time). The patients in the dexmedetomidine group received dexmedetomidine during mechanical ventilation, and other sedatives, such as propofol and midazolam, were added as needed. Patients in the control group received sedatives other than dexmedetomidine towards the same target of RASS. Fentanyl was used to achieve the Behavioral Pain Scale goal of <5 in both groups. The detailed methods and results of the DESIRE trial are reported elsewhere. The ethical review boards of all relevant institutions approved the study protocol, and all participants provided written informed consent prior to enrollment.

In this subanalysis, we included patients with length of ICU stay ≥24 h. We excluded patients with length of ICU stay <24 h, those who were delirious on the first day of ICU admission, and those with sustained coma throughout the ICU stay.

All patients were screened for delirium at least once daily by nurses using the Confusion Assessment Method for the Intensive Care Unit (CAM‐ICU). Identical to the study that recalibrated the PRE‐DELIRIC, delirium was defined as at least one positive variable in CAM‐ICU during the patient’s ICU stay.

For the calculation of the PRE‐DELIRIC, we collected 10 variables within the first 24 h after ICU admission. These variables comprised age, Acute Physiology and Chronic Health Evaluation II (APACHE II) score, presence of coma, admission route, presence of infection, presence of metabolic acidosis, morphine dose on the first day, sedative usage on the first day, blood urea nitrogen, and incidence of urgent admission. We calculated APACHE II score within 24 h after ICU admission. Coma was defined as RASS score −4 or lower throughout the first day of ICU admission. If the patients in a coma were given sedatives on the first day, they were classified as “drug‐induced coma” patients. If the patients in a coma had central nervous system Sequential Organ Failure Assessment (SOFA) subscore ≥3, they were classified as “miscellaneous coma” patients. If the patients in a coma had both factors, they were classified as “mixed coma” patients. Metabolic acidosis was defined as pH < 7.35 and HCO3 ‐ < 24 mmol/L by the arterial blood gas analysis on the first day. We used i.v. fentanyl as analgesic and we converted fentanyl dose to morphine dose in the following ratio: i.v. morphine 33 mg = i.v. fentanyl 1 mg. Sedative usage was defined as using propofol or midazolam on the first day. All patients in this cohort intrinsically fulfilled two criteria of presence of infection and urgent admission.

Statistical analysis

We expressed continuous variables as means ± standard deviations or medians and interquartile ranges. We expressed categorical variables as numbers and percentages. We compared delirious patients with non‐delirious patients using Fisher’s exact test for categorical variables and the t‐test or Wilcoxon rank sum test for continuous variables. We used the area under the receiver operating characteristic curve (AUROC) to assess the discrimination of PRE‐DELIRIC for predicting delirium. For exploratory purposes, we also assessed the AUROC in subgroups: SOFA score (>8 [median] and ≤8), and presence or absence of each organ failure on the first day. Each organ failure was defined as ≥3 SOFA subscore. We evaluated the calibration graphically by plotting the observed incidence of delirium in each ten‐percentile group of expected incidence from PRE‐DELIRIC. All analyses were undertaken using JMP Pro software version 12.2 (SAS Institute, Cary, NC, USA).

Results

From a total of 201 patients in the DESIRE study, we enrolled 158 patients in this subanalysis after excluding 43 patients (Fig. 1). The median PRE‐DELIRIC score was 0.41, and delirium was observed in 63 patients (40%) during their ICU stay (Table 1). Delirious patients had lower incidence of abdominal infection, were less immunocompromised, had higher incidences of mixed coma on the first day, and longer lengths of ICU stay. The PRE‐DELIRIC score distribution is shown in Figure S1.

Fig. 1

Patient flowchart. DESIRE, Dexmedetomidine for Sepsis in Intensive Care Unit Randomized Evaluation; ICU, intensive care unit.

Table 1

Characteristics and outcomes of mechanically ventilated patients with sepsis in the intensive care unit (ICU)

Characteristic	Analyzed patients (n = 158)	Delirious patients (n = 63)	Non‐delirious patients (n = 95)	P‐value
Age, years; mean ± SD	69.4 ± 14.0	68.9 ± 13.8	69.7 ± 14.7	0.7100
Male sex, n (%)	99 (63)	40 (63)	59 (62)	1.0000
Body weight, kg; mean ± SD	57.0 ± 13.6	56.6 ± 16.0	57.2 ± 11.7	0.8000
APACHE II score, median (IQR)	22 (16–28)	23 (17–29)	22 (16–27)	0.3500
SOFA score, median (IQR)	8 (6–11)	8 (5–11)	8 (6–11)	0.5500
Emergency surgery, n (%)	66 (42)	21 (33)	45 (47)	0.1000
Site of infection
Abdomen, n (%)	60 (38)	15 (24)	45 (47)	0.0007
Thorax, n (%)	53 (34)	20 (32)	33 (35)
Urinary tract, n (%)	13 (8)	5 (8)	8 (8)
Skin and soft tissue, n (%)	12 (8)	7 (11)	5 (5)
Others, n (%)	20 (13)	16 (25)	4 (4)
Lactate level, mmol/L; median (IQR) †	3.5 (1.8–5.1)	3.1 (1.7–4.6)	3.7 (1.8–5.5)	0.3600
Comorbidity
Immunocompromised, n (%) ‡	25 (16)	4 (6)	21 (22)	0.0080
Chronic hemodialysis, n (%)	12 (8)	3 (5)	9 (9)	0.3600
Chronic respiratory disorder, n (%)	7 (4)	2 (3)	5 (5)	0.4200
Chronic heart failure, n (%)	4 (3)	0 (0)	4 (4)	0.1500
Liver insufficiency, n (%)	1 (1)	0 (0)	1 (1)	1.0000
BUN, mg/dL; median (IQR)	32 (22–54)	37 (24–52)	31 (21–55)	0.3000
Dosage of fentanyl on the first day, μg/day; median (IQR)	520 (269–761)	494 (265–864)	520 (270–757)	0.9000
Coma on first day §
Drug‐induced, n (%)	17 (11)	6 (10)	11 (12)	0.8000
Miscellaneous, n (%)	10 (6)	6 (10)	4 (4)	0.2000
Mixed, n (%)	14 (9)	10 (16)	4 (4)	0.0200
Propofol or midazolam give on first day, n (%)	99 (63)	38 (60)	61 (64)	0.7400
Metabolic acidosis, n (%) ¶	55 (35)	26 (41)	29 (31)	0.1800
PRE‐DELIRIC score, median (IQR)	0.41 (0.30–0.50)	0.46 (0.33–0.57)	0.38 (0.29–0.47)	0.0400
Length of ICU stay, days; median (IQR)	8 (5–13)	11 (7–15)	6 (4–11)	0.0004
28‐day mortality, n (%)	25 (16)	11 (17)	14 (15)	0.6600

APACHE II, Acute Physiology and Chronic Health Evaluation II; BUN, blood urea nitrogen; Drug‐induced coma, patients with coma given sedatives on the first day; IQR, interquartile range; Miscellaneous coma, patients with coma and central nervous system SOFA subscore ≥ 3; Mixed coma, patients with coma given sedatives on the first day and central nervous system SOFA subscore ≥ 3; PRE‐DELIRIC, prediction model for delirium in ICU patients; SD, standard deviation; SOFA, Sequential Organ Failure Assessment.

Lactate level was missing for three patients.

Defined as patients with immunosuppression from chemotherapy, radiation therapy, long‐term or recent high‐dose steroids, or immunodeficiency (e.g. leukemia, lymphoma, or AIDS).

Defined as ≤ −4 Richmond Agitation Sedation Scale score throughout the first day of ICU admission.

Defined as pH < 7.35 and HCO3 ‐ < 24 mmol/L by arterial blood gas analysis on the first day.

The AUROC of PRE‐DELIRIC was 0.60 (95% confidence interval [CI], 0.50–0.69) for predicting delirium (Fig. 2). If we use the cut‐off of the PRE‐DELIRIC score as 0.44, the sensitivity and specificity are 0.57 and 0.68, respectively. In the subgroup analysis, the subgroups with more severe organ dysfunction (SOFA score >8) showed fair discrimination of PRE‐DELIRIC (AUROC 0.65; 95% CI, 0.51–0.77). The subgroup with circulatory failure also showed fair discrimination (AUROC 0.63; 95% CI, 0.51–0.74) (Fig. 3). The calibration plot of expected and observed incidence of delirium could not show good calibration of PRE‐DELIRIC (Fig. S2).

Fig. 2

Receiver operating characteristic curve of a prediction model for delirium in intensive care unit patients (PRE‐DELIRIC) in mechanically ventilated patients with sepsis. The area under the receiver operating characteristic curve of PRE‐DELIRIC for predicting delirium is 0.60 (95% confidence interval, 0.50–0.69).

Fig. 3

Area under the receiver operating characteristic curve (AUROC) of a prediction model for delirium in intensive care unit patients (PRE‐DELIRIC) in each subgroup of mechanically ventilated patients with sepsis. Organ failure is defined as Sequential Organ Failure Assessment (SOFA) subscore ≥3. The numbers of patients with liver failure (n = 1) and coagulation failure (n = 13) were too small to undertake subgroup analyses. Data are shown as AUROC (95% confidence interval). CNS, central nervous system.

Discussion

In this study, the prevalence of delirium during ICU stay was approximately 40% in mechanically ventilated patients with sepsis. Discrimination of PRE‐DELIRIC in this cohort was suboptimal (AUROC 0.60). This study did not show good calibration of PRE‐DELIRIC in patients under mechanical ventilation.

Previous studies reported higher discrimination (AUROC 0.74–0.84) than in our study. , , However, they included an overall ICU population, and many of them were planned ICU admission of postoperative patients. Approximately half of the patients in the original development study were actually non‐urgently admitted to ICU. Furthermore, a recent prospective observational study from Hong Kong that included an ICU subpopulation admitted after cardiac surgery also reported better discrimination of PRE‐DELIRIC (AUROC 0.75). Compared with our study, most of the study cohort (over 80%) were planned postoperative patients. The discrepancy of the results between our study and previous studies could derive from differences between urgent and non‐urgent ICU admissions.

Another possible explanation is that the discrepancy derives from the specific ICU population of patients with sepsis. All patients in our study intrinsically fulfilled the important criterion of presence of infection, which might hinder the predictive ability of PRE‐DELIRIC. Sepsis‐associated delirium, also known as sepsis‐associated encephalopathy, could have distinct characteristics from other causes. Both impaired brain perfusion and excess inflammatory response could play an important role in the pathogenesis of sepsis‐associated encephalopathy. Clinical study also shows that circulatory index, such as heart rate and serum lactate level, are associated with the presence of sepsis‐associated encephalopathy. The PRE‐DELIRIC model does not include variables regarding circulatory and inflammatory parameters, which could lead to poor predictive ability of PRE‐DELIRIC in patients with sepsis. In line with this, our study shows fair discrimination of PRE‐DELIRIC after stratification by the presence of circulatory failure. The PRE‐DELIRIC model could have application for septic patients with circulatory failure.

Interestingly, our study shows that immunocompromised patients are less likely to develop delirium than immunocompetent patients. A previous large prospective observational study also showed that the rate of immunocompromised patients was lower in patients developing sepsis‐associated encephalopathy than in patients without sepsis‐associated encephalopathy (25.2% versus 35.2%). Elevated serum cortisol levels were shown in another prospective observational study to be associated with brain dysfunction in patients with sepsis, which suggests the importance of the immune system in development of sepsis‐associated encephalopathy. Immunocompromised patients could be at low risk of sepsis‐associated delirium by alleviation of excess inflammatory response. This hypothesis requires confirmation in future studies.

Discrimination and calibration of PRE‐DELIRIC on delirium were not well shown in mechanically ventilated patients with sepsis in this study. These results imply that PRE‐DELIRIC cannot be used in this emergently admitted septic subpopulation to stratify the risk of delirium, or to implement selective preventive strategies in patients at high risk of delirium. The prevalence rate of delirium in our study was higher (40%) than those reported in previous studies that enrolled overall ICU populations (approximately 20%). , Furthermore, other studies reported much higher prevalence of delirium in ICU patients with sepsis (50–60%). , Non‐selective preventive strategies for all patients might be better in mechanically ventilated patients with sepsis.

Our study has several limitations. It was a post‐hoc analysis of a previous randomized controlled trial and the study cohort was derived from the DESIRE trial, which was not designed for the validation of PRE‐DELIRIC. However, the main inclusion and exclusion criteria were set in concordance with those of previous studies (e.g. included patients with ≥24 h length of ICU stay, and excluded patients with length of ICU stay <24 h, patients who were delirious on the first day of ICU admission, and patients with sustained coma throughout their ICU stay). , , We also applied the same definition of delirium as a previous study. The patient selection and outcome measurement are therefore not substantially different from previous studies, which provides validity of our results. The size of our cohort was relatively small for validation of prediction models, but this study was a post‐hoc analysis of a randomized clinical trial and the eligibility of patients and outcome measurements were more robust than previous observational studies.

Conclusions

The PRE‐DELIRIC model could not predict delirium in mechanically ventilated patients with sepsis.

Disclosure

Approval of the research protocol: This study was a post hoc analysis of the DESIRE trial, which was a randomized controlled trial undertaken among eight ICUs in Japan. The participating institutions were Osaka City University, Osaka City General Hospital, Hyogo College of Medicine, Saga University Hospital, National Hospital Organization Kyoto Medical Center, Sapporo Medical University, Yamaguchi Grand Medical Center, and Wakayama Medical University. The ethical review boards of all relevant institutions approved the study protocol.

Informed consent: Written informed consent was obtained from all participants or legally authorized guardians before study enrollment.

Registry and registration no. of the trial: This trial was registered on ClinicalTrials.gov (NCT01760967) on 1 January, 2013.

Animal studies: N/A.

Conflict of interests: None.

Fig. S1. Distribution of the PRE‐DELIRIC score.

Fig. S2. Observed and expected incidence of delirium.

Click here for additional data file.