Development of an ICU discharge instrument predicting psychological morbidity: a multinational study.

PURPOSE: To develop an instrument for use at ICU discharge for prediction of psychological problems in ICU survivors.
METHODS: Multinational, prospective cohort study in ten general ICUs in secondary and tertiary care hospitals in Sweden, Denmark and the Netherlands. Adult patients with an ICU stay ≥ 12 h were eligible for inclusion. Patients in need of neurointensive care, with documented cognitive impairment, unable to communicate in the local language, without a home address or with more than one limitation of therapy were excluded. Primary outcome was psychological morbidity 3 months after ICU discharge, defined as Hospital Anxiety and Depression Scale (HADS) subscale score ≥ 11 or Post-traumatic Stress Symptoms Checklist-14 (PTSS-14) part B score > 45.
RESULTS: A total of 572 patients were included and 78% of patients alive at follow-up responded to questionnaires. Twenty percent were classified as having psychological problems post-ICU. Of 18 potential risk factors, four were included in the final prediction model after multivariable logistic regression analysis: symptoms of depression [odds ratio (OR) 1.29, 95% confidence interval (CI) 1.10-1.50], traumatic memories (OR 1.44, 95% CI 1.13-1.82), lack of social support (OR 3.28, 95% CI 1.47-7.32) and age (age-dependent OR, peak risk at age 49-65 years). The area under the receiver operating characteristics curve (AUC) for the instrument was 0.76 (95% CI 0.70-0.81).
CONCLUSIONS: We developed an instrument to predict individual patients' risk for psychological problems 3 months post-ICU, http://www.imm.ki.se/biostatistics/calculators/psychmorb/ . The instrument can be used for triage of patients for psychological ICU follow-up. TRIAL REGISTRATION: The study was registered at clinicaltrials.gov, NCT02679157.

Take-home message

Introduction

Several million patients are treated in intensive care units (ICUs) in Europe yearly [1]. While the majority survive, many patients suffer from psychological, physical and cognitive problems after ICU discharge [2-6], collectively termed the post-intensive care syndrome (PICS) [7]. Problems can persist and impede the individuals’ ability to return to life as it was prior to the episode of critical illness, including returning to work or other major activities [8, 9]. The psychological PICS components depression, post-traumatic stress (PTS) and anxiety occur in approximately one of three ICU survivors [2-4].

Increased awareness of PICS has led to an expansion of ICU follow-up [10]. Follow-up has mainly been limited to patients with longer ICU length of stay (LOS) [11] or prolonged duration of mechanical ventilation [12], due to a poorly evidenced assumption that these are the strongest risk factors for PICS.

In a lack of standardized, simple methods to identify high-risk patients that might benefit from psychological follow-up, selection of ICU survivors for follow-up is largely expert opinion-based [10, 13]. A more accurate risk prediction instrument, applicable for general ICU survivors could be of value, in order to improve triage of patients for interventional follow-up. The aim of this study was to develop a screening instrument for use at ICU discharge, in order to assess individual patients’ risk for psychological problems 3 months post-ICU. The study was published as an abstract at the 38th International Symposium on Intensive Care and Emergency Medicine [14].

Methods

This prospective multi-center cohort study was performed in ten ICUs in Sweden, Denmark and the Netherlands. We followed the TRIPOD guidelines for reporting of multivariable prediction model development studies [15]. The study protocol is provided in the electronic supplementary material (ESM). Regional ethical review boards in participating countries approved the study, which was performed in accordance with the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments. All study participants gave informed consent. The study was registered at clinicaltrials.gov, NCT02679157.

Participants

All patients ≥ 18 years old were consecutively screened for inclusion for up to 3.5 months, depending on the starting date for each center, during a period from January to June 2016. Patients with an ICU stay ≥ 12 h (≥ 24 h for elective postoperative care patients; those with stays < 24 h were not considered true ICU patients) were eligible for inclusion. Exclusion criteria were need for neurointensive care, previously documented cognitive impairment, lack of a formal home address, inability to communicate in the language of the study site, or ICU admission solely for an elective procedure (e.g. epidural, central line placement). Patients with more than one limitation of treatment (e.g. a do-not-resuscitate order) and moribund patients were also excluded.

Outcome

The primary outcome was psychological problems 3 months after discharge from the ICU, assessed with the Hospital Anxiety and Depression Scale (HADS) and the Post-Traumatic Stress Symptoms Checklist-14 (PTSS-14). Patients scoring above the predefined cut-off in any of the questionnaires were considered to be a case. Secondary outcome was mental health-related quality of life (HRQOL), assessed with the RAND-36. All questionnaires are validated in the languages of participating countries.

HADS is a validated questionnaire for critically ill patients [16]. It consists of 14 questions; 7 screening for symptoms of depression and 7 for symptoms of anxiety [16]. Each question generates 0–3 points. A subscale score ≥ 11 was used as a cut-off for substantial symptoms of depression or anxiety [17]. PTSS-14 is a validated questionnaire assessing PTS symptoms in ICU survivors [18]. Part A has four questions regarding traumatic memories from the ICU stay, answered yes or no. Part B consists of 14 questions screening for ongoing stress symptoms, each scored from 1 (never) to 7 (always). A part B score > 45 was used as a cut-off for substantial symptoms of PTS [18]. RAND-36 is a 36-item questionnaire assessing HRQOL consisting of 8 domains that can be divided into 2 component scores, the mental health component score (MCS) and the physical health component score [19]. Scores range from 0 to 100, a higher score indicating better HRQOL. We compared scores between patients classified as cases and non-cases in the four mental domains and the MCS as secondary outcomes.

Patients alive 3 months after ICU discharge received and returned the questionnaires by postal mail. Non-responders received a reminder phone call after 2 weeks. A new set of questionnaires was sent out if the original ones were missing or if patients did not respond to the phone call.

Predictors

Potential predictors were selected after searching the literature and a consensus discussion with a panel of experts. For a list of reviewed studies and more detail on definitions and categorizations of predictors, see the ESM. Data were collected at ICU discharge from the patient data management systems, medical charts, patients or their next-of-kin. The selection rendered 18 risk factors divided into 3 categories: (1) premorbid risk factors, (2) in-ICU risk factors and (3) ICU discharge risk factors.

Premorbid risk factors were age, sex, educational level, employment status, physical comorbidities assessed with the Charlson Comorbidity Index [20] and box 1 of the Simplified Acute Physiology Score (SAPS) III, being the caretaker of a child < 18 years old and having a history of psychological problems.

In-ICU risk factors were admission diagnosis (surgical, medical or trauma), acute or elective hospital admission, severity of illness assessed with the Acute Physiology and Chronic Health Evaluation (APACHE) II, ICU LOS, severe sepsis/septic shock, days with coma, hours with invasive ventilator treatment and agitation/agitated delirium during ICU stay.

ICU discharge risk factors were symptoms of depression, evaluated with the modified patient health questionnaire 2 (PHQ-2) [21]. Traumatic ICU memories were assessed with PTSS-14 part A [18]. Patients were asked about their perception of social support after hospital discharge: “Do you have a family member or a close friend who cares about you and your health who can help you when you leave the hospital?” Patients unable to answer questions at discharge were assessed at the regular ward within 24 h.

Statistical methods

Sample size

The projected patient recruitment was 800 patients, with an expected loss-to-follow-up ratio of 30%. With 30% of patients having the primary outcome, this would render a case/predictor quotient of 9.3, in the range of recommended 5–10 cases per predictor [22].

Missing data

Missing data for single items < 10% was considered negligible. Missing data due to non-responding was handled with inverse probability weighting, to minimize selection bias due to non-response [23]. Potential predictors for the probability of being a responder were analyzed in a univariable regression model. Factors with a p value < 0.1 were included in a multivariable regression model where possible interactions were analysed. The generated weighted model was applied to all the subsequent analyses.

Patients with missing items in the HADS and PTSS-14 questionnaires were regarded as non-cases if their total score remained below the predefined case cut-off after replacing all the missing questionnaire items with their respective largest possible value. Patients were regarded as cases if their score was above the cut-off after replacing all the missing items with their respective smallest possible value. Patients with ambiguous scores despite replacement were not included in the final analyses.

Statistical analysis methods

All statistical analyses were performed with STATA version 14.2 (StataCorp, College Station, TX, USA). Two-sided significance level was set to 0.05. Continuous variables were presented with medians and interquartile ranges (IQR) and categorical variables with numbers and percentages. For comparisons of categorical variables, the Chi-square test or Fisher’s exact test were used. The Mann–Whitney U test was used for continuous variables.

Associations between each risk factor and being a case were investigated with univariable logistic regression analysis. Risk factors with a significance level of p < 0.1 were included in a multivariable logistic regression model. Continuous predictors were visually evaluated with locally weighted scatterplot smoothing and tested numerically with restricted cubic splines to assess the shape of the potential relationship with the outcome. Thereafter, a supervised, non-automatic stepwise selection was performed, removing the predictors with significance level < 0.05 one at a time and re-estimating the model, generating the predictive instrument. The area under the receiver operating characteristics curve (AUC) was used to assess the predictive value of the model. For calibration of the model, the observed risk of psychological problems was plotted against predicted risk across 30% strata. Arbitrary risk groups were created with 0–29, 30–59 and ≥ 60% risk of having the outcome, and negative and positive predictive values were calculated. The model was internally validated in 500 bootstrap samples, and a shrinkage factor of regression coefficients was estimated to adjust for possible over-optimism of the predictive model.

Additional analysis

Given that ICU LOS is the eligibility criterion of choice in many ICU follow-up clinics [11, 13, 24, 25], the association between ICU LOS and psychological problems was examined. The predictive value was assessed with an AUC.

Results

Of 2193 screened patients, 572 patients were included in the study. Fifty-five patients died before follow-up and 78% (n = 404) responded to the questionnaires (Fig. 1). Among included patients, 61% were men with a median age of 65 years and median APACHE II score of 18. The proportion of patients on mechanical ventilation was 60% and median duration of mechanical ventilation was 50 h. Median ICU LOS was 62 h (Table 1).

Fig. 1

Flow chart of inclusion and exclusion

Table 1

Patient characteristics

Patient characteristic	Sweden (n = 300)	The Netherlands (n = 166)	Denmark (n = 106)	Total population (n = 572)
Age	65 (49–74)	63 (56–71)	66 (59–73)	65 (53–73)
Male sex	179 (60)	103 (62)	69 (65)	351 (61)
Somatic comorbidities (CCI score)	4 (1–5)	3 (2–5)	4 (2–6)	4 (2–5)
APACHE II score	18 (13–24)	16 (12–22)	21 (16–26)	18 (13–23)
Admission diagnosis
Medical	123 (41)	55 (33)	40 (38)	218 (38)
Surgical	156 (52)	92 (55)	53 (50)	301 (53)
Trauma	21 (7)	19 (11)	12 (11)	52 (9)
Mechanical ventilation	153 (51)	143 (86)	47 (44)	343 (60)
Duration of mechanical ventilation (h)	68 (19–174)	34 (8–118)	24 (12–101)	50 (13–137)
ICU LOS (h)	66 (27–142)	67 (43–188)	47 (22–99)	62 (30–140)

Data are presented as medians (interquartile range) and proportions, n (%) where appropriate. Duration of mechanical ventilation calculated for mechanically ventilated patients only

CCI Charlson Comorbidity Index, APACHE Acute Physiology And Chronic Health Evaluation, ICU LOS intensive care unit length of stay

Three months after ICU discharge, the prevalence of psychological problems was 20% (n = 80). Eleven percent (n = 43) suffered from substantial symptoms of depression, 10% (n = 39) had symptoms of anxiety and 13% (n = 54) had symptoms of PTS. As many as 70% of patients were classified as cases scored above a cut-off in more than one questionnaire; 46% scored above a cut-off in two symptom domains and 24% above a cut-off in all three domains.

Missing data

Sixteen percent of responders had missing data for any of the predictors analyzed in the univariable model, of which 10% had only one missing variable. Ten percent of responders had incomplete primary outcome data, with missing items in the follow-up questionnaires. After item replacement as described above, five patients had ambiguous total scores and were not included in the analyses. For a complete description of missing variables and the development of the weighted model for non-response, see the ESM.

Model development

Prevalence of predictors and their univariable associations are shown in Table 2. Five predictors [previous psychological problems, age, lack of social support, traumatic memories (PTSS-14 A) and symptoms of depression (modified PHQ-2) at ICU discharge] were associated with psychological problems 3 months after ICU discharge (p < 0.1) and included in the multivariable logistic regression model. All predictors except previous psychological problems were significantly associated with the adverse outcome (p < 0.05). Thus, the final predictors were age (OR age-dependent, peak risk at age 49–65 years, Fig. S1, Table S1, ESM), lack of social support (OR 3.28, 95% CI 1.47–7.32), traumatic memories (OR 1.44, 95% CI 1.13–1.82) and symptoms of depression (OR 1.29, 95% CI 1.10–1.50) assessed at ICU discharge (Table 3). Previous psychological problems did not improve the final regression model. This predictor was strongly associated with psychological symptoms at ICU discharge (p < 0.001 for both PHQ-2 and PTSS-14 part A).

Table 2

Categorization of potential predictors for post-ICU psychological problems (cases) and no psychological problems (non-cases) and predictors’ univariate associations

Predictor	Categorization	Cases (n = 80)	Non-cases (n = 319)	Univariate association (p value)†
Age		64 (54–72)	65 (56–73)	NA
Male sex		47 (59)	201 (62)	> 0.1
Education level	Elementary school	20 (25)	75 (24)	> 0.1
	Senior high school	35 (44)	143 (45)
	University/college	24 (30)	88 (28)
Employment status pre-ICU	Unemployed	1 (1)	12 (4)	> 0.1
	Sick-leave	13 (16)	30 (9)
	Retired	42 (53)	168 (53)
	Student	0	6 (2)
	Employed	23 (29)	95 (30)
Somatic comorbidities (CCI)		3 (2–5)	4 (2–5)	> 0.1
Caretaker of children < 18 years old		11 (14)	32 (10)	> 0.1
Admission diagnosis	Medical	28 (35)	116 (36)	> 0.1
	Surgical	46 (58)	178 (56)
	Trauma	6 (8)	24 (8)
Admission severity of illness (APACHE)		18 (14–23)	18 (13–23)	> 0.1
SAPS III box 1		20 (13–24)	19 (14–23)	> 0.1
Psychological problems pre-ICU		26 (33)	58 (18)	< 0.01*
Type of hospital admission	Acute	59 (74)	237 (74)	> 0.1
	Elective	21 (26)	82 (26)
Agitation		14 (18)	63 (20)	> 0.1
Severe sepsis		21 (26)	66 (21)	> 0.1
Duration of coma		0 (0–1)	0 (0–1)	> 0.1
ICU LOS (h)		61 (38–162)	54 (26–138)	> 0.1
Mechanical ventilation		52 (65)	198 (62)	> 0.1
Duration of mechanical ventilation (h)		22 (11–103)	49 (9–132)	> 0.1
Depressive symptoms (PHQ-2)		2 (0–4)	0 (0–2)	< 0.001*
Traumatic memories (PTSS-14A)		2 (1–3)	1 (0–2)	< 0.001*
Lack of social support		15 (19)	26 (8)	< 0.05*

Data are presented as medians (interquartile range) and proportions, n (%) as appropriate. Duration of mechanical ventilation calculated only for mechanically ventilated patients

NA not applicable, ICU intensive care unit, CCI Charlson Comorbidity Index, APACHE Acute Physiology and Chronic Health Evaluation, SAPS Simplified Acute Physiology Score, LOS length of stay, PHQ-2 Patient Health Questionnaire-2, PTSS-14A Post-Traumatic Symptoms Checklist 14A

*Statistically significant p value

†p values for the univariate association between the predictor and the outcome

Table 3

Odds ratios and confidence intervals for risk factors included in the multivariable model

Risk factor	Odds ratio	95% Cl	p value	Regression coefficient
Lack of social support	3.28	1.47–7.32	< 0.01	15.71
Psychological problems pre-ICU	2.17	1.22–3.85	> 0.05
Depressive symptoms (PHQ-2)	1.29	1.10–1.50	< 0.01	3.39 per point
Traumatic memories (PTSS-14A)	1.44	1.13–1.82	< 0.01	4.84 per point
Age	Separate table			0–58

Regression coefficients for risk factors in the final predictive model

CI confidence interval, CPAx Chelsea Critical Care Physical Assessment tool, ICU intensive care unit, PHQ-2 patient health questionnaire, PTSS-14A post-traumatic symptoms checklist 14 part A

The final model AUC was 0.76 (95% CI 0.70–0.81; Fig. S2, ESM). The final instrument with questions, scores and a graph to obtain patients’ individual risk is depicted in Fig. 2. The instrument is available on-line: http://www.imm.ki.se/biostatistics/calculators/psychmorb/. The positive predictive value for a high-risk group with ≥ 60% predicted risk of later psychological problems was 0.83 (95% CI 0.37–0.98). The negative predictive value for a low-risk group with < 30% predicted risk for an adverse outcome was 0.84 (95% CI 0.79–0.88). Positive and negative predictive values for different risk groups are shown in Table S2 of the ESM. For calibration of the model, see Fig. S3 of the ESM.

Fig. 2

The psychological risk prediction instrument for use at ICU discharge

Internal validation

With internal validation in 500 bootstrap samples, the AUC was 0.73 (95% CI 0.73–0.74). The shrinkage factor for the final model was 0.89. The regression coefficients can be multiplied by that value to provide more reliable predictions for new patients. A maximum shrinkage factor of 1 indicates that no over-optimism occurs.

Secondary outcome

There were clinically significant differences in all RAND-36 mental domains as well as MCS between patients with and without psychological problems post-ICU. Cases had worse MCS, median 33 (IQR 22–43), than non-cases, median 72 (IQR 53–84), p < 0.001 (Fig. S4, ESM). For scores in the four mental domains, see Table S3 of the ESM.

The AUC for ICU LOS as a predictor was 0.49 (95% CI 0.42–0.56; Fig. S5, ESM).

Discussion

In this multinational study, we developed a clinical instrument for use at ICU discharge, predicting individual patients’ risk for adverse psychological outcome after ICU stay. The predictive value assessed as AUC was 76%. Substantial psychological problems post-ICU were associated with a significant reduction in mental HRQOL. The instrument provides clinicians with a simple and practical bedside tool for quantitative assessment of the 3-month risk for adverse psychological outcome.

The need for screening instruments for early identification of ICU survivors at risk for later morbidity has been called for in stakeholder’s meetings and national guidelines [7, 10, 26]. Previous studies have addressed this but with limited generalizability due to subgroup selection of patients [12]. Other studies have screened patients at later time points [18, 27] which may be less practical compared to screening at ICU discharge when all patients can be assessed. Predictive values in previous studies have been moderate and are based on smaller cohorts [28].

We found an increased risk for psychological morbidity in the middle age. This middle-age risk peak (Fig. S1) is not readily detected merely comparing median and IQR data (Table 2) for cases and non-cases. Previous studies of age as a risk factor for psychological problems post-ICU have shown varying results [2, 3]. Being middle aged may imply unaccounted risks for psychological reactions to critical illness. Some studies indicate that the middle age is associated with the lowest ratings of happiness and life satisfaction [29]. The finding of depressive symptoms and traumatic memories in the ICU as predictors of later psychological problems is in concordance with previous studies [30-32]. Perceived lack of social support as a risk factor is also consistent with previous research [30, 33]. The only significant predictor in the univariable analysis that did not further improve the regression model was previous psychological problems, which has been associated with later psychological morbidity in ICU survivors [3, 31, 34]. We found a strong association between previous psychological problems and psychological symptoms at ICU discharge, indicating that previous psychological problems are likely well accounted for in the psychological assessment at ICU discharge.

The final instrument is conceptually aligned with previous expert-based recommendations in the British National Institute for Health and Care Excellence (NICE) guidelines, suggesting psychological assessment before ICU discharge [10]. The predictive instrument offers clinicians a quantitative screening tool for use at ICU discharge. This first step in identifying risk patients can be followed by reevaluation and subsequent interventions. With a feasible number of specific questions, the instrument can aid the clinician to estimate the individual patient’s future risk for psychological problems, which may facilitate intervention trigger thresholds.

The instrument clearly outperforms ICU LOS as a method to triage patients for psychological follow-up. ICU LOS is the most common criterion for follow-up after ICU stay in Sweden, Denmark and the Netherlands [11, 13, 24, 25].

Strengths and limitations

Our study is multinational, prospective and includes medical and surgical patients with a broad range of ICU LOS, increasing its generalizability. To our knowledge, it is the largest psychological ICU follow-up study to date. The follow-up response rate of 78% can be considered high in an ICU survivor population. The instrument has been internally validated but needs external validation in different ICU cohorts.

A potential limitation with this kind of study is missing data. Data missing due to non-response was managed with inverse probability weighting, a recognized strategy for handling loss-to-follow-up that may not be missing completely at random [23]. A substantial proportion of patients had an ICU stay shorter than 12 h, and more patients than anticipated were transferred to other ICUs before discharge, rendering a smaller study cohort than originally projected. Based on prior agreements with study sites and resource constraints, the study was stopped at the agreed time; hence fewer participants than projected were included. Potential over-fitting of predictors due to the smaller study size was assessed, rendering a high shrinkage factor indicating little over-optimism. Fewer participants than expected had psychological problems, which could partly be explained by the stricter cut-offs for the HADS subscales, using ≥ 11 (probable case) rather than the often-used cut-off of ≥ 8 (possible case). Another limitation is the time anchoring of the PHQ-2 questionnaire, assessing symptoms during the past days, an aspect that needs consideration for patients with short ICU stays. Using self-administered questionnaires instead of psychological interviews was due to limited resources. However, the PTSS-14 and the HADS are validated questionnaires screening for psychological symptoms in ICU survivors [17, 18].

Future perspectives

The instrument facilitates identification of high-risk populations for interventional follow-up trials. This step may reduce dilution of potential beneficial effects of follow-up, a problem that may have contributed to the lack of effects in previous ICU follow-up interventions [35-40]. The instrument has fair predictive accuracy in a mixed ICU survivor cohort, does not require special equipment or specially trained staff and enables concentration of ICU follow-up to patients most likely to benefit from post-ICU interventions. Such interventions can be initiated early and potentially reduce long-term psychological morbidity and improve HRQOL.

Conclusions

We developed a screening instrument for psychological morbidity 3 months after ICU stay. The instrument, freely available online, predicts psychological problems in ICU survivors more accurately than existing methods. It can be used to identify high-risk patients for follow-up already at ICU discharge, facilitating early interventions and improved long-term psychological outcome in ICU survivors.

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOC 41 kb)

Supplementary material 2 (DOC 46 kb)

Supplementary material 3 (DOC 49 kb)

Supplementary material 4 (DOC 47 kb)

Supplementary material 5 (DOC 46 kb)

Supplementary material 6 (DOC 43 kb)

Supplementary material 7 (DOC 29 kb)

Supplementary material 8 (DOC 29 kb)

Supplementary material 9 (DOC 110 kb)

Supplementary material 10 (DOC 80 kb)

Effective methods to prospectively identify patients likely to suffer from psychological problems in the months after intensive care are lacking. In this prospective multi-center study, we developed a simple online instrument, for use at ICU discharge, rendering a fairly precise individual risk for psychological problems 3 months after ICU discharge.