Subjective score and outcomes after discharge from the intensive care unit: a prospective observational study.

Background

Survivors of critical illness are frequently vulnerable to new complications after intensive care unit (ICU) discharge, including readmission and short-term mortality.[1] Risk factors for such adverse events following patient discharge from the ICU should be identified to avoid increases in morbidity and/or mortality. Several risk factors associated with worse outcomes have been studied, such as age, nighttime discharge, admission source, and transfer to a high-dependency unit.[2,3] In a recent retrospective cohort of patients with oncohematological diseases, we found that ICU readmissions increased mortality by 10-fold and were associated with male sex, emergency surgery, a longer hospital stay before ICU transfer, and invasive mechanical ventilation.[4] Most of these risk factors are associated with a higher severity of illness at ICU admission or previous chronic health problems. However, the means by which to combine their relevancy and weight into a straightforward decision is still unknown. Additionally, some previous studies focused on data solely at ICU admission. Assessment at ICU discharge would likely allow for better prediction of later outcomes.[4]

Physicians usually rely on their clinical judgment to decide whether the patient is ready to be discharged from the ICU and to choose the proper discharge facility (e.g., the ward or high-dependency unit).[5] Some previous data suggest that death might be more accurately predicted by an ICU physician’s subjective impression than by usual scoring systems.[6] In fact, the Sabadell score is a subjective tool for prediction of the post-ICU prognosis that exhibits good discriminative ability.[7,8] Nevertheless, its validation is restricted to developed countries, and it may not fully apply to low- and middle-income countries.

The primary objective of this study was to evaluate the ability of the Sabadell score to predict worse outcomes after ICU discharge in a tertiary-care hospital located in a middle-income country.

Methods

This was a prospective cohort analysis of patients admitted to an “open format” mixed medical-surgical ICU. We characterized our unit and hospital in a previous report.[4] Briefly, it is a private tertiary-care hospital in São Paulo, Brazil; cardiac surgical patients are managed in another unit within our hospital. Admission and discharge decisions are made after discussion between the patient’s attending physician and the intensive care physician. This study was approved by the local institutional ethics committee of Hospital Sirio-Libanes, which waived the requirement for informed consent because of the observational design of the study (CAAE: 21503913.5.0000.5461).

The study population comprised consecutive adult patients (>18 years of age) who were admitted from August 2014 to May 2015 and discharged alive. The exclusion criteria were an ICU length of stay of <12 h, pregnancy, and patient unsuitability for ICU readmission (death on the unit or transfer to another hospital or to palliative care). Data were retrieved from the ICU administrative database (Sistema Epimed™; www.epimedmonitor.com)[9,10] and included age, sex, Simplified Acute Physiology III Score,[11,12] admission source, diagnosis, comorbidities, resource use during ICU stay, frequency of nighttime and weekend discharges, Sequential Organ Failure Assessment (SOFA) score at ICU admission and discharge,[13] C-reactive protein and lactate levels at ICU admission and discharge, and hospital mortality. Sepsis was defined according to a previous consensus definition at the time of data collection.[14] Readmission was defined as ICU admission of a patient who had been previously admitted to the ICU during the same hospitalization. If multiple readmission episodes occurred, only the first was considered for the present analysis. Based on the time of ICU discharge, the patients were categorized into daytime (7:00 am to 6:59 pm) and nighttime (7:00 pm to 6:59 am) discharge and into weekday (Monday–Friday) and weekend (Saturday and Sunday) discharge.

At ICU discharge, the attending intensivist scored the patient’s subjective prognosis as reported in the Sabadell score.[7,8] Mutually exclusive groups of patients were defined as follows: score of 0, good prognosis in the long term (>6 months); score of 1, poor prognosis in the long term (>6 months) and suitable for ICU readmission without restrictions; score of 2, poor prognosis in the short term (<6 months) and with debatable suitability for ICU readmission; and score of 3, not expected to survive the hospital stay. The Sabadell score is entirely subjective and integrates the physician’s knowledge and impression about the patient’s condition, previous performance before ICU admission, and medical history during the healthcare facility stay. The ward team was blind to the Sabadell score; likewise, the ICU physicians who evaluated requests for ICU readmission were blind to the Sabadell score (the decision regarding ICU readmission was at the discretion of the ward physician only).

Statistical analysis

Data were analyzed with IBM SPSS Statistics for Windows, Version 20.0 (IBM Corp., Armonk, NY, USA). Normally distributed data were tested with the Kolmogorov–Smirnov test. Data are presented as the mean (standard deviation) or median [25th–75th percentile] accordingly. Categorical variables are presented as absolute number (percentage). Parametric variables were compared between groups with an unpaired Student’s t-test or analysis of variance, and nonparametric variables were compared between groups using the Mann–Whitney test or Kruskal–Wallis test. Categorical data were compared with the chi-squared test. All statistics were two-tailed, and a p-value of <0.05 was considered statistically significant.

A composite endpoint that included ICU readmission or unexpected ward death was assessed. A readmission event is strongly associated with an increased risk of hospital mortality[3,4] and is usually applied as a metric of quality of care.[15] Thus, combining readmission with ward death into a composite endpoint sounds plausible, is clinically meaningful, avoids competing risk between these variables, and increases the statistical power of the study to evaluate risk factors for these events.[16] A multivariate logistic regression analysis was performed, with worse outcomes as the dependent factor. Variables with a p-value of <0.1 in the univariate analysis were included in the logistic model. Multicollinearity was excluded with the variance inflation factor before modeling.[17] The model was refined using the backward stepwise method, excluding the least significant variable at each step if the significance level was >0.05. As a sensitivity analysis, the same procedure was performed for ICU readmission. Calibration and discrimination of the prediction model were evaluated with the Hosmer–Lemeshow goodness-of-fit test and the area under the curve (AUC), respectively. Survival curves were constructed with the Kaplan–Meier method censored at 90 days and compared with the log-rank test.

Results

Of 425 patients admitted to the ICU and discharged alive during the study period, readmission occurred in 43 patients (10.1% of discharged patients) and death occurred in 19 patients (4.5%). The characteristics of the study group are presented in Table 1. At ICU admission, patients who developed worse outcomes after ICU discharge were older, were sicker, were more frequently male, had a nonelective surgical reason for admission, were usually from the ward, more frequently had sepsis, were less frequently independent during daily activities, more frequently required invasive procedures, had higher SOFA scores both at ICU admission and discharge, had higher C-reactive protein values at admission, and higher Sabadell scores at discharge.

Table 1.

Patient characteristics at intensive care unit admission and discharge.

	All patients (n = 425)	Without worse outcomes (n = 373)	Worse outcomes (n = 52)	p-value*
Age, years	66.7 ± 18.2	66.0 ± 18.3	71.7 ± 17.1	0.037
Male	211 (49.6)	177 (47.5)	34 (65.4)	0.018
SAPS III	40 [31–51]	38 [29–50]	50.5 [39.3–56.8]	<0.001
Admission type				0.002
Medical	207 (48.7)	170 (45.6)	37 (71.2)
Emergency surgery	24 (5.6)	21 (5.6)	3 (5.8)
Elective surgery	194 (45.6)	182 (48.8)	12 (23.1)
Admission source				0.001
Ward	40 (9.4)	29 (7.8)	11 (21.2)
Emergency room	122 (28.7)	104 (27.9)	18 (34.6)
Operating room	218 (51.3)	203 (54.4)	15 (28.8)
Intermediate care	23 (5.4)	19 (5.1)	4 (7.6)
ICU discharge during weekends	80 (18.8)	70 (18.8)	10 (19.2)	1.00
ICU discharge during the nighttime	140 (32.9)	123 (33.0)	17 (32.7)	1.00
Length of hospital stay before ICU admission, days	1 [0–1]	1 [0–1]	1 [0–7.8]	0.49
Non-oncohematological comorbidities				0.54
0	139 (32.7)	123 (33.0)	16 (30.8)
1	123 (28.9)	109 (29.2)	14 (26.9)
≥2	160 (37.6)	138 (37.0)	22 (42.3)
Independence during daily activities	309 (72.7)	282 (75.6)	27 (51.9)	0.001
Oncohematological condition	155 (36.5)	133 (35.7)	22 (42.3)	0.36
Sepsis at ICU admission	61 (14.4)	42 (11.3)	19 (36.5)	<0.001
Tracheostomy	10 (2.4)	9 (2.4)	1 (1.9)	0.26
Mechanical ventilation during ICU stay	64 (15.1)	51 (13.7)	13 (25.0)	0.039
Vasoactive drug during ICU stay	104 (24.5)	87 (23.3)	17 (32.7)	0.17
Dialysis during ICU stay	9 (2.1)	5 (1.3)	4 (7.7)	0.016
Total SOFA score at ICU admission	2 [1–4]	2 [0–3]	3 [1–5]	<0.001
Total SOFA score at ICU discharge	1 [0–2]	1 [0–2]	2 [1–3]	0.001
CRP at ICU admission, mg/dL	3.9 [1.0–9.2]	3.6 [0.8–8.5]	5.3 [2.5–11.8]	0.015
CRP at ICU discharge, mg/dL	5.2 [1.6–10.4]	4.9 [1.6–10.4]	6.4 [1.9–10.6]	0.36
Lactate at ICU admission, mmol/L	1.7 [1.2–2.6]	1.7 [1.2–2.6]	1.8 [1.1–3.1]	0.99
Lactate at ICU discharge, mmol/L	1.1 [0.9–1.6]	1.1 [0.9–1.6]	1.2 [0.9–1.6]	0.68
Sabadell score	0 [0–1]	0 [0–1]	1 [1–2]	<0.001
Hospital death	19 (4.5)	---	---
ICU readmission	43 (10.1)	---	---

Data are presented as n (%), mean ± standard deviation, or median [25th–75th percentile].

SAPS III, Simplified Acute Physiology III Score; SOFA, Sequential Organ Failure Assessment; CRP, C-reactive protein; ICU, intensive care unit.

*p-value for comparison between patients with and without worse outcomes.

At ICU discharge, the attending physicians classified the patients as follows: score of 0, 219 (51.5%); score of 1, 134 (31.5%); score of 2, 65 (15.3%); and score of 3, 7 (1.6%). The baseline characteristics and ICU procedures were different between the two groups (Table 2). Ward mortality was 4.5% among all patients and differed for each Sabadell score group: score of 0, 0.5%; score of 1, 2.2%; score of 2, 16.9%; and score of 3, 57.1% (p < 0.001) (Figure 1). The readmission rates were also different for each group: score of 0, 4.6%; score of 1, 14.2%; score of 2, 20.0%; and score of 3, 14.3% (p = 0.001). When considering any unwanted outcome, the observed incidences were as follows: score of 0, 5.0%; score of 1, 14.9%; score of 2, 26.2%; and score of 3, 57.1% (p < 0.001).

Table 2.

Clinical characteristics according to the Sabadell score at intensive care unit discharge.

	Sabadell 0 (n = 219)	Sabadell 1 (n = 134)	Sabadell 2 (n = 65)	Sabadell 3 (n = 7)	p-value
Age, years	61.0 ± 18.9	70.8 ± 15.4	77.2 ± 14.0	71.7 ± 17.2	<0.001
Male	98 (44.7)	69 (51.5)	40 (61.5)	4 (57.1)	0.11
SAPS III	33 [27–42]	45 [37–51]	54 [48–62]	57 [49–66]	<0.001
Admission type					<0.001
Medical	73 (33.3)	72 (53.7)	56 (86.2)	6 (85.7)
Emergency surgery	14 (6.4)	9 (6.7)	1 (1.5)	---
Elective surgery	132 (60.3)	53 (39.6)	8 (12.3)	1 (14.3)
Admission source					<0.001
Ward	13 (5.9)	13 (9.7)	12 (18.5)	2 (28.6)
Emergency room	51 (23.3)	40 (29.9)	27 (41.5)	4 (57.1)
Operating room	145 (66.2)	63 (47.0)	9 (13.8)	1 (14.3)
Intermediate care	3 (1.4)	8 (5.9)	12 (18.5)	---
ICU discharge during weekends	45 (20.5)	15 (11.2)	18 (27.7)	2 (28.6)	0.025
ICU discharge during the nighttime	57 (26.0)	50 (37.3)	29 (44.6)	4 (57.1)	<0.001
Length of hospital stay before ICU admission, days	1 [0–1]	1 [0–2]	1 [0–7]	0 [0–19]	0.55
Non-oncohematological comorbidities#					<0.001
0	88 (40.2)	34 (25.4)	15 (23.1)	2 (28.6)
1	69 (31.5)	43 (32.1)	9 (13.8)	2 (28.6)
≥2	60 (27.4)	57 (42.5)	40 (61.5)	3 (42.9)
Independence during daily activities	196 (89.5)	92 (68.7)	20 (30.8)	1 (14.3)	<0.001
Oncohematological condition	52 (23.7)	70 (52.2)	30 (46.2)	3 (42.9)	<0.001
Sepsis at ICU admission	15 (6.8)	22 (16.4)	21 (32.3)	3 (42.9)	<0.001
Tracheostomy	1 (0.5)	3 (2.2)	5 (7.7)	1 (14.3)	<0.001
Mechanical ventilation during ICU stay	20 (9.1)	30 (22.4)	14 (21.5)	---	0.002
Vasoactive drug during ICU stay	40 (18.3)	41 (30.6)	22 (33.8)	1 (14.3)	0.013
Dialysis during ICU stay	1 (0.5)	5 (3.7)	3 (4.6)	---	0.082
Total SOFA score at ICU admission	1 [0–2]	2 [1–4]	4 [2–5]	5 [4–7]	<0.001
Total SOFA score at ICU discharge	1 [0–2]	1 [1–3]	3 [1–4]	7 [4–8]	<0.001
CRP at ICU admission, mg/dL	2.4 [0.7–6.6]	4.8 [0.9–10.7]	5.6 [2.2–11.6]	8.4 [3.9–26.2]	<0.001
CRP at ICU discharge, mg/dL	4.4 [1.5–10.0]	5.6 [1.9–10.7]	4.2 [1.1–11.7]	6.9 [5.9–11.0]	0.32
Lactate at ICU admission, mmol/L	1.8 [1.2–2.6]	1.7 [1.1–2.6]	1.6 [1.1–2.1]	1.6 [0.9–2.4]	0.68
Lactate at ICU discharge, mmol/L	1.1 [0.9–1.6]	1.2 [0.8–1.7]	1.2 [0.9–1.4]	1.2 [1.2–1.8]	0.72
Hospital death	1 (0.5)	3 (2.2)	11 (16.9)	4 (57.1)	<0.001
ICU readmission	10 (4.6)	19 (14.2)	13 (20.0)	1 (14.3)	0.001

Data are presented as n (%), mean ± standard deviation, or median [25th–75th percentile].

SAPS, Simplified Acute Physiology III Score; SOFA, Sequential Organ Failure Assessment; CRP, C-reactive protein; ICU, intensive care unit.

Figure 1.

Survival curves in the ward for patients in each group according to the Sabadell score.

In the multivariate analysis (Table 3), independent risk factors for worse outcomes were sepsis at ICU admission and Sabadell scores at ICU discharge. The Hosmer–Lemeshow test was not statistically significant for the final model. The AUC was 0.73 [95% confidence interval (CI), 0.66–0.81; p < 0.001]. The sensitivity analysis with ICU readmission as the outcome identified the same independent predictors, but a Sabadell score of 3 was no longer significant (Table 4). The Hosmer–Lemeshow test was not statistically significant for the readmission model. The AUC was 0.71 (95% CI, 0.63–0.79; p < 0.001).

Table 3.

Factors associated with worse outcomes after intensive care unit discharge in the multivariate analysis.

Parameter	OR	95% CI	p-value
Sepsis	2.73	1.34–5.55	0.005
Sabadell score 1 versus 0	2.66	1.18–6.02	0.019
Sabadell score 2 versus 0	3.99	1.63–9.75	0.002
Sabadell score 3 versus 0	15.89	2.97–84.87	0.001

OR, odds ratio; CI, confidence interval

Table 4.

Factors associated with intensive care unit readmission in the multivariate analysis.

Parameter	OR	95% CI	p-value
Sepsis	2.93	1.39–6.16	0.005
Sabadell score 1 versus 0	2.77	1.19–6.48	0.019
Sabadell score 2 versus 0	3.19	1.23–8.31	0.017
Sabadell score 3 versus 0	2.61	0.26–26.14	0.42

OR, odds ratio; CI, confidence interval.

Discussion

The main finding of our single-center prospective observational cohort study is the applicability of a subjective score (Sabadell score) to prediction of unfavorable events after ICU discharge. Because these events suggest poor discharge decision-making and are associated with a longer hospital stay, increased resource consumption, and greater morbidity, tools with which to correctly identify patients who are ready to be discharged are required.

Our ward mortality rate is similar to that in a previous multicenter study that validated the Sabadell score in Spain[8] and is in line with a recent meta-analysis that aimed to estimate hospital mortality among patients discharged alive from the ICU.[18] However, we observed a readmission rate of 10.1%, which is higher than that in most previously published studies[19-21] as well as in the above-mentioned meta-analysis, which suggested that readmission rates generally range from 4% to 6% in critically ill patients.[18] Our high readmission rate might have been due to case-mix differences (e.g., the presence of a step-down unit in our hospital, a high proportion of patients with oncohematological diseases in our cohort, different end-of-life practices in middle-income countries wherein some patients who should receive palliative care in the wards are readmitted). In fact, Kramer et al.[20] recently acknowledged the problem with case-mix and readmission rates. In-hospital mortality adjustment nullified the differences in standardized mortality between ICUs with high and low rates of readmission. Thus, quality indicators and predictive tools might work differently across facilities. This is one of the reasons why we decided to study the validity of the Sabadell score in our institution.

Notably, the Sabadell score was originally described to predict ward mortality.[7,8] However, ICU readmission is associated with post-ICU death.[3,4,8] A recent European Society of Intensive Care Medicine report suggests that this metric can serve as a quality indicator[15] because some deaths associated with readmission are thought to be preventable.[22] Therefore, the relationship between the Sabadell score and readmission as a possible endpoint was plausible. Actually, our observed mortality associated with Sabadell score strata is similar to that in a previous report,[8] and the observed AUC is similar to that described for the Stability and Workload Index for Transfer (SWIFT) score, a predictive tool for ICU readmission.[23] A recent systematic review of published tools to predict adverse events (i.e., ICU readmission, death after ICU discharge, or medical emergency team activation) revealed a single tool (the Minimizing ICU Readmission score) that could predict both post-ICU discharge mortality and readmission.[24] The Minimizing ICU Readmission score includes data related to the severity of illness (Simplified Acute Physiology II score at ICU admission, central venous catheter use during the hospital stay, SOFA score at ICU discharge, and systemic inflammatory response syndrome in the last 2 days before ICU discharge) and the discharge policy (discharged from the ICU at night). Interestingly, this score has a discrimination ability similar to our findings (AUC, 0.74; 95% CI, 0.68–0.79).[25] Finally, our sensitivity analysis for ICU readmission, which should be viewed only as a hypothesis-generating analysis, identified the same predictors; however, a Sabadell score of 3 was no longer significant, probably because ICU readmission was not considered adequate for these patients (e.g. end-of-life planning). It should be stressed that in our study, the ward physician was solely responsible for the decision regarding ICU readmission, and he/she was unaware of the Sabadell score and the objective of the present study.

Clinical decision-making is frequently required of critical care practitioners and should be performed in a timely fashion.[26] In many instances, there is a lack of solid evidence-based support for such decisions,[27] especially related to ICU discharge policies. Some previous publications suggest that subjective physician impressions are more accurate in the prediction of outcomes after critical illness than are contemporary prognostic scores.[28-30] A systematic review suggested that physicians’ predictions were more accurate than scoring systems’ predictions (area under the summary receiver operating characteristic curve: 0.85 ± 0.03 for physicians and 0.63 ± 0.06 for scoring system predictions, p = 0.002).[6] Although scoring systems might reduce undesirable variability among health practitioners, an experienced physician usually subconsciously integrates all relevant information (e.g., history, clinical parameters at ICU admission, current state at ICU discharge, and presence of vulnerability). Therefore, their subjective impression about a patient’s prognosis could be as accurate or even better in the prediction of outcomes. Our results corroborate this hypothesis as previously demonstrated in a multicenter Spanish cohort.[8]

Our study has several limitations. First, it was a single-center prospective analysis of a private tertiary-care oncology center, which might limit the generalizability of our findings and our sample size. Thus, our results are hypothesis-generating, and caution is required when interpreting them. However, this is the first analysis involving a cohort derived from a middle-income country, and the concordance with previous publications is reassuring. Second, we only studied admission and discharge factors associated with outcomes. A patient’s history in the ward probably influences his or her hospital outcome, but this was not evaluated in our study. Third, the composite endpoint of ICU readmission and ward death gives the same importance for both outcomes. Some may argue that the relative importance of each of the components might be different for patients and clinicians. Although we acknowledge that this might be true, readmission is a relevant outcome. Furthermore, the composite endpoint increases the power of our study to detect potentially relevant risk factors and avoids the competing risks of readmission and death.[16] Fourth, a reliability evaluation of the Sabadell score for repeatability or agreement between physicians was not carried out. A subjective score might have great variability according to the doctor who applies it. Finally, we only asked intensive care physicians to predict survival and readmission. Recent data suggest that nurses’ perceptions are also relevant.[31]

Conclusion

In our cohort of critically ill patients discharged alive from the ICU, sepsis and the Sabadell score were identified as independent risk factors for worse outcomes. Future studies should validate the clinical utility of this score as a clinical decision-making tool.

List of abbreviations

ICU, intensive care unit; SOFA, Sequential Organ Failure Assessment; AUC, area under the curve.