Inclusion of 'ICU-Day' in a Logistic Scoring System Improves Mortality Prediction in Cardiac Surgery.

BACKGROUND Prolonged intensive care unit (ICU) stay is a predictor of mortality. The length of ICU stay has never been considered as a variable in an additive scoring system. How could this variable be integrated into a scoring system? Does this integration improve mortality prediction? MATERIAL AND METHODS The 'modified CArdiac SUrgery Score' (CASUS) was generated by implementing the length of stay as a new variable to the 'additive CASUS'. The 'logistic CASUS' already considers this variable. We defined outcome as ICU mortality and statistically compared the three CASUS models. Discrimination, comparison of receiver operating characteristic curves (DeLong's method), and calibration (observed/expected ratio) were analyzed on days 1-13. RESULTS Between 2007 and 2010, we included 5207 cardiac surgery patients in this prospective study. The mean age was 67.2 ± 10.9 years. The mean length of ICU stay was 4.6 ± 7.0 days and ICU mortality was 5.9%. All scores had good discrimination, with a mean area under the curve of 0.883 for the additive and modified, and 0.895 for the 'logistic CASUS'. DeLong analysis showed superiority in favor of the logistic model as from day 5. The calibration of the logistic model was good. We identified overestimation (days 1-5) and accurate (days 6-9) calibration for the additive and 'modified CASUS'. The 'modified CASUS' remained accurate but the 'additive CASUS' tended to underestimate the risk of mortality (days 10-13). CONCLUSIONS The integration of length of ICU stay as a variable improves mortality prediction significantly. An 'ICU-day' variable should be included into a logistic but not an additive model.

Background

No improvement of disease severity after cardiac surgery extends the duration of intensive care unit (ICU) stay. However, the definition of a prolonged postoperative ICU stay in cardiac surgery is controversial. It varies from greater than 24 hours to more than 14 days after ICU admission [1-6]. Nevertheless, a prolonged ICU stay is related to operative and postoperative problems, stands as a surrogate for morbidity [7], and is actually a strong predictor of mortality, as experienced in the praxis [4,5,7-9]. The association of prolonged ICU stay and mortality was merely investigated but never considered in intensive care risk stratification models. To the best of our knowledge, the ICU length of stay (ICULOS) was never included as a variable in an additive scoring system, in spite of its predictive power.

When our group published the ‘additive CArdiac SUrgery Score’ (additive CASUS) (Table 1) in 2005, by combining descriptors of mortality and multiorgan dysfunction, the importance of ICULOS was disregarded [10]. The ‘additive CASUS’ was evaluated in 3230 patients, recently validated in a population of 6007 patients [11], and showed high accuracy in other independent patient subsets [12-14].

Table 1

Additive and ‘modified CASUS’.

		Organ system	Descriptor	Score points
				0	1	2	3	4
Additive CASUS	Modified CASUS	Respiratory	PaO2 /FiO2 (mmHg/%)	Extubated	>250	151–250	75–150	<75
		Renal	Creatinine (mg/dl)	<1.2	1.2–2.2	2.3–4.0	4.1–5.5	>5.5
			CVVH/dialysis	No				Yes
		Liver	Bilirubin (mg/dl)	<1.2	1.2–3.5	3.6–7.0	7.1–14.0	>14.0
		Cardiovascular	PAR=HR×CVP/MAP	<10.1	10.1–15.0	15.1–20.0	20.1–30.0	>30.0
			Lactic acid (mmol/l)	<2.1	2.1–4.0	4.1–8.0	8.1–12.0	>12.0
			IABP	No				Yes
			VAD	No				Yes
		Coagulation	Platelets ×10≥/μL	> 120	81–120	51–80	21–50	<21
		Central nervous	Neurologic state	Normal		Confused	Sedated	Diffuse neuropathy
		ICU-day/ICULOS		1 or 2	3–5	6–8	9–12	≥13

CVVH – continuous venovenous hemofiltration; FiO2 – fraction of inspired oxygen; IABP – intraaortic balloon pump; ICU – intensive care unit; ICULOS – ICU length of stay; PAR – pressure-adjusted heart rate (=HR: heart rate × CVP: central venous pressure/MAP: mean arterial blood pressure); PaO2 – partial oxygen pressure; VAD – ventricular assist device.

In 2012, we published the ‘logistic CASUS’ (Table 2) based on a logistic formula that calculates mortality prediction in percentage [15]. Different predictors of organ dysfunction gained a specific weight by their β-coefficients. This score is not only the first logistic model for cardiac surgical ICU patients, but is also the first system that integrates the ‘ICU-day’ as a variable [15].

Table 2

‘Logistic CASUS’.

Descriptor		β-coefficient
PaO2/FiO2 (mmHg/%)		−0.0013498
Serum creatinine (mg/dl)		0.3063634
CVVH/dialysis	No	0
	Yes	0.4109223
Serum bilirubin (mg/dl)		0.2214784
PAR=HR×CVP/MAP		0.0642751
Lactic acid (mmol/l)		0.2115760
IABP	No	0
	Yes	0.6770239
VAD	No	0
	Yes	2.2976240
Platelets ×10≥/μL		−0.0042251
Neurologic state	Normal	0
	Confused	0.4736275
	Sedated	0.7020852
	Dif. neuropath.	1.4715528
ICU-day	1	0
	2	0.0097085
	3	0.8372058
	4	1.0426010
	5	1.2561380
	6	1.4975238
	7	1.6260023
	8	1.9023001
	9	2.1064412
	10	2.2268852
	11	2.2597632
	12	2.3782868
	13–∞	2.5289064
Constant		−5.6412079

CVVH – continuous venovenous hemofiltration; Dif. neuropath. – diffuse neuropathy; FiO2 – fraction of inspired oxygen; IABP – intraaortic balloon pump; ICU – intensive care unit; PAR – pressure-adjusted heart rate (=HR: heart rate × CVP: central venous pressure/MAP: mean arterial blood pressure); PaO2 – partial oxygen pressure; VAD – ventricular assist device.

In this study, 2 questions were addressed: What is the best way to integrate the ICULOS into a scoring system? Does the integration of ICULOS improve mortality prediction? To answer these questions, we devised the ‘modified CASUS’ (Table 1) by integrating ‘ICU-day’ as a new variable into our additive model. Then, we statistically compared the ‘additive CASUS’, the ‘modified CASUS’, and the ‘logistic CASUS’ in a large cardiac surgery subset of patients.

Material and Methods

General information

The study is based on an evaluation of prospectively collected data of all consecutive adult patients admitted to our ICU after cardiac surgery between January 1st 2007 and December 31st 2010. It was approved by the Institutional Review Board of our university (approval number: 2809-05/10). We considered only the first admission for patients who were readmitted to the ICU during the study period. Outcome was defined as ICU mortality. Please find further information on the data collection in our previous study [15].

Score calculation of the ‘modified CASUS’

We calculated the ‘modified CASUS’ by adding ICULOS as an additional variable to the ‘additive CASUS’ to assess prolonged ICU stay in an individual patient. The new score (‘modified CASUS’) contains, accordingly, 11 variables. The total number of points ranges from 0 to 44.

The new variable (ICULOS) was divided into 5 categories. Each category was affiliated with points. During the first 48 hours, the points remain 0 for each patient, but the maximum number of points is 4 in case an ICU stay exceeds 12 days. Table 1 demonstrates the additive and the ‘modified CASUS’.

Statistical analyses

Discrimination was evaluated with receiver operating characteristic curves (ROC). The area under the curve (AUC) was used to indicate the discriminative ability of the parameters. For the comparison of the new ‘modified CASUS’ with the additive and the ‘logistic CASUS’, we used the method of DeLong [16]. Calibration was analyzed according to the observed versus expected ratio (O/E ratio). An O/E ratio of 1.0 indicates perfect calibration, while an O/E ratio <1 or >1 implies overestimation and underestimation, respectively. Discrimination statistics focus on an individual patient’s outcome, whereas calibration assesses the general population level. A value of p<0.05 was defined as significant. Statistical analyses were performed from ICU day 1 (operative day) (n=5207) to day 13 (n=353). Please find detailed information on the applied statistical methods in our previous study [15].

Results

Population characteristics

This study included 5207 ICU patients admitted over a period of 4 years; 37.6% were female and the mean age was 67.2±10.9 years. The types of surgical procedures are shown in Table 3. The mean length of ICU stay was 4.6±7.0 days. ICU mortality was 5.9% (n=305). The preoperative mean additive EuroSCORE was 6.3±3.7 and mean logistic EuroSCORE was 9.9±12.9. No sex-based differences were present. There were no missing data in this study.

Table 3

Operations during the study period.

Operations	Number	%
CABG	2755	52.9
Isolated valve surgery	1187	22.8
Combined CABG and valve surgery	692	13.3
Aortic surgery	323	6.2
Cardiac transplantation	52	1.0
Others (including congenital heart defects, tumors, pulmonary embolectomy, assist device implantation)	198	3.8
Total	5207	100.0

CABG – coronary artery bypass grafting.

Results of the statistical analyses

Table 4 summarizes the discriminatory power, the comparison of the 3 models’ AUC curves by DeLong’s method, and the calibration of the models. The 3 scores had good discrimination on all days, with AUCs of ≥0.838 for the additive and ‘modified CASUS’, and ≥0.855 for the ‘logistic CASUS’. The best results of all scores were on day 2 (0.957). The additive and ‘modified CASUS’ demonstrated the smallest AUCs on day 12, whereas the ‘logistic CASUS’ showed the smallest AUC on day 10. In general, the ‘logistic CASUS’ showed better AUC curves compared to the other 2 models, throughout the statistical analysis.

Table 4

Statistical results.

ICU-day (patients)	Scoring model	ROC-analysis		De Long	O/E ratio
		AUC	95%-CI	p-value
1 (5207)	Add_CASUS	0.905	0.887–0.924	1.0000	0.6207
	Log_CASUS	0.908	0.888–0.928	0.5963	1.0011
	Mod_CASUS	0.905	0.887–0.924		0.6211
2 (5159)	Add_CASUS	0.957	0.938–0.976	1.0000	0.6118
	Log_CASUS	0.957	0.938–0.977	0.9934	0.9924
	Mod_CASUS	0.957	0.938–0.976		0.6109
3 (2372)	Add_CASUS	0.935	0.920–0.950	1.0000	0.7888
	Log_CASUS	0.939	0.928–0.949	0.6460	0.9976
	Mod_CASUS	0.935	0.925–0.946		0.8264
4 (1612)	Add_CASUS	0.912	0.904–0.920	1.0000	0.8289
	Log_CASUS	0.921	0.907–0.936	0.2691	0.9984
	Mod_CASUS	0.912	0.902–0.921		0.8419
5 (1165)	Add_CASUS	0.893	0.886–0.901	1.0000	0.8691
	Log_CASUS	0.911	0.905–0.918	0.0005	0.9986
	Mod_CASUS	0.893	0.886–0.901		0.8967
6 (888)	Add_CASUS	0.888	0.883–0.894	1.0000	0.9277
	Log_CASUS	0.902	0.896–0.908	0.0039	1.0015
	Mod_CASUS	0.888	0.881–0.896		0.9416
7 (713)	Add_CASUS	0.878	0.874–0.882	1.0000	0.9688
	Log_CASUS	0.892	0.886–0.898	0.0018	0.9989
	Mod_CASUS	0.878	0.871–0.885		0.9883
8 (601)	Add_CASUS	0.857	0.853–0.862	1.0000	1.0232
	Log_CASUS	0.881	0.875–0.886	<.0001	1.0016
	Mod_CASUS	0.857	0.851–0.864		0.9967
9 (506)	Add_CASUS	0.856	0.851–0.860	1.0000	1.0897
	Log_CASUS	0.871	0.868–0.875	<.0001	0.9989
	Mod_CASUS	0.856	0.850–0.862		1.0008
10 (448)	Add_CASUS	0.846	0.841–0.859	1.0000	1.1113
	Log_CASUS	0.855	0.852–0.858	0.0072	1.0007
	Mod_CASUS	0.846	0.840–0.852		1.0089
11 (415)	Add_CASUS	0.852	0.847–0.856	1.0000	1.1146
	Log_CASUS	0.866	0.863–0.869	<.0001	1.0018
	Mod_CASUS	0.852	0.846–0.858		1.0123
12 (379)	Add_CASUS	0.838	0.834–0.842	1.0000	1.1621
	Log_CASUS	0.857	0.854–0.860	<.0001	1.0010
	Mod_CASUS	0.838	0.832–0.844		1.0212
13 (353)	Add_CASUS	0.856	0.851–0.861	1.0000	1.2015
	Log_CASUS	0.878	0.875–0.881	<.0001	1.0022
	Mod_CASUS	0.856	0.851–0.861		1.0322

Add.: additive; AUC – area under curve; CI – confidential interval; ICU – intensive care unit; Log. – logistic; Mod. – modified; O/E – observed/expected; p-value – comparison of the new modified CASUS with the additive and the logistic CASUS according to DeLong’s method; ROC – receiver operating characteristic.

For the ICU days 1 up to 4, the differences between AUCs were not significant. However, the DeLong analysis showed a significant p-value in favor of the logistic model from day 5 up to day 13. The AUCs of the additive and ‘modified CASUS’ were identical on all days and therefore had p-value=1.000 according to DeLong’s method.

The calibration of the logistic model was good on all days and better than the additive and ‘modified CASUS’, especially during the early postoperative period (Figure 1). Both scores overestimated the mortality on day 1 up to day 5 (O/E <0.9). From day 6 up to day 9, the additive and ‘modified CASUS’ calibrated well (O/E 0.9–1.1). We detected a positive effect by integrating ‘ICU-day’ as a new variable in the ‘modified CASUS’ because its calibration remained appropriate (O/E <1.1) from day 10 up to day 13. The ‘additive CASUS’ tended to underestimate the risk of mortality from day 10 to day 13 (O/E >1.1) (Figure 1).

Figure 1

Calibration of the 3 CASUS models. ICU – intensive care unit; O/E – observed/expected.

Figure 2 shows the patients’ distribution to survivors and non-survivors. The 5 groups were arranged in a manner similar to the ‘modified CASUS’ categories according to the length of ICU stay. It can be clearly seen that the portion of non-survivors increased the longer an ICU stay persisted: 2.2% for patients discharged within 48 hours versus 22.7% for those admitted for longer than 12 days.

Figure 2

Survivors and non-survivors according to length of ICU stay. ICU – intensive care unit; n – number of patients.

Discussion

Definition and incidence of prolonged ICU stay

The definition of a prolonged ICU stay in cardiac surgery is controversial. Uncomplicated surgery and uneventful ICU stay allow an ICU discharge on the second day, when the pathophysiological sequels of the cardiac operation fade away [4-7]. This is in conformity to our discharge policy (Table 4), which points to a prolonged ICU stay in 45.6% of our patients. Nevertheless, some studies define prolonged ICU stay as from 7 up to 14 days after cardiac surgery [1-3].

Lagercrantz et al. defined prolonged ICU stay as more than 10 days, and consequently reported it for 3.5% of their study population [17]. In contrast to this, Silberman et al. set the cutoff at 48 hours and included 73% of their patients in the prolonged ICU stay group [7]. Accordingly, the incidence of prolonged ICU stay after cardiac surgery diverges widely depending on its definition [1-5,7-9,17-27].

Risk factors for a prolonged ICU stay

During recent years, a large number of pre-, intra-, and postoperative risk factors for prolonged ICU stay were identified. In a study of 10759 patients, Bucerius et al. postulated that preoperative predictors of prolonged ICU stay were: cardiac, pulmonary, and renal failure; advanced age; diabetes; hypertension; peripheral and cerebrovascular disease; and atrial fibrillation [4,5,7,8,17-19,26,28].

The urgency, duration, and complexity of cardiac surgery and a prolonged cardio-pulmonary-bypass time were identified as intraoperative risk factors for an extended ICU admission [4,5,7,8,18,20].

According to Hein et al., postoperative cardiac failure is an independent risk factor for a prolonged ICU stay [19]. Moreover, low ejection fraction was postulated as a risk factor by several authors [7,18,19,26]. This status is often accompanied by the need for high-dose catecholamine therapy and intra-aortic balloon pump, which are both risk factors themselves [4,8,18-20]. In addition to respiratory failure [4,7,19,20,28], renal insufficiency was also identified as a risk factor [4,5,7,8,17,19,20]. Other strong postoperative predictors are blood transfusion, re-exploration [4,18], stroke, and sepsis [7]. Benetis et al. postulated the readmission to ICU as a cofactor of significantly higher mortality after cardiac surgery [29]. Furthermore, the rare complications of deep sternal wound infection and post-sternotomy mediastinitis might lead to a prolonged ICU stay and end in increased mortality [30]. Finally, a postoperative peripheral cytopenia in patients after orthotropic heart transplantation might extend the duration of intensive care unit stay [31].

Influence of prolonged ICU stay on short-term survival

Patients that require a prolonged ICU stay have a higher short-term mortality [3-5,7-9,18], due to cardiac reasons [7,17] or multiorgan dysfunction [2,17,21-24]. Heimrath et al. reported a short-term mortality after prolonged ICU stay of 10%, which was significantly higher than the 1.2% for patients after non-prolonged ICU stay [5]. Hein et al. observed a significantly increased ICU mortality in patients with an ICU stay longer than 3 days in comparison to patients with ICULOS <3 days [19]. Hassan et al. postulated prolonged ICU stay as a predictor of short-term mortality, with an odds ratio of 20.9 [8]. Mahesh et al. reported that patients who required a prolonged ICU stay had a significantly higher ICU mortality compared to the control group (10% versus 0.6%, respectively; p-value <0.001) [4]. Soppa et al. subdivided patients with prolonged ICU stay into 2 groups (group A: 5–10 days; group B: >10 days). ICU mortality was 13% for group A compared to 20% for group B [9]. In 2013, Silberman et al. compared mortality rates according to ICULOS after cardiac surgery [7]. They distributed 6385 patients into 3 groups (group 1: <2 days; group 2: 3–14 days; group 3: >14 days). The early postoperative mortality rates were significantly different (2%, 8%, and 40%, respectively).

The incidence of short-term mortality diverges widely depending on the definition of prolonged ICU stay [3,18,28]. In Figure 2 we display the correlation of increased mortality and long ICU stay in our patient population.

‘ICU-day’ as a variable in an additive scoring system

According to several authors, the ICULOS is a strong predictor of mortality [4,5,7-9]. However, the ICULOS has never been included as a variable in an additive scoring system.

The general approach to assess the influence of a long ICU stay on mortality is by dividing a population into groups. Although there is no consistent definition of prolonged ICU stay, this division is based on 1 or several cut-off points [1-5,7-9,17-27]. This has the potential risk of wrongly converting the continuous nature of a prolonged ICU stay into a dichotomous or categorical unit and might therefore lead to inaccurate mortality prediction.

By integrating ‘ICU-day’ as a categorical variable into an additive model, we created the ‘modified CASUS’. Due to the new variable, we identified a more accurate mortality prediction in calibration statistics on patients with a long ICU stay. We failed to demonstrate an improved outcome prediction on the level of an individual patient, since the discrimination of additive and ‘modified CASUS’ was identical.

Despite its high influence on mortality, the lack of improved individual outcome prediction might be why ICULOS was never considered as a variable in an additive scoring system.

‘ICU-day’ as a variable of a logistic model

In accordance with our results, Silberman et al. postulated a proportional increase of negative outcome as a consequence of prolonged ICU stay [7]. Especially among elective surgical patients, this effect arises not immediately after cardiac surgery, but after some days, when organ dysfunctions evolve [32].

‘ICU-day’ is an established variable of the ‘logistic CASUS’ [15]. The score’s logistic character maintains the continuous nature of a prolonged ICU stay and its progressive influence on mortality. Table 2 captures the increasing weight of the ‘ICU-day’ variable by demonstrating the parameter’s β-coefficients, which remain consistent after the 13th postoperative day (Figure 3) [15].

Figure 3

Variable ‘ICU-day’ in ‘logistic CASUS’: course of β-coefficients from day 1 to day 25. ICU – intensive care unit.

During the study’s conception phase, we calculated the ‘logistic CASUS’ with and without the ‘ICU-day’ variable to detect its particular effect on mortality prediction. This approach was later dismissed because it necessitated the total recalculation of the score’s β-coefficients, which affects not only the ‘ICU-day’ variable, but also the other 10 parameters and the recalculated β-coefficients would have generated misleading results.

The ‘logistic CASUS’ showed a significantly improved accuracy in mortality prediction as from day 5, which demonstrates the importance of the ‘ICU-day’ variable during a long ICU stay. The improved accuracy applies to the levels of the individual patient and the general population. Thus, the ‘logistic CASUS’ is superior to the other 2 models.

Conclusions

We detected a significant influence of prolonged ICU stay on adverse outcome. This effect arises on the fifth day. To significantly improve mortality prediction, the length of ICU stay should be considered and ICULOS should be included as a variable in postoperative scoring systems. The significant improvement of mortality prediction for an individual patient was observed in a logistic model, but not in an additive ICU scoring system. We therefore recommend the integration of ‘ICU-day’ into a logistic score. In this regard, additive systems might be replaced by logistic models, which could gain the user’s attention.