Use of EuroSCORE as a predictor of morbidity after cardiac surgery.

OBJECTIVE: To evaluate the use of the EuroSCORE as a predictor of postoperative morbidity after cardiac surgery.
METHODS: We retrospectively analyzed the charts of 900 patients operated on and admitted to the intensive care unit postoperatively at the Royal Portuguese Hospital of Recife. We included all patients with complete medical records, excluding those who died during surgery, underwent transplantation or correction of congenital heart disease. We evaluated the development of respiratory infection, cerebrovascular accident, and dialysis-dependent renal failure, and the EuroSCORE was compared in terms of the three complications using the Mann-Whitney test. The calibration model for predicting the morbidities being studied was evaluated using the test set of Homer-Lemeshow goodness. The accuracy of the model was assessed using the area under the ROC curve (AUROC).
RESULTS: The model showed good calibration in predicting respiratory infection, acute renal failure and stroke (P=0.285, P=0.789, P=0.45, respectively), with good accuracy for respiratory infection (AUROC=0.710 and P<0.001) and dialysis-dependent renal failure (AUROC=0.834 and P<0.001), but no accuracy to predict stroke (AUROC=0.519). The high-risk patients were more likely to develop respiratory infection (OR=9.05, P<0.001) and dialysis-dependent renal failure (OR=39.6, P<0.001). The probability of developing respiratory infection and dialysis-dependent renal failure was less than 10% with EuroSCORE up to 7 and more than 70% with EuroSCORE greater than 15.
CONCLUSION: EuroSCORE proved to be a good predictor of major postoperative morbidity in cardiac surgery: respiratory and dialysis-dependent renal failure.

INTRODUCTION

Risk stratification has become more relevant in cardiac surgery practice with the use of specific scores, which are important tools to measure risk, analyze quality of care, and evaluate costs [. Thus, several scores have been developed and applied to predict mortality in cardiac surgery [. Calculating surgical risk, i.e., death, is relatively simple since the response variable "death" is an obvious excluding variable. However, studying causes of death is often a lot more complex as there are multiple variables, making it difficult to develop specific scores to predict mortality. In addition, the relationship between the development of a complication and death does not always hold true; nonetheless, it can prolong hospital stay and change patients' quality of life [.

In cardiac surgery, when the following three major events are present, there is greater risk of death: onset of respiratory tract infection (RTI), preoperative cerebrovascular accident (CVA), and dialysis-dependent renal failure (DDRF) [. Besides being associated with higher mortality rates, those events are the leading cause of readmission to the intensive care unit, increasing hospital costs [.

Even though it is known that those adverse events can definitely contribute to unfavorable results in cardiac surgeries, there are no specific scores derived from major studies capable of predicting the chances of developing such complications and, hence, capable of predicting morbidity.

This study sets out to evaluate the EuroSCORE as a predictor of morbidity, since it is simple and practical, has a reduced number of variables, and is widely used throughout the world, having been validated several times, including in our midst, with good results [.

METHODS

This cross-sectional observational study was developed at the Thoracic Surgery Recovery Unit of the Royal Portuguese Hospital of Recife. Medical records of 900 out of 1036 patients operated on from July 1, 2008 to July 30, 2009 were analyzed. Patients with complete medical records were included in the study and patients who died during surgery or who had undergone either cardiac transplant or correction of congenital heart disease were excluded. Data was collected from electronic medical records and inserted into an Excel spreadsheet containing all the variables included in the study. A specific calculator was used to obtain the EuroSCORE, classifying patients into the following three risk groups, according to additive score values: high, medium, and low. Variables studied were EuroSCORE values, death, and development of RTI, DDRF or CVA, the occurrence of the latter being considered in the period between surgery and hospital discharge. RTI was diagnosed according to clinical and radiological criteria, and it was confirmed by quantitative culture of tracheal aspirate containing colony counts above one million. Patients who developed the need to have compulsory renal replacement therapy after surgery were considered as having DDRF. Patients with CVA should have their diagnosis confirmed by recent findings of brain damage as evidenced by non-contrast computed tomography scan performed 72 hours after suspicion of the event. Continuous variables were expressed as means and/or medians and standard deviation. Categorical variables were expressed as their relative and absolute frequencies.

The non-parametric Mann-Whitney test was applied to compare EuroSCORE in terms of death, CVA, RTI, and DDRF. Calibration of EuroSCORE was measured by the Hosmer-Lemeshow goodness of fit test for logistic regression models, in which response was hospital mortality or morbidities (CVA, RTI, and DDRF) and the independent variable was EuroSCORE. Accuracy was assessed by the area under the ROC curve (receiver operating characteristic curve), built based on sensitivity (correct prediction of death) and 1 - specificity (correct prediction of survival), which were calculated for every score value studied. The relationship between the risk groups classified by the score and the development of the aforementioned complications was measured by chi-square and Fischer's exact test, as indicated. P-values of P<0.05 were considered statistically significant to reject the null hypothesis. The Statistical Package for the Social Sciences 16.0 (SPSS) was the software used for analysis.

The project was approved by the Ethics Committee of the Royal Portuguese Hospital of Recife.

RESULTS

The sample studied consisted of 900 patients, with a mean age of 57.6 ± 13.9 years, ranging from 11 to 86 years old, and 518 (57.6%) being male. Mean EuroSCORE was 2.76 ± 2.27.

Most patients had undergone isolated coronary artery bypass grafting (67%), followed by isolated valve surgery (26%), and coronary artery bypass grafting associated with another cardiac surgery (4%) (Figure 1).

Fig.1

Characteristics of the sample according to the surgery performed

Tipo de cirurgia realizada = surgery performed

Valvular isolada = isolated valve

Cirurgia da Aorta = Aortic surgery

CRM isolada = isolated coronary artery bypass grafting

CRM associada = coronary artery bypass grafting associated with another cardiac surgery

Outras cirurgias = Another surgeries

Population distribution in the high, medium, and low EuroSCORE risk groups was 12.1%, 38.4%, and 49.4%, respectively.

Prevalence of complications was 4.8% and sample mortality was 4.1%. In addition, EuroSCORE values were higher among patients who died. Mortality in the group with complications was significantly higher than in the group without complications (Table 1).

Table 1

Incidence of complications and mortality.

Variable	n (%)
Comorbidities
RTI	30 (3.3%)
DDRF	17 (1.9%)
CVA	15 (1.7%)
Complications	43 (4.8%)
General mortality	37 (4.1%)
Mortality in the group without complications	1.8%
Mortality in the group with complications	47.7% *

P<0.0001 chi-square.

CVA = preoperative cerebrovascular accident; DDRF = dialysis-dependent renal failure; RTI = respiratory tract infection

Analysis of the predictive ability of EuroSCORE for morbidities

Respiratory tract infection

A comparison of score values revealed higher values for patients who developed RTI than those who did not, as shown in Table 2.

Table 2

Comparison of EuroSCORE according to RTI.

Death	N	Mean	Median	Standard Deviation	Minimum	Maximum
No	870	2.68	2.00	2.17	0	15
Yes	30	5.13	4.50	3.59	0	13
Total	900	2.76	3.00	2.27	0	15

P-value < 0.001 (Mann-Whitney test). RTI = respiratory tract infection

The predictive model for RTI showed good calibration, as presented in Table 3, and good accuracy, determined by analysis of the area under the ROC curve (Figure 2).

Table 3

RTI observed and predicted using EuroSCORE as predicting variable in the groups defined according to the Hosmer-Lemeshow test.

Contingency table for the Homer-Lemeshow test
		RTI = no		RTI = yes
		Observed	Expected	Observed	Expected	Total
	1	148	150.583	4	1.417	152
	2	171	169.740	1	2.260	172
	3	121	119.746	1	2.254	122
Step 1	4	144	144.162	4	3.838	148
	5	131	131.065	5	4.935	136
	6	107	107.205	7	6.795	114
	7	48	47.499	8	8.501	56

Chi-square (5) = 6.221 (P-value = 0.285). RTI = respiratory tract infection

Fig. 2

Comparison of EuroSCORE in patients with and without RTI

The area under the ROC curve was 0.710 (CL 95%, 0.600 - 0.821), with P-value<0.001. Results obtained show that EuroSCORE was good in discriminating between patients with and without RTI (Figure 3).

Fig. 3

Graph of the ROC curve for RTI

Dialysis-dependent renal failure

Just like RTI, higher EuroSCORE values were found among patients who developed DDRF than in those who did not develop DDRF, as shown in Table 4 and Figure 4.

Table 4

Comparison of EuroSCORE according to DDRF.

Death	N	Mean	Median	Standard Deviation	Minimum	Maximum
No	883	2.68	2.00	2.16	0	12
Yes	17	6.88	6.00	3.94	1	15
Total	900	2.76	3.00	2.27	0	15

P-value = 0.001 (Mann-Whitney test). DDRF = dialysis-dependent renal failure

Fig. 4

Comparative Box-plot of EuroSCORE according to DDRF

The predictive model for DDRF showed good calibration, as presented in Table 5, and excellent accuracy, determined by analysis of the area under the ROC curve (Figure 2).

Table 5

DDRF observed and predicted using EuroSCORE as predicting variable in the groups defined according to the Hosmer-Lemeshow test.

Contingency table for the Homer-Lemeshow test
		DDRF = no		DDRF = yes
		Observed	Expected	Observed	Expected	Total
	1	152	151.723	0	0.277	152
	2	171	171.467	1	0.533	172
	3	122	121.358	0	0.642	122
Step 1	4	146	146.681	2	1.319	148
	5	133	133.951	3	2.049	136
	6	110	110.209	4	3.791	114
	7	49	47.611	7	8.389	56

Chi-square (5) = 2.419 (P-value = 0.789). DDRF = dialysis dependent renal failure

The area under the ROC curve was 0.834 (CL 95%, 0.738 - 0.930), with P-value<0.001. Results show that EuroSCORE was good in discriminating between patients with and without DDRF (Figure 5).

Fig. 5

Graph of the ROC curve for DDRF

Cerebrovascular accident

With regard to occurrence of CVA, there was no difference in EuroSCORE values between patients who developed this complication and those who did not (P=0.484; Mann-Whitney test). In terms of the predictive model for this morbidity, despite good calibration (P=0.45), it was not accurate to discriminate between patients who did and did not develop CVA in the postoperative period (area under the ROC curve = 0.519), as shown in Figure 3.

Analysis of the relationship between the risk group and the chance of developing postoperative complications

It was observed that patients in the high-risk group, according to EuroSCORE values, had greater chances of developing RTI (OR: 9.05) and DDRF (OR: 39.96), as presented in Tables 6 and 7. However, the same association could not be said of the occurrence of CVA.

Table 6

Crossing of EuroSCORE with RTI

			RTI		Total	OR
			Não	Sim		(CL 95%)
	>Low	N	439	6	445	1.0
Risck		%	98.7%	1.3%	100.0%
Group	Medium	N	334	12	346	2.63
		%	96.5%	3.5%	100.0%	(0.98 - 7.08)
	High	N	97	12	109	9.05
		%	89.0%	11.0%	100.0%	(3.32 - 24.70)

P-value < 0.001 (chi-square). RTI = respiratory tract infection

Table 7

Crossing of EuroSCORE and DDRF.

			RTI		Total	OR
			No	Yes		(CL 95%)
	Low	N	444	1	445	1.0
Risck		%	99.8%	0.2%	100.0%
Group	Medium	N	339	7	346	9.17
		%	98.0%	2.0%	100.0%	(1.1 - 74.9)
	High	N	100	9	109	39.96
		%	91.7%	8.3%	100.0%	(5.0 - 78.9)

P< 0.001. DDRF = dialysis-dependent renal failure

Cut-off points for greater specificity and sensitivity for prediction of RTI and DDRF are listed in Table 8.

Table 8

Cut-off points and sensitivity and specificity values.

Variable	Cut-off points	Sensitivity	Specificity
RTI	≥ 3.5	66.7%	67.1%
DDRF	≥ 3.5	82.4%	66.9%

DDRF = dialysis-dependent renal failure; RTI = respiratory tract infection

The probability of developing RTI and DDRF based on EuroSCORE was determined, varying from less than 10% to more than 70% (Tables 9 and 10).

Table 9

Probability of developing RTI according to EuroSCORE.

EuroSCORE	RTI Probability
0 - 7	< 10%
8 - 9	10% - 19%
10	20 - 29%
11 - 12	30 - 39%
13	40 - 49%
14	50 - 59%
15	60 - 69%
>15	≥ 70%

RTI = respiratory tract infection

Table 10

Probability of developing DDRF according to EuroSCORE.

EuroSCORE	DDRF Probability
0 - 7	< 10%
8 - 9	10% - 19%
10	20 - 29%
11	30 - 39%
12	40 - 59%
13	60 - 69%
>14	≥ 70%

DDRF = dialysis-dependent renal failure

DISCUSSION

The use of risk scores in surgical practice is a valid strategy, not only to measure risk but also to analyze and compare results. Risk scores are related to factors inherent to the model itself, the population for which they were developed, and the characteristics of the population in which they will be applied. As a result, the majority of scores show conflicting results in the prediction of risks. In addition, the large number of variables of a score also reduces its effectiveness when applied to distinct populations [. On the other hand, scores that are validated in diverse populations, such as the EuroSCORE, tend to show better results.

The prevalence of complications (RTI, DDRF, and CVA) was similar to what has been described in the literature. Likewise, general mortality in the sample studied was also similar to previous results, drawing attention to the high mortality rate, an important fact that indicates the importance of both early identification of patients at risk for developing those complications and aggressive intervention to reverse this situation [.

Even though the prevalence of complications was similar to what has been described in the literature, the use of strict diagnostic criteria to characterize complications may have left patients who had only clinical diagnoses out of the RTI and CVA groups, which could lead to underdiagnosis of those complications.

By using the score to predict RTI, it was observed that patients with higher EuroSCORE values had greater chances of developing a complication, which could be considered an obvious outcome, but the score tool now presents numeric data. The model showed good calibration to that end, especially in patients with medium to high EuroSCORE values. The same findings were obtained when predicting the development of DDRF, with higher values (P=0.789).

In terms of discriminating power, assessed by the area under the ROC curve, the model was also deemed adequate, with predicting values for RTI and DDRF of 0.710 and 0.834, respectively, both statistically significant. In practice, these data translates into good ability to discriminate which patients will definitely develop complications among those with chances of developing them.

Similar to the model calibration, the values for predictive ability were significant for both RTI and DDRF. However, the results of the DDRF group were more robust, probably because of better characterization of the patients belonging to that group, and since diagnostic criteria is exclusive, i.e., whether or not the patient developed DDRF, there are no confounding variables in the diagnosis of the complication.

The opposite was observed when the score was used to predict CVA. The score was not successful, showing neither effective calibration nor effective predictive ability, probably due to the diagnostic method used: the computed tomography. Even though computed tomography was performed 72 hours after suspected clinical diagnosis, the scan might not show ischemic lesions and nuclear magnetic resonance is more accurate. However, due to the characteristics of the patients, such as hemodynamic instability at the time of the exam, the decision was made not to perform a longer exam that could put the patient at risk.

Finally, it is interesting to observe the exponential growth in the chance of developing RTI and DDRF as the EuroSCORE values increase, with sensitivity and specificity cut-off points starting at score 3.5. In patients with EuroSCORE values above this cut-off point, it allows for the possibility of implementing protective measures, directing human and material resources, and defining the best surgical strategy so that the occurrence of complications with potential mortality can be reduced.

The results and validations of the EuroSCORE II were published during this study [; however, we decided to continue using EuroSCORE I because of it has numerous validations, including in our midst, it is easy to use, and it is widely known to all professionals involved in the care of patients undergoing cardiac surgery, such as intensivists, surgeons, and cardiologists.

The publication of studies showing the lack of superiority of EuroSCORE II over EuroSCORE I [ supports the decision to use EuroSCORE I and emphasizes the need for constant recalibration of risk scores, especially when used in populations other than the ones they were developed for.

CONCLUSION

EuroSCORE proved to be a good predictor for major postoperative morbidities in cardiac surgery (RTI and DDRF). However, in this study, it could not predict the development of CVA. The chances of developing RTI and DDRF increase as additive EuroSCORE values increase, making it possible to identify high-risk patients and to start implementing preventive measures and early intervention.

Abbreviations, acronyms & symbols
AUROC	Area under the ROC curve
CVA	Cerebrovascular accident
EuroSCORE	European System for Cardiac Operative Risk Evaluation
DDRF	Dialysis-dependent renal failure
RTI	Respiratory tract infection
ROC	Receiver Operating Characteristic
SPSS	Statistical Package for the Social Sciences
ITU	Intensive therapy unit

Authors' roles & responsibilities
INGA	Main author
FRMN	Coordination of data collection, data and bibliographic referencesorganization
TGA	Tabulation and organization of data, partial drafting of the final text