Validation of EuroSCORE II risk model for coronary artery bypass surgery in high-risk patients.

INTRODUCTION: Determining operative mortality risk is mandatory for adult cardiac surgery. Patients should be informed about the operative risk before surgery. There are some risk scoring systems that compare and standardize the results of the operations. These scoring systems needed to be updated recently, which resulted in the development of EuroSCORE II. In this study, we aimed to validate EuroSCORE II by comparing it with the original EuroSCORE risk scoring system in a group of high-risk octogenarian patients who underwent coronary artery bypass grafting (CABG).
MATERIAL AND METHODS: The present study included only high-risk octogenarian patients who underwent isolated coronary artery bypass grafting in our center between January 2000 and January 2010. Redo procedures and concomitant procedures were excluded. We compared observed mortality with expected mortality predicted by EuroSCORE (logistic) and EuroSCORE II scoring systems.
RESULTS: We considered 105 CABG operations performed in octogenarian patients between January 2000 and January 2010. The mean age of the patients was 81.43 ± 2.21 years (80-89 years). Thirty-nine (37.1%) of them were female. The two scales showed good discriminative capacity in the global patient sample, with the AUC (area under the curve) being higher for EuroSCORE II (AUC 0.772, 95% CI: 0.673-0.872). The goodness of fit was good for both scales.
CONCLUSIONS: We conclude that EuroSCORE II has better AUC (area under the ROC curve) compared to the original EuroSCORE, but both scales showed good discriminative capacity and goodness of fit in octogenarian patients undergoing isolated coronary artery bypass grafting.

Introduction

Determining operative mortality risk is mandatory for all cardiac operations. Patients have to be informed preoperatively about the risk factors. Some risk scoring systems are used to compare and standardize the results of the operations. The European System for Cardiac Operation Risk Evaluation (EuroSCORE) is a risk model described in 1999 [1]. For more than a decade, this risk model has been used widely and validated innumerable times, demonstrating wonderful goodness of fit [2, 3]. Although there are many risk models used globally, risk scoring systems are relatively outdated. Therefore, the update of scoring systems was required, so EuroSCORE II was published on May 2010 [4]. EuroSCORE II has demonstrated a discriminative capacity similar to EuroSCORE (AUCEuroSCORE II = 0.81 vs. AUCEuroSCORE = 0.78), and good calibration (x2 HL [EuroSCORE II] = 15.48; p = 0.0505) [5].

In this study, we aimed to validate EuroSCORE II in comparison with the original EuroSCORE in a group of octogenarian patients with high preoperative risk who underwent isolated coronary artery bypass grafting (CABG).

Material and methods

In this study we included only octogenarian high-risk patients who underwent CABG from January 2000 to January 2010. Redo and concomitant procedures were excluded. Patients’ data were collected and analyzed retrospectively. Cardiovascular risk score of all patients was calculated by additive and logistic EuroSCORE and EuroSCORE II according to the criteria described by the EuroSCORE taskforce [6]. Patients were classified in three groups by additive EuroSCORE. All patients had a minimum score of 5 due to their age. So patients with an additive EuroSCORE of 5 to 8 were considered to have low risk, 8 to 10 moderate risk and higher than 10 high risk. We compared the observed mortality with the expected mortality according to logistic EuroSCORE and EuroSCORE II, which was calculated online [7].

Patient characteristics are shown in Table I. Additive EuroSCORE, logistic EuroSCORE and EuroSCORE II models were compared based on sensitivity and specificity. Sensitivity and specificity were assessed by receiver operating characteristic (ROC) analysis and the calibration of both scales was assessed by the Hosmer-Lemeshow (HL) test. Calibration was considered to be poor if the HL test was significant. The discrimination measures the capacity of a model (in this case additive and logistic EuroSCORE and EuroSCORE II) to differentiate the individuals of a sample who suffer an event (in this case death) and those who do not. The discriminative capacity of the three scales was estimated by means of ROC curves [8]. For the statistical analysis, the Statistical Package for Social Sciences (SPSS) version 15.0 (SPSS, Inc., Chicago, IL, USA) for Windows was used. A p-value < 0.05 was considered significant.

Tab. I

Patient characteristics

EuroSCORE II	EuroSCORE
Patient related factors
Age: 81.43 ± 2.21 (80-89)	Age: 81.43 ± 2.21 (80-89)
Female: 39 (37.1%)	Female: 39 (37.1%)
Peripheral arteriopathy: 7 (6.6%)	Peripheral arteriopathy: 7 (6.6%)
COPD: 20 (19.0%)	COPD: 20 (19.0%)
Diabetes on insulin: 20 (19.%)
Poor mobility: 4 (3.8%)	Neurological dysfunction: 9 (8.6%)
Renal impairment	Cr > 200 µmol/l: 17 (16.2%)
Dialysis: 11 (10.5%)
CC < 50: 5 (4.7%)
CC > 50: 1 (0.95%)
Cardiac related factors
Active endocarditis: 0	Active endocarditis: 0
Recent AMI: 17 (16.1%)	Recent AMI: 17 (16.1%)
NYHA class
II: 63 (60%)
III: 34 (32.4%)
IV: 8 (7.6%)
CCS4: 6 (5.7%)	Unstable angina: 17 (16.1%)
LVEF (%)
> 50: 58 (52.5%)	> 50: 58 (52.5%)
31-50: 39 (37.2)	31-50: 39 (37.2)
21-30: 7 (6.7%)	< 30: 8 (7.6%)
< 20: 1 (0.95%)
Pulmonary artery pressure	Pulmonary artery pressure > 60 mmHg: 2 (1.9%)
31-55 mmHg: 20 (18.6%)
> 55 mmHg: 3 (2.8%)
Procedure
Critical condition: 8 (7.6%)	Critical condition: 8 (7.6%)
Re-operation: 0
Thoracic aorta: 0
Emergency
Emergency: 4 (3.5%)	Emergency: 5 (4.8%)
Emergent Salvage: 1 (0.95%)
Weight of procedure
CABG: 105 (100%)	Surgery other than isolated CABG: 0
	VSD post AMI: 0

COPD – chronic obstructive pulmonary disease, AMI – acute myocardial infarction, LVEF – left ventricular ejection fraction, CABG – coronary artery bypass grafting, VSD – ventricular septal defect

Results

We considered 105 CABG operations on high-risk octogenarian patients for this study from January 2000 to January 2010. The mean (standard deviation; SD) age of the patients was 81.43 ± 2.21 (range: 80-89) years; 39 (37.1%) of them were female.

The two scales showed good discriminative capacity in the global patient sample, with the area under the ROC curve (AUC) being higher for EuroSCORE II (0.772, 95% CI: 0.673-0.872) (Fig. 1). The goodness of fit was good for both scales (Table II). In the low-risk subgroup all scales had good discriminative capacity with EuroSCORE II still being better than others (AUC: 0.774; 0.776; 0.816). However, in the moderate- and high-risk subgroups all scales showed poor discriminative capacity (Figs. 2–4).

Fig. 1

ROC curves for all patients

Tab. II

Hosmer-Lemeshow test for EuroSCORE II

Step	χ2	df	Sig.
1	1.995	7	0.960

Fig. 2

ROC curves for low-risk patients

Fig. 4

ROC curves for high-risk patients

ROC curves for moderate-risk patients

Benchmarking of our institutional mortality rates revealed worse prediction upon EuroSCORE II scoring compared to EuroSCORE (Fig. 5).

Fig. 5

Observed, EuroSCORE and EuroSCORE II expected mortality

Discussion

The development of EuroSCORE II eliminated insufficiencies observed in EuroSCORE such as low prevalence of octogenarians and valve surgery. Additionally, due to the progress in cardiac surgery, the impact of renal function on mortality decreased. Finally, EuroSCORE II was capable of predicting hospital mortality after major cardiac surgery with an excellent discriminative capacity (AUC = 0.81, 95% CI: 0.78-0.83) [5]. Alcazar et al. validated EuroSCORE II on 3798 patients, concluding that EuroSCORE II was a good discriminative method but with poor calibration [9]. Nashef et al. also advocated this conclusion with 5553 cases [5]. Howell and colleagues reported EuroSCORE II to be a model with poor calibration (p = 0.035) and original EuroSCORE to have a statistically significantly better model fit (the difference in AIC was –5.66; p = 0.017) in high-risk patients [10].

By applying both logistic models on the whole group, no statistically significant differences were observed comparing AUCEuroSCORE and AUCEuroSCORE II (Fig. 1). We compared the patients grouped according to additive EuroSCORE, and finally neither model did well, with statistically insignificant AUC results (Figs. 2–4). But our subgroups were statistically different and the numbers were small. On the other hand, when ROC analysis was applied to the whole study group, both models did well (Fig. 1), and also we observed that EuroSCORE II had better discriminative values. Parallel to our results, Chalmers et al. validated EuroSCORE II with 5576 subjects and concluded that EuroSCORE II has good discriminative capacity and good calibration (C-statistic 0.87 and HL p = 0.6) [11]. Also Akgul et al. reported a good C-statistic value of EuroSCORE II compared to the original EuroSCORE (0.992 [95% CI: 0.977-0.998] for logistic EuroSCORE and 0.990 [95% CI: 0.975-0.997] for EuroSCORE II) and in the subgroup of high risk (additive EuroSCORE > 6) they found that again EuroSCORE II was better (0.857 [95% CI: 0.691-0.954] for logistic EuroSCORE and 0.961 [95% CI: 0.829-0.998] for EuroSCORE II) [12].

In our study, we observed that the original EuroSCORE overestimates compared to EuroSCORE II, but we had high mortality rates compared to STS (Society of Thoracic Surgeons) results (20% and 6.8% respectively) and both risk models (Fig. 1). Chalmers et al. claim that EuroSCORE II has better calibration for cumulative sum survival (CUSUM) curves [11]. In the medical literature, there are papers supporting the results of the original EuroSCORE for the Turkish population, but no study specifically analyzed high-risk patients [13-16]. At this point EuroSCORE II needs to be validated in more cases nationally and subgroups of low prevalence and high-risk patients.

This study was conducted in a single center with multi-surgeon operations. Analysis of a single institution's results has limitations and may not represent national and international practice and outcomes. Also the study was designed to collect data retrospectively, and was conducted on a small population with particular properties.

Conclusions

We consider that EuroSCORE II has a better AUC (area under the ROC curve) compared to the original EuroSCORE but both scales showed good discriminative capacity and goodness of fit on octogenarian patients undergoing isolated coronary artery bypass grafting.