Comparison of Prognostic Value Among 4 Risk Scores in Patients with Acute Coronary Syndrome: Findings from the Improving Care for Cardiovascular Disease in China-ACS (CCC-ACS) Project.

BACKGROUND Accurate risk assessment and prospective stratification are of great importance for treatment of acute coronary syndrome (ACS). However, the optimal risk evaluation systems for predicting different type of ACS adverse events in Chinese population have not been established. MATERIAL AND METHODS Our data were derived from the Improving Care for Cardiovascular Disease in China-ACS (CCC-ACS) Project, a multicenter registry program. We incorporated data on 44 750 patients in the study. We compared the performance of the following 4 different risk score systems with regard to prediction of in-hospital adverse events: the Global Registry for Acute Coronary Events (GRACE) risk score system; the age, creatinine and ejection fraction (ACEF) risk score system, and its modified version (AGEF), and the Canada Acute Coronary Syndrome (C-ACS) risk assessment system. RESULTS Admission AGEF risk score was a better prognosis index of potential for in-hospital mortality for patients with ST segment elevation myocardial infarction (STEMI) than GRACE risk score (AUC: 0.845 vs 0.819, P=0.012), ACEF (AUC: 0.845 vs 0.827, P=0.014), C-ACS (AUC: 0.845 vs 0.767, P<0.001). In patients with non-ST segment-elevation acute coronary syndrome (NSTE-ACS), there was no statistically significant difference between the GRACE risk scale and AGEF (AUC: 0.853 vs 0.832, P=0.140) for in-hospital death. CONCLUSIONS AGEF risk score showed a non-inferior utility compared with the other 3 scoring systems in estimating in-hospital mortality in ACS patients.

Background

Despite remarkable improvements in the treatment of acute coronary syndromes (ACS), the mortality rate is still poor, at 5–10% according to some reports [1-3]. The risk of further cardiovascular complications following ACS is substantial [4]. As such, ACS is a pivotal public health issue throughout the world. Therefore, accurate risk assessment and prospective stratification are of great importance for clinical management of ACS.

Clinical manifestations, electrocardiograms, biochemical analyses, and other quantifiable factors have been used to determine risk and management options for patients with ACS. A number of models and scores of varying degrees of complexity have been used in various studies to identify patients at high risk. Current European Society of Cardiology (ESC) clinical guidelines and the American College of Cardiology Foundation/American Heart Association (ACC/AHA) advocate use of Global Registry for Acute Coronary Events (GRACE) risk scores for risk assessment stratification [5-7]. Many studies have demonstrated the accuracy of GRACE risk scores for prediction of ACS-related mortality in hospital and during follow-up after discharge [8-10]. However, this risk model contains numerous independent variables, which limits its utility. Several simple cardiovascular risk scores have been proposed in recent years, including the age, creatinine, and ejection fraction (ACEF) risk score [11-13], AGEF, a modified version of the ACEF score [14,15], and Canada Acute Coronary Syndrome (C-ACS) score [16]. These simplified risk models eliminate “overfitting” of many independent variables. However, these risk scores were originally designed for different purposes. The ACEF was designed to predict in-hospital outcomes and the C-ACS was designed to predict longer-term outcomes, while the AGEF was designed to assess contrast-induced nephropathy. In clinical practice, the use of these scores is often generalized to other ailments, and they are often not used for their original purposes. Furthermore, GRACE, ACEF, AGEF, and C-ACS have not been compared in large patient cohorts in China.

This study aimed to compare the predictive and discriminatory abilities of these 4 risk scores with respect to in-hospital outcomes for ACS patients. Our findings in this study are built upon a collaboration between the American Heart Association (AHA) and Chinese Society of Cardiology (CSC): Improving CCC Project (Care for Cardiovascular Disease in China).

Material and Methods

Study Design and Population

Data from a multicenter registry project focusing on upgrading the quality of treatment and nursing for ACS patients were used in this study. The study setting and facilities strategy of the CCC project are provided at length in a previous publication [17]. In each hospital, the first 20–30 ACS inpatient cases in each month were consecutively recruited to this study. Clinical information was acquired using a standard data-gathering website (Oracle Clinical Remote Data Capture, Oracle). Patient characteristics, medical histories, symptoms on arrival, in-hospital treatments and procedures, discharge medications, and secondary prevention information were collected. During November 2014 and June 2017, 63 641 patients diagnosed as having ACS from 150 hospitals were enrolled in the project. Of these patients, 44 750 were incorporated in this research after excluding 18 891 (3.3%) patients due to lack available admission serum creatinine data, left ventricular ejection fraction (LVEF) data, and GRACE risk scores (Figure 1).

Figure 1

Flow diagram for the selection of the study population.

Definitions and Risk Scores

ST segment elevation myocardial infarction (STEMI), in line with the 2010 STEMI guideline [18], was defined as the existence of typical stethalgia and accompanying symptoms lasting ≥30 minutes but <12 hours. In addition, there had to be at least 2 contiguous leads with ST segment elevation ≥1 mm or a new or undetermined duration of left branch bundle block with a ≥2-fold increase in cardiac enzymes (troponin I or T). Non-ST segment elevation (NSTE)-ACS was determined on the basis of the primary discharge diagnosis of non-ST segment elevated myocardial infarction (NSTEMI) and unstable angina. Non-ST segment elevated myocardial infarction ACS was defined in line with the diagnostic and management guidelines published by the CSC [19]. The diagnostic criteria for unstable angina were as follows: (1) ischemic symptoms at rest or variant angina; new-onset (ie, within 1 month) angina; ischemic symptoms became more frequent, severe, or prolonged, or did not respond to nitroglycerin in recent months for patients with stable angina; (2) myocardial ischemia detected by electrocardiogram or other examination; (3) coronary artery stenosis ≥70% with a need for coronary intervention [17]. Hypertension was diagnosed when there was a high blood pressure history, taking antihypertensive medicine, and accompanied by systolic blood pressure (SBP) ≥140 mmHg, or diastolic blood pressure (DBP) ≥90 mmHg. Diabetes was defined as having a history of a diabetes, taking hypoglycemic agents during prior hospitalization, or glycated hemoglobin A1c concentration 6.5% and over at discharge.

GRACE risk scores consist of medical history, findings at hospital presentation, and findings during hospitalization. Components include age, heart rate, Killip class, SBP, cardiac arrest, ST segment deviation, serum creatinine, and cardiac biomarker status [10]. ACEF scores were estimated with the following equation, available in the publication in which the model was defined: Age/EF (%)+1 (if preoperative serum creatinine value >2.0 mg/dL) [11]. AGEF risk scores were estimated with the equation age/EF (%)+1 point for each 10 mL/min decreased in creatinine clearance (CrCl) below 60 mL/min/1.73 m2 (up to 6 points) [14]. LVEF was the ejection fraction value recorded before the index percutaneous coronary intervention (PCI). C-ACS risk scores were assigned based on whether the heart rate exceeded 100 beats per minute, age 75 years and older, systolic blood pressure lower than 100 mm Hg, or Killip grade II–IV [16]. If the answer is yes, 1 point was scored for each item.

To compare differences among the 4 risk scores, we classified patients into tertiles using the data collected in this study. Patients in tertiles I, II, and III were defined as low-, moderate-, and high-risk patient populations, respectively. The study endpoints were all-cause mortality and in-hospital major adverse clinical events (MACEs). Major adverse clinical events were set to any combined with cardiogenic death, recrudescent myocardial infarction, stent thrombogenesis, and apoplexy. In addition, in-hospital major bleeding [20] was also recorded, which was defined as hemorrhage in brain and retroperitoneum, a 4 g/dL and over reduction in hemoglobin levels, or hemorrhage requiring transfusion and surgical management.

Statistics Analysis

Continuous data are expressed as mean±standard deviations (SD) or median and quartile ranges. Categorical data are exhibited as counts and percentages. The area under the ROC curve (AUC) and 95% confidence intervals (CI) among different risk scores for predicting adverse events were calculated by using receiver an operating characteristic (ROC) model. SPSS software program version 19.0 (SPSS, Inc.; Chicago, IL, USA) for Windows was used for statistical processing in our study. Bilateral P<0.05 was deemed as significance in statistical analysis.

Results

Baseline Clinical Characteristics

After excluding 1948 patients with missing values for serum creatinine, 13 651 patients due to insufficient data for LVEF, and 3292 patients due to lack of GRACE risk scores, 44 750 patients were incorporated in this study, of whom 64.6% were diagnosed with STEMI and 35.4% presented with NSTE-ACS (NSTE-ACS). The average age of the sample population was 62.63±12.39 years, and 75.6% were males. The baseline clinical characteristics of the study population is provided in Table 1. PCI was performed in 10 149 (64.1%) patients in the NSTE-ACS group and 23 757 (82.1%) patients in the STEMI group. The majority of the patients underwent dual antiplatelet treatment with full anticoagulation. During hospitalization, death occurred in 468 patients (1.0%). Major adverse clinical events occurred in 1510 (9.5%) patients during the hospitalization period in the NSTE-ACS group and in 3079 (10.6%) patients in the STEMI group. Major bleeding happened in 846 (2.9%) patients in the STEMI group during hospitalization and 310 (2.0%) patients in the NSTE-ACS group.

Table 1

Baseline characteristics and complications of the hospital survivors with ACS.

Clinical variables	NSTE-ACS (n=15825)	STEMI (n=28925)
Age	64.9±11.9	61.4±12.5
Sex
Male	11063 (69.9%)	22759 (78.7%)
Female	4762 (30.1%)	6166 (21.3%)
Previous medical history
Smoke	5926 (37.4%)	14325 (49.5%)
Diabetes	4205 (26.6%)	5803 (20.1%)
Hypertension	9608 (60.7%)	14349 (49.6%)
Dyslipidemia	1917 (12.1%)	1911 (6.6%)
MI	1807 (11.4%)	1474 (5.1%)
PCI	2003 (12.7%)	1273 (4.4%)
CABG	141 (0.9%)	60 (0.2%)
Stroke	1797 (11.4%)	2620 (9.1%)
Atrial fibrillation	631 (4.0%)	431 (1.5%)
Cardiac arrest before admission	76 (0.5%)	399 (1.4%)
SBP	135.4±22.9	127.2±23.3
DBP	78.8±13.7	77.7±14.7
HR	76.6±15.8	78.0±16.3
Body weight	67.7±11.2	68.6±11.4
Killip ≥2	4751 (30.0%)	8121 (28.1%)
Serum creatinine, mg/dl	1.0±0.8	1.0±0.66
Hemoglobin, g/L	133.5±20.6	138.2±19.9
LVEF	57.6±10.2	53.6±10.0
Log hospital stays	10.5±5.8	11.0±5.7
In-hospital medication
Aspirin	14752 (93.2%)	28051 (97%)
Clopidogrel	13351 (84.4%)	22378 (77.4%)
Ticagrelor	1919 (12.1%)	76315 (26.3%)
β-blocker	9629 (608%)	15896 (55.0%)
ACEI/ARB	8188 (51.7%)	14158 (49.0%)
Statin	14884 (94.1%)	27572 (95.3%)
Glycoprotein IIb/IIIa inhibitor	2914 (18.4%)	11792 (40.8%)
C-ACS scores	0.64±0.78	0.63±0.79
ACEF scores	1.22±0.47	1.23±0.45
AGEF scores	1.34±0.61	1.31±0.57
GRACE scores	135.91±37.22	147.81±32.48
In-hospital events
Re-infarction	50 (0.3%)	109 (0.4%)
AHF	1360 (8.6%)	2746 (9.5%)
Stroke	107 (0.7%)	195 (0.7%)
Major bleeding	310 (2.0%)	846 (2.9%)
Any bleeding	472 (3.0%)	1171 (4.1%)
Death	137 (0.9%)	331 (1.1%)
MACEs	1510 (9.5%)	3079 (10.6%)

NSTE-ACS – non-ST segment-elevation acute coronary syndrome; STEMI – ST segment elevation myocardial infarction; MI – myocardial infarction; PCI – percutaneous transluminal coronary intervention; CABG – coronary artery bypass grafting; SBP – systolic blood pressure; DBP – diastolic blood pressure; HR – heart rate; LVEF – left ventricular ejection fraction; ACEI/ARB – angiotensin-converting enzyme inhibitor/angiotensin receptor blocker; C-ACS – Canada Acute Coronary Syndrome risk score; ACEF – the age, creatinine and ejection fraction risk score; AGEF – the modified version of ACEF risk score; GRACE – the Global Registry for Acute Coronary Events risk scores; AHF – acute heart failure; MACEs – major adverse clinical events.

Based on the 4 risk scores, the in-hospital death (%) rates for STEMI patients in the low-, moderate-, and high-risk groups were 0.3, 1.2, and 4.2, respectively, based on C-ACS risk score (P<0.001, Figure 2B); the mortality rates were 0.1, 0.5, and 2.8, respectively, based on ACEF risk score (P<0.001, Figure 2B); the mortality rates were 0.1, 0.4, and 2.7, respectively, based on AGEF risk score (P<0.001, Figure 2B); and the mortality rates were 0.2, 0.4, and 2.6, respectively, based on GRACE risk score (P<0.001, Figure 2B). Similar results were observed in NSTE-ACS patients (Figure 2A). Incidence of MACEs and major hemorrhage during hospitalization stratified by different risk scores are shown in Supplemenatry Figures 1 and 2, indicating that the incidences in high-risk patients were significant higher.

Figure 2

Rates of in-hospital death in the low-, moderate-, and high-risk groups, according to the GRACE, ACEF, AGEF, and C-ACS risk scores.

Risk Model Discrimination

Table 2 shows the ROC curve comparison for in-hospital adverse events. The predictive accuracies of the 4 risk scores are presented in Figure 3. In the NSTE-ACS group, the AUCs for in-hospital death were 0.853, 0.827, 0.832, and 0.766 for GRACE, ACEF, AGEF, and C-ACS risk score, respectively (Figure 3A). The C-ACS score exhibited the lowest predictive ability. AUC differences for C-ACS and ACEF, AGEF, and GRACE were 0.060, 0.066, 0.087, respectively. The abilities of GRACE, ACEF, and AGEF risk models to assess in-hospital deaths in the NSTE-ACS group were insignificant (Table 2).

Table 2

ROC Comparison for death, MACEs, bleeding, according to the C-ACS, ACEF, AGEF, and GRACE risk score.

		NST-ACS				STEMI
	Comparison (A vs B)	AUC-A%	AUC-B%	ΔAUC%	P value	AUC-A%	AUC-B%	ΔAUC%	P value
Death	C-ACS vs ACEF	76.6	82.7	6.1 (2.0 to 10.0)	0.003	76.7	82.7	6.0 (3.4 to 8.7)	<0.001
	C-ACS vs AGEF	76.6	83.2	6.6 (2.8 to 10.4)	<0.001	76.7	84.5	7.8 (5.4 to10.2)	<0.001
	C-ACS vs GRACE	76.6	85.3	8.7 (5.4 to 12.0)	<0.001	76.7	81.9	5.2 (3.5 to 7.1)	<0.001
	ACEF vs AGEF	82.7	83.2	0.5 (−1.3 to 2.4)	0.559	82.7	84.5	1.8 (0.3 to 3.1)	0.014
	ACEF vs GRACE	82.7	85.3	2.6 (−0.3 to 5.7)	0.082	82.7	81.9	0.8 (−1.5 to 3.1)	0.486
	AGEF vs GRACE	83.2	85.3	2.1 (−0.7 to 4.9)	0.140	84.5	81.9	2.6 (0.6 to 4.5)	0.012
MACEs	C-ACS vs ACEF	75.1	77.7	2.6 (1.2 to 4.0)	<0.001	71.1	71.9	0.8 (−0.4 to 1.8)	0.190
	C-ACS vs AGEF	75.1	77.6	2.5 (1.1 to 3.9)	<0.001	71.1	72.1	1.0 (−0.2 to 2.0)	0.094
	C-ACS vs GRACE	75.1	79.7	4.6 (3.5 to 5.6)	<0.001	71.1	73.0	1.9 (1.1 to 2.6)	<0.001
	ACEF vs AGEF	77.7	77.6	0.1 (−0.5 to 0.7)	0.669	71.9	72.1	0.2 (−0.2 to 0.6)	0.374
	ACEF vs GRACE	77.7	79.7	2.0 (0.7 to 3.1)	0.002	71.9	73.0	1.1 (0.2 to 2.0)	0.022
	AGEF vs GRACE	77.6	79.7	2.1 (0.9 to 3.2)	<0.001	72.1	73.0	0.9 (0.0 to 1.8)	0.049
Bleeding	C-ACS vs ACEF	57.8	64.1	6.3 (2.9 to 9.8)	<0.001	58.0	57.6	0.4 (−1.7 to 2.4)	0.744
	C-ACS vs AGEF	57.8	65.0	7.2 (3.9 to 10.6)	<0.001	58.0	58.2	0.2 (−1.8 to 2.3)	0.828
	C-ACS vs GRACE	57.8	64.6	6.8 (4.1 to 9.6)	<0.001	58.0	59.4	1.4 (−0.1 to 3.0)	0.062
	ACEF vs AGEF	64.1	65.0	0.9 (−0.5 to 2.3)	0.192	57.6	58.2	0.6 (−0.2 to 1.4)	0.149
	ACEF vs GRACE	64.1	64.6	0.5 (−2.3 to 3.3)	0.724	57.6	59.4	1.8 (0.1 to 3.5)	0.038
	AGEF vs GRACE	65.0	64.6	0.4 (−2.4 to 3.2)	0.765	58.2	59.4	1.2 (−0.5 to 2.9)	0.155

ROC – receiver operating characteristics; NSTE-ACS – non-ST segment-elevation acute coronary syndrome; STEMI – ST segment elevation myocardial infarction; C-ACS – Canada Acute Coronary Syndrome risk score; ACEF – age, creatinine and ejection fraction risk score; AGEF – modified version of ACEF risk score; GRACE – Global Registry for Acute Coronary Events risk scores; MACE – major adverse clinical events.

Figure 3

Receiver operating characteristics (ROC) curves showing the discriminative ability of the risk scales for the predictive ability of in-hospital death in patients with NSTE-ACS (A) and STEMI (B).

In the STEMI group, the AGEF risk model (AUC=0.845; 95% CI 0.825–0.864; P<0.001, Figure 3B) exhibited better predictive potential for in-hospital mortality than GRACE risk score on admission (AUC=0.819; 95% CI 0.796–0.842; P<0.001, Figure 3B), ACEF risk score (AUC=0.827; 95% CI 0.806–0.849; P<0.001, Figure 3B), or C-ACS risk score (AUC=0.767, 95% CI 0.740, 0.793; P<0.001; Figure 3B). The AUC differences between the C-ACS model and ACEF, AGEF, and GRACE were 0.061, 0.078, 0.053, respectively (Table 2).

Supplementary Figures 3 and 4 summarize the discriminative ability of these 4 risk models to predict MACEs and major hemorrhage. GRACE risk scores exhibited greater predictive power for in-hospital MACEs than the other 3 risk scores in both the STEMI and NSTE-ACS groups. In addition, GRACE and AGEF risk scores exhibited similar discriminative ability for major bleeding.

Subgroup Analysis

For mortality, the prognostic power of different scores was compared in patients who did, or did not, undergo PCI. In patients receiving PCI, the C-ACS score exhibited a lower inpatient death discrimination ability than GRACE (AUC: 0.759 vs 0.834, P=0.015), ACEF (AUC: 0.759 vs 0.811, P=0.015), and AGEF (AUC: 0.759 vs 0.827, P=0.014) risk scores. Similar results were observed in patients who did not undergo PCI (Figure 4). Moreover, the predictive power of each risk score system for MACEs and major bleeding were also compared (Table 3, Supplementary Figures 5, 6).

Figure 4

Receiver operating characteristics (ROC) curves showing the discriminative ability of the risk assessments of in-hospital death in patients undergoing PCI (A) or non-PCI (B).

Table 3

Subgroup analysis of ROC curve comparison for in-hospital adverse events.

	Comparison (A vs B)	PCI				Non-PCI
		AUC-A%	AUC-B%	ΔAUC%	P value	AUC-A%	AUC-B%	ΔAUC%	P value
Death	C-ACS vs ACEF	75.9	81.1	5.2(1.8 to 8.7)	0.003	72.8	79.7	6.9(3.9 to 9.9)	<0.001
	C-ACS vs AGEF	75.9	82.7	6.8(3.8 to 9.9)	<0.001	72.8	80.5	7.7(4.7 to 10.6)	<0.001
	C-ACS vs GRACE	75.9	83.4	7.5(5.1 to 9.9)	<0.001	72.8	79.5	6.7(4.6 to 8.8)	<0.001
	ACEF vs AGEF	81.1	82.7	1.6(−0.1 to 3.3)	0.072	79.7	80.5	0.8(−0.8 to 2.4)	0.354
	ACEF vs GRACE	81.1	83.4	2.3(−0.7 to 5.2)	0.133	79.7	79.5	0.2(−2.3 to 2.6)	0.883
	AGEF vs GRACE	82.7	83.4	0.7(−1.9 to 3.2)	0.608	80.5	79.5	0.9(−1.2 to 3.2)	0.404
MACEs	C-ACS vs ACEF	70.3	71.2	0.9(−0.3 to 2.1)	0.157	72.8	74.8	2.0(0.7 to 3.3)	0.003
	C-ACS v AGEF	70.3	71.3	1.0(−0.3 to 2.1)	0.132	72.8	74.6	1.8(0.5 to 3.1)	0.007
	C-ACS vs GRACE	70.3	73.4	3.1(2.3 to 3.9)	<0.001	72.8	76.4	3.6(2.6 to 4.5)	< 0.001
	ACEF vs AGEF	71.2	71.3	0.1(−0.4 to 0.5)	0.858	74.8	74.6	0.2(−0.4 to 0.8)	0.516
	ACEF vs GRACE	71.2	73.4	2.2(1.2 to 3.3)	<0.001	74.8	76.4	1.6(0.4 to 2.7)	0.009
	AGEF vs GRACE	71.3	73.4	2.1(1.2 to 3.2)	<0.001	74.6	76.4	1.8(0.6 to 2.9)	0.002
Bleeding	C-ACS vs ACEF	57.7	59.1	1.4(−0.8 to 3.5)	0.234	57.2	58.9	1.7(−1.7 to 5.1)	0.322
	C-ACS vs AGEF	57.7	59.2	1.5(−0.6 to 3.6)	0.168	57.2	60.6	3.4(0.1 to 6.6)	0.045
	C-ACS vs GRACE	57.7	60.6	2.9(1.2 to 4.4)	<0.001	57.2	62.8	5.6(3.3 to 8.0)	<0.001
	ACEF vs AGEF	59.1	59.2	0.1(−0.6 to 0.9)	0.628	58.9	60.6	1.7(0.0 to 3.3)	0.048
	ACEF vs GRACE	59.1	60.6	1.5(−0.2 to 3.3)	0.090	58.9	62.8	3.9(1.1 to 6.7)	0.005
	AGEF vs GRACE	59.2	60.6	1.4(−0.4 to 3.1)	0.135	60.6	62.8	2.2(−0.4 to 5.0)	0.101

ROC – receiver operating characteristics; PCI – percutaneous transluminal coronary intervention; C-ACS – Canada Acute Coronary Syndrome risk score; ACEF – age, creatinine and ejection fraction risk score; AGEF – modified version of ACEF risk score; GRACE – Global Registry for Acute Coronary Events risk scores; MACE – major adverse clinical events.

Discussion

Risk stratification is crucially important for optimum management of ACS, and patients with the greatest risk for death or recurrent ischemic incidents could benefit from further investigation and management. Validation of ACS risk scale is critical for diagnosis and improved quality of care. There is considerable heterogeneity between different models due to differences in accuracy and predictive potential for adverse prognosis in ACS. Our study is the first to compare the 4 validated risk scales for determining prognosis among ACS patients. In our research, in patients with STEMI, AGEF showed significantly better ability to predict in-hospital death, while the predictive utility of GRACE and ACEF in NSTE-ACS patients was not significantly different. Overall, AGEF showed non-inferior ability to GRACE and ACEF in predicting ACS inpatient death.

The most wide-ranging implemented risk score is the GRACE score [5], which was derived from a large prospective evaluation. The external validity of the GRACE scale has been evaluated in prospective testing of patients from GRACE and Global Use of Strategies to Open Occluded Coronary Arteries IIb (GUSTO-IIb) trial database, which included patients with unstable angina and STEMI. This scale was produced to evaluate the risk of mortality at 6 months [6]. In addition, the GRACE risk score seemed to accurately discriminate between survivors and non-survivors at long-term follow-up [21]. In contrast, the GRACE scoring system has been used for prediction of other in-hospital MACEs [22,23]. Luo et al [24] indicated that GRACE risk score was an excellent predictor of post-myocardial infarction new-onset atrial fibrillation.

The GRACE risk model consists of 8 factors and is difficult to evaluate, which limits its use in clinical practice. A simpler risk score is necessary for broader clinical use. Furthermore, some factors of the GRACE risk model are categorical binary variables. Each of these factors require inclusion of definitions in the model. These definitions are specified for each risk score, but different operators interpret these variables differently, resulting in different final risk scores [11]. Previous studies have resulted in generation of several new risk scoring systems with differing levels of complexity for identification of high-risk patients. Risk scores should be determined using simple linear formulas for predicting mortality or morbidity at the bedside without the need for calculators or other methods of assistance [25].

The ACEF score, a convenient tool, was used for mortality prediction in patients receiving elective cardiac interventions. The predictive value of this scale has been validated in numerous myocardial revascularization scenarios beyond bypass grafting. Wykrzykowska et al showed that ACEF risk score could be used in ACS patients receiving PCI [26]. Di Serafino et al found that ACEF risk score was a valuable tool for determination of outcomes of patients receiving coronary chronic total occlusion (CTO) PCI [27]. They also found that ACEF risk score helped to recognize patients free of harm despite the unsuccessful percutaneous treatment of the CTO.

The 3 risk factors used to determine the ACEF risk score are continuous variables [11]. Use of fewer variables results in an easier calculation, and eliminates overfitting by numerous indexes in low-incident populations. Two of the risk factors (age and serum creatinine value) are not subject to personal estimation. This results in standardized assessment of this risk prediction model.

Glomerular filtration rate (GFR) and CrCl are more accurate indicators of kidney function than is serum creatinine. Two of the most widely implemented formulas for calculating kidney function are the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) equation and the Modification of Diet in Renal Disease (MDRD) Study formula [28,29]. Capodanno et al showed that including GFR or CrCl into ACEF risk score yielded superior results to the original serum creatinine-based in patients undergoing PCI [15]. Kalaycı et al found that this modified ACEF risk score had good ability to predict adverse cardio-cerebrovascular outcomes after 1-year follow-up [30]. Our study also showed that AGEF risk score is a better predictor of all-cause death in patients with STEMI than the other 3 risk scores in duration of hospital stay. Andò et al investigated 481 STEMI patients and found that AGEF score was an accurate prognosticator of contrast-induced nephropathy [14].

The main merit of the C-ACS score is the use of only socio-demographic and simple hemodynamic data, and not an electrocardiogram (ECG), blood sample, or a calculator. It is a useful tool for stratifying risk levels of ACS patients in many settings, including the emergency system or at home [31]. In the present study, the C-ACS score had lower prognostic ability than the other 3 models in ACS patients. This could be attributed to the non-involvement of renal function in C-ACS scale, which is a pivotal prognostic maker in cardiovascular disease [32,33]. Furthermore, classification of cardiac function is very subjective and may have influenced the results.

Several factors may account for the low in-hospital mortality in the STEMI and NSTE-ACS groups in our study. First, the included hospitals are all tertiary hospitals, which means they have advanced equipment and therapy, and provide the highest level of care. Studies have shown that acute myocardial infarction mortality varies by more than 3 times among hospitals at different levels in China [34]. Second, in each hospital, only the first 20–30 ACS inpatient cases in each month were consecutively recruited to this study. Third, some patients who died during transport were not included in this study. Similar phenomena can be found in other registration ACS studies in China. Jiyan Chen et al included 8197 adults who underwent PCI for NST-ACS in 5 hospitals from 2010–2014, and found that the in-hospital all-cause death was 0.2% [35]. Mengxuan Chen et al analyzed data on 2128 STEMI patients between 2010 November and 2016 October, showing that the in-hospital mortality rate after PCI was 1.6% [36]. Accordingly, the in-hospital mortality rate is relatively low among the registered ACS patients in China. However, the identification of high-risk patients is still important, and the long-term outcomes and other adverse events within this population warrant further study.

Limitations

This was a retrospective study based on prospectively collected information. Missing data and confounding factors might have influenced the results. Furthermore, the CCC-ACS project only recruits ACS patients in the highest level of public hospital, and there is no information on patients who died before arriving at the hospital. This may lead to potential selection bias in this cohort study, thereby reducing the in-hospital mortality rates of patients. Thus, the patients and outcomes of this study may not reflect experiences elsewhere in the healthcare system. More precise results need to be validated in a broader range of care settings. In addition, AUC and 95% CI were used to compare predicted values of different scores, but the results also found that ROC curves intersected at some points. Since the size of the overall AUC was compared in this study, and the AGEF risk score had the largest AUC, it must be acknowledged that AGEF risk score may be less sensitive and specific than the other 3 models at some points. Finally, the study lacked follow-up data. Therefore, the predictive capability of the different scores with regard to long-term prognosis was not evaluated.

Conclusions

The predictive and discriminatory abilities of different risk scores with respect to in-hospital clinical outcomes in Chinese ACS patients were compared in our study. In STEMI patients, AGEF was significantly superior to GRACE and ACEF in predicting in-hospital death, while there was no significant difference for the NSTE-ACS group. Overall, AGEF risk score showed a non-inferior utility to the other 3 scoring systems in predicting in-hospital mortality in ACS patients.

Rates of in-hospital MACEs in the low-, moderate-, and high-risk groups, according to the GRACE, ACEF, AGEF, and C-ACS risk scores.

Rates of in-hospital major bleeding in the low-, moderate-, and high-risk groups, according to the GRACE, ACEF, AGEF, and C-ACS risk scores.

Received operating characteristic (ROC) curves showing the discriminative ability of the risk scales for the predictive ability of in-hospital MACEs (A: NSTE-ACS; B: STEMI).

Received operating characteristic (ROC) curves showing the discriminative ability of the risk scales for the predictive ability of in-hospital major bleeding (A: NSTE-ACS; B: STEMI).

Received operating characteristic (ROC) curves showing the discriminative ability of the risk assessments of in-hospital MACEs in patients undergoing PCI (A) or non-PCI (B).

Received operating characteristic curves showing the discriminative ability of the risk assessments of in-hospital major bleeding in patients undergoing PCI (A) or non-PCI (B).