Mortality prediction algorithms for patients undergoing primary percutaneous coronary intervention.

Mortality risk of ST-segment elevation myocardial infarction (STEMI) patients shows high variability. In order to assess individual risk, a number of scoring systems have been developed and validated. Yet, as treatment approaches evolve over time with improving outcomes, there is a need to build new risk prediction algorithms to maintain/increase prognostic accuracy. One of the most relevant improvements of therapy is primary percutaneous coronary intervention (PCI). We overview the characteristics and discriminative performance of the most studied and some recently constructed mortality risk models that were validated in patients with STEMI who underwent primary PCI. 2020 Journal of Thoracic Disease. All rights reserved.

Introduction

Mortality risk of ST-segment elevation myocardial infarction (STEMI) patients shows high variability. In order to assess individual risk, a number of mathematical models (scoring algorithms) have been developed. Yet, as treatment approaches evolve over time with improving outcomes and as ever older patients with complex disease patterns are treated invasively, there is a need to build new risk prediction algorithms to maintain/increase prognostic accuracy. One of the most relevant improvements of therapy is primary percutaneous coronary intervention (PPCI), since, compared with fibrinolysis, it further reduces mortality. Therefore, it is the treatment of choice according to both American and European guidelines (1-3). Prediction algorithms may provide useful information for patients/relatives and help physicians to allocate hospital resources. Moreover, they may contribute to an improved quality of care as they can be used for risk adjustment in inter-organizational comparisons of health care providers with different case mixes. They also enable intra organizational quality monitoring. Furthermore, risk models may be helpful in clinical trial design identifying patients with the needed risk profile thereby increasing statistical power/reducing sample size and costs.

Methods

PubMed (https://pubmed.gov) was searched for English language mortality risk models that were developed using (at least in part) data of STEMI patients (). Other risk prediction algorithms (e.g., different derivations of the SYNTAX score, which was developed excluding cases with myocardial infarction) were not considered (4,5). After identifying the models, we sought for their external validation studies. Only reports with populations involving STEMI and primary PCI as a treatment modality (at least partly) were analyzed.

Table 1

Characteristics of the derivation data set of mortality risk models

Risk model	TIMI	PAMI	Zwolle	CADILLAC	APEX-AMI	NCDR CathPCI	AR-G	EH STEMI PCI	Dynamic TIMI	GRACE 2.0 (1-year death model)	ALPHA
First author	Morrow	Addala	De Luca	Halkin	Stebbins	Peterson	Chin	de Mulder	Amin	Fox	Hizoh
Year of publication	2000	2004	2004	2005	2010	2010	2011	2011	2013	2014	2017
Clinical setting	STEMI	STEMI	STEMI	STEMI	STEMI	Stable CAD + ACS	NSTEMI + STEMI	STEMI	STEMI	ACS	STEMI
Treatment	Thrombolysis	PPCI	PPCI	PPCI	PPCI	PCI	Not specified	PPCI	Thrombolysis	Not specified	PPCI
Source of data	Multi Center RCT	Multi Center RCT + Registry	Single Center Registry	Multi Center RCT	Multi Center RCT	Multi Center Registry	Multi Center Registry	Multi Center Registry	Multi Center RCT	Multi Center Registry	Single Center Registry
Cardiogenic shock	Excluded	Excluded	NR	Excluded	Included	Included	Included	Included	Excluded	Included	Included
Time of end point	30 days	6 months	30 days	1 year	90 days	In-hospital	In-hospital	In-hospital	1 year	1 year	30 days
Size (n)	14,114	3,252	1,791	2,082	5,745	181,775	65,668	4,091	19,121	32,037	750
Number of events	946	164	65	90	271	2,254	3,218	220	988	2,422	57
Mortality (%)	6.7	5.0	3.6	4.3	4.7	1.2	4.9	5.4	5.2	7.6	7.6
C-Statistic (95% CI)	0.78 (NR)	0.78 (NR)	0.91 (NR)	0.79 (NR)	0.82 (NR)	0.91 (NR)	0.85 (NR)	0.86 (NR)	0.76 (NR)	0.83 (NR)	0.88 (0.85 to 0.92)

ACS, acute coronary syndrome; ALPHA, age, life support, pressure, heart rate, access site; APEX AMI, Assessment of Pexelizumab in Acute Myocardial Infarction; AR-G, Acute Coronary Treatment and Intervention Outcomes Network Registry-Get With the Guidelines; CAD, coronary artery disease; CADILLAC, Controlled Abciximab and Device Investigation to Lower Late Angioplasty Complications; CI, confidence interval; EH, EuroHeart; GRACE, Global Registry of Acute Coronary Events; NCDR CathPCI, National Cardiovascular Data Registry for Catheterization Percutaneous Coronary Intervention; NR, not reported; NSTEMI, non-ST-segment elevation myocardial infarction; PAMI, primary angioplasty in myocardial infarction; PCI, percutaneous coronary intervention; PPCI, primary percutaneous coronary intervention; RCT, randomized controlled trial; STEMI, ST-segment elevation myocardial infarction; TIMI, thrombolysis in myocardial infarction.

General characteristics of risk predicting algorithms

The general characteristics of the derivation and validation studies of the analyzed mortality risk models are summarized in .

Table 2

Validation studies of mortality risk models in patients with ST-segment elevation myocardial infarction

Author, year	Study characteristics						C-Statistic (95% CI)
	Clinical setting	Treatment	Source of data	Time of end point	Event rate (%)		TIMI, n=10	PAMI, n=6	Zwolle, n=5	CADILLAC, n=6	APEXAMI, n=1	NCDR CathPCI, n= 2	AR-G, n=2	EH STEMI, PCI, n=1	Dynamic TIMI, n=2	GRACE, n=17	ALPHA, n=2
Morrow, 2001	STEMI	PPCI	Multi Center Registry	In-hospital	NR/15,348 (NA)		0.80 (NR)
De Luca, 2004	STEMI	PPCI	Multi Center Registry	30 days	27/747 (3.6)				0.90 (NR)
Halkin, 2005	STEMI	PPCI	Multi Center RCT	30 days	24/900 (2.7)		0.70 (NR)	0.78 (NR)	0.74 (NR)	0.81 (NR)
Halkin, 2005	STEMI	PPCI	Multi Center RCT	1 year	39/900 (4.3)		0.69 (NR)	0.77 (NR)	0.74 (NR)	0.78 (NR)
Lev, 2008	STEMI	PPCI	Multi Center Registry	30 days	31/855 (3.6)		0.72 (NR)	0.74 (NR)		0.82 (NR)						0.47 (NR)*
Lev, 2008	STEMI	PPCI	Multi Center Registry	1 year	50/855 (5.8)		0.75 (NR)	0.75 (NR)		0.81 (NR)						0.48 (NR)*
Elbarouni, 2009	STEMI	Not specified	Multi Center Registry	In-hospital	171/3,186 (5.4)											0.83 (0.80–0.86)*
Peterson, 2010	STEMI	PPCI	Multi Center Registry	In-hospital	NR/39,889 (NA)							0.88 (NR)
Abu-Assi, 2010	STEMI	Not specified	Multi Center Registry	In-hospital	178/2,344 (7.6)											0.86 (0.83–0.89)*
Abu-Assi, 2010	STEMI	Not specified	Multi Center Registry	6 months	126/2,165 (5.8)											0.79 (0.75–0.83)*
Chin, 2011	NSTEMI/STEMI	Not specified	Multi Center Registry	In-hospital	800/16,336 (4.9)								0.84 (NR)
de Mulder, 2011	STEMI	PCI	Multi Center Registry	In-hospital	203/3,969 (5.1)									0.89 (NR)
Yusufali, 2011	STEMI	Not specified	Multi Center Registry	In-hospital	144/2,986 (4.8%)											0.86 (NR)*
Selverajah, 2012	STEMI	Not specified	Multi Center Registry	30 days	522/4,701 (11.1)		0.79 (0.77–0.81)
Raposeiras-Roubín, 2012	STEMI	Not specified	Multi Center Registry	In-hospital	141/1,443 (9.8)								0.89 (0.87–0.92)			0.91 (0.88–0.93)*
Méndez-Eirín, 2012	STEMI	PPCI/Rescue PCI	Multi Center Registry	30 days	83/1,503 (5.5)		0.87 (0.85–0.89)	0.81 (0.79–0.83)		0.90 (0.88–0.91)						0.90 (0.89–0.92)*
Méndez-Eirín, 2012	STEMI	PPCI/Rescue PCI	Multi Center Registry	1 year	105/1,130 (9.3)		0.85 (0.83–0.87)	0.81 (0.78–0.83)		0.87 (0.84–0.89)						0.85 (0.83–0.87)*
Amin, 2013	STEMI	PCI	Multi Center RCT	1 year	48/1,829 (2.6)										0.81 (NR)
Timóteo, 2013	STEMI	PPCI	Multi Center Registry	In-hospital	33/607 (5.4)		0.84 (0.77–0.92)									0.92 (0.87–0.96)*
Timóteo, 2013	STEMI	PPCI	Multi Center Registry	30 days	38/607 (6.3)		0.83 (0.76–0.90)									0.88 (0.82–0.95)*
Fox, 2014	STEMI	Not specified	Multi Center Registry	1 year	NR/1,558 (NA)											0.84 (NR)*
Fox, 2014	STEMI	Not specified	Multi Center Registry	3 years	NR/1,558 (NA)											0.82 (NR)*
Fujii, 2014	STEMI	PPCI	Multi Center Registry	In-hospital	54/412 (13.1)											0.95 (NR)**
Fujii, 2014	STEMI	PPCI	Multi Center Registry	1 year	64/412 (15.5)											0.92 (NR)**
Abelin, 2014	STEMI	PPCI	Multi Center Registry	30 days	39/501 (7.8)		0.81 (0.74–0.87)	0.75 (0.68–0.82)	0.80 (0.73–0.87)							0.84 (0.78–0.90)*
Littnerova, 2015	STEMI	PPCI	Multi Center Registry	6 months	24/593 (4.0)		0.72 (0.70–0.85)	0.77 (0.65–0.80)	0.81 (0.73–0.88)	0.82 (0.75–0.88)					0.81 (0.73–0.89)	0.85 (0.78–0.93)*
Littnerova, 2015	STEMI	PPCI	Multi Center Registry	1 year	43/593 (7.3)		0.73 (0.70–0.85)	0.77 (0.66–0.80)	0.80 (0.72–0.87)	0.82 (0.76–0.89)					0.81 (0.74–0.89)	0.86 (0.80–0.93)*
Littnerova, 2015	STEMI	PPCI	Multi Center Registry	2 years	53/593 (8.9)		0.68 (0.64–0.79)	0.72 (0.61–0.76)	0.72 (0.64–0.80)	0.76 (0.69–0.83)					0.75 (0.67–0.83)	0.79 (0.72–0.86)*
Littnerova, 2015	STEMI	PPCI	Multi Center Registry	3 years	63/593 (10.6)		0.66 (0.64–0.78)	0.71 (0.59–0.73)	0.69 (0.61–0.76)	0.74 (0.67–0.80)					0.73 (0.67–0.80)	0.77 (0.70–0.83)*
Parenica, 2016	STEMI	PPCI	Multi Center Registry	1 year	40/593 (6.7)											0.85 (NR)*
Huang, 2016	STEMI	Not specified	Multi Center Registry	1 year	8/378 (2.1)											0.94 (NR)**
Timóteo, 2016	ACS	PCI	Multi Center Registry	In-hospital	96/2,148 (4.5)							0.87 (0.83–0.91)				0.94 (0.91–0.96)*
Hizoh, 2017	STEMI	PPCI	Multi Center Registry	30 days	41/505 (8.1)		0.82 (0.75–0.89)	0.78 (0.70–0.88)	0.81 (0.74–0.88)	0.83 (0.79–0.92)	0.86 (0.79–0.92)					0.86 (0.80–0.92)**	0.87 (0.81–0.93)
Hizoh, 2017	STEMI	PPCI	Multi Center Registry	1 year	73/505 (14.5)												0.84 (0.80–0.89)
Yu, 2017	NSTEMI/STEMI	PCI	Multi Center Registry	1 year	29/728 (4.0)					0.73 (0.70–0.76)						0.74 (0.71–0.77)*
Hizoh, 2018	STEMI	PPCI	Multi Center Registry	30 days	383/5,203 (7.4)		0.81 (0.79–0.83)									0.87 (0.85–0.89)**	0.86 (0.84–0.88)

*, using the GRACE 1.0 model; **, using the GRACE 2.0 model. ACS, acute coronary syndrome; ALPHA, Age, life support, pressure, heart rate, access site; APEX AMI, Assessment of Pexelizumab in Acute Myocardial Infarction; AR-G, Acute Coronary Treatment and Intervention Outcomes Network Registry-Get With the Guidelines; CADILLAC, Controlled Abciximab and Device Investigation to Lower Late Angioplasty Complications; EH, EuroHeart; GRACE, Global Registry of Acute Coronary Events; NA, not applicable; NCDR CathPCI, National Cardiovascular Data Registry for Catheterization Percutaneous Coronary Intervention; NR, not reported; PAMI, primary angioplasty in myocardial infarction; PCI, percutaneous coronary intervention; PPCI, primary percutaneous coronary intervention; RCT, randomized controlled trial; STEMI, ST-segment elevation myocardial infarction; TIMI, thrombolysis in myocardial infarction.

Clinical setting

Most of the studied risk predicting algorithms were constructed using data of patients with STEMI exclusively (6-13) (). Yet, the “Acute Coronary Treatment and Intervention Outcomes Network Registry-Get With the Guidelines” (AR-G) model was derived from mixed data of STEMI and non-STEMI cases (14), whereas the “Global Registry of Acute Coronary Events” GRACE 2.0 score was developed using data of all three types (i.e., STEMI, non-STEMI, and unstable angina pectoris) of acute coronary syndrome (ACS) (15). In contrast, the developers of the “National Cardiovascular Data Registry for Catheterization Percutaneous Coronary Intervention” NCDR CathPCI model used data of all coronary artery disease (CAD) patients who underwent PCI, regardless of disease acuity (16). Similarly to the training data sets, clinical settings of the validation studies also vary ().

Table 3

Characteristics and composition of mortality risk models

Risk model	TIMI	PAMI	Zwolle	CADILLAC	APEX-AMI	NCDR CathPCI	AR-G	EH STEMI PCI	Dynamic TIMI	GRACE 2.0	ALPHA
Presentation Characteristics
Age	+	+	+	+	+	+	+	+	+	+	+
Gender								+
Body weight/BMI	+							+	+
Heart rate	+	+			+		+		+	+	+
Systolic blood pressure	+				+		+		+	+	+
Heart failure on presentation						+	+
Killip class/cardiogenic shock/hemodynamic instability	+	+	+	+	+	+	+	+	+	+
ECG localization (STEMI)	+	+	+		+				+
ST-segment deviation (qualitative or quantitative)					+		+			+
Ischemia time	+		+					+	+
Cardiac arrest on or prior to admission										+	+
Timing of PCI						+
History of diabetes mellitus	+	+						+	+
History of hypertension	+								+
History of angina pectoris	+								+
History of stroke								+
History of CABG								+
History of CHF						+
History of chronic lung disease						+
History of PAD						+	+
Smoking status								+
Procedural data
Vascular access site											+
2/3 vessel disease			+	+				+
Pre-procedural TIMI flow								+
Final TIMI flow			+	+
Infarct related artery								+
Bifurcation lesion								+
Type-C lesion								+
Laboratory and imaging studies
Elevated necrosis biomarkers							+			+
Renal function				+	+	+	+			+
Anemia				+
LVEF				+
In-hospital events
Recurrent myocardial infarction									+
Stroke									+
Major bleeding									+
CHF/shock									+
Arrhythmia									+
Renal failure									+
Number of predictors	10	5	6	7	7	8	9	14	16	8	5

ALPHA, Age, Life support, Pressure, Heart rate, Access site; APEX AMI, Assessment of Pexelizumab in Acute Myocardial Infarction; AR-G, Acute Coronary Treatment and Intervention Outcomes Network Registry-Get With the Guidelines; BMI, body mass index; CABG, coronary artery bypass graft surgery; CADILLAC, Controlled Abciximab and Device Investigation to Lower Late Angioplasty Complications; CHF, congestive heart failure; EH, EuroHeart; GRACE, Global Registry of Acute Coronary Events; LVEF, left ventricular ejection fraction; NCDR CathPCI, National Cardiovascular Data Registry for Catheterization Percutaneous Coronary Intervention; PAD, peripheral artery disease; PAMI, primary angioplasty in myocardial infarction; PCI, percutaneous coronary intervention; STEMI, ST-segment elevation myocardial infarction; TIMI, thrombolysis in myocardial infarction.

Treatment modality

In the development set of the “Thrombolysis in Myocardial Infarction” (TIMI) risk score, the oldest of the analyzed algorithms (6), and the derivative “Dynamic TIMI” model (12) patients were treated with fibrinolysis. In contrast, newer scores used primary PCI (7-11,13) or PCI (16), depending on the clinical setting, as a treatment modality. Nevertheless, in the AR-G algorithm for STEMI and non STEMI (14) and in the GRACE 2.0 model developed for all types of the ACS (15), the therapeutic modality was not specified. Likewise, treatment was not restricted to primary PCI/PCI in many of the validation studies ().

Source of data

Older models often used randomized controlled trials (RCTs) as a data source for derivation (6,7,9,10,12) (). While RCTs generally have excellent data quality, due to the strict inclusion and exclusion criteria, their participants may not be fully representative of the whole population. Moreover, in the case of some models, patients with cardiogenic shock were completely excluded from the derivation data set (6,7,9,12). Missing or under-representation of prognostically important factors in the derivation data set may result in a systematic misestimation of the regression coefficients and biased prediction, limiting the generalizability of the algorithm. Newer models, however, usually employed derivation data from single- or multi-center registries representing “real-world” patients (8,11,13-16). Likewise, the risk prediction algorithms were mostly validated using registries ().

Prediction end point

Three of the studied scores were constructed to predict the risk of in-hospital deaths (11,14,16), whereas three used 30-day mortality risk as an outcome measure (6,8,13). The “Assessment of Pexelizumab in Acute Myocardial Infarction” (APEX-AMI) (10) and the “Primary Angioplasty in Myocardial Infarction” (PAMI) (7) models were developed for forecasting 90-day and 6-month mortality risks, respectively. For long-term (1 year) prognosis, the” Controlled Abciximab and Device Investigation to Lower Late Angioplasty Complications” (CADILLAC) (9), the dynamic TIMI (12), and the GRACE 2.0 (15) scores were developed. Latter was also designed to predict in-hospital, 6-month, and 3-year mortality risks. Irrespective of the outcome measure used for derivation, most validation studies used in hospital, 30-day, and/or 1-year mortality risks as prediction end points ().

Predictors/time of assessment

Some of the models use exclusively predictors that are available at presentation like demographic and historical data, presentation and ECG characteristics (6,7) (“admission model”), while others also make use of findings/results of the coronary intervention and/or more time consuming imaging/laboratory studies/in-hospital events assessing risk later during the hospital stay (8-11,13-16) or only at the time of discharge (12) (“discharge model”) (, ). The most common variables used in the models are age, which is a predictor in each of the studied models (6-16), Killip class/presence of cardiogenic shock/hemodynamic instability (6-12,14-16), heart rate (6,7,10,12-15) and systolic blood pressure at admission (6,10,12-15), ECG localization of the infarction (6-8,10,12), renal function (9,10,14-16), ischemia time (6,8,11,12), and history of diabetes mellitus (6,7,11,12). Each of the variables presented in was independently associated with mortality being parts of one or more models. Yet, researchers have to maintain a balance between including too many predictors and model parsimony. Omitting better treatment options, such as primary PCI (6,12) and/or under-representation of other important prognostic factors (e.g., cardiogenic shock) (6,7,9,12) may cause biased prediction. On the other hand, using too many variables may result in loss of precision in the estimation of the coefficients and the predictions of new responses.

Figure 1

Composition of mortality risk scores. Height of the bars shows the number of predictors needed for calculation of the score. Color of the predictor groups corresponds with the time needed for the availability of predictors: blue: variables that are available at or soon after admission (presentation characteristics and procedural data); orange: laboratory and imaging studies requiring some more time; green: in hospital events that can only be assessed at the time of discharge. True admission models are the TIMI and PAMI scores, whereas dynamic TIMI can only be calculated at the time of discharge. With the exception of the GRACE 2.0 and ALPHA models, there is a trend that newer algorithms became more complex with more predictors. TIMI, thrombolysis in myocardial infarction; PAMI, primary angioplasty in myocardial infarction.

Usage

Classic risk scores used points usually derived from the odds/hazard ratios of the predictors and provided mainly relative risk classes. Modern models, however, make use of more complex statistical methods allowing non-linear associations between continuous predictors and the outcome (e.g., APEX-AMI, AR-G, GRACE 2.0, ALPHA), which makes “manual” calculations somewhat difficult. Hence, these algorithms sometimes come with an online calculator/mobile app providing both relative and absolute risks (e.g., GRACE 2.0, ALPHA).

Characteristics of individual risk models

The characteristics of the derivation and validation studies of the analyzed mortality risk models are summarized in and . Here we give a short description of each of the analyzed algorithms sorted by the treatment modality used in the derivation data set and publication date.

Scores without treatment specification

AR-G

The “Acute Coronary Treatment and Intervention Outcomes Network (ACTION) Registry-Get With the Guidelines (GWTG)” (AR-G) score was constructed using data of both STEMI and non STEMI patients to forecast in hospital deaths (14). Similarly to the GRACE models for all three types of ACS, treatment modality was not specified. Yet, most patients (~80%) were treated with PCI, some 14% with fibrinolysis, whereas around 6% did not get reperfusion therapy. The score consists of 7 patient related and 2 laboratory parameters (, ). Though the predictive ability of the model was similarly good in both the training and validation data sets (14), the score has been poorly validated externally (17).

GRACE 2.0

The GRACE 2.0 models (two distinct algorithms for 1-year and 3-year mortality risks) were derived from an international multi-center registry of 32,037 and 1,274 ACS patients, respectively (15). Together with the GRACE 1.0 model, they are capable of predicting in-hospital, 6-month, 1-year, and 3-year mortality risks. Besides 6 parameters that are available at the presentation it evaluates serum creatinine and cardiac necrosis biomarker levels (, ; in the “mini” version of the model creatinine level may be substituted by the history of renal failure). The GRACE 1.0 and 2.0 scores have been extensively validated in the STEMI setting (13,15,17-31). An online calculator is available at https://www.outcomes-umassmed.org/grace/acs_risk2/index.html.

Fibrinolysis scores

TIMI

The thrombolysis in myocardial infarction score was developed from data of a multi-center RCT on STEMI patients who were treated with fibrinolysis (6). The model employs 10 variables that are all readily available at admission (, ). After the GRACE score, TIMI is the second most validated model (9,13,18,19,24,25,29,30,32,33).

Dynamic TIMI

Similarly to the TIMI model, the dynamic TIMI score has also been constructed using data of a multi-center RCT: the 10 predictors of the TIMI risk score have been amended by 6 major clinical in-hospital events such as myocardial infarction, arrhythmia, major bleeding, stroke, congestive heart failure/shock, and renal failure (, ) (12). Therefore, the dynamic TIMI risk score can only be calculated at the time of discharge. This RCT-derived score has been relatively poorly validated (12,19).

Primary PCI/PCI scores

PAMI

The “Primary Angioplasty in Myocardial Infarction” risk score was developed using a mixed population of 3,252 patients from the multi-center PAMI RCTs and the registry arm of the PAMI-2 study to forecast 6-month mortality risk (7). Despite the registry arm, cases with cardiogenic shock were excluded. Likewise to the TIMI model, it solely employs 5 simple parameters that are all accessible at presentation (, ). Being the oldest primary PCI model, the PAMI score is well evaluated (9,13,18,19,25,29).

Zwolle

The Zwolle risk score was developed using a single-center registry of 1,791 STEMI patients treated with primary PCI (8). The model consists of 6 variables: besides basic demographics, presentation and ECG characteristics it also makes use of 2 procedural parameters: the presence of triple vessel disease and final TIMI flow (, ). The Zwolle risk score was validated in several studies (8,9,13,18,19).

CADILLAC

The risk model was derived to predict 1-year mortality risk from data of the “Controlled Abciximab and Device Investigation to Lower Late Angioplasty Complications” multi-center trial, excluding high-risk cases (9). The algorithm employs 7 variables including procedural and laboratory parameters: age, Killip class, baseline left ventricular ejection fraction (LVEF) as assessed by ventriculography before PCI, presence of triple vessel disease, final TIMI flow grade, anemia, and renal failure (, ). Thus, this is the only model that makes use of three predictor categories i.e., presentation characteristics, procedural data, and imaging/laboratory studies. Though LVEF less than 40% was the most important predictor in the model [odds ratio =3.50, 95% confidence interval (CI): 2.07–5.75], ventriculography is rarely performed before/during primary PCI limiting the practical value of the original model. Nevertheless, LVEF can also be estimated using echocardiography as it was done in some of the validation studies (13,19,23). Despite the exclusion of high-risk patients from the derivation data set, the model performs well in registry-based validation analyses as well (9,13,19,23,25,29).

APEX-AMI

Characteristics of the “Assessment of Pexelizumab in Acute Myocardial Infarction” (APEX-AMI) score are shown in (10). The model has 6 variables that are available at admission and renal function as a laboratory parameter (, ). The c-statistic for 90-day mortality in the derivation set was 0.82. Though the APEX-AMI model underwent internal validation, to our knowledge, only one external validation study was performed showing good predictive ability (13).

NCDR CathPCI

The NCDR CathPCI is a simplified, user-friendly model with 7 patient-related (pre-procedural) parameters and 1 laboratory variable (glomerular filtration rate) to predict in-hospital mortality (16). It was derived from the National Cardiovascular Data Registry using data of both elective and acute PCI procedures (, ). Besides external temporal validation using the same registry, it also underwent a fully external evaluation showing good predictive ability (16,22).

EuroHeart STEMI PCI

The EuroHeart ST-segment elevation myocardial infarction PCI model (EH STEMI PCI) was constructed using a multi-center registry to predict in-hospital mortality of patients treated with primary PCI (11). Besides 9 patient-related parameters (this is the only algorithm that contains gender as a predictor), it includes 5 procedural variables (, ). Though the model seems to exhibit excellent discriminatory power, to our knowledge it did not undergo fully external validation (it was only validated using the very same registry randomly divided into derivation and validation data sets) (11).

ALPHA

Transradial primary PCI has been shown to reduce mortality risk in several randomized trials and is gaining popularity worldwide (34-37). In light of that, current European guidelines on STEMI and myocardial revascularization give a class IA indication for routine radial access (2,3). To reflect this change in practice, a new risk model including vascular access site has been constructed for predicting 30-day mortality in patients treated with primary PCI (13). The ALPHA model consists of 5 simple predictors such as Age, need for Life support (i.e., cardiac arrest on or prior to admission), systolic blood Pressure and Heart rate at admission, and vascular Access site (, ). Besides internal validation, the model underwent temporal and fully external evaluations both suggesting high discriminatory power: c-statistic =0.87 (95% CI: 0.81–0.93) and 0.86 (95% CI: 0.84–0.88), respectively. Moreover, the predictive power of the score was stable for up to one year (13,24). With only 5 parameters that are available at or soon after the presentation the score can be calculated early using the online calculator (https://alphascore.org) providing both relative risk class and absolute 30-day mortality risk. ALPHA is the only mortality risk model that includes the access site as a variable representing contemporary PCI practice. Using the model, patients who may benefit most from transradial access may be identified at presentation. Then, even before coronary angiography, when the actual vascular access site is still not known, estimating the absolute risks for the two approaches and subtracting the radial from the femoral one, the resulting difference equals the absolute risk reduction that is attributable to transradial access. Despite these advantages, the model still awaits further (international) validation.

Comparative validation

Halkin et al. compared the performance of the CADILLAC score (9) with that of the TIMI (6), PAMI (7), and Zwolle (8) models in 900 patients of the Stent-Primary Angioplasty in Myocardial Infarction (Stent-PAMI) trial () (38). Though the authors state that the CADILLAC score compared favorably with these previous models in prognostic performance, data of pairwise comparisons has not been reported (9).

Lev et al. analyzed the TIMI (6), PAMI (7), CADILLAC (9), and GRACE (15) models in 855 hemodynamically stable patients from a single-center registry (25). According to the authors, the CADILLAC, TIMI, and PAMI risk scores all had relatively high predictive accuracy for 30-day and 1-year mortality (), with slight superiority of the CADILLAC score. Surprisingly, the discriminative ability of the GRACE model was not found to be statistically significant. Nevertheless, the results of pairwise comparisons have not been published.

Using a single-center registry, Raposeiras-Roubín et al. studied the AR-G (14) and GRACE (15) models in STEMI patients, who were only partly treated with primary PCI, with in-hospital mortality risk as an outcome measure (). They found no statistical difference in the predictive performance of the two scores (17).

Méndez-Eirín et al. studied the TIMI (6), PAMI (7), CADILLAC (9), and GRACE (15) models in a single-center cohort of STEMI patients who were treated with primary or rescue PCI, using mortality risks at 30 days and 1 year as end point () (29). They found that the TIMI, CADILLAC, and GRACE scores had greater discriminatory power for both 30-day and 1-year mortality than the PAMI model. Also, at 30 days, the GRACE model predicted statistically better than the TIMI model.

The TIMI (6) and GRACE (15) scores were also compared by Timóteo et al. using 607 patients of a single-center registry (). With that sample size the GRACE score was found to have a better predictive performance for in-hospital but not for 30-day mortality, despite numerical difference (30).

Abelin et al. analyzed the discriminative abilities of the TIMI (6), PAMI (7), Zwolle (8), and GRACE (15) models in a single-center cohort of 501 STEMI patients treated with primary PCI () (18). With that sample size, there was no statistically significant difference regarding the predictive accuracy of the TIMI, GRACE, and Zwolle scores for 30-day mortality risk, but the GRACE model was superior to the PAMI algorithm (P<0.01).

Littnerova et al. analyzed the capability of the TIMI (6), dynamic TIMI (12), PAMI (7) Zwolle (8), CADILLAC (9), and GRACE (15) scores to predict mortality from 6 months up to 3 years in 593 STEMI patients of a single-center registry who underwent primary PCI (19). The best predictive values for long-term mortality risk were obtained by the GRACE algorithm, followed by the CADILLAC, Zwolle, and dynamic TIMI models. In contrast, the TIMI and PAMI risk scores were less good at long-term predictions (). Nevertheless, not all pairwise comparisons were made, the predictive value of the models was compared with that of the GRACE score as a reference.

The GRACE (15) and the NCDR CathPCI (16) algorithms were compared by Timóteo et al. using a single-center cohort of 2,148 ACS patients (of whom 70.9% had STEMI) treated with PCI (22). The authors found that the predictive power of the GRACE model for in hospital mortality risk was statistically greater than that of the NCDR CathPCI score ().

More recently, Hizoh et al. comparatively determined the c-statistic of the TIMI (9), PAMI (7), Zwolle (8), CADILLAC (9), APEX-AMI (10), GRACE 2.0 (15), and ALPHA (13) models for 30-day mortality risk using a single-center registry cohort of 505 patients (). The ALPHA, GRACE 2.0, APEX-AMI, and CADILLAC models predicted 30-day mortality risk better than the PAMI score [ALPHA vs. PAMI: difference =0.10 (95% CI: 0.03–0.16), P=0.005; GRACE 2.0 vs. PAMI: difference =0.09 (95% CI: 0.03–0.15), P=0.004; APEX-AMI vs. PAMI: difference =0.08 (95% CI: 0.02–0.15), P=0.01; CADILLAC vs. PAMI: difference =0.08 (95% CI: 0.01–0.14), P=0.02], the remaining comparisons revealed no statistically significant differences. The same group also compared the predictive performance of the ALPHA, GRACE 2.0, and TIMI models for 30 day risk of death in 5,203 patients using a national multi-center registry (24). The analysis showed a high discriminatory power of the GRACE 2.0 model: c-statistic =0.87 (95% CI: 0.85–0.89). Similarly, the ALPHA score performed well with a c-statistic of 0.86 (95% CI: 0.84–0.88). The difference between the two algorithms was not statistically significant (P=0.19). In contrast, the predictive ability of the TIMI score was somewhat weaker with a c-statistic of 0.81 (95% CI: 0.79–0.83). Compared with the GRACE 2.0 and ALPHA models, the difference was statistically significant (P<0.0001, in both comparisons). Thus, the predictive ability of the ALPHA score was similar to that of the more complex GRACE 2.0 model whereas both models performed statistically better than the TIMI score from the fibrinolysis era.

Limitations

In the present review we studied the discriminatory power of the models using receiver operating characteristic (ROC) curve analysis and the c-statistic (39,40). Though this approach is popular and widely accepted, it has some drawbacks. ROC analysis may present an overly optimistic picture of the model on data sets with a class imbalance (i.e., numbers of controls and cases differ substantially), like in the present validation studies. In such situations, presentation of the so called “precision recall curve” would be more appropriate, since these calculations do not make use of the true negatives, they are only concerned with the correct prediction of the less frequent positive events (41). Unfortunately, neither of the studies give such information. Moreover, we did not analyze calibration of the models, because in some works no information is available on that or authors report the result of the Hosmer-Lemeshow test, which is not considered to be an appropriate measure of model fit because of limited power and poor interpretability (39,40).

Also, we did not perform a (network) meta analysis for several reasons. The studied populations were substantially heterogeneous: different clinical settings (STEMI vs. non-STEMI and STEMI vs. ACS), various treatment modalities (primary PCI vs. fibrinolysis vs. no reperfusion therapy), different baseline risks (populations of randomized clinical trials vs. registry data), diverse prediction end points (the inherently heterogeneous in-hospital vs. 30-day vs. 6-month vs. 1-year mortality risks) that might have introduced substantial bias into the results of a network meta-analysis. Moreover, some authors did not provide a 95% confidence interval for the point estimate of the c-statistic which would be necessary for meta-analyses.

Conclusions

Mortality prediction algorithms are useful tools for patients, physicians, and clinical researchers that are also essential for quality control. Though the extensively validated GRACE model was not particularly derived from data of invasively treated STEMI patients, it also performs well in the era of transradial primary PCI. Similarly, the Zwolle, CADILLAC, APEX-AMI, and ALPHA models, that were constructed using primary PCI data, all seem to have comparable discriminative abilities. In contrast, the admission model TIMI, which was developed in the fibrinolysis era, might have less predictive power. Finally, the primary PCI admission model PAMI is likely the weakest among the comparatively studied risk models concerning discriminatory ability. Despite a large number of models, as treatment approaches evolve over time with improving outcomes and as ever older patients with complex disease patterns are treated invasively, new or updated risk prediction algorithms are needed to maintain/increase prognostic accuracy.

The article’s supplementary files as