Development and Validation of a Prediction Rule for Major Adverse Cardiac and Cerebrovascular Events in High-Risk Myocardial Infarction Patients After Primary Percutaneous Coronary Intervention.

Background and Aims: We aimed to develop a clinical prediction tool to improve the prognosis of major adverse cardiac and cerebrovascular events (MACCE) among high-risk myocardial infarction (MI) patients undergoing primary percutaneous coronary intervention (PCI).
Methods: The present study was a prospective and observational study. A total of 4151 consecutive MI patients who underwent primary PCI at Fuwai Hospital in Beijing, China (January 2010 and June 2017) were enrolled. Forty-eight patients without follow-up data were excluded from the study. The pre-specified criteria (Supplementary Information 1) were chosen to enroll MI patients at high risk for MACCE complications after PCI.
Results: The full model included seven variables, with a risk score of 160 points. Derivation and validation cohort models predicting MACCE had C-statistics of 0.695 and 0.673. The area under the curve (AUC) of the survival receiver operating characteristic curve (ROC) for predicting MACCE was 0.991 and 0.883 in the derivation and validation cohorts, respectively.
Conclusion: The predicted model was internally validated and calibrated in large cohorts of patients with high-risk MI receiving primary PCI to predict MACCE and showed modest accuracy in the derivation and validation cohorts.

Introduction

Early primary percutaneous coronary intervention (PPCI) has been established as the first-line treatment for patients with acute myocardial infarction (MI).1 A randomized trial of moderate size2–7 showed a significant increase in major adverse cardio cerebral events (MACCE) post PPCI. Indeed, residual atherothrombotic risk remains substantial in high-risk patients and lesion subsets, including those older than 65 years, with renal dysfunction, diabetes mellitus (DM), thrombotic target lesions, and multi-vessel disease. The Framingham Heart Study investigators have developed various cardiovascular disease risk prediction models that identify high-risk patients more precisely than the conventional classification. It is beneficial and effective that pretreatment risk factors to reduce the risk of cardiovascular disease in patients who are evaluated as being high-risk with multivariable prediction equations than treating patients with high levels of single risk factors.8 However, few tools have been provided to assess the incidence of MACCE among high-risk MI patients undergoing PPCI to guide long-term risk management. Using these specific data elements, a new risk score was created, with which we sought to 1) define major independent predictors of MACCE among high-risk MI patients post PPCI and 2) develop and validate a full pre-procedure risk prediction model that can be adapted to individuals based on precision medicine and healthcare decisions. We present the following article in accordance with the TRIPOD reporting check. The main purpose of predictive models is to support medical decision-making. Therefore, it is important to determine the target population of the predictive models to develop and validate a representative database. At the same time, the target population must be clearly described to evaluate or validate the model performance and identify the target for its proper usage. Including these variables minimizes the risk of being unduly penalized for hospitals that disproportionately care for patients with these high-risk characteristics. Ultimately, this model could improve long-term treatment safety at centers with higher than expected risk-adjusted MACCE rates. Therefore, we consider it necessary to establish a predicting model for identifying high-risk patients.

Materials and Methods

Study Population – Enrollment and Randomization

This observational, prospective cohort study analyzed data from Fuwai Hospital, National Center for Cardiovascular Diseases, Peking Union Medical College, and the Chinese Academy of Medical Sciences. In total, 4151 consecutive MI patients who underwent PPCI at Fuwai Hospital, Beijing, China, between January 2010 and June 2017 were selected for inclusion. Forty-eight patients without follow-up data were excluded from this study. For enrollment in the study, patients were required to meet at least one of the clinical inclusion criteria and one of the angiographic inclusion criteria, but not none of the exclusion criteria, as shown in Figure 1. The clinical criteria were as follows: 1) adult patients ≥65 years of age; 2) female sex; 3) documented peripheral artery disease (PAD) or coronary artery disease (CAD)/PAD revascularization; 4) diabetes mellitus; 5) chronic kidney disease; and 6) troponin positivity. The angiographic inclusion criteria were 1) multivessel coronary artery disease; 2) target lesion requiring a total stent length >30 mm; 3) thrombotic target lesion; 4) bifurcation lesions; 5) left main (≥50%) or proximal left anterior descending (LAD) (≥70%) lesion; and 6) calcified target lesion. These prespecified criteria were selected to enroll patients with MI at a high risk for MACCE complications after PPCI. Elements are included in validated risk scores for ischemia, bleeding, or both types of complications after PPCI.10–18 R software was used to divide the derivation and validation cohorts randomly and proportionally (70%:30%). All patients were referred to the coronary catheterization center with a diagnosis of MI fulfilling the criteria for PPCI according to guidelines.19,20 The study was approved by the Ethics Committee of Fuwai Hospital, and all enrolled patients provided written informed consent for coronary angiography and PPCI. Patient records, including demographics, medical history, physical examination, blood test results, electrocardiography (ECG), echocardiography data, and discharge medication regimens, were reviewed. The primary endpoints were identified and extracted from recordings of hospital, clinical notes, laboratory report in the event of MACCE by physicians who in charge of follow-up blinded to the clinical and laboratory information via telephone enquiry and health records which get permission from the Review Board of Fuwai Hospital. Blood testing was performed at the clinical laboratory of Fuwai Hospital. The experimental protocols and the process for obtaining informed consent were approved by the appropriate ethics review committee of Fuwai Hospital. This study conformed to the principles outlined in the Declaration of Helsinki. Informed written consent was obtained from the patients prior to the inclusion.

Figure 1

Study sample selection flow diagram.

Definitions and Primary Outcome

The primary outcome of this analysis was MACCE, which was defined as the composite of all-cause death, recurrent MI, stroke (including ischemic stroke and hemorrhagic stroke), heart failure, or target-vessel revascularization. Hypertension was defined as blood pressure ≥140/90 mmHg on three occasions at rest or a previous diagnosis of hypertension and current use of antihypertensive drugs. DM was defined according to the 75-g oral glucose tolerance test (OGTT), and patients were diagnosed with DM if they met one of the following criteria: (i) a fasting plasma glucose level of ≥7.0 mmol/L, (ii) a 2-h value of ≥11.1 mmol/L in the 75-g OGTT, and (iii) a casual plasma glucose level of ≥11.1 mmol/L. Dyslipidemia, guided by the 2019 ESC/EAS Guidelines9 for the management of dyslipidemia, was defined as a lipid profile consisting of the following abnormalities either singly or in combination: A triglyceride level ≥150 mg/dL, low-density lipoprotein cholesterol level ≥100 mg/dL, total cholesterol level ≥200 mg/dL, and high-density lipoprotein cholesterol level <40 mg/dL. Height and weight were measured by trained medical staff, and body mass index was calculated as weight (kg)/height squared (m2). The no-reflow phenomenon was defined as thrombolysis in MI (TIMI) flow grade <3 after PPCI. Stroke was defined by the World Health Organization (WHO) Multinational Monitoring of Trends and Determinants in Cardiovascular Diseases (MONICA) standards, as a rapidly developing focal or general brain dysfunction that lasts for more than 24 h or causes death, excluding non-vascular causes (such as trauma, metabolic disorders, tumors, and any neurological abnormalities caused by CNS infection). According to the imaging examination in the first week of onset, the neurologist-diagnosed stroke included subarachnoid hemorrhage, intracranial hemorrhage, cerebral thrombosis, and cerebral embolism; hemorrhagic stroke included subarachnoid and intracranial hemorrhages; and ischemic stroke included cerebral thrombosis and embolism. Transient ischemic attack (TIA) and chronic cerebrovascular diseases were not included. The outcomes of this study included initial stroke only. Investigators collected data, including head CT and head MRI, and hospital records from patients during their hospitalization.

Statistical Analysis

The normal distribution of outcome variables was confirmed using the Kolmogorov–Smirnov test. Categorical variables are summarized as frequencies (percentages) and were compared using Pearson’s chi-squared tests. Continuous variables are presented as medians and were compared using an independent t-test. The characteristics of the derivation and validation cohorts are shown in Table 1. The study population was randomly split into a development sample consisting of 70% admissions and a validation sample consisting of the remaining 30% admissions. The baseline patient characteristics and variables from coronary angiography and diagnostic catheterization were considered candidate variables and were prespecified. Candidate variables had <1.8% missing data, except for the use of Intra-aortic balloon pump (IABP) (27%), apolipoprotein A (29.2%), and uric acid (12.6%). To account for missing values in baseline serum IABP (27%) and ApoA (29.2%), covariates for each model were identified in an iterative process that multiple imputation with automated variable selection. Five imputations were generated from the original dataset, and using multivariate normal regression was used to substitute missing values for IABP and ApoA within each impute in the first step.

Table 1

The Characteristics of Derivation Cohort and Validation Cohort

Variables	Derivation Cohort (n=2384)			Validation Cohort (n=1020)
	MACCE (n=578)	No MACCE (n=1806)	P value	MACCE (n=251)	No MACCE (n=769)	P value
Age (years)	65.1246	58.8632	<0.0001	64.7331	58.6736	<0.0001
Male [%(n)]	405 (70.07%)	1412 (78.18%)	<0.0001	177 (70.52%)	577 (75.03%)	0.1572
Height (cm)	167.7623	168.4187	0.0718	168.0256	168.2891	0.6317
Weight (kg)	73.1232	73.8718	0.2545	73.6537	73.5102	0.8778
BMI (kg/m2)	25.8476	25.9597	0.5528	25.9623	25.8643	0.7119
Heart rate (beats per minute)	79.5655	77.3499	0.0033	80.5840	76.1917	0.0001
SBP (mmHg)	124.6982	124.3561	0.7031	121.6345	126.1836	0.1725
DBP (mmHg)	73.0745	74.4174	0.0315	72.3109	74.2051	0.0470
Hypertension[%(n)]	385 (66.61%)	1069 (59.19%)	0.0015	172 (68.53%)	465 (60.47%)	0.0221
Diabetes[%(n)]	234 (40.48%)	677 (37.49%)	0.1966	98 (39.04%)	277 (36.02%)	0.3884
Hyperlipidemia[%(n)]	515 (89.10%)	1672 (92.58%)	0.0082	226 (90.04%)	727 (94.54%)	0.0125
Smoking[%(n)]	342 (62.75%)	1083 (66.08%)	0.1580	149 (63.14%)	455 (65.56%)	0.4998
Previous PCI[%(n)]	98 (16.96%)	273 (15.12%)	0.2885	44 (17.53%)	131 (17.04%)	0.8567
Previous CABG[%(n)]	11 (1.90%)	19 (1.05%)	0.1101	8 (3.19%)	6 (0.78%)	0.0044
Atrial fibrillation[%(n)]	48 (8.30%)	85 (4.71%)	0.0010	28 (11.16%)	41 (5.33%)	0.0014
CKD[%(n)]	64 (11.07%)	146 (8.08%)	0.0274	36 (14.34%)	59 (7.67%)	0.0016
Laboratory examinations
HDL-cholesterol (mg/dl)	1.6936	1.7091	0.7880	1.6525	1.7000	0.5692
LDL-cholesterol (mg/dl)	2.6939	2.7461	0.2561	2.7078	2.7219	0.8267
Triglycerides (mg/dl)	1.0528	1.0565	0.7904	1.0606	1.0419	0.3483
LPA (g/L)	272.53	265.12	0.5234	262.53	255.06	0.6753
hs-CRP	7.9750	7.6508	0.1712	8.5646	7.1744	0.0001
D-dimer	0.8345	0.5736	0.0009	1.0090	0.6265	0.0122
Crea	85.5148	81.4937	0.0005	87.459	80.507	0.0006
eGFR	87.7076	90.0583	0.5702	96.147	92.221	0.5744
Discharge medication regimen
Statin[%(n)]	533 (95.01%)	1682 (93.86%)	0.3128	221 (90.57%)	711 (92.70%)	0.2814
Aspirin[%(n)]	554 (98.75%)	1782 (99.44%)	0.0923	237 (97.13%)	759 (98.96%)	0.0399
Clopidogrel	493 (87.88%)	1353 (75.50%)	<0.0001	205 (84.02%)	575 (74.97%)	0.0034
Ticagrelor[%(n)]	65 (11.84%)	426 (23.81%)	<0.0001	37 (15.48%)	184 (24.02%)	0.0054
ACEI [%(n)]	304 (54.19%)	1134 (63.28%)	0.0001	137 (56.15%)	501 (65.32%)	0.0097
ARB[%(n)]	54 (9.63%)	165 (9.21%)	0.7661	18 (7.38%)	52 (6.78%)	0.7488
Beta-blockers[%(n)]	479 (85.38%)	1574 (87.83%)	0.1287	211 (86.48%)	681 (88.79%)	0.3290
Diuretic[%(n)]	197 (35.12%)	531 (29.63%)	0.0142	96 (39.34%)	193 (25.16%)	<0.0001
Spironolactone[%(n)]	142 (25.31%)	407 (22.71%)	0.2039	61 (25.00%)	151 (19.69%)	0.0758
P2Y12 inhibitors	558 (99.47%)	1778 (99.22%)	0.5474	242 (99.18%)	759 (98.96%)	0.7588
Endpoint events
All caused death [%(n)]	145 (25.09%)	0 (0.00%)	<0.0001	68 (27.09%)	0 (0.00%)	<0.0001
Recurrent MI [%(n)]	81 (14.09%)	0 (0.00%)	<0.0001	37 (14.74%)	0 (0.00%)	<0.0001
Revascularization [%(n)]	346 (60.07%)	0 (0.00%)	<0.0001	144 (57.37%)	0 (0.00%)	<0.0001
Heart failure [%(n)]	23 (3.99%)	0 (0.00%)	<0.0001	11 (4.40%)	0 (0.00%)	<0.0001
Ischemic stroke [%(n)]	47 (8.15%)	0 (0.00%)	<0.0001	19 (7.60%)	0 (0.00%)	<0.0001
Hemorrhagic stroke [%(n)]	8 (1.39%)	0 (0.00%)	<0.0001	3 (1.20%)	0 (0.00%)	<0.0001
Coronary angiography
Bifurcation lesion [%(n)]	189 (33.69%)	630 (35.16%)	0.5246	74 (30.33%)	272 (35.46%)	0.1409
Multi-vessel lesions [%(n)]	487 (86.81%)	1339 (74.72%)	<0.0001	218 (89.35)	559 (72.99)	<0.0001
LM lesion [%(n)]	58 (10.34%)	104 (5.80%)	0.0002	23 (9.43%)	48 (6.26%)	0.0916
PTCA	504 (89.84%)	1566 (87.39%)	0.1193	215 (88.11%)	681 (88.79%)	0.7731
Thrombus aspiration	208 (37.08%)	784 (43.75%)	0.0052	100 (40.98%)	323 (42.11%)	0.7556
Coronary stent implantation	484 (86.27%)	1593 (88.90%)	0.0923	207 (84.84%)	680 (88.66%)	0.1130
The use of IABP	74 (13.19%)	168 (9.38%)	0.0094	35 (14.34%)	68 (8.87%)	0.0137

Note: Continuous data are presented as mean, categorical variables are presented as % (n).

Abbreviations: BMI, body mass index; SBP, systolic blood pressure; DBP, diastolic blood pressure; PCI, percutaneous coronary intervention; CABG, coronary artery bypass grafting; CKD, chronic kidney disease; HDL-C, high-density lipoprotein cholesterol; LDL-C, low-density lipoprotein cholesterol; TG, triglyceride; LPA, lipase activator; hs-CRP, high sensitive C-reactive protein; eGFR, estimated glomerular filtration rate; ACEI, angiotensin-converting enzyme inhibitor; ARB, angiotensin receptor blocker; MACCE, major adverse cardiovascular cerebrovascular event.

The variables included in the least absolute shrinkage and selection operator (LASSO) regression are shown in and are used to screen the independent variables to draw the corresponding nomogram model (). LASSO selects variables for sample data based on the penalty method. By compressing the original coefficients, the originally small coefficients are directly compressed to 0, so that the corresponding variables of these coefficients are regarded as non-significant variables and the non-significant variables are directly discarded.

We developed a predicting model using all potential predictive variables selected by LASSO regression. We also developed a risk prediction score by taking the regression coefficients from the pre-procedure model and assigning them an integer weighted associated with the risk factors. The corresponding nomogram model is drawn according to the regression coefficient of the selected independent variables. For the variables selected in the nomogram model, the values of different variables can correspond to different scores on the integral line at the top of the nomogram (the score range is 0–160 points) through the projection of the vertical line, and the total score can be obtained by adding up the scores corresponding to the values of each variable. The cumulative occurrence probability of MACCE events in 3 and 5 years can be obtained from the total score on the prediction line at the bottom of the nomogram.

In order to reduce the over-fitting bias, the self-sampling method is used to verify the nomogram model. The Harrell’s C-statistic was used to compare discrimination between derivation cohort and validation cohort including 3-year and 5-year. Calibration plots were used to access goodness of fit. We draw the survival receiver operator characteristic curve (survival ROC curve) by R language. Survival ROC curves export the best cut-off values and divided into low-risk group and high-risk group by R language. We conducted the K-M survival analysis between two groups and export the discrepancy result of the analysis. The subgroup of the K-M curves included the all-caused death, recurrence MI and stroke during 3 and 5 years. The cox regression analysis was performed using the survival package. Furthermore, due to the number of cases are sufficient, we conducted the k-fold cross validation in order to optimize the model and find the superparameter value which makes the model generalization performance optimal.

Standard statistical metrics of model and discrimination performance (R2, Harrell’s C statistic) were calculated. The calibration and discrimination performance of equations developed in the derivation sub-cohort was assessed in the validation sub-cohort and compared with the performance of models developed in the entire cohort; baseline survival functions and hazard ratios were also compared. Indicators of internal verification include c-index and calibration degree, which, respectively, represent the prediction accuracy and prediction consistency of the nomogram prediction model. The degree of calibration is represented by a calibration graph. Decision Tree is a learning technique of machine for data classification in a fixed set of classes and has been explored systematically with success in many classification problems. Supposing that data from variant calling have various features of ease-discretization, decision trees turn into an outstanding deterministic model for information search. The present work aims to classify variants in establishing the predicting model among high-risk population. In this work, we describe a decision tree modeling to improve the accuracy of the risk factor identification process. We expect that this procedure can be used to support the screening of identified risk factors. The technique of decision tree is based on a hierarchical structure of “if-then” rules which produce bifurcations and promote a tree shape. The classification consists of traverse a tree through its edges from the root and executes a test over variable.

The main software used for statistical analysis in this study used survival and rms package in R language version I 386 3.6.3. Other analyses were performed using SPSS version 20.0 (IBM Corp., Armonk, NY, United States). All p-values were two-tailed, and statistical significance was set at p < 0.05.

Results

Patient Demographics of Derivation and Validation Cohort

Between January 1, 2010, and June 30, 2017, 4151 men and women underwent PPCI at Fuwai Hospital. Of these, 48 people were excluded from the study. After applying the inclusion criteria, 3404 high-risk MI patients were included. Meeting at least one clinical and at least one angiographic criteria to be eligible were enrolled in the study. There were 578 high-risk MI subjects that had MACCE events in derivation cohort after undergoing PPCI procedures, yielding a MACCE event rate of 24.24%. Of these events, 25.09% were all-caused death; whereas 14.09% were detected due to recurrence MI, 60.07% by revascularization, 3.99% by heart failure, 8.15% by ischemic stroke and 1.39% were hemorrhagic stroke. The mean follow-up duration was 3 years. Patients must have met at least one clinical and at least one angiographic criterion to be eligible for treatment in the study. The derivation and validation cohorts comprised 2384 and 1020 patients, respectively, which were used in these analyses by random allocation (Figure 1). Candidate variables had <1.8% missing data, except for the use of IABP (27%), apolipoprotein A (29.2%), and uric acid (12.6%). Table 1 displays the baseline patient, procedural, and hospital characteristics of the development and validation samples.

Screening Risk Factors for MACCE by the LASSO Method

Baseline patient characteristics and variables from coronary angiography and diagnostic catheterization were considered candidate variables and were all prespecified (). These variables were filtered using the LASSO regression method, and the filtering and cross-validation processes for the independent variables are shown in Figure 2A1 and A2, respectively. Lambda.1se is the lambda value of the optimal efficiency model within the standard error range, which provides a model with excellent performance.

Figure 2

The establishment procedure of the model. (A1 and A2) Least absolute shrinkage and selection operator (LASSO) regression. The filtering and cross-validation processes of independent variables are shown in (A1 and A2) respectively. Lambda.1se is the lambda value of the optimal efficiency model in the standard error range which gives a model with excellent performance. (B) Forest plot by using the multivariable COX regression; HR, hazard ratio; CABG, coronary artery bypass grafting. (C) Decision tree flow diagram. The binary decision diagram of the variables was shown in (C and D) the risk score nomogram. The score, ranging from 0 to 160, assigned points as follows: for patients younger than 40 years, 100 points; for age 40 to younger than 50 years, 80 points; for age 50 to younger than 60 years, 60 points; for age 60 to younger than 70 years, 40 points; for age 70 to younger than 80 years, 20 points; for patients 80 years or older,0; for Killip II, 7.68; for Killip III, 15.36; for Killip IV, 23.03; for EF at admissio≦50%, 4.62; for previous history of CABG, 20; for in-stent restenosis, 4.81; for stent thrombosis, 9.62; for without complete revascularization, 3.45; for multi-vessel lesion, 18. Age group, 1 stand for age less than 40 years/ 2 stand for age range from 40 to 50 years/ 3 stand for age range from 50 to 60 years/4 stand for age range from 60 to 70 years/ 5 stand for age range from 70 to 80 years/ 6 age stand for age more than 80 years. Killip classification, 1= Killip I, 2= Killip II, 3= Killip III, 4=Killip IV. EF, 0 stands for >50%, 1 stands for less than 50%. History of CABG, 1=with, 0=without; type of lesion, 1= De novo lesion, 2=restenosis, 3= stent thrombosis; complete revascularization, 0=without, 1=with; multivessel disease, 1=with, 0=without. Histogram refers to the score distribution in the derivation cohort. For the variables selected in the nomogram model, the values of different variables can correspond to different scores on the integral line at the top of the nomogram (the score range is 0–160 points) through the projection of the vertical line, and the total score can be obtained by adding up the scores corresponding to the values of each variable. The cumulative occurrence probability of MACCE in 30 days, 3 year, 5 year and 7 years can be obtained from the total score on the prediction line at the bottom of the nomogram. (E) Elements of clinical prediction score and distribution of score among high-risk MI patients who underwent PPCI. (E) The left side has shown the clinical prediction score variable and corresponding points. The right side has shown the graph of distribution of the clinical prediction score.

Establishment of the Risk Prediction Model

Seven independent variables (age subgroup, Killip classification, ejection fraction, history of coronary artery bypass grafting (CABG), type of lesions, complete revascularization at admission, and multi-vessel disease of the coronary artery) were included in the predictive model. The forest plot of the variables obtained by the multivariate Cox regression is shown in Figure 2B, and the binary decision diagram of the variables is shown in Figure 2C. Multivariate Cox regression by full model from observed data is shown in and . Furthermore, multivariate Cox regression by stepwise selected model from observed data is shown in and . ROC curve analysis and optimal threshold analysis of prediction model are shown in .

It is necessary to convert the classification variables into factorization and then use the as.matrix () function to convert the data from the non-matrix format to the matrix format before the R language “glmet” package can call the data. According to the nomogram model (Figure 2D), the score predicting project included seven variables as predictive factor variables.

Clinical Prediction Score

A simplified risk score was generated to predict MACCE events. The scores, ranging from 0 to 160, were as follows: for patients <40 years, 100 points; for age 40 to <50 years, 80 points; for age 50 to <60 years, 60 points; for age 60 to <70 years, 40 points; for age 70 to <80 years, 20 points; for patients 80 years or older, 0; for Killip II, 7.68; for Killip III, 15.36; for Killip IV, 23.03; for ejection fraction (EF) at admission ≤50%, 4.62; for previous history of CABG, 20; for in-stent restenosis, 4.81; for stent thrombosis, 9.62; for without complete revascularization, 3.45; and for multivessel lesion, 18 (Figure 2E). The elements of the clinical prediction score and distribution of scores among high-risk MI patients who underwent PPCI are shown in Figure 2E.

Performance of the Risk Score Model

The MACCE risk prediction model had good discrimination in both the development and validation samples (c-index, development sample 0.695; validation sample 0.673). The calibration plot for the full model is shown in Figure 3A–D. There was high concordance between the risk predicted by the models and observed MACCE events. Calibration was indicated by the estimated risk of survival from Kaplan–Meier analysis, in which the gray line represents perfect calibration. Figure 3E and F shows survival (time-dependent) receiver operating characteristic (ROC) curves for the discriminatory value of the 3- and 5-year evaluation performance of the risk prediction model. The cut-off points of the 3- and 5-year survival ROC curves were 0.22663 and 0.09733, respectively, and the area under the curve (AUC) was 0.991 and 0.931, respectively. The cut-off points of the 3- and 5-year survival ROC curves were –0.35597 and –0.35597, and the AUCs were 0.883 and 0.883 in the validation cohort, respectively. We have drawn the ROC curves of the Grace risk score model among the derivation cohort (N = 2384) in the as follows. The AUC of the Grace risk score model is 0.672. Furthermore, we have the drawn the ROC curves of the new established model in the manuscript and the AUC was 0.767. The AUC of the new model is larger than the grace score model, which showed excellent performance and indicated that the risk score was helpful for discrimination in the clinical prediction models. shows the decision curve analysis of the 3- and 5-year survival in the derivation and validation cohorts.

Figure 3

The internal validation of the model. (A–D) Risk score calibration in the derivation cohort and the internal validation cohort; the stroke events risk score of 3-year (A) 5-year (B) in the derivation cohort and 3-year (C) 5-year (D) in the validation cohort. Calibration is shown as the estimated risk against survival from Kaplan- Meier analysis. Gray line=perfect calibration. (E) showed the survival ROC curve of derivation cohort (AUC=0.767, p<0.001). (F) showed the survival ROC curve of validation cohort (AUC=0.696, p<0.001). AUC, area under the curve; ROC, survival receiver operating characteristic; TP, true positive; FP, false positive.

Survival ROC curves export the best cut-off values and are divided into relatively low-risk and high-risk groups using R language. We conducted Kaplan–Meier survival analysis (Figure 4A–D) and exported the discrepancy results. In the group predicting MACCE, the two groups displayed significant differences in both the derivation cohort (p < 0.001) and validation cohort (p < 0.001), as shown in Figure 4A–D. In the subgroup predicting all-cause death, there was a remarkable difference (p < 0.001) between the high-risk and relatively low-risk groups in both the development and validation groups (). Furthermore, when the endpoint was recurrent MI, the log-rank p-value was <0.02 in the 3-year derivation cohort and <0.01 in the 5-year Kaplan–Meier curve in the derivation cohort (). We also found distinct discrepancies in predicting stroke events (p < 0.05) () in the 3- and 5-year development and validation cohorts. has shown the one-fold cross validation and the AUC was 0.744 and the confidence interval was 0.645–0.789. Furthermore, we took a 10-fold cross validation. First, we divided the data into ten parts, and then we took the first data set as the training set. Finally, the fitting model is obtained and the ten-fold cross validation was 0.670.

Figure 4

K-M survival analysis; in the group of predicting MACCE events, the two groups displayed significant difference in both derivation cohort (p<0.001) and validation cohort (p<0.001) shown in (A–D).

Discussion

This study developed a clinical prediction score based on clinical and coronary angiography indices to predict the incidence of long-term MACCE among patients with high-risk MI who underwent PPCI. The MACCE prediction risk model had good discrimination in both the development and validation samples (c-index, validation sample 0.673; development sample 0.695). For patients who were divided into the relatively low-risk group and the high-risk group by the best cut-off values in the prediction model study (derivation cohort), the relatively high-risk group had a significantly greater likelihood of occurrence of all-cause death, recurrence of MI, heart failure, ischemic stroke, hemorrhagic stroke, and revascularization compared to the low-risk group. These results suggest that it may be possible to identify patients with a discordant incidence of MACCE. Although the prediction score is expected to be applied to the patients represented by the enrollment criteria, inconsistency in evaluating treatment risks and benefits and adjusting treatment according to personal data provides opportunities for further optimization of the results to maximize benefits and reduce harm. However, few equivalent scores are available for use in high-risk patients with acute MI undergoing PPCI. For these patients, cardiac imaging, coronary angiography, and advanced biomarkers are routinely available at admission; therefore, it is convenient to include them in a score for this setting for long-term management.

Many patient characteristics are correlated with the long-term incidence of MACCE. Many of the predictive elements that we identified have been previously shown to be predictive of MACCE events. For instance, age is consistently associated with an increased risk of MACCE incidence,21 as are other variables, such as Killip classification, EF at admission, history of CABG, and multi-vessel lesions of the coronary artery.22 In addition to these factors, we also identified unique variables not present in other predictive models, such as complete revascularization and the type of lesion detected by coronary angiography. The addition of such forecasting factors is noteworthy as the acuity of clinical presentation is generally not as severe as that of previous models that merely include clinical characteristics. It is important to minimize the risk of inappropriate care and management of patients with MI with high-risk characteristics who undergo PPCI by including these factors. Bleeding complications, including hemorrhagic stroke after PPCI, are common and are correlated with an increased short- and long-term risk of mortality.23,24 The use of vascular closure devices, bivalirudin, and the radial approach, known as bleeding avoidance strategies, has been proposed to reduce hemorrhage among higher-risk patients.25–28 Therefore, this model can be used to predict the long-term incidence of MACCE in high-risk MI patients post-PCI, identify leaders and laggards, and ultimately improve the long-term prognosis of PPCI by making healthcare decisions during follow-up and encouraging the adoption of corresponding measures at admission.

According to the present model, the score predicting model included seven variables of predictive factors, including age, Killip classification, ejection fraction, history of CABG, type of lesions, complete revascularization at admission, and multi-vessel disease of the coronary artery. Previous analyses29,30 have also shown that risk factors, including age, atrial fibrillation, female sex, Killip classification, and chronic disease, can predict the incidence of stroke within 12-months of PPCI, which is generally in accordance with classic risk variables in the general population. Stroke, including ischemic and hemorrhagic, is a devastating complication with high MACE and mortality rates following PPCI. Similarly, a database derived from the British Cardiovascular Intervention Society (BCIS) reported that ischemic stroke was independently associated with both 30-day mortality and in-hospital MACE after adjusting for baseline clinical and procedural demographics.31 Previously, Luke et al32 demonstrated that the incidence of stroke among outpatients following PPCI is higher in younger patients than in the general population. This is coincident with our score projection that age <40 years contributed to greater weight compared to other age groups. Furthermore, at the beginning of the cardiac catheterization period (1970–1980s), the incidence of cerebrovascular events ranged from 0.03% to 0.06%33 compared to 0.18–0.44% during the following years. The increased incidence of stroke-complicating PPCI might account for the extended use of PCI and coronary angiography, especially among patients with severe vascular calcification.34 The time of risk assessment post-event and cardiovascular disease are the main factors to evaluate the performance of the risk score for secondary prevention. CALIBER35 enrolled 102,023 patients with stable CAD and developed a risk score to identify patients at high-risk assist with management decisions.

The reason for using relatively old data from 2010 to 2017 was due to the duration of funding. Based on the national Polish PCI registry database,36 the study has enrolled 132,686 patients in high-volume interventional cardiology centers in the National Registry of Drug Eluting Stents (NRDES) Registry from October 2010 to October 2011 in Poland. The results showed a large selection bias for Drug Eluting stents with regard to angiographic and demographic characteristics in ACS. The implantation of drug-eluting stents was linked to ischemic outcomes similar to those of bare metal stents at the 1-year follow-up. In the present study, PPCI was performed via the radial or femoral access. Thrombus aspiration was performed to reduce thrombus burden and restore antegrade coronary flow. However, the proportion of radial and femoral access surgeries was not counted and the radial experience of the operator was not evaluated. In a previous study37, 539 invasive cardiologists from 151 invasive cardiology centers in Poland were included. The results suggested that the highest-proficiency operators in radial access were correlated with a higher risk of periprocedural stroke, mortality, and other complications at the access site during angiography via femoral access. In both stable and acute situations, sufficient experience and utilization of radial access are associated with adverse outcomes of PCIs performed via the femoral artery. The time of the procedure in the present study included on- and off-hours. A previous study38 demonstrated a higher rate of periprocedural mortality for PCIs performed during off-hours among 99,783 cohorts. They found that more professional and sufficiently trained operators performed PPCI during regular work hours compared to off-hours. Furthermore, the influence of fatigue and reduced operator experience among invasive interventions should be considered. Thus, the confounding factors of the procedure time should be controlled for in further investigations. A previous study39 suggested that the radial approach was associated with lower periprocedural mortality than the femoral approach in 3565 consecutive patients with MI. The radial approach appears to be a worthwhile option in technical situations. Furthermore, some comprehensive insights into the real-world clinical practice of MI were provided in this study. Radial professional and experienced operators may be critical in decreasing the number of periprocedural complications. The literature40 drew the conclusion that a reduction in the risk of periprocedural mortality was observed in greater experienced radial operators.

Study Limitation

Several limitations of our study should be considered when interpreting the results. Based on the clinical and angiographic inclusion and exclusion criteria of the trial, as well as the single-center study design, the score project of the model should be interpreted with the understanding that patients enrolled in clinical trials may not be completely representative of those cared for in the routine practice of PPCI. Therefore, the analysis should be regarded as exploratory despite the predetermination of the score variables, and the predictive score should be used with circumspection until further external validation has been performed. Optimal and suitable long-term management of the procedure and care should be administered independently of the patient’s score to reduce overall MACCE. Furthermore, the extent and severity of granular measures of atherosclerosis were unavailable, and the situation of receiving ticagrelor or other antiplatelet combinations may, in part, affect the discrimination of the cohort and have a different risk–benefit relationship.41 Finally, cerebrovascular events were determined by contacting the participants, followed by validation through medical records. Although it is likely to cover almost all hemorrhagic and ischemic strokes, stroke incidence may be underestimated if the patients are asymptomatic and not admitted to hospital.

Conclusion

In summary, we developed a risk prediction model for estimating the long-term (3- and 5-year) incidence of MACCE based on clinical parameters and indices of coronary angiography, which were suitable for high-risk patients with MI who underwent PPCI. The scoring can be implemented alongside further medical investigations to support therapeutic decision-making. This model requires further prospective assessment to evaluate the potential impacts on patient management, as well as external validation in other cohorts.