Derivation and validation of a combined in-hospital mortality and bleeding risk model in acute myocardial infarction.

BACKGROUND: In the potent new antiplatelet era, it is important issue how to balance the ischemic risk and the bleeding risk. However, previous risk models have been developed separately for in-hospital mortality and major bleeding risk. Therefore, we aimed to develop and validate a novel combined model to predict the combined risk of in-hospital mortality and major bleeding at the same time for initial decision making in patients with acute myocardial infarction (AMI).
METHODS: Variables from the Korean Acute Myocardial Infarction Registry (KAMIR) - National Institute of Health (NIH) database were used to derive (n = 8955) and validate (n = 3838) a multivariate logistic regression model. Major adverse cardiovascular events (MACEs) were defined as in-hospital death and major bleeding.
RESULTS: Seven factors were associated with MACE in the model: age, Killip class, systolic blood pressure, heart rate, serum glucose, glomerular filtration rate, and initial diagnosis. The risk model discriminated well in the derivation (c-static = 0.80) and validation (c-static = 0.80) cohorts. The KAMIR-NIH risk score was developed from the model and corresponded well with observed MACEs: very low risk (0.9%), low risk (1.7%), moderate risk (4.2%), high risk (8.6%), and very high risk (23.3%). In patients with MACEs, a KAMIR-NIH risk score ≤ 10 was associated with high bleeding risk, whereas a KAMIR-NIH risk score > 10 was associated with high in-hospital mortality.
CONCLUSION: The KAMIR-NIH in-hospital MACEs model using baseline variables stratifies comprehensive risk for in-hospital mortality and major bleeding, and is useful for guiding initial decision making.

Introduction

In the modern era, new antiplatelet agents, such as potent P2Y12 inhibitors and glycoprotein IIb-IIIa inhibitors, have substantially decreased in-hospital mortality by reducing the ischemic burden after acute myocardial infarction (AMI) [1], [2]. However, these agents also have increased bleeding risk, which became an obstacle to use more new antiplatelet agents and to further improve clinical outcome [1], [2], [3], [4], [5], [6]. Accordingly, it became important issue how to balance the ischemic risk and bleeding risk. Previous studies have reported several risk prediction models, regarding in-hospital mortality and major bleeding [7], [8], [9], [10], [11], [12], [13], [14]. However, although the majority of predictors overlapped in each model, the in-hospital mortality and major bleeding risk have still been assessed separately in order to improve initial decision making in the acute setting of AMI. Moreover, it was often difficult to discriminate the risk at the same time because these risk prediction models were not sufficient to reflect the reality of the patient’s whole risk regarding in-hospital mortality and major bleeding. Unfortunately, there was few risk prediction models reflecting ischemic and bleeding risk at the same time, and guiding initial decision making for antiplatelet selection and interventional strategy. Therefore, we aimed to develop and validate a novel combined model to predict the combined risk of in-hospital mortality and major bleeding at the same time for initial decision making in contemporary AMI populations.

Method

Study design and patient population

The Korean AMI Registry (KAMIR) – National Institute of Health (NIH) is a Korean, prospective, open, observational, multicenter online registry of AMI with the support of the NIH since November 2011. The flow diagram of the study is shown in Supplementary Fig. 1. From November 2011 to November 2015, 13,516 patients with AMI were recruited from the KAMIR-NIH. AMI was diagnosed based on characteristic clinical presentation, serial changes on electrocardiogram indicating infarction or injury, and an increase in cardiac enzyme levels [15].

Data about patients and procedural details at the time of admission were collected and followed prospectively at each hospital. Data were recorded on a web page-based report form with electronical encryption in the NIH database. This research was supported by a fund (2013-E63005-02) from the Research of Korea Centers for Disease Control and Prevention. The protocol was approved by the ethics committee of each participating institution, and all patients provided written informed consent prior to participation.

Among baseline clinical data, initial hospital presentations were available in 12,973 patients. Of these, patients were divided by simple random sampling into a derivation cohort (70% of the total) for model development, and a validation cohort (30% of the total) for model validation. Vital signs were determined at the time of hospital presentation. Killip class on admission was assessed by the attending physician based on signs and symptoms of heart failure at the time of presentation. The baseline estimated glomerular filtration rate (eGFR) was calculated by the Modification of Diet in Renal Disease (MDRD) Study equation. Initial diagnosis was made by the attending physician based on the electrocardiogram at presentation.

Clinical outcomes

Major adverse cardiovascular event (MACE) was defined as a composite of mortality and major bleeding during hospitalization. Mortality was defined as all-cause mortality during hospitalization. Major bleeding was defined as an absolute Hgb decrease of ≥5 g/dL (baseline to nadir), absolute Hct decrease of ≥15% (baseline to nadir), and intracranial hemorrhage.

Statistical analyses

Data are expressed as mean ± standard deviation for continuous variables and percentages for categorical variables. All comparisons between baseline variables were assessed using the Student’s t-test for continuous variables and the Pearson’s chi-square test for categorical variables. As this model was primarily designed to assist with decision making for treatment selection, variables were limited to those known at the time of initial hospital presentation in order to avoid influences related to treatment strategy, which occurred after hospital presentation. Univariate analyses were performed to determine the predictors for in-hospital MACEs. The accuracy of each variable in predicting in-hospital MACEs was tested using receiver operating characteristics curve (ROC) analysis. The values of area under the curve (AUC) were used to rank the variables according to their predictive accuracy. All variables with p values <0.05 on univariate analysis were tested in a multivariate model to determine the independent predictors of in-hospital MACEs. We selected seven covariates for the final regression model on the basis of the strength of statistical significance (i.e. large adjusted chi-squared values) and clinical significance. Among seven variables, age, systolic blood pressure, heart rate, serum glucose, and eGFR were entered into multivariate analysis as continuous variables, and then categorized into several groups. Killip class and initial diagnosis were entered into multivariate analysis as categorical variables. The logistic generalized estimating equations method with exchangeable working correlation matrix was used to account for within-hospital clustering because patients at the same hospital were more likely to have similar responses, relative to patients at other hospitals. This method produced estimates similar to those from logistic regression, but variances were adjusted for the correlation of outcomes within a hospital. The discriminative performance of all the models was calculated by c statistics. The accuracy of calibration was assessed by plotting the predicted versus observed in-hospital MACEs according to population deciles of predicted risk.

The KAMIR-NIH in-hospital mortality and major bleeding risk score was created by assigning weighted integers to each variable (on the basis of the coefficient of each variable) in the final KAMIR-NIH in-hospital mortality and major bleeding model. The final risk score was calculated by calculating the sum of the individual weighted values. Using this as a continuous variable, the predicted probability of in-hospital mortality and major bleeding was plotted against the KAMIR-NIH in-hospital mortality and major bleeding risk score. The risk score was also divided into quintiles in order to compare the observed in-hospital mortality and major bleeding rates across categories: very low risk (≤5), low risk (6 to 10), moderate risk (11 to 15), high risk (16 to 20), and very high risk (>20). Both the KAMIR-NIH in-hospital mortality and major bleeding model and the KAMIR-NIH risk score were then tested in the validation cohort and also in the following clinically relevant patient subgroups in both the derivation and validation cohorts: male, female, patients ≥75 years of age, patients <75 years of age, ST-segment elevation MI, and non-ST segment elevation MI. For all analyses, a two-sided p-value of <0.05 was considered statistically significant. Statistical analysis was performed using SPSS software (version 20.0, SPSS Inc., Chicago, IL, USA).

Results

The baseline characteristics of the study subjects are shown in Table 1. There were no significant differences between the derivation and validation cohorts. The mean age of the subjects was 64 years, and 73.0% were men. Approximately 47% of the population presented with ST-segment elevation MI. The in-hospital MACEs rate was 5.0% in both the derivation and validation cohorts. In univariate analysis (Table 2), the mean age, heart rate at admission, and Killip class >1 were significantly higher, male sex and current smoker status were significantly less frequent, and body mass index, systolic blood pressure, and left ventricular ejection fraction were significantly lower in patients with in-hospital MACEs compared to the patients without. A previous history of hypertension and diabetes mellitus were more common, whereas hyperlipidemia was less common among patients with in-hospital MACEs than among patients without. Laboratory data indicated that the serum glucose and CK-MB concentrations were significantly higher, whereas the eGFR was significantly lower in patients with in-hospital MACEs. The prescription rates of aspirin, oral anticoagulants, beta-blockers, angiotensin converting enzyme inhibitors/angiotensin type II receptor blockers, and statins were significantly lower in patients with in-hospital MACEs.

Table 1

Baseline characteristics of derivation and validation cohorts.

Variables	Derivation cohort(n = 8,955)	Validation cohort (n = 3,838)	P value
Demographics
Age, year-old	64.0 ± 12.6	64.1 ± 12.7	0.790
Male (%)	6613 (73.8)	2799 (72.9)	0.280
Body mass index (kg/m2)	24.0 ± 3.5	24.0 ± 3.5	0.648
Initial presentation
Systolic blood pressure (mmHg)	130.8 ± 28.4	131.1 ± 28.1	0.501
Heart rate (beats/min)	79.2 ± 19.2	79.1 ± 18.8	0.911
Killip class > 1 (%)	1898 (21.2)	846 (22.0)	0.284
Past medical history
Hypertension (%)	4570 (51.0)	1980 (51.6)	0.564
Diabetes mellitus (%)	2603 (29.1)	1083 (28.2)	0.331
Hyperlipidemia (%)	1010 (11.3)	423 (11.0)	0.672
Previous coronary artery disease (%)	1479 (16.5)	648 (16.9)	0.609
Current smoking (%)	3460 (38.6)	1456 (37.9)	0.455
Laboratory findings
Serum glucose (mg/dL)	168.5 ± 81.1	168.3 ± 81.7	0.891
eGFR (ml/min)	84.4 ± 29.6	83.4 ± 29.8	0.092
CK-MB (ng/mL)	108.2 ± 166.0	107.2 ± 152.0	0.749
Left ventricular ejection fraction (%)	52.0 ± 11.4	52.1 ± 11.2	0.438
Initial diagnosis			0.323
ST-elevation myocardial infarction (%)	4201 (46.9)	1764 (46.0)
Non-ST elevation myocardial infarction (%)	4754 (53.1)	2074 (54.0)
Medical therapy
Aspirin (%)	8900 (99.4)	3813 (99.3)	0.807
Clopidogrel (%)	7018 (78.4)	3007 (78.3)	0.978
Prasugrel (%)	1079 (12.0)	451 (11.8)	0.634
Ticagrelor (%)	1925 (21.5)	831 (21.7)	0.845
Beta-blockers (%)	7225 (80.7)	3051 (79.5)	0.122
ACE inhibitors (%)	4043 (45.1)	1747 (45.5)	0.700
ARBs (%)	2833 (31.6)	1201 (31.3)	0.701
Statins (%)	8002 (89.4)	3386 (88.2)	0.06

Data expressed as mean ± SD or number (percent).

*Estimated by MDRD formula.

eGFR = estimated glomerular filtration rate; ACE = angiotensin converting enzyme; ARB = angiotensin type II receptor blocker.

Table 2

Univariate analysis for major adverse cardiovascular events in hospital in the derivation cohort.

Variables	MACE		P-value
	No (n = 8505)	Yes (n = 450)
Demographics
Age (years)	63.7 ± 12.5	71.3 ± 12.1	<0.001
Male (%)	6327 (74.4)	286 (63.6)	<0.001
Body mass index (kg/m2)	24.1 ± 3.4	23.2 ± 3.8	<0.001
Presentation
Systolic blood pressure (mmHg)	132.1 ± 27.7	113.3 ± 30.2	<0.001
Heart rate (beats/min)	78.6 ± 18.7	89.3 ± 26.1	<0.001
Killip class > 1 (%)	1647 (19.4)	251 (55.8)	<0.001
Medical history
Hypertension (%)	4296 (50.5)	274 (60.9)	<0.001
Diabetes mellitus (%)	2424 (28.5)	179 (39.8)	<0.001
Hyperlipidemia (%)	977 (11.5)	33 (7.3)	0.007
Previous coronary artery disease (%)	1404 (16.5)	75 (16.7)	0.930
Current smoking (%)	3348 (39.4)	112 (24.9)	<0.001
Laboratory findings
Serum glucose (mg/dL)	165.7 ± 77.5	222.5 ± 119.2	<0.001
eGFR (ml/min)	85.4 ± 29.0	64.4 ± 33.2	<0.001
CK-MB (ng/mL)	105.7 ± 160.8	155.4 ± 239.5	<0.001
Left ventricular ejection fraction (%)	52.3 ± 11.1	42.8 ± 13.6	<0.001
Initial diagnosis			<0.001
ST-elevation myocardial infarction (%)	3932 (46.2)	269 (59.8)
Non-ST elevation myocardial infarction (%)	4573 (53.8)	181 (40.2)
Medication during hospitalization
Aspirin (%)	8467 (99.6)	433 (96.2)	<0.001
Clopidogrel (%)	6681 (78.6)	337 (74.9)	0.066
Prasugrel (%)	1037 (12.2)	42 (9.3)	0.069
Ticagrelor (%)	1840 (21.6)	85 (18.9)	0.167
Beta-blockers (%)	7091 (83.4)	134 (29.8)	<0.001
ACE inhibitors (%)	3961 (46.6)	82 (18.2)	<0.001
ARBs (%)	2800 (32.9)	33 (7.3)	<0.001
Statins (%)	7836 (92.1)	166 (36.9)	<0.001

Data expressed as mean ± SD or number (percent).

MACE = major adverse cardiovascular events; eGFR = estimated glomerular filtration rate; ACE = angiotensin converting enzyme; ARB = angiotensin type II receptor blocker.

In multivariate analysis, we determined the seven most statistically significant variables, including age, Killip class, systolic blood pressure, heart rate at admission, glucose, eGFR, and initial diagnosis, to be included in the final model (Table 3). The c statistic for the final model was 0.81 in the derivation cohort and 0.80 in the validation cohort. The accuracy of calibration of the KAMIR-NIH in-hospital MACE model was very good (Fig. 1). Furthermore, the model discriminated and calibrated well when tested in key patient subgroups, including males (c-static = 0.82) versus females (c-static = 0.80), age < 75 (c-static = 0.80) versus ≥ 75 years (c-static = 0.74), and ST-segment elevation MI (c-static = 0.80) versus non-ST segment elevation MI (c-static = 0.80) in the validated cohort (Supplementary Fig. 2).

Table 3

Multivariate analysis: factors associated with in-hospital mortality and major bleeding.

Variables	Derivation cohort		Validation cohort
	χ2	OR (95% CI)	OR
Killip class	573.5	1.563 (1.417–1.724)	1.578 (1.364–1.827)
eGFR	216.4	0.991 (0.988–0.995)	0.988 (0.982–0.994)
Serum glucose	209.8	1.002 (1.001–1.003)	1.002 (1.001–1.004)
Systolic blood pressure	190.8	0.985 (0.981–0.989)	0.986 (0.980–0.991)
Age	158.2	1.039 (1.030–1.049)	1.032 (1.017–1.046)
Heart rate	132.6	1.018 (1.013–1.022)	1.015 (1.008–1.022)
Initial diagnosis	31.5	1.693 (1.367–2.098)	1.341 (0.969–1.584)
c statistic		0.80	0.80

OR = Odds ratio; CI = confidence interval; eGFR = estimated glomerular filtration rate.

Fig. 1

Comparison of predicted versus observed major adverse cardiac events rate for the validation cohort.

The KAMIR-NIH risk score is shown in Fig. 2, and the plot shows the association between the KAMIR-NIH risk score and the predicted probability of in-hospital MACEs in the derivation cohort. The KAMIR-NIH risk score showed very good discrimination among patients with various degrees of risk for in-hospital MACEs in both the derivation and validation cohorts (c statistics of 0.80 and 0.79, respectively). In the derivation cohort, most patients had a total KAMIR-NIH risk score from 6 to 10 (n = 3614, 40.4%) or 11 to 15 (n = 2244, 25.1%). Fig. 3 shows the observed in-hospital MACE rates increasing across the risk score categories in the derivation and validation cohorts (P for trend <0.001). For the entire derivation cohort, the rates of in-hospital MACE were 0.6% (very low risk; KAMIR-NIH risk score ≤5), 1.7% (low risk; KAMIR-NIH risk score from 6 to 10), 4.1% (moderate risk; KAMIR-NIH risk score from 11 to 15), 9.0% (high risk; KAMIR-NIH risk score from 16 to 20), and 22.0% (very high risk; KAMIR-NIH risk score >20).

Fig. 2

The Korean Acute Myocardial Infarction Registry – National Institute of Health prediction score and nomogram for in-hospital mortality and major bleeding.

Fig. 3

Rate of observed in-hospital mortality and major bleeding across the Korean Acute Myocardial Infarction Registry – National Institute of Health risk score categories in the derivation and validation cohort.

In patients with MACEs, a KAMIR-NIH risk score ≤10 (very low risk and low risk) was associated with high bleeding risk, whereas a KAMIR-NIH risk score >10 (moderate risk, high risk, and very high risk) was associated with high in-hospital mortality (Supplementary Fig. 3).

Discussion

There are several principle findings of this large observational study. First, we derived and validated a combined in-hospital mortality and major bleeding risk model in patients with AMI. Second, we determined seven independent variables associated with in-hospital mortality and major bleeding. Third, our novel KAMIR-NIH risk prediction model performed well in a validation cohort and in various key subgroups. Fourth, our concise score system can facilitate early risk evaluation of both in-hospital mortality and major-bleeding at the same time. Fifth, the KAMIR-NIH risk score can guide the initial management strategy such as selection of new antiplatelet agents.

To the best of our knowledge, there was few risk models to consider both in-hospital mortality and bleeding risk at the same time. Although several risk models for in-hospital mortality and major bleeding have been developed for patients with AMI [7], [8], [9], [10], [11], [12], [13], [14], each model was developed separately for in-hospital mortality and major bleeding risk. Therefore, these previous models are somewhat inconvenient to apply to real clinical practice, and they do not accurately reflect the overall risk to the patient. The KAMIR-NIH risk score not only provides integrated information on in-hospital mortality and major bleeding but also discriminates their risk according to various degrees.

The most intriguing finding of this study is that the KAMIR-NIH risk score system uses seven variables that are available at the time of hospital presentation. In this regard, we believe that the KAMIR-NIH risk score can play an important role in predicting clinical risk and determining the treatment strategy for several reasons. First, the treatment strategy for AMI should be selected with an individual’s baseline ischemic risk and bleeding risk. Some risk models consider treatment modalities (i.e., glycoprotein IIb-IIIa inhibitors and the use of hemodynamic support devices) as variables in determining their risk, which limits their universal adoption [12], [16]. Therefore, to overcome this limitation, it is necessary to develop a model using baseline clinical factors for increased generalizability.

Second, the initial presentation could be a type of barometer that is reflective of the real risk to the patient. Initial presentation after AMI is mainly determined by the amount of myocardial ischemia and necrosis. If myocardial necrosis affects a sufficiently large amount of the myocardium, left ventricular contractility can be reduced, thereby decreasing the cardiac output and increasing the left ventricular pressure. These effects result in pulmonary congestion and an abrupt worsening of renal function. Myocardial necrosis also activates the sympathetic nervous system and the renin-angiotensin-aldosterone system. Therefore, baseline factors could be a good indicator for risk prediction.

Third, initial presentation could enable early risk stratification, which can facilitate the establishment of a fast and safe clinical pathway. A previous study reported that 40% of patients were at a higher than average risk of death and major bleeding [14]. However, these dual high-risk patients were undertreated, both in terms of acute pharmacotherapy and early invasive angiography. Moreover, despite the high-risk of bleeding, they received excessive doses of antithrombotic medications, paradoxically [14]. There were also substantial variations in treatment patterns in patients with similar risk for both in-hospital mortality and major bleeding as a result of the clinicians’ preference and not by clinical evidence. Therefore, we suggest a KAMIR-NIH risk score-guided management algorithm based on the results of the current study (Graphic abstract). According to the KAMIR-NIH risk score, we can adjust the type and dose of antithrombotic medications, as well as the timing and modalities of the interventional strategy. We believe that our combined mortality and bleeding risk prediction model based on the initial presentation could optimize the risk–benefit ratio in the acute phase of AMI.

We did not compare the KAMIR-NIH risk model directly against existing models because it was not our intention to replace them. Furthermore, it would be challenging to perform a direct comparison because of differences in the patient population and variables in the models. For instance, GRACE investigators included patients with unstable angina as well as those with AMI. In addition, the CRUSADE bleeding risk model was developed by studying an older cohort that predominantly included patients with non-ST segment elevation MI [17], [18]. Rather, we are aimed to create a combined risk prediction model that could be applied to all patients with AMI, including ST-segment elevation MI and non-ST segment elevation MI, thereby providing an integrated decision-making. Moreover, previous risk models are unable to guide the initial decision making process.

Study limitations

Our research has several limitations to consider. First, since the KAMIR-NIH was an observational study, we cannot completely exclude the possibility of selection bias. The participating hospitals in KAMIR-NIH are larger tertiary referral centers and are more likely to have percutaneous and surgical revascularization capabilities. Therefore, baseline characteristics, treatment patterns, and outcomes in these hospitals may not accurately reflect those of the other hospitals. Second, we were unable to control unmeasured factors, such as individual patient general health status (frailty and physical disability), and other non-cardiac, co-existing disease, which may have affected the outcomes. Third, we performed internal validation, not external validation, and as such, further studies are required to validate our risk model across various registries. Fourth, some patients experienced both major bleeding and in-hospital death. However, it was uncertain that these patients died because of major bleeding in our registry. Fifth, although all-comers with AMI were enrolled in the KAMIR-NIH registry, 723 patients with missing data including initial hospital presentations were excluded from this study. Therefore, complete case analysis excluding patients with any missing data would have some possibility to lead to bias and larger standard errors. However, these limitations should not undermine strengths of this study, which includes a cohort that is representative of the patients that are encountered in day-to-day clinical practice.

Conclusion

The KAMIR-NIH risk score system represents a simple but comprehensive and accurate risk assessment tool for combined in-hospital mortality and major bleeding risk in AMI by using only baseline variables. This novel model could be useful and practical for initial assessment, guiding clinical decision-making, and early risk stratification for AMI

Funding

This research was supported by a fund (2013-E63005-02) by Research of Korea Centers for Disease Control and Prevention.

CRediT authorship contribution statement

Hong Nyun Kim: Conceptualization, Investigation, Writing – original draft. Jang Hoon Lee: Conceptualization, Methodology, Investigation, Formal analysis, Data curation, Writing – review & editing. Hyeon Jeong Kim: Data curation. Bo Eun Park: Data curation. Se Yong Jang: Investigation, Data curation. Myung Hwan Bae: Investigation, Data curation. Dong Heon Yang: Investigation, Data curation. Hun Sik Park: Investigation, Data curation. Yongkeun Cho: Investigation, Data curation. Myung Ho Jeong: Investigation, Data curation, Project administration, Funding acquisition. Jong-Seon Park: Investigation, Data curation. Hyo-Soo Kim: Investigation, Data curation. Seung-Ho Hur: Investigation, Data curation. In-Whan Seong: Investigation, Data curation. Myeong-Chan Cho: Investigation, Data curation. Chong-Jin Kim: Investigation, Data curation. Shung Chull Chae: Investigation, Data curation, Supervision. : .

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.