Readily accessible risk model to predict in-hospital major adverse cardiac events in patients with acute myocardial infarction: a retrospective study of Chinese patients.

OBJECTIVE: Rapid, accurate identification of patients with acute myocardial infarction (AMI) at high risk of in-hospital major adverse cardiac events (MACE) is critical for risk stratification and prompt management. This study aimed to develop a simple, accessible tool for predicting in-hospital MACE in Chinese patients with AMI.
DESIGN: Retrospective review of deidentified medical records.
SETTING: 38 urban and rural hospitals across diverse economic and geographic areas in China (Beijing, Henan Province and Jilin Province). PARTICIPANTS: 15 009 patients discharged from hospital with a diagnosis of AMI. MAIN OUTCOME MEASURE: The primary outcome was MACE occurrence during index hospitalisation. A multivariate logistic regression model (China AMI Risk Model, CHARM) derived using patient data from Beijing (n=7329) and validated with data from Henan (n=4247) and Jilin (n=3433) was constructed to predict the primary outcome using variables of age, white cell count (WCC) and Killip class. C-statistics evaluated discrimination in the derivation and validation cohorts, with goodness-of-fit assessed using Hosmer-Lemeshow statistics.
RESULTS: The CHARM model included age (OR: 1.06 per 1-year increment, 95% CI 1.05 to 1.07, p<0.001), WCC (OR per 109/L increment: 1.10 (95% CI 1.07 to 1.13), p<0.001) and Killip class (class II vs class I: OR 1.34 (95% CI 0.99 to 1.83), p=0.06; class III vs class I: OR 2.74 (95% CI 1.86 to 3.97), p<0.001; class IV vs class I: OR 14.12 (95% CI 10.35 to 19.29), p<0.001). C-statistics were similar between the derivation and validation datasets. CHARM had a higher true positive rate than the Thrombolysis In Myocardial Infarction score and similar to the Global Registry of Acute Coronary Events (GRACE). Hosmer-Lemeshow statistics were 5.5 (p=0.703) for derivation, 41.1 (p<0.001) for Henan, and 103.2 for Jilin (p<0.001) validation sets with CHARM, compared with 119.6, 34.0 and 459.1 with GRACE (all p<0.001).
CONCLUSIONS: The CHARM model provides an inexpensive, accurate and readily accessible tool for predicting in-hospital MACE in Chinese patients with AMI. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

We used data from a contemporary cohort recruited from a geographically and socioeconomically diverse set of hospitals. The China Acute Myocardial Infarction Risk Model (CHARM) model provides a simple, inexpensive, accurate and readily accessible tool for predicting in-hospital major adverse cardiac events in Chinese patients with acute myocardial infarction, which could facilitate first-line clinical practice, especially that of emergency and cardiovascular departments in different levels of urban and rural hospitals in China.

Our findings need to be interpreted with appropriate caution because of the inherent limitations of retrospective study design; thus, missing data and incorrectly recorded data are a limitation.

Although we adjusted whether a reperfusion therapy was performed, we did not consider whether Thrombolysis In Myocardial Infarction III flow was achieved in the model.

It is not unusual in China, especially in rural areas, for severely ill patients to be discharged home to die. Consequently, in-hospital mortality may have been underestimated.

Moreover, the CHARM score can only be calculated with the aid of a computer or smart phone app.

Introduction

Mortality due to coronary heart disease (CHD) steadily increased in China between 1990 and 2016, with the age-standardised mortality rate rising by 25.3% from 110.0 per 100 000 persons in 1990 to 137.7 per 100 000 in 2016.1A rapidly expanding ageing population in recent years has driven this increasing prevalence of CHD.1 2 According to the National Bureau of Statistics, China had 254 million people aged 60 years or above at the end of 2019, accounting for 18.1% of the country’s population,3 and the Ministry of Civil Affairs estimates this number will exceed 300 million in the next 5 years, leading to a subsequent increase in myocardial infarction. The China Patient-centred Evaluative Assessment of Cardiac Events study4 showed that the number of hospital admissions for ST-segment elevation myocardial infarction (STEMI) increased significantly between 2001 and 2011. There was no decrease in the risk of in-hospital mortality over the same period, however.4

Rapid and accurate risk stratification at the time of presentation enables clinicians to manage patients with acute myocardial infarction (AMI) promptly and properly. Acquiring a quick quantitative prognostic assessment of patients’ outcomes is important for clinicians to make individualised treatment decisions and to communicate with the patients and their families. Risk prediction is recommended in clinical practice guidelines for the management of AMI in Europe,5 the USA6 and also in China7 8.

Several risk models have been developed to predict the risk of death in patients presenting with AMI.9–12 One of the most widely used models is the Thrombolysis In Myocardial Infarction (TIMI) risk score,11 which incorporates seven variables (age ≥65 years, use of aspirin within the last week, ≥2 angina episodes within ≤24 hours, elevated serum cardiac biomarkers, ST-segment deviation, coronary stenosis >50%,≥3 risk factors for heart disease) in patients presenting with acute non-ST elevation acute coronary syndrome (NSTEACS), to predict the risk of in-hospital death or myocardial infarction (MI). Similarly, the Global Registry of Acute Coronary Events (GRACE) uses eight clinical parameters to predict death and MI in hospital, after 6 months12 13 and over the longer term.14 A major limitation of these two scores is that several variables may be unavailable at the time of patient presentation, with a time lag of several hours. Moreover, neither of these scoring systems were developed in the same environment and differences in race and quality of medical care are strongly associated with differential risk. The aim of our study is to develop a simple and readily accessible risk model (China AMI Risk Model, CHARM) to predict in-hospital major adverse cardiac events (MACE) in patients with AMI and to validate its performance in a Chinese patient population.

Methods

Design and setting

Our study was conducted at 38 urban and rural hospitals in Beijing (n=18), Henan province (n=10) and Jilin province (n=10) that routinely admit more than 100 patients with suspected AMI per year (online supplemental table 1).

Study population

Each hospital enrolled the first five consecutive patients hospitalised for STEMI and the first five consecutive patients hospitalised for NSTEACS during each calendar month from 1 January 2008 to 31 December 2015. Adult patients (aged ≥18 years) with a final diagnosis of STEMI or non-ST-segment elevation MI (NSTEMI) identified at the time of death or discharge were included in this analysis.

The diagnosis of AMI was determined on basis of local discharge diagnosis, ECG and cardiac markers, including creatine kinase MB isoenzymes and troponin T/I. If the local diagnosis was not definitive, cardiologists not involved in data abstraction at the coordinating centre reviewed the medical records to establish diagnosis.

Patients with potentially lethal diseases (eg, incurable cancer, decompensated cirrhosis, multisystem organ failure), and those who died within 10 min of arrival at the hospital were excluded. To be included in the analysis a patient was required to have data for age, sex, MACE and location of residence.

Patient and public involvement

Patients or the public were not involved in the design, conduct, reporting or dissemination plans of our research.

Data collection

Patient charts were reviewed by trained investigators on a standardised case report form. Deidentified data were collected, including demographic characteristics, clinical features, initial assessment and diagnosis of AMI, any therapy received for reperfusion (thrombolysis or primary percutaneous coronary intervention or coronary artery bypass grafting), medications commenced after admission and those ongoing at the time of death or discharge, the final diagnosis (STEMI, NSTEMI) and any in-hospital MACE after admission.

Primary outcome

The primary outcome of the study was the occurrence of MACE, which included cardiovascular death, MI and ischaemic stroke occurring during the index hospitalisation. The definition of cardiovascular death, MI and ischaemic stroke are in line with the 2014 American College of Cardiology/American Heart Association guidelines of key data elements and definitions for cardiovascular endpoint events in clinical trials.15

Data preprocessing

Several data preprocessing steps were applied before modelling. First, we eliminated patients with missing information for age or location of residence. Second, outliers were detected and removed using criteria based on clinical knowledge (online supplemental table 2). We then imputed missing values for other risk factors by multiple imputation utilising the multivariate imputation by chained equations (‘mice’) package in R.16 Continuous variables were imputed by predictive mean matching, binary variables were imputed by logistic regression, and a proportional odds model was used for ordinal variables.17–19

Statistical analysis

Risk factors were selected by combining variables that were statistically significantly associated with the outcome in the univariate analysis (p<0.01), as well as variables identified by clinicians as being important. Univariate analysis was conducted using the χ2 test for categorical variables and Wilcoxon rank sum test for continuous variables. All p values were two tailed, and p<0.05 was considered statistically significant. All analyses were performed with R (V.3.4.1).

A risk model was constructed to predict the primary outcome using the variables selected. A multivariate stepwise logistic regression model was fitted to balance model discrimination and the statistical significance of predictors.

We evaluated the discrimination of the model by using C-statistics on both the derivation set and the two validation sets. Calibration was assessed by constructing a 10-decile plot of observed vs predicted risk, and the Hosmer-Lemeshow goodness-of-fit test.

The performance of our model was compared with the TIMI and GRACE models. Each patient in the dataset was assigned a TIMI score20 and a GRACE probability12 using the GRACE multivariate logistic regression model. C-statistics were calculated from the receiver operating characteristic (ROC) curves for each model to compare the prediction performance of the different models. Calibration curves, net reclassification indices (NRI) and integrated discrimination improvements (IDI) were also prepared, but were compared only with GRACE. This is because calibration and decision curves are only applicable to probability scores, while NRI and IDI are meaningful only when compared with risk scores using the same scales.21–23

NRI quantifies the extent of correct reclassification when using the new model (CHARM) compared with the GRACE score. Various cutoffs (5%–10%) based on actual MACE rates across the different locations of residence (4.9%–9.1%), were applied in calculating NRI. IDI quantifies the improved discrimination of events and nonevents using the new model. A positive value for both NRI and IDI suggests the CHARM model performs better.

Results

The disposition of patients enrolled in the study is shown in figure 1, and their baseline characteristics are shown in table 1 and online supplemental table 3. Our study recruited 28 766 in-hospital ACS patients between 1 January 2008 and 31 December 2015, among whom 12 597 patients were from Beijing, 8637 were from Henan province, and 7478 were from Jilin province. Location of residence was missing for 54 patients. A total of 13 757 patients were excluded from the analysis (1055 for missing the mandatory criteria of age, sex, ACS type, MACE status and/or location of residence, 743 for being outliers, and 11 959 with unstable angina; online supplemental table 2). Thus, a total of 15 009 patients, including 2248 with NSTEMI and 12 761 with STEMI, were included in the analysis.

Figure 1

Patient disposition. ACS, acute coronary syndrome; AMI, acute myocardial infarction; MACE, major adverse cardiac events.

Table 1

Baseline characteristics

Characteristic	All (N=15 009)	Beijing (N=7329)	Henan (N=4247)	Jilin (N=3433)
Sex (male), % (n/N)	71.2 (10 691/15 009)	72.0 (5277/7329)	74.1 (3145/4247)	66.1 (2269/3433)
Age (years), mean (SD)	62.4 (12.9)	62.6 (13.1)	61.8 (12.9)	62.7 (12.3)
Medical cost type (self-pay), % (n/N)	10.2 (1474/14 432)	5.7 (400/7021)	14.2 (567/4007)	14.9 (507/3404)
Current smoking, % (n/N)	48.3 (6651/13 763)	51.6 (3659/7090)	44.6 (1680/3765)	45.1 (1312/2908)
Time from symptom onset to hospital arrival (hours), mean (SD)	12.1 (23.4)	14.3 (26.8)	8.4 (15.4)	11.5 (23.1)
Systolic blood pressure (mm Hg), mean (SD)	129.3 (24.2)	128.5 (22.5)	127.4 (24.4)	133.4 (26.7)
Heart rate (bpm), mean (SD)	77.5 (18.0)	77.6 (17.1)	76.6 (17.5)	78.5 (20.2)
Medical history, % (n/N)
Established coronary artery disease	25.3 (2200/8686)	29.7 (1396/4704)	15.8 (342/2167)	25.5 (462/1815)
Hypertension	55.8 (7836/14 033)	61.9 (4362/7051)	48.5 (1859/3834)	51.3 (1615/3148)
Diabetes mellitus	26.1 (3507/13 449)	31.5 (2134/6778)	20.1 (724/3610)	21.2 (649/3061)
Hyperlipidaemia	31.3 (1669/5330)	41.0 (1484/3617)	10.8 (142/1314)	10.8 (43/399)
Heart failure	2.5 (112/4394)	2.6 (62/2417)	1.1 (14/1264)	5.0 (36/713)
Ischaemic stroke	26.3 (1967/7467)	25.0 (1064/4262)	21.6 (451/2092)	40.6 (452/1113)
Chronic kidney disease	6.3 (314/4966)	8.4 (241/2879)	2.6 (32/1230)	4.8 (41/857)
White cell count (×109/L), mean (SD)	9.8 (3.5)	9.5 (3.4)	10.1 (3.5)	10.1 (3.6)
Haemoglobin (g/L), mean (SD)	135.8 (19.3)	135.9 (19.8)	134.2 (17.8)	137.8 (19.7)
SCr (μmol/L), mean (SD)	84.2 (46.4)	87.2 (50.1)	78.2 (37.8)	85.0 (47.2)
LDL-C (mmol/L), mean (SD)	2.8 (0.9)	2.7 (0.9)	2.7 (0.8)	2.9 (1.0)
AMI type, % (n/N)
STEMI	85 (12 761/15 009)	80.6 (5904/7329)	91.2 (3874/4247)	86.9 (2983/3433)
NSTEMI	15 (2248/15 009)	19.4 (1425/7329)	8.8 (373/4247)	13.1 (450/3433)
Killip class, % (n/N)
I	60.8 (8078/13 282)	63.6 (4492/7067)	52.6 (1505/2862)	62.1 (2081/3353)
II	25.1 (3339/13 282)	25.5 (1803/7067)	29.5 (844/2862)	20.6 (692/3353)
III	7.0 (927/13 282)	6.0 (423/7067)	8.1 (231/2862)	8.1 (273/3353)
IV	7.1 (938/13 282)	4.9 (349/7067)	9.9 (282/2862)	9.2 (307/3353)
Outcome, % (n/N)
MACE	5.9 (888/15 009)	5.0 (368/7329)	4.9 (207/4247)	9.1 (313/3433)

ACS, acute coronary syndrome; AMI, acute myocardial infarction; bpm, beats per minute; LDL-C, low density lipoprotein cholesterol; MACE, major adverse cardiovascular events; NSTEMI, non-ST segment elevation acute myocardial infarction; STEMI, ST segment elevation myocardial infarction.

The model was derived using data from patients enrolled in Beijing (n=7 329) and validated using data from patients enrolled in Henan province (n=4247) and Jilin province (n=3433). A total of 368 (of 7329; 5.0%) patients in Beijing, 207 (of 4247; 4.9%) in Henan, and 313 (of 3433; 9.1%) in Jilin experienced in-hospital MACE. The number of missing values for each variable considered in the model is shown in online supplemental table 4.

The derived risk model included three predictors of age (OR 1.06 per 1-year increment, 95% CI 1.05 to 1.07, p<0.001; OR 1.34 per 5-year increment, 95% CI 1.28 to 1.40, p<0.001), white cell count (WCC) (OR 1.10 per 109/L increment, 95% CI 1.07 to 1.13, p<0.001) and Killip class (OR 1.34 for class II vs class I; 95% CI 0.99 to 1.83, p=0.060; OR 2.74 for class III vs class I; 95% CI 1.86 to 3.97, p<0.001; OR 14.12 for class IV vs class I; 95% CI 10.35 to 19.29, p<0.001) (table 2). For an individual patient, the risk can be calculated using the formula:

Table 2

Multivariate logistic regression model for in-hospital prediction of major adverse cardiovascular events in patients with acute myocardial infarction

Variable		OR	95% CI for OR		P value
Age, per 1-year increase		1.06	1.05	1.07	<0.001
Age, per 5-year increase		1.34	1.28	1.40	<0.001
WCC, per 109/L increase		1.10	1.07	1.13	<0.001
Killip class	I (reference)
	II	1.34	0.99	1.83	0.060
	III	2.74	1.86	3.97	<0.001
	IV	14.12	10.35	19.29	<0.001
Constant					<0.001

WCC, white cell count.

where Killip score=0, 0.295, 1.007 and 2.647 for Killip class=I, II, III and IV, respectively. The prevalence of these risk factors in the Beijing, Henan and Jilin cohorts is shown in online supplemental table 5.

ROC curves for the three models show that the CHARM model has a consistently high true positive rate that is generally higher than TIMI and similar to GRACE for all three datasets (figure 2). C-statistics were similar for the model derivation and validation datasets, and for STEMI patients and NSTEMI patients (online supplemental table 5).

Figure 2

ROC curves for the China ACS risk model (CHARM), and the TIMI and GRACE models. ACS, acute coronary syndrome; GRACE, Global Registry of Acute Coronary Events; ROC, receiver operating characteristic; TIMI, thrombolysis In myocardial infarction.

Model calibration by deciles is illustrated in figure 3. The Hosmer-Lemeshow statistic for our model was 5.5 (p=0.703) for the derivation set, 41.1 (p<0.001) for the Henan and 103.2 for the Jilin (p<0.001) validation datasets compared with 119.6, 34.0, and 459.1 for the GRACE probability model in the Beijing, Henan, and Jilin sets (all p<0.001). The calibration result shows that the goodness-of-fit character for our model is satisfactory and better than or comparable to the GRACE model.

Figure 3

Calibration of the China ACS risk model (CHARM). Observed MACE risk is calculated as the observed proportion of patients with primary outcome. Predicted mace risk is calculated as the average predicted risk score. HL statistic, Hosmer-Lemeshow test statistic; GRACE, Global Registry of Acute Coronary Events; MACE, major adverse cardiac event.

The comparison of IDI and NRI values between the model derivation and validation sets for CHARM and GRACE is shown in online supplemental table 5. The IDI values are consistently positive in the derivation and validation datasets, suggesting that CHARM has improved discrimination over GRACE. The NRIs are positive in most cases across different cut-off values, suggesting consistent correct reclassification of CHARM over GRACE (online supplemental table 5).

Discussion

In this study, we developed and validated a simplified risk prediction model (CHARM) in Chinese patients with AMI. The resulting score incorporates three readily available demographic and disease-related characteristics, including age, Killip class and WCC.

These three variables included in the CHARM model could be acquired within several minutes after patients’ present. The CHARM model also exhibited excellent discrimination capacity with respect to predicting in-hospital MACE risk in different practice settings in China. It categorises patients with AMI into groups that span a wide range of risk for MACE (more than 10-fold), with the lowest score correlated with a <1% risk, whereas the highest score correlated with a >10% risk. Thus, the CHARM model could be highly useful for rapid risk stratification, potentially facilitating individualised treatment decision making and promoting doctor–patient communication. This is especially important in the emergency and cardiovascular departments of basic level hospitals with limited access to troponin testing, echocardiography or primary coronary angiography. Even for tertiary hospitals with access to these tools, risk stratification within a few minutes could accelerate subsequent diagnosis and treatment selection.

Our model is consistent with the TIMI risk score11 and GRACE score12–14 in using age and cardiac function as risk predictors, two factors proven to be predictors of adverse events in previous studies. However, these two widespread scores incorporate certain variables that are typically unavailable immediately at the time of patient presentation; the subsequent time lag of several hours or more is a major limitation for these scores.

Being the only laboratory marker, WCC was incorporated into the CHARM model as a potential predictor of in-hospital MACE in patients with AMI for the first time. Inflammatory bursts after MI may disrupt the fibrous cap of atherosclerotic lesions and increase the likelihood of reinfarction. WCCs are quickly overwhelmed in response to AMI and the leucocyte profile changes drastically.24 Therefore, WCCs are a biologically plausible marker of adverse events in patients with AMI. Previous studies25 26 have observed that ACS patients with elevated WCCs were at higher risk of mortality and recurrent AMI. CAMI score27 and CAMI-NSTEMI score28 models from recent studies were relatively complicated, with both including more than ten markers besides WCCt to predict risk of in-hospital mortality in patients with STEMI or NSTEMI, resulting in a time lag of several hours, similar to that of the TIMI and GRACE score.

Although troponin elevation is also a powerful risk marker and is incorporated in other risk models,11 12 29 it is absent from our model. This is because our study only included patients with AMI; thus, every patient had elevated troponin, which reduces its predictive ability to identify patients at risk of adverse events. Furthermore, models that incorporate troponin, such as the TIMI and GRACE score experience a time lag of more than half an hour, which was considered to be unsatisfactory, particularly in the emergency setting.

It is well accepted that a model with c-statistics of 0.8 has adequate discrimination for clinical use.30 The c-statistics values for the CHARM model are approximately 0.8 or higher in both the derivation cohort and the two validation cohorts of AMI patients. Moreover, the CHARM model used only three baseline characteristics, which can be easily ascertained when a patient presents to the emergency room. WCC, the only laboratory test required, is simple to obtain, inexpensive and promptly available in a few minutes. These attributes support the wider application of the CHARM model in first-line clinical practice, especially that of emergency and cardiovascular departments in both resource-limited basic level hospitals as well as tertiary hospitals.

A previous study reported the GRACE risk score to underestimate in-hospital mortality in a large cohort of Chinese patients.31 Our analysis revealed that the GRACE risk score also underestimated in-hospital MACE in the Beijing, Henan and Jilin cohorts. The CHARM model had good calibration in the Henan cohort, with a Hosmer-Lemeshow statistic of 41. However, it consistently underestimated in-hospital rates of MACE in the Jilin cohort. This finding is notable because one explanation for the underestimation of the CHARM and GRACE score could relate to differences in clinical practice patterns. Patients in one cohort are less likely than others to receive evidence-based treatments and have higher rates of in-hospital complications.

Strengths and limitations

The strengths of the present analysis include its use of data from a contemporary cohort recruited from a geographically and socioeconomically diverse set of hospitals. Nonetheless, our findings need to be interpreted with appropriate caution because of an inherent limitation of the retrospective nature of the study; thus, missing data and incorrectly recorded data are a limitation, although we provided a rigorous training programme for investigators and included strict quality control procedures during the chart review. Although we adjusted whether a reperfusion therapy was performed, we did not consider whether TIMI III flow was achieved in the model. It is not unusual in China, especially in rural areas, for severely ill patients to be discharged home to die, which would have resulted in an underestimation of in-hospital mortality. Moreover, the score can only be calculated with the aid of a computer or smart phone app.

Conclusions

In conclusion, the CHARM model developed in this study, provides a simple, inexpensive, accurate and readily accessible tool for predicting in-hospital MACE in Chinese patients with AMI. The CHARM model could facilitate rapid risk stratification and decision making in first-line clinical practice, especially that of emergency and cardiovascular departments in different levels of hospitals in China.