Prognostic values of 4 risk scores in Chinese patients with chest pain: Prospective 2-centre cohort study.

Four risk scores for stratifying patients with chest pain presenting to emergency departments (EDs) (namely Thrombolysis in myocardial infarction [TIMI], Global registry for acute coronary events [GRACE], Banach and HEART) have been developed in Western settings but have never been compared and validated in Chinese patients. We aimed to find out to the number of MACE within 7 days, 30 days, and 6 months after initial ED presentation, and also to compare the prognostic performance of these scores in Chinese patients with suspected cardiac chest pain (CCP) to predict 7-day, 30-day, and 6-month major adverse cardiac events (MACE).A prospective 2-center observational cohort study of consecutive patients presenting with chest pain to the EDs of 2 university hospitals in Guangdong and Hong Kong from 17 March 2012 to 14 August 2013 was conducted. Patients aged ≥18 years with suspected CCP but without ST-segment elevation myocardial infarction (STEMI) were recruited.Of 833 enrolled patients (mean age 65.1 years, SD14.5; 55.6% males), 121 (14.5%) experienced MACE within 6 months (4.8% with safety outcomes and 10.3% with effectiveness outcomes). The HEART score had the largest area under the receiver operating characteristic (ROC) curve for predicting MACE at 7-day, 30-day, and 6-month follow-up [area under curve (AUC) = 0.731, 0.726, and 0.747, respectively. The HEART score also had the largest AUC for predicting effectiveness outcome (AUC = 0.715, 0.704, and 0.721, respectively). However, there was no significant difference in AUC between HEART and TIMI scores. Banach had the largest AUC for predicting safety outcome (AUC = 0.856, 0.837, and 0.850, respectively).The HEART score performed better than the GRACE and Banach scores to predict total MACE and effectiveness outcome in Chinese patients with suspected CCP, whereas the Banach score best predicted safety outcomes.

Introduction

Chest pain is the second most common chief complaint of patients presenting to an emergency department (ED) and coronary heart disease (CHD) is one of the most common causes of chest pain. The World Health Organization reported that coronary artery disease is the most common cause of death worldwide, accounting for 7.2 million deaths.[ Therefore, the early identification of CHD in chest pain patients is important. Nevertheless, there is only a small proportion of ED chest pain patients with diagnostic electrocardiograms (ECGs) on presentation. It is clearly important not to discharge patients with acute coronary syndrome (ACS). However, non-ACS patients are unnecessarily admitted to hospitals leading to a heavy burden on hospital resources. These highlight the imperfections of current clinical diagnosis and risk stratification of patients who present to the ED with chest pain.

In the United States, the mortality of CHD was still high. 379, 559 Americans die of CVD and caused ≈1 of every 6 about one in every six deaths was caused by CHD in 2010.[ In Hong Kong, heart disease is the third leading cause of death, ∼11.8 persons on average died from CHD per day in 2014.[ In mainland China, it accounts for 22% of cardiovascular deaths in urban areas and 13% in rural areas. There is a significant difference in outcome of CHD between Chinese patients and patients with other origins, such as Western and Indian.[ This may be associated with the difference in lifestyle and body habitus between different ethnicities.[

The need to stratify patients presenting to the ED with cardiac chest pain accurately and rapidly is increasing. Risk stratification allows for more accurate decision making and is an important step in the assessment of suspected ACS patients. Several risk scores have been developed and locally validated in order to achieve a prompt, precise and cost-saving clinical decision process.[ Comparative validation of different risk scores has been rarely reported in either retrospective or prospective studies.[ To our knowledge, there is no prospective study comparing the predictive performance of the Thrombolysis in Myocardial Infarction (TIMI) score, the Global Registry for Acute Coronary Events (GRACE) score, the Banach score and the HEART score for patients with suspected ACS in the Chinese setting.

The aim of this study was to find out the number of MACE within 7 days, 30 days, and 6 months after initial ED presentation, and also to compare the prognostic performance of TIMI, HEART, Banach and GRACE scores in Chinese patients presenting with suspected cardiac chest pain (CCP) for predicting MACE at 7-day, 30-day, and 6-month follow-up.

Patients and methods

Study design

This prospective observational cohort study compared 4 risk scores including TIMI, GRACE, Banach, and HEART scores in patients presenting to EDs with suspected cardiac chest pain. Ethical approval was obtained from the joint Chinese University of Hong Kong-New Territories East Cluster Clinical Research Ethics Committee in Hong Kong and the Institutional Review Board in Guangzhou. Written informed consent was obtained from each patient or patient's relative after a verbal and written explanation of the study was provided. Patients were informed that they could withdraw from the study at any time.

Settings

This study was conducted in the EDs of the 2nd Affiliate Hospital of Guangzhou Medical University (AHGZMU) in Guangzhou and the Prince of Wales Hospital (PWH) in Hong Kong. PWH is located in the New Territories in Hong Kong. PWH is a university hospital with 1400 beds. It sees >150,000 new ED patients per annum and serves a local population of ∼800,000 people. AHGZMU is located at Hai Zhu district of Guangzhou in Guangdong province, about 100 miles north of Hong Kong. It is an academic hospital with 1200 beds. More than 180,000 new patients per year present to the ED, and it serves a local population of ∼1.32 million people.

Inclusion and exclusion criteria

Consecutive ED patients ≥18 years old, with a chief complaint of chest pain or discomfort, were recruited from 21 May 2012 to 3 March 2013 in PWH and from 17 March 2012 to 14 August 2013 in AHGZMU. Patients were excluded if they were non-Chinese or had a clearly noncardiac cause of chest pain, such as spontaneous pneumothorax. Confirmed ST-segments elevation myocardial infarction (STEMI) patients at ED presentation were also excluded as they do not have undifferentiated chest pain. Patients unable or unwilling to provide informed consent or unable to be contacted after discharge were excluded as well.

Data collection

Demographic data, the characteristics of the chest pain (such as location, feature, and radiation), vital signs, medical history, family history of CAD, ECG results, and contact information to facilitate subsequent follow-up were collected and recorded in a computerized database. Data was also obtained from the Clinical Management System (CMS) in PWH and Health Insurance Information Management System (HIIMS) in AHGZMU. All data required to calculate the TIMI, HEART, Banach and GRACE scores were also collected. Each score was assessed by a doctor who is blinded to the outcome of the patients. The standardized data collection form for the 4 scores are shown in Appendices 1–4.

Sample size calculation

According to our previous study, the 30-day MACE rates in Chinese patients presenting to ED with cardiac chest pain could have been as high as the upper limit of the 95% confidence interval in the low-risk group, that is, 13%, and as low as the lower limit of the 95% confidence interval in the high-risk group, that is, 23%. To achieve adequate power to address the objectives by using 2-tailed alpha of 0.05 and a power of 80%.The calculation formula for sample size is shown in Appendix 9.

The minimum sample size required per group is 230. We aim to recruit an extra 30% in case for unforeseen circumstances and thus at least 299 (230 × 1.3 = 299) patients are required per group. Therefore, the minimum sample size in this study is 598 (299 × 2 groups = 598).

Follow-up

Subsequent visits to ED, hospital readmission for evaluation of chest pain, and all cardiac procedures performed were retrieved from CMS in PWH and HIIMS in AHGZMU, and verified via telephone at 7-day, 30-day, and 6-month follow-up after initial presentation. Furthermore, death, myocardial infarction, readmission for ACS, and all cardiac testing and coronary revascularization procedures were also obtained via CMS and HIIMS.

Definitions

MACE is defined as a composite of safety and effectiveness outcomes. Safety outcomes include all-cause mortality (including cardiac death and sudden cardiac death), cardiac arrest, readmission with myocardial infarction and cardiogenic shock. Effectiveness outcomes consist of coronary revascularization (including percutaneous coronary intervention and coronary artery bypass grafting), ventricular arrhythmia needing intervention and high-degree artioventricular block needing intervention (including percutaneous radiofrequency ablation and pacemaker implantation).[ ACS is an umbrella term for a spectrum of symptoms that are compatible with acute myocardial ischaemia,[ consisting of unstable angina, NSTEMI and STEMI.[ The definitions for the individual risk scores are in Appendices 1–4.

Chest pain protocol

Guidelines recommend that patients with possible ACS but with a normal initial ECG and cardiac markers should be observed in a chest pain unit, where continuous cardiac monitoring and repeated measurement of cardiac markers are available.[ In this study, the first ECG was performed at triage and cardiac troponin T (cTnT) was analyzed. Those patients with a normal initial ECG and cTnT were required to stay in the ED for at least 6 hours to have a second cTnT sample analyzed and a second ECG performed. Patients were admitted if the cTnT levels were elevated or if the ECG showed abnormalities. Patients were discharged after 6 hours if they were pain free with 2 negative troponin tests (cTnT < 0.03 μg/L) and 2 ECGs without any new changes.

Outcomes

The primary outcomes were the number of MACE within 7 days, 30 days, and 6 months, and the prognostic performance of the 4 risk scores for predicting MACE including safety and effectiveness outcomes at 7-day, 30-day, and 6-month follow-up. The secondary outcomes for each patient were the presence or absence of MACE within 7 days, 30 days, and 6 months after initial ED presentation.

Statistical analysis

Continuous variables were presented as median, interquartile range, mean ± standard deviation as appropriate and categorical variables were expressed as frequencies. Statistical analysis was performed using SPSS v17.0 (SPSS Inc, IL) and Medcalc v9.5 (MedCalc Software, Mariakerke, Belgium). Categorical variables were compared using Pearson's chi-square test. Receiver operator characteristic (ROC) curves were plotted to determine the areas under the curves, sensitivities, specificities, and corresponding 95% confidence intervals (CIs) of the 4 risk scores. The ROC curve analyses were also utilized to determine the optimal cut-off values of the 4 risk scores. Statistical significance was set at P < 0.05.

Results

Study population and baseline characteristics

Patients were recruited from 17 March 2012 to 14 August 2013. The flowchart of the patient recruitment is shown in Fig. 1. There were 1274 eligible patients. In total, 418 patients were excluded due to missing the onset time, refusal to join the study, unable to sign the consent and non-ACS patients, leaving 856 patients for inclusion to the study. Among the 856 patients, 833 patients completed 6-month follow-up, and thus, their data were used for analysis. The mean age of the 833 patients was 65.1 ± 14.5 years and 55.3% were males. The baseline characteristics of the study population are presented in Table 1.

Figure 1

The flowchart of the patient recruitment. ACS = acute coronary syndrome, FU = follow-up, STEMI = ST-segment elevation myocardial infarction.

Table 1

Baseline characteristics of the study population (N = 833).

Clinical outcomes

The 7-day, 30-day, and 6-month rates of MACE for all patients were 8.4%, 10.8%, and 14.5%, respectively (Table 2). For the safety outcomes, the 7-day, 30-day, and 6-month event rates were 2.2%, 3.4%, and 6.7%, respectively. For the effectiveness outcomes, the 7-day, 30-day, and 6-month event rates were 7.2%, 8.9%, and 10.3%, respectively. No patients had a cardiac arrest and no patient required thrombolysis.

Table 2

Clinical outcomes for overall population at different time points.

Four risk scores for predicting 7-day, 30-day, and 6-month MACE

Figure 2 shows the total number of patients and the number with adverse outcomes in each risk stratum of the 4 risk scores. Increases in risk stratum and the number of total MACE corresponded best for all risk scores at 6 months. Increases in risk stratum and the number of safety outcomes corresponded well for GRACE and Banach at 7 days, 30 days, and 6 months, but for TIMI this was only apparent at 6 months. Only the HEART score had a better agreement with the number of effectiveness outcomes at 6 months.

Figure 2

Four risk scores for predicting 7-day, 30-day, and 6-month total MACE (A), safety (B) and effectiveness (C) outcomes in patients with suspected cardiac chest pain. GRACE = Global Registry of acute coronary events, TIMI = thrombolysis in myocardial infarction.

The distributions of patients in each score are shown in Appendices 5–8.

Comparison of the prognostic values of 4 risk scores

The comparison of the prognostic values of the 4 risk scores on total MACE, safety, and effectiveness outcomes at 7 days, 30 days, and 6 months was based on ROC analysis (Fig. 3). The HEART score had a larger AUC than the other scores for predicting the total MACE at 7 days (AUC = 0.731, 95%CI 0.699–0.761), 30 days (AUC = 0.726, 95%CI 0.694–0.756), and 6 months (AUC = 0.747, 95%CI 0.716–0.776). However, there was no statistically significant difference in AUC between the HEART and the TIMI scores. At the optimal cut-off value (>5), the specificities and sensitivities of the HEART score were 83.2% (95%CI 80.4–85.8%) and 52.9% (95%CI 40.6–64.9%) at 7 days and 83.7% (95%CI 80.9–86.3%) and 48.9% (95%CI 38.2–59.7%) at 30 days (Table 3).

Figure 3

Receiver operating characteristics curves of 4 risk scores for predicting 7-day, 30-day, and 6-month total MACE, safety and effectiveness outcomes. GRACE = Global Registry of acute coronary events, TIMI = thrombolysis in myocardial infarction.

Table 3

Prognostic performances of different risk scores for predicting 7-day, 30-day, and 6-month MACE.

The Banach and GRACE scores outperformed the TIMI and HEART scores for predicting safety outcomes at 7 days (Banach: 0.856, 95%CI 0.830–0.879; GRACE: 0.839, 95%CI 0.812–0.863), 30 days (Banach: 0.837, 95%CI 0.811–0.862; GRACE: 0.825, 95%CI 0.798–0.851), and 6 months (Banach: 0.850, 95%CI 0.824–0.874; GRACE: 0.843, 95%CI 0.816–0.867). Banach had a relatively high specificity of 92.1% (95%CI 90.0–93.8%) and a moderate sensitivity of 72.7% (95%CI 39.0–94.0%) at the optimal cut-off value (>3) at 7-day follow-up. GRACE had relatively high specificities of 92.2% (95%CI 89.6–93.5%) and 92.6% (95%CI 90.6–94.3%) and moderate sensitivities of 72.7% (95%CI 39.0–94.0%) and 66.7% (95%CI 41.0–86.7%) at the optimal cut-off value (>165) at 7-day and 30-day follow-up. It also had a high specificity of 91.9% (95%CI 89.8–93.7%) and a moderate sensitivity of 64.1% (95%CI 47.2–78.8%) at the optimal cut-off value (>160) at 6-month follow-up.

The HEART score also had the largest AUC for predicting effectiveness outcomes at 7 days (AUC = 0.715, 95%CI 0.683–0.746), 30 days (AUC = 0.704, 95%CI 0.672–0.735), and 6 months (AUC = 0.721, 95%CI 0.690–0.752). However, again, there was no statistically significant difference in AUC between the TIMI and the HEART scores. At the optimal cut-off value (>3), the sensitivities and specificities were 89.2% (95%CI 79.8–95.2%) and 40.4% (95%CI 36.9–44.05%) at 30 days and 89.5% (95%CI 81.1–95.1%) and 41.0% (95%CI 37.4–44.6%) at 6 months.

Discussion

This is the first large-scale prospective validation study on a Chinese population to evaluate the prognostic values of 4 commonly used cardiac risk-stratification tools to predict MACE.

Chest pain patients presenting to EDs create uncertainty for all treating physicians. The decision to discharge a patient where ACS cannot be excluded may result in life-threatening outcomes; on the other hand, admission of patients with atypical chest pain can lead to unnecessary medical treatment and costs. In this study, MACE was divided into safety and effectiveness outcomes. Clinicians are more concerned about safety outcomes, whereas hospital administrators may pay more attention to effectiveness outcomes.

The original HEART score performed optimally with respect to its discriminatory power to predict the total MACE and effectiveness outcome.[ Our study showed the HEART score have good predictive values for total MACE and effectiveness outcome at 7-day, 30-day, and 6-month. HEART score is designed specifically for early risk stratification of patients presenting to the ED with chest pain of suspected cardiac origin. Each element in the HEART score has a certain predictive value toward the occurrences of clinical end points.[ It also suggested that the HEART score provided the clinicians with a quick and reliable predictor of outcome, without computer-required calculation. So far no study has used HEART to predict effectiveness outcome separately. In this study, we showed that the HEART score had a slightly lower predictive value compared to a multinational validation study, which used the ASPECT database of chest pain patients.[ The study demonstrated the AUC of the HEART score for MACE was 0.83 during 6-week follow-up, similar to our safety outcome.[ However, our study showed the AUCs for safety outcomes were 0.696 at 30 days and 0.755 at 6 months. On the other hand, the challenge in the utilization of the HEART score is lack of exact definitions for a patient's history. The patient history criteria published in 2005[ is only classified as highly suspicious, moderately suspicious, and slightly suspicious. The developers of the score suggested that patient history was subject to personal interpretations.[ Better and more clear criteria were amended and published in 2013.[ The definition of the HEART score in this study was based on the new criteria.

The TIMI score showed a little lower predictive value of total MACE and effectiveness outcomes compared to the HEART score. The TIMI score was derived from 2 large fibrinolytic therapy studies and includes ECG and clinical features.[ It is an externally validated and widely used structured risk assessment method to predict adverse outcomes in patients with non-ST-segement elevation myocardial infarction (NSTEMI), unstable angina (UA), and undifferentiated chest pain.[ In our previous study, we investigated the prognostic performance of the TIMI score in Chinese patients presenting to the ED with undifferentiated chest pain for adverse cardiac events within 30 days. However, 0.7% of patients with very low risk (defined as TIMI 0) still had a MACE within 30 days.[ In the present study, we also found that there were 1.1–1.7% false negatives in patients with TIMI score 0 within 6 months. The lower predictive value of the TIMI score may be associated with ignorance of patients’ clinical histories.[

The Banach score based on a large registry of Polish ACS patients is a new score with 12 parameters (Appendix 4). It incorporates the entire spectrum of patients with ACS. It reveals good predictive value, moderate complexity, and good utility.[ To our knowledge, there are no studies investigating its clinical value in the ED and at 6 months.[ In our study, the Banach score performed optimally to predict safety outcomes within 6 months. Using the cut-off value recommended by the score's developers (<1), there were no safety MACEs. The Banach score included 2 novel predictors: “sudden cardiac arrest” (also in GRACE RS) and “pathological Q wave on admission ECG.” NSTEMI patients complicated with ventricular arrhythmias were at much higher risk of in-hospital and 6-month death than those without.[ Patients with Q waves were at a higher risk of cardiovascular death.[

The GRACE score is based on a prospective, multicenter, global registry of patients across the entire spectrum of ACS.[ Most studies have demonstrated that the GRACE score stratifies the risk of ACS patients in the ward setting,[ whereas few studies have investigated its prognostic value in patients with chest pain presenting to the ED.[ This tool is complex and requires a computer for calculation. Many parameters of the GRACE score would be missed in the ED setting.[ The GRACE score showed a little lower prognostic values for predicting total MACE and effectiveness outcomes in our study. Lyon et al. has demonstrated that both GRACE and TIMI scores were effective in accurately stratifying risk in patients with chest pain presenting to the ED, with an AUC of 0.80 for the GRACE score and 0.78 for the TIMI score.[

Limitations

First, though this was a 2-center study, both centers were located in South China. Consequently, it may not be possible to generalize our findings to other hospitals in other parts of China and the world. Second, some patients presenting to ED were not be able to be assessed immediately. This might lead to recall bias. Their data in CMS and HIIMS were checked in order to reduce bias. Third, some patients might not recognize the presence of cardiac risk factors, such as hypertension and hyperlipoidemia. Patients’ risks in each of the score groups might be underestimated although this reflects the reality of ED risk assessment. Last but not least, some patients might refuse coronary revascularisation because of financial problem or perceived complications of the procedure. Therefore, the rate of effective outcome might be slightly underestimated. As these potential errors might be found in all the groups, the trend of the predictive powers of the scores should not be affected significantly.

Conclusion

This study compared 4 independent risk scores in a large cohort of unselected ED patients with possible cardiac chest pain. The HEART score performed better than the GRACE and Banach scores for predicting total MACE and effectiveness outcomes, whereas the Banach score best predicted safety outcomes.

Acknowledgment

The authors thank Dr. Yishen Wang for reviewing the statistical analyses.