Development and validation of a risk score for chest pain with suspected non-ST-segment elevation acute coronary syndrome.

OBJECTIVE: To investigate a new risk score for acute chest pain with suspected non-ST-segment elevation acute coronary syndrome (NSTE-ACS).
METHODS: Patients who suffered from Chest pain and suspected NSTE-ACS were enrolled as subjects. Predictor variables had been analyzed, and a bootstrap technique was used to evaluate the internal validity of the model, and external validation had been assessed for a prospective cohort study.
RESULTS: Thousand five hundred and sixty-eight patients had been included in this study. Six predictor variables were found to be significant and were used to develop the model. The C-statistic of the model was 0.83, and internal validation revealed the stability of the model and the absence of over-optimism. Patients were given different triage recommendations, and the risk score was prospectively validated.
CONCLUSIONS: A risk score may be a suitable method for assessing the risk of major adverse cardiac events and aiding patient triage in emergency departments among patients with suspected NSTE-ACS.

INTRODUCTION

Acute chest pain is common in emergency department (ED) (Alghamdi et al., 2019). However, it is difficult to differentiate acute chest pain with suspected ACS from other chest pain syndromes. Guidelines recommend using risk scores for early stratification of suspected ACS based on disease history, symptoms, electrocardiogram (ECG), and biomarkers (Amsterdam et al., 2014; Ibanez et al., 2018). Many risk scores could help to identify chest pain reason, including the TIMI, GRACE, HEART, and MACS scores (Alimohammadi et al., 2021; Body et al., 2014; Marcusohn et al., 2020; Ziaee et al., 2019). However, an effective score has not been developed specifically for chest pain combined suspected NSTE‐ACS.

This study aims to investigate a new risk score for chest pain with suspected NSTE‐ACS based on logistic regression analysis. High‐sensitivity troponin I (hs‐TnI), as an alternative to conventional troponin, is assessed as a predictor variable in the new risk score.

METHODS

Design

The retrospective cohort study was performed from January to June 2019. The prospective cohort study was then performed from August to October 2019. Serum samples of hs‐TnI (Enhanced Accu Troponin I; Beckman‐Coulter Inc., Brea, California, USA) were tested by chemiluminescence assay. Patients with chest pain in the ED of an urban academic tertiary hospital in Qinhuangdao, China, had been recruited. The hospital had more than 80,000 patients per year. This study had been approved by the hospital ethics committee, and all patients signed the informed consent.

Study population

Patients aged over 18 were included due to chest pain (or associated symptoms, such as discomfort, pressure, or tightness of the chest), where there was a suspicion of NSTE‐ACS. The time from chest pain to arrival at the ED was more than two hours for each of the selected patients.

Patients were excluded from the study if the chest pain was not due to NSTE‐ACS clearly (e.g., trauma, pulmonary embolism, ST‐elevation acute coronary syndrome, aortic dissection, or arrhythmia), and the clinical data were incomplete, combined with terminal disease, pregnant women, did not provide informed consent.

Candidate predictor variables

Data collected on arrival and retrieved from the case report form were used to develop the risk score. A total of 10 baseline characteristics were the candidate predictor variables. The continuous variables were age and the number of CAD risk factors, such as hypertension, hypercholesterolemia, diabetes, active smoking (within the previous month), family history of CAD, and obesity. The four dichotomous variables were sex, known CAD, stroke, and elevated hs‐TnI. The two rank variables were chest pain history and ECG.

Chest pain history was classified into three ranks by two researchers in the derivation cohort, treating physician in the validation cohort. Patients were assigned the first rank, a slight suspicion of CAD, presented with right‐sided chest pain or worsened on inspiration. Patients assigned the third rank, a high suspicion of CAD, presented with central or left‐sided chest pain which radiated to the throat, jaw, shoulders, back, or arms, or exhibited associated diaphoresis, dyspnea, nausea, or vomiting. Patients were assigned to the second rank, a moderate suspicion of CAD, if they exhibited both slight and highly suspicious elements. A third opinion was sought in cases where there was disagreement over classification.

Similarly, the ECG taken in the ED was also classified into three ranks. If the ECG was normal according to Minnesota criteria, the first rank, a slight suspicion of CAD, was assigned (Blackburn et al., 1960). The third rank, classified as a high suspicion of CAD, was allocated if the ECG showed significant ST‐segment depression or elevation in two or more contiguous leads. If the ECG indicated a suspicion of CAD but did not meet the third rank criteria, it was allocated to the second rank, a moderate suspicion of CAD. Again, a third opinion was sought in cases where there was disagreement over classification.

Endpoint

The endpoint was major adverse cardiac events, included acute myocardial infarction (AMI), coronary artery bypass graft (CABG), percutaneous coronary intervention (PCI), and all‐cause death. As an observational study about risk stratification used in the ED, short‐term follow‐up periods of three months were sufficient to evaluate initial disposition decisions and provide patients time to comply with specialist referrals. Therefore, MACEs occurring within three months after admission. AMI’s definition referred to the fourth universal definition of myocardial infarction (Thygesen et al.,) and consisted of evidence of myocardial ischemia together with a rise or fall in hs‐TnI values. The follow‐up was three months by telephone interview.

Statistical analysis

SPSS 20.0 (SPSS Inc.) was used for data analysis. Continuous variable was presented as mean ± standard deviation (SD). Discontinuous variables were expressed as percentages. Univariate logistic regression model was used for predictor variable score analysis, and p < .20 was selected for testing in a multivariate stepwise logistic regression model. Variables that p < .05 were retained in the final model.

After developing the multivariate regression model, the regression coefficients were rounded to the nearest whole multiple of the smallest coefficient to obtain a simple and appropriate weighting for each variable. C‐statistic was used for the discriminative power of the model. A Z‐score was used to compare the difference between two C‐statistics, and the internal validation was assessed by a bootstrap technique (Steyerberg et al., 2001).

The chi‐square test was used for discontinuous data comparison. The homogeneity of the derivation and validation cohorts was tested by least‐squares linear regression analysis. p < .05 was considered statistically significant.

RESULTS

General characteristics

Among 1568 consecutive patients with acute chest pain in ED, 335 patients were omitted according to exclusion criteria, and twenty‐four patients were lost to follow‐up. At the end, 1189 eligible patients had been enrolled (Figure 1). The baseline characteristics of the derivation cohort had been listed in Table 1.

FIGURE 1

Flow chart for participants

TABLE 1

Baseline characteristics of the derivation cohort

Age, yrs	60.5 ± 8.7
Female sex	371 (31.2%)
ECG
Slight suspicion	459 (38.6%)
Moderate suspicion	546 (45.9%)
High suspicion	184 (15.5%)
History of chest pain
Slight suspicion	97 (8.2%)
Moderate suspicion	595 (50.4%)
High suspicion	497 (41.8%)
Hypercholesterolemia	142 (12.0%)
Diabetes mellitus	335 (28.2%)
Hypertension	762 (64.1%)
Current smoking	369 (31.0%)
Obesity	165 (12.7%)
Family history of CAD	157 (13.2%)
Stroke	137 (11.5%)
Known CAD	359 (30.2%)
Elevated hs‐TnI	256 (21.5%)

Data are mean ± SD or n (%).

Abbreviations: CAD, coronary artery disease; ECG, electrocardiogram; hs‐TnI, high‐sensitivity troponin I; SD, standard deviation.

Development of the score

The patient's age and no. of CAD risk factors were dichotomized by finding the point of maximum discrimination through analysis of the receiver operating characteristic curves. Patient age was analyzed in five‐year increments, from 30 to 80 years, and the C‐statistic for each age group was found to range from 0.50 to 0.52. The C‐statistic was highest at an age cutoff point of 60 years old. The C‐statistic ranged between 0.50 and 0.58 and was highest where the cutoff point was two CAD risk factors. So, a cutoff of a number of two CAD risk factors was used in the score.

Of the 10 candidate predictor variables, three (known CAD, peripheral arterial disease, and carotid artery disease) did not achieve a significant level in the univariate regression model. Of the seven residual variables, stroke did not achieve a significant level in the multivariate regression model. We screened out six predictor variables by multivariate regression analysis and formed the final set of predictor variables (Table 2).

TABLE 2

Candidate predictor variables of the model in univariate and multivariate logistic regression analysis

Candidate predictor variables	Univariate analysis			Multivariate analysis
	β coefficient	p value	OR (95% CI)	β coefficient	p value	OR (95% CI)
Age, ≥60 years	0.187	.094	1.21 (0.97–1.50)	0.429	.002	1.54 (1.16–2.03)
Male sex	0.878	<.001	2.41 (1.90–3.05)	0.836	<.001	2.31 (1.71–1)
History of chest pain
Slight suspicion	0	0	1 (reference)	0	0	1 (reference)
Moderate suspicion	2.430	<.001	11.36 (5.48–23.52)	2.503	<.001	12.22(5.34–27.94)
High suspicion	3.693	<.001	40.17 (19.03–84.78)	3.421	<.001	30.06 (13.08–71.67)
ECG
Slight suspicion	0		1 (reference)	0		1 (reference)
Moderate suspicion	1.126	<.001	3.08 (2.42–3.92)	0.895	<.001	2.45 (1.85–3.24)
High suspicion	3.017	<.001	20.42 (10.32–40.41)	2.236	<.001	9.36 (4.09–21.54)
2 or more risk factors*	0.641	<.001	1.90 (1.52–2.37)	0.427	.002	1.53 (1.16–2.02)
Elevated hs‐TnI	3.520	<.001	33.80 (18.62–61.22)	2.801	<.001	16.47 (8.79–30.83)
Known CAD	0.154	.209	1.17 (0.92–1.48)	NA	NA	NA
Stroke	0.300	.099	1.35 (0.95–1.93)	NA	NA	NA

Abbreviations: CAD, coronary artery disease; CI, confidence interval; ECG, electrocardiogram; hs‐TnI, high‐sensitivity troponin I; NA, not applicable; OR, odds ratio.

Risk factors: hypertension, hypercholesterolemia, diabetes mellitus, family history of coronary artery disease, current smoking (<1 month), and obesity (body mass index ≥30 kg/m2).

The regression coefficient of two or more CAD risk factors was the smallest, and its weighting was one point. The regression coefficients of the other variables were rounded to the nearest whole multiple of the smallest coefficient to obtain their weightings. The final score ranged from 0 to 24 points (Table 3).

TABLE 3

Weightings of the predictor variables in the final score

Predictor variables	Weightings
Age, ≥60 years	1
Male sex	2
History of chest pain
Slight suspicion	0
Moderate suspicion	6
High suspicion	8
ECG
Slight suspicion	0
Moderate suspicion	2
High suspicion	5
2 or more risk factors*	1
Elevated hs‐TnI	7

Abbreviations: CAD, coronary artery disease; ECG, electrocardiogram; hs‐TnI, high‐sensitivity troponin I.

*Risk factors: hypertension, hypercholesterolemia, diabetes mellitus, family history of coronary artery disease, current smoking (<1 month), and obesity (body mass index ≥30 kg/m2).

The C‐statistic of new score in all patients was 0.84 (95% confidence interval (CI): 0.81–0.86), in elderly subgroups (≥65 years old), 95% CI was 0.80 (95% CI: 0.76–0.84), in female subgroups, 95% CI was 0.84 (95% CI: 0.81–0.88), and in diabetes mellitus subgroups, 95% CI was 0.82 (95% CI: 0.78–0.86). The receiver operating characteristic curves are shown in Figure 2A. With the increase in scores, the incidence of events increased (p < .001 by chi‐square for trend). The C‐statistic from the bootstrap analysis was also 0.84 (95% CI: 0.81–0.88).

FIGURE 2

Receiver operating characteristic curves in all patients and three subgroups (elderly, female, and diabetes mellitus) in the derivation cohort (a) and validation cohort (b)

Prospective validation of the score

The prospective validation of the score was performed from August to October 2019. The predictor variables in the final score were prospectively collected. The treating physician classified both history of chest pain and ECG into three ranks according to previous standards. Finally, 523 chest pain patients had been recruited and 198 patients had MACE. Event rates increased significantly with increasing score (p < .001). The validation and derivation cohorts showed a homogenous pattern (p = .981). The C‐statistic for the score in the validation cohort was similar to the derivation cohort (p = .892). The C‐statistic for the score in elderly subgroups (age ≥65 years), female subgroups, and diabetes mellitus subgroups was 0.84, 0.86, and 0.79, respectively (Figure 2B).

Exploring the utility of the score

The classification was assessed in the validation cohort, the rate of MACE in the low‐, intermediate‐, and high‐risk groups was 1.5% (1/65), 17.7% (35/197), and 63.5% (165/260), respectively (p < .001 by chi‐square test) (Table 4).

TABLE 4

Classifications of chest pain patients in the derivation cohort

Classification	Score	Patients, n (%)	MACE (n)	Rate of MACE (%)*
Low risk	0–4	142 (11.9%)	PCI (2)	1.4%
Intermediate risk	5–8	416 (35.0%)	PCI (73), CABG (2)	18.0%
High risk	9–24	631 (53.1%)	AMI (152),PCI(212), CABG (49), Death (9)	62.9%

Abbreviations: AMI, acute myocardial infarction; CABG, coronary artery bypass graft; MACE, major adverse cardiac events; PCI, percutaneous intervention.

*The rate of MACE in the three groups was significantly different (p < .001 by chi‐square test).

DISCUSSION

Previously, the PURSUIT, GRACE, TIMI, and FRISC scores had been used for acute coronary syndrome (Alimohammadi et al., 2021; Blackburn et al., 1960; Marcusohn et al., 2020; Ziaee et al., 2019). However, the suitability of TIMI and GRACE scores remains debatable (Holly et al., 2013). MACS scores were developed for possible cardiac chest pain (Body et al., 2014), and the HEART score was developed specifically for suspected NSTE‐ACS (Alimohammadi et al., 2021; Backus et al., 2013; Cortés et al., 2020). The weightings of these scores were assigned without consideration for the prediction of adverse cardiac events.

This study specifically developed a new risk score for chest pain due to suspected NSTE‐ACS. The new risk score was termed CHEST‐A. In the multivariable model, the three most powerful predictors were history of chest pain, ECG, and elevated hs‐TnI, which was consistent with guidelines (Amsterdam et al., 2014; Ibanez et al., 2018). Chest pain history and ECG were trichotomous rather than dichotomous. A hs‐TnI assay is elevated 2 h after symptom onset in acute myocardial infarction (Cullen et al., 2013; Zaninotto et al., 2020). To our knowledge, this is the first risk score describing hs‐TnI as an independent predictor for suspected NSTE‐ACS. In our study, two or more traditional risk factors remained statistically significant in the final model, and its prognostic value was weaker than those of chest pain history, ECG, and elevated hs‐TnI.

In risk score development, it is necessary to balance simplicity and ease of use with complexity and accuracy based on statistical analysis. The TIMI score is easy to use; however, its discrimination is poor (C‐statistic = 0.65) (Antman et al., 2000). Although GRACE score is excellent (C‐statistic = 0.83) (Granger et al., 2003), it is complex and usually calculated using a computer. The HEART score is simple to use with excellent discrimination (C‐statistic = 0.80) (Alimohammadi et al., 2021), but score development is not based on statistical analysis. In our study, the score predictors could easily be assessed in the ED during initial patient evaluation. They were trichotomous or dichotomous and given appropriate weightings, which made them simple and easy to use. The discrimination of the score was excellent in all patients and was better than GRACE score and TIMI score. Thus, the score effectively balances simplicity and ease of use with complexity and accuracy, based on statistical analysis, and internal validation revealed a stable model with no over‐optimism in predictive accuracy. In addition, the CHEST‐A score was prospectively validated in a new cohort and the results were also excellent.

Nevertheless, the score cannot be used directly in clinical practice. Therefore, we divided it into low‐, intermediate‐, and high‐risk boundaries as ≤2%, >2% but<20%, and ≥20%, respectively. In this study, the MACE rate was significantly different in the three groups and could be used as complement patient triage in the ED. In low‐risk group, patients could be discharged early and safely. In the intermediate‐risk group, patients should be stayed in the ED for further clinical evaluation. In the high‐risk group, patients should be immediately received invasive therapy. The recommendations in this study are simple to apply in the ED and could aid rapid initial triage, reduce crowding, and enable effective medical resource assignment.

This study, however, has certain limitations. The prevalence of low‐risk and high‐risk chest pain was relatively rare and common, respectively, in the study population of our ED. Therefore, the score still needs further research in different disease prevalence. Although all predictors were trichotomous or dichotomous and given appropriate weightings, the score is modestly complex and may be difficult to remember and calculate, which could limit its use in the ED.

CONCLUSIONS

The new risk score is validated for chest pain combined suspected NSTE‐ACS, based on logistic regression analysis. It may serve as a method to assess MACE risk and aid patient triage in the ED.

CONFLICT OF INTEREST

The authors declare that they have no competing interests.

AUTHOR CONTRIBUTIONS

Chun‐Peng Ma involved in conception and design. Xue‐Fei Dong involved in administrative support. Xiao‐Li Liu involved in provision of study materials or patients. Jian‐Shuang Feng involved in collection and assembly of data and data analysis and interpretation. All authors involved in manuscript writing and final approval of the manuscript.

ETHICS APPROVAL

This study was conducted in accordance with the declaration of Helsinki. This study was conducted with approval from the Ethics Committee of our hospital.

CONSENT TO PARTICIPATE

Written informed consent was obtained from all participants.