Comparative prospective study of the performance of chest pain scores and clinical assessment in an emergency department cohort in Singapore.

OBJECTIVE: Chest pain scores allow emergency department (ED) physicians to identify low-risk patients for whom discharge can be safely expedited. Although these have been extensively validated in Western cohorts, data in patients of Asian heritage are lacking. This study aimed to determine the accuracy of HEART, ED Assessment of Chest Pain Score (EDACS), and Global Registry of Acute Coronary Events (GRACE) in risk-stratifying which chest pain patients are at risk of major adverse cardiovascular events within 30 days (composite of all-cause mortality, acute myocardial infarction and coronary revascularization).
METHODS: This single-center prospective cohort-study that enrolled 1200 patients was conducted by a large urban tertiary center in Singapore. Chest pain scores were reported before disposition by research assistants blinded to the physician's clinical assessment. Outcomes were assessed independently by a blinded cardiologist and emergency physician, while another cardiologist adjudicated in the case of discrepancies.
RESULTS: Of the 1195 patients analyzed, 135 (11.3%) suffered major adverse cardiovascular events within 30 days. HEART, which ruled out major adverse cardiovascular events in 52.8% of patients with 88.1% sensitivity, and EDACS, which ruled out major adverse cardiovascular events in 57.5% of patients with 83.7% sensitivity, proved comparable to clinical judgment that ruled out major adverse cardiovascular events in 73.0% of patients with 85.5% sensitivity. GRACE was weaker-ruling out major adverse cardiovascular events in 79.2% of patients with a dismal sensitivity of 45.0%. The correlation-statistic for HEART (79.4%) was superior to EDACS (69.9%) and GRACE (69.2%).
CONCLUSIONS: HEART more accurately identified low-risk chest pain patients in an Asian ED, demonstrating comparable performance characteristics to clinical judgment. This has major implications on the use of chest pain scores to safely expedite disposition decisions for low-risk chest pain patients.

INTRODUCTION

Background

Chest pain is among the leading causes for emergency department (ED) presentations. Emergency physicians must identify which high‐risk patients require urgent admission and further testing for acute coronary syndrome, such as angiography, and which low‐risk patients can be safely discharged expediently to avoid unwarranted hospitalizations and unnecessary testing.

In 2015, Mahler et al found that the HEART (history, electrocardiogram [ECG], age, risk factors, and troponins) pathway significantly reduced hospitalization rates and length of stay while maintaining a low miss‐rate for major adverse cardiovascular events. Although the HEART pathway has gained traction for its ease of use, other emergency physicians favor a slew of other chest pain scores such as Global Registry of Acute Coronary Events (GRACE), ED Assessment of Chest Pain Score (EDACS), Vancouver Chest Pain Rule, or Troponin‐Only Manchester Acute Coronary Syndrome score. Importantly, although the American Heart Association has published scientific statements to support the use of scores to guide acute coronary syndrome rule‐out, the decision of which particular score to favor has been left to the discretion of individual emergency physicians and institutions.

Importance

Despite a long history of attempts to derive and validate chest pain scores, few studies have done head‐to‐head comparisons that directly pit the performance of various scores against each other or against unstructured clinical judgment. Moreover, although the literature favors chest pain scores as useful unbiased risk‐stratification tools, results have not been reproduced consistently—partly attributable to the varying demographics and disease burden of different populations as well as the varying coronary catheterization rates across cardiology departments of various institutions. It also remains unclear if chest pain scores can be safely applied in patients of predominantly Asian heritage, because most scores were derived and validated in Western populations such as the United States, Australia, New Zealand, and the Netherlands.

Goals

Therefore, this study aimed to evaluate the diagnostic accuracies of HEART, EDACS, and GRACE in identifying which chest pain patients in an Asian ED were at high risk of major adverse cardiovascular events within 30 days. This head‐to‐head comparison of the various scores was further benchmarked against clinical judgment to address the knowledge gap regarding the use of chest pain scores.

We hypothesize that a score incorporating some element of clinical gestalt (such as HEART, which requires the physician to determine if various aspects are slightly/moderately/highly suspicious) would prove superior to scores that do not incorporate clinical gestalt (such as EDACS and GRACE).

METHODS

Study design and setting

This was a prospective cohort study, conducted from October 2015 to November 2017, which comprised 1200 patients. This single‐center study was conducted in the Singapore General Hospital, a large urban tertiary hospital in Singapore with an annual census of 135,000 patients that had dedicated cardiology and cardiothoracic specialty services.

Participant selection

ED patients had to present with a chief complaint of chest pain or angina‐equivalent symptoms, and their physician had to order an ECG and troponin for acute coronary syndrome rule‐out to be eligible. Full‐time trained research assistants used non‐probability sampling methods (continuous convenience sampling) for subject recruitment. Subjects enrolled had to be above 21 years old with capacity to provide written informed consent. Subjects with ST‐elevation myocardial infarction or poor premorbid status with a life expectancy less than a year were excluded.

Measurements

Trained full‐time research assistants interviewed each patient and their families directly and cross‐referenced their findings with the documentation entered into the electronic health records by the treating physician to complete a comprehensive case report form detailing the chest pain characteristics, physical examination findings, and ECG interpretation for each patient. Data were obtained before patient disposition to ensure that research assistants were blinded to the eventual diagnosis and disposition. The cut‐offs used (GRACE ≥109, EDACS ≥16, and HEART ≥4 ) were derived from the original studies of the respective scores.

For the purpose of this study, patient history was specifically classified based on the narrative provided by the patient to the research assistants: typical chest pain was defined as “substernal chest pain/discomfort” “provoked by exertion/emotional stress,” and “relieved by rest/nitroglycerine” and assigned 2 points under the “History” subsection of the HEART score; atypical chest pain as 2 of the abovementioned criteria and was assigned 1 point, whereas chest pain with none of these was deemed non‐specific and given no points. This is the same definition adopted by the Coronary Artery Disease consortium (comprising 18 different hospitals across Europe and United States) to estimate the pre‐test probability of coronary artery disease. The chest pain characteristics that made up the individual variables of EDACS (diaphoresis, pain radiating to arm/shoulder/neck/jaw, pain occurred/worsened with inspiration, and pain reproducible by palpation) were also prospectively sought by the research assistants from each patient.

The patients and emergency physicians were blinded to the chest pain scores obtained by the research assistants. None of the scoring tools were made available to the emergency physician, nor was there any extraneous influence exerted by the study investigators on the emergency physician to use any score in their decisionmaking process. The eventual admitting diagnosis and disposition were left to the emergency physicians’ discretion and were used as surrogate measures of clinical judgment. Patients admitted to cardiology telemetry‐monitored beds with a provisional impression of acute coronary syndrome were deemed “high risk” by clinical judgment. Patients discharged directly from ED (including those who were discharged after extended ED observation, which involved 3 sets of serial troponins and ECGs obtained over an 8‐hour observation period without any provocative or invasive cardiac stress tests) and patients admitted to non‐cardiology beds (eg, medical) or admitted under provisional non‐cardiac diagnoses were deemed “low risk.” Abscondments and discharges against medical advice were excluded from analyses of the performance characteristics of clinical judgment.

Outcomes

Outcome assessment was performed by 2 independent clinicians (a cardiologist and an emergency physician) who were blinded to the results of the chest pain scores, with a second cardiologist engaged to resolve any discrepancies. All subjects on follow‐up were both contacted by standardized telephone interviews and had their medical records reviewed after 30 days for chest pain, angina‐equivalent symptoms, or clinical events relevant to adjudication.

The Bottom Line

Some patients suffering “low risk chest pain” go on to have serious cardiac events. This study compared three chest pain scores, HEART, EDACS, and GRACE in a single tertiary hospital in Singapore. HEART most accurately identified those patients safe for ED discharge.

For the purposes of this study, major adverse cardiovascular events were defined as a composite of 3 outcomes: all‐cause mortality, acute myocardial infarction, or coronary revascularization. Myocardial infarction was defined based on the Third Universal Definition of Myocardial Infarction, and the cardiac biomarker used in this institution was the high‐sensitivity troponin‐T assay. The primary outcomes are the sensitivities and percentage rule‐outs (specificities) of the various chest pain scores in identifying major adverse cardiovascular events, with the correlation statistic (C‐statistic) as a secondary outcome.

Ethics

This study conformed to the principles set out in the Declaration of Helsinki. This study was nested in the “Evaluation of High‐Sensitivity Troponin‐I in the Management of Patients with Chest Pain in the Emergency Department” prospective cohort study that was led by the study's co‐investigators. Registered with the United States’ National Institutes of Health National Library of Medicine ClinicalTrials.gov (NCT02789904), the original cohort study primarily aimed to investigate the validity of high‐sensitivity troponin‐I against troponin‐T while setting up a comprehensive chest pain registry. Relevant data (ie, chest pain characteristics, supplementary history, physical exam, ECG findings, disposition, and eventual outcomes) was extracted from the registry and analyzed for the purposes of this study. Although the original study was funded by Abbott and Beckman‐Coulter, the sponsors did not play any role in study construct, data collection, data analysis, or paper writing.

Statistical analysis

Clinical characteristics were summarized using mean ± SD for continuous data and proportion for categorical data. Receiver‐operating‐characteristic curves were generated, with calculation of area‐under‐curve (Figure 2) and estimation of sensitivity, specificity, positive predictive value, and negative predictive value for 30‐day major adverse cardiovascular events performed to determine the diagnostic accuracy of each score. STATA version 15 was used in the analysis. Logistic regression was used in the determination of area‐under‐curve.

FIGURE 2

Area under ROC curves for the various chest pain scores

RESULTS

Characteristics of study subjects

A total of 1200 patients were enrolled, of which 5 withdrew consent after recruitment (Figure 1). An estimated 300 patients were assessed for eligibility but not enrolled because of the time‐sensitive nature of the condition studied. Of the 1195 patients analyzed, 42 patients (3.5%) absconded or discharged against medical advice at the index ED visit, but these patients were still followed up at 30 days via telephone call and by reviewing their medical records.

FIGURE 1

Flowchart describing enrolment and outcomes of chest pain patients in the ED

All patients had their electronic medical records reviewed 30 days after their index visit and attempts were also made to contact all patients by telephone follow‐up as an additional measure. Although 456 patients (38.2%) could not be contacted via telephone follow‐up, this is unlikely to have significant impact because researchers remained aware of return visits to other hospitals and were still able to verify outcomes because of the centralized cluster‐based nature of the electronic health care records system in Singapore.

The cohort recruited had a high burden of chronic diseases, with 73.4% (877 patients) having at least 1 of hypertension, hyperlipidemia, or diabetes mellitus, and 20.9% (250 patients) having all 3 (Table 1). Because this hospital had a dedicated cardiology and cardiothoracic unit, a significant proportion of patients (17.2%) had pre‐existing ischemic heart disease as well.

TABLE 1

Demographics of chest pain patients

Demographic data	No MACE (n, (%))	MACE at 30 days (n, (%))
Total no. of patients	1060 (88.7)	135 (11.3)
Mean age (y)	55.9 (SD 11.7)	59.5 (SD 9.2)
Sex
Male	694 (65.5)	123 (91.1)
Female	366 (34.5)	12 (8.9)
Race
Chinese	573 (54.1)	70 (51.9)
Malay	219 (20.7)	22 (16.3)
Indian	224 (21.1)	39 (28.9)
Others	44 (4.2)	4 (3.0)
Chronic diseases
DM	316 (29.8)	52 (38.5)
HTN	550 (51.9)	103 (76.3)
HLD	600 (56.6)	96 (71.1)
Nil chronic diseases	302 (28.5)	16 (11.9)
DM + HTN + HLD	204 (10.3)	40 (29.6)
Smoking history
Current smoker	204 (19.3)	39 (28.9)
Ex‐smoker	182 (17.2)	39 (28.9)
Never smoker	673 (63.6)	57 (42.2)
Family history of acute myocardial infarction	479 (45.2)	61 (45.2)
Past medical history of acute myocardial infarction	165 (15.6)	40 (29.6)
Disposition
Discharged from ED (includes observation unit)	561 (52.9)	15 (11.1)
Absconded or discharged against medical advice	38 (3.6)	4 (3.0)
Admitted inpatient with non‐cardiac diagnoses	185 (17.5)	4 (3.0)
Admitted inpatient to Cardiology for acute coronary syndrome	276 (26.0)	112 (83.0)

Abbreviations: DM, diabetes mellitus; ED, emergency department; HLD, dyslipidemia; HTN, hypertension; MACE, major adverse cardiovascular events.

Main results

The eventual major adverse cardiovascular events rate at 30 days in this study was 11.3% (135 patients). HEART had the highest sensitivity of 88.1% (95% confidence interval [CI] = 81.5–92.6) among the chest pain scores, comparable to clinical judgment at 85.5% sensitivity (95% CI = 78.3–90.6) (Tables 2, 3, and 4). Of the 135 patients who suffered major adverse cardiovascular events within 30 days, HEART missed 16 patients (11.9%) whereas EDACS missed 22 (16.3%). The C‐statistic for HEART (79.4%) (95% CI = 0.76–0.83) was also higher than EDACS (69.9%) (95% CI = 0.66–0.74) and GRACE (69.2%) (95% CI = 0.65–0.74), affirming that HEART is superior to both EDACS and GRACE at discriminating which chest pain patients are at increased risk of major adverse cardiovascular events.

TABLE 2

Test characteristics of the various chest pain scores (n = 1195)

	HEART	EDACS	GRACE
Sensitivity (%)	88.1 (81.5–92.6)	83.7 (76.5–89.0)	45.2 (37.0–53.6)
Specificity (%)	52.8 (49.8–55.8)	57.5 (54.5–60.4)	78.9 (76.3–81.2)
NPV (%)	97.2 (95.9–98.6)	96.5 (95.1–97.9)	91.9 (90.1–93.6)
PPV (%)	19.2 (16.1–22.3)	20.0 (16.7–23.3)	21.4 (16.6–26.2)
AUC (%)	79.4 (76–83)	69.9 (66–74)	69.2 (65–74)

TABLE 3

Test characteristics of the various chest pain scores and clinical judgment (patients who absconded or discharged against medical advice excluded; n = 1153)

	Clinical judgment	HEART	EDACS	GRACE
Sensitivity (%)	85.5 (78.3–90.6)	87.8 (80.9–92.4)	84.0 (76.6–89.3)	45.0 (36.8–53.6)
Specificity (%)	73.0 (70.2–75.6)	53.0 (50.0–56.1)	57.6 (54.6–60.6)	79.2 (76.6–81.5)
NPV (%)	97.5 (96.4–98.6)	97.1 (95.7–98.5)	96.6 (95.1–98.0)	91.8 (90.0–93.6)
PPV (%)	28.9 (24.4–33.4)	19.3 (16.2–22.5)	20.3 (16.9–23.6)	21.7 (16.8–26.6)

A total of 42 patients who discharged against medical advice or absconded were excluded from analyses of the performance characteristics of clinical judgment. Data for HEART/EDACS/GRACE were re‐analyzed with the 42 patients excluded as part of sensitivity analysis.

TABLE 4

Test characteristics of the various chest pain scores and clinical judgment (only patients admitted for further evaluation included; n = 577)

	Clinical judgment	HEART	EDACS	GRACE
Sensitivity (%)	96.6 (91.1–98.9)	91.4 (84.6–95.4)	84.5 (76.7–90.0)	49.1 (40.2–58.1)
Specificity (%)	40.1 (35.8–44.7)	38.2 (33.9–42.7)	45.8 (41.3–50.3)	70.5 (66.2–74.5)
NPV (%)	97.9 (95.8–99.9)	94.6 (91.4–97.9)	92.1 (88.7–95.6)	84.6 (81.0–88.2)
PPV (%)	28.9 (24.4–33.4)	27.1 (22.7–31.5)	28.2 (23.4–32.9)	29.5 (23.1–36.0)

Secondary results and sensitivity analyses

Abscondments and discharges against medical advice were excluded from analyses in pre‐planned sensitivity analyses. The results were consistent with the main study findings and did not yield any major disparity—HEART with 87.8% sensitivity remained superior to clinical judgment (85.5% sensitivity). Subgroup analyses of only hospitalized patients revealed that clinical judgment (that ruled out 40.1% of patients with 96.6% sensitivity) outperformed HEART (that ruled out 38.2% of patients with 91.4% sensitivity) and EDACS (that ruled out 45.8% of patients with 84.5% sensitivity).

LIMITATIONS

The history component of HEART (as developed in the original 2008 Backus study or the 2015 Mahler study ) is inherently subjective, because it depends on the physician's evaluation of certain qualitative clinical parameters (such as chest pain characteristics). The use of research coordinators (who may have less clinical acumen than physicians) invariably leads to concerns for whether they demonstrate similar inter‐observer reliability to physicians. The Coronary Artery Disease Consortium definition of typical versus atypical and non‐cardiac chest pain was deliberately adopted by the study investigators as it is standardized, well‐validated, and objective. All of the research coordinators underwent scenario‐based training before data collection under the direction of the primary investigators and received detailed instructions on how each aspect of the scores should be evaluated. The first few cases for each research coordinator were conducted together with and under the direct supervision of the more experienced chief research coordinator. All of the case report forms submitted by the research coordinators had to be audited personally by the study investigators, and meetings between the research coordinators and the primary investigators were conducted at regular intervals to discuss the audit findings and ensure the robustness of the data collected. Although the inter‐observer variability of the research assistants was not specifically assessed for this study, research by Cruz et al had reaffirmed that trained research assistants tasked with prospective data collection of subjective chest pain characteristics in ED patients exhibit comparable inter‐rater reliability to physicians. The research coordinators in this study also had access to the patients’ electronic health records to cross‐reference their findings to that of the treating physician.

It is difficult to evaluate for incorporation bias arising from inherent, independent use of chest pain scores by some physicians, because the decisionmaking process of the emergency physicians were not captured in this study. Currently, few attending emergency physicians in this institution use chest pain scores in their daily practice. The typical approach to chest pain evaluation remains largely based on unstructured clinical gestalt, because department guidelines advise admitting typical angina patients to Cardiology, offering atypical/non‐specific chest pain patients with cardiovascular risk‐factors an extended observation protocol, and discharging patients with atypical symptoms and no cardiovascular risk‐factors with an outpatient cardiology review.

As no invasive or provocative stress tests are available as part of either initial assessment or observation protocol in this ED, the discharge/admitting diagnoses closely represent, and are a good surrogate for, unstructured clinical judgment of the attending emergency physicians based solely on receipt of initial/serial cardiac markers. In addition, department guidelines mandate that any patient for whom the emergency physician is concerned of having acute coronary syndrome (ie, “high‐risk” by clinical judgment) must be admitted to a telemetry‐monitored bed in Cardiology.

However, the retrospective methods by which clinical judgment was derived meant that patients who absconded or discharged against medical advice could not be analyzed in the clinical judgment arm, but were included for the chest pain risk scores (that were prospectively sought). Sensitivity analyses were performed to mitigate concerns of bias, because this subgroup is more likely to have incomplete evaluations and possibly higher rates of adverse outcomes arising from non‐adherence. The data were re‐analyzed with these 42 patients (3.5%) excluded from all arms, with no significant resultant difference in the results.

DISCUSSION

Head‐to‐head comparisons of chest pain scores are scarce, and it remains contentious which is superior. Although HEART remains the dominant risk stratification score in the United States, the American Heart Association and European Society of Cardiology reserve judgment on which has the best performance. Chapman et al found that HEART combined with the European Society of Cardiology's 3‐hour pathway (that calls for serial troponins at 0‐ and 3‐ hour) has the best test characteristics of 99.7% sensitivity when pitted against EDACS (99.2%) and GRACE (99.0%), whereas another multicenter study by Stopyra et al reported that EDACS identified 10% more patients as low‐risk, with a nearly identical negative predictive value as HEART. A comparison of 4 decision aids (EDACS, TIMI, HEART, and T‐MACS) by Body et al found that EDACS ruled‐out myocardial infarction in 48.3% of patients, outperforming T‐MACS (46.5%) and HEART (34.9%). Few studies have examined the utility of chest pain clinical decision rules in a predominantly Asian population: a small single‐center study in Hong Kong by Yang et al found that both HEART and EDACS had 100% sensitivity, but was inadequately powered to find any differences between HEART and EDACS with only 231 patients recruited.

The major adverse cardiovascular events rate in this study (11.3%) is similar to that found in other studies (9%–17%, depending on country and institution). , , , , HEART was found to be the most reliable of the chest pain scores, EDACS trailed closely, and GRACE fared the poorest. This was likely because GRACE, like TIMI, was first developed for patients with established acute coronary syndrome rather than undifferentiated chest pain. HEART exhibited comparable performance characteristics to clinical judgment, despite the latter often entailing a much more rigorous and lengthy evaluation (including an extended 8‐hour period of observation for serial troponins and ECGs). In addition, our postulation that a score that incorporates clinical gestalt (such as HEART) would prove superior to others that did not (EDACS, GRACE) was affirmed by the study findings, although further validation is needed as this was a single‐center study.

This study demonstrated that HEART more accurately identified low‐risk chest pain patients in an Asian ED safe for expedited discharge, compared to EDACS and GRACE. HEART proved comparable to clinician judgment for chest pain risk stratification, EDACS trailed closely behind, whereas GRACE is significantly weaker as a chest pain score or accelerated diagnostic protocol.

These findings are promising and have major implications on disposition decisions, because low‐risk chest pain patients may potentially be safely discharged within a shorter turnaround time in future, without resorting to extensive serial testing and observation for acute coronary syndrome rule‐out.

AUTHOR CONTRIBUTIONS

HJGT, FG, JWCT, SHL, and MEHO conceived the study concept and design and were involved in data acquisition. MN, HJGT, FG, and JWCT analyzed and interpreted the data. MN, HJGT, FG, JWCT, MEHO, and RP were involved in the drafting and critical revision of the manuscript. MN accepts final responsibility for the accuracy of the manuscript.

CONFLICT OF INTEREST

JWCT has grants from Beckman‐Coulter, Abbott Diagnostics, Medtronic, Bayer, Roche, and Amgen, AZ. MEHO has patents relating to use of heart rate variability and artificial intelligence for risk stratification of chest pain. MEHO is also scientific advisor for TIIM Healthcare SG. The other authors declare that they do not have any competing interests.

AVAILABILITY OF DATA AND MATERIALS

The datasets generated and/or analyzed during the current study are not publicly available due to patient confidentiality issues but are available from the corresponding author on reasonable request.