Enhanced triage for patients with suspected cardiac chest pain: the History and Electrocardiogram-only Manchester Acute Coronary Syndromes decision aid.

OBJECTIVES: Several decision aids can 'rule in' and 'rule out' acute coronary syndromes (ACS) in the Emergency Department (ED) but all require measurement of blood biomarkers. A decision aid that does not require biomarker measurement could enhance risk stratification at triage and could be used in the prehospital environment. We aimed to derive and validate the History and ECG-only Manchester ACS (HE-MACS) decision aid using only the history, physical examination and ECG.
METHODS: We undertook secondary analyses in three prospective diagnostic accuracy studies that included patients presenting to the ED with suspected cardiac chest pain. Clinicians recorded clinical features at the time of arrival using a bespoke form. Patients underwent serial troponin sampling and 30-day follow-up for the primary outcome of ACS. The model was derived by logistic regression in one cohort and validated in two similar prospective studies.
RESULTS: The HE-MACS model was derived in 796 patients and validated in cohorts of 474 and 659 patients. HE-MACS incorporated age, sex, systolic blood pressure plus five historical variables to stratify patients into four risk groups. On validation, 5.5 and 12.1% (pooled total 9.4%) patients were identified as 'very low risk' (potential immediate rule out) with a pooled sensitivity of 99.5% (95% confidence interval: 97.1-100.0%).
CONCLUSION: Using only the patient's history and ECG, HE-MACS could 'rule out' ACS in 9.4% of patients while effectively risk stratifying remaining patients. This is a very promising tool for triage in both the prehospital environment and ED. Its impact should be prospectively evaluated in those settings.

Background

Recent advances in biomarker technology have enabled earlier reassurance for patients who present to the Emergency Department (ED) with symptoms that are compatible with an acute coronary syndrome (ACS). There is now convincing evidence that serial cardiac troponin testing can help clinicians to rule out ACS over as little as 2 h with a contemporary assay [1,2] and 1 h with a high sensitivity assay [3]. It may even now be possible to ‘rule out’ acute myocardial infarction (AMI) with a single blood test, obviating the need for serial sampling [4,5].

The Manchester Acute Coronary Syndromes (MACS) and Troponin-only MACS (T-MACS) decision aids are validated tools that can both ‘rule in’ ACS, ‘rule out’ ACS and risk stratify remaining patients after a single blood test in the ED [6,7]. Both MACS and T-MACS calculate the probability that a patient has ACS and use that probability to stratify patients into one of four risk groups. The key difference between these models is that MACS incorporates two biomarkers (cardiac troponin and heart-type fatty acid binding protein), whereas T-MACS incorporates only cardiac troponin. However, both rely on biomarker measurement.

In the prehospital environment and when patients arrive in the ED for triage, the ability to test biomarker concentrations is not usually available. A decision aid that can ‘rule in’, ‘rule out’ and accurately risk stratify patients with suspected ACS without requiring biomarker measurement would have substantial advantages in these settings. Our primary objective was to derive and externally validate a decision aid, called ‘history and electrocardiogram MACS’ (HE-MACS), to achieve this.

Methods

Design and setting

We undertook this work in three stages: a derivation study followed by two external validation studies. We elected not to pool data from the two validation studies because of the potential for important heterogeneity between the studies. The derivation study took place at Manchester Royal Infirmary (an inner city university-affiliated hospital with a regional cardiology service); validation study took place at Stepping Hill Hospital, Stockport (a suburban community hospital) and validation study was the Bedside Evaluation of Sensitive Troponin study. This is a multicentre study incorporating an extensive programme of research. For this analysis, we included patients from Manchester Royal Infirmary and St George’s NHS Trust, London, UK. Each of the three studies is a prospective diagnostic accuracy study. All analyses presented here are secondary analyses.

Participants

In each study, we included adult patients presenting to the ED with suspected cardiac chest pain. We excluded patients with another medical condition necessitating hospital admission and those whose symptoms had occurred over 24 h (derivation study and validation study 1) or 12 h (validation study 2) prior to arrival. In validation study 2 we excluded patients who were diagnosed with ST-elevation myocardial infarction in the ED. Full details of the methodology for the derivation study and validation study 1 have previously been reported [6,7]. All patients provided written informed consent and the studies were approved by the National Research Ethics Service.

Data collection and processing

Clinical data were recorded by the treating clinician in the ED using a bespoke case report form (Supplementary Appendix, Supplemental digital content 1, ). ECGs were also interpreted by the treating clinician. All patients underwent serial cardiac troponin testing. In the derivation study and validation study 1, samples were drawn on arrival in the ED and 12 h after symptom onset. In validation study 2, samples were drawn on arrival and 3 h later. The troponin assays in use were cardiac troponin T (99th percentile 0.01 ng/ml, 10% coefficient of variation at 0.03 ng/ml, Roche Elecsys 4th generation; Roche Diagnostics, GmBH, Penzberg, Germany) in the derivation study and high sensitivity cardiac troponin T (99th percentile 14 ng/l, 10% coefficient of variation at 12 ng/l, Roche Elecsys 5th generation, Roche Diagnostics) in both validation studies. Patients were followed up after 30 days by telephone and chart review. If patients were persistently uncontactable we contacted their general practitioner.

Outcomes

The primary outcome for this analysis was a diagnosis of ACS. Patients were considered to have ACS if they had AMI or a major adverse cardiac event (MACE) within 30 days. The diagnosis of AMI was adjudicated by two independent investigators with disagreements being resolved by discussion. AMI was defined in accordance with the universal definition of myocardial infarction, requiring a rise and/or fall of cardiac troponin with at least one concentration above the 99th percentile of the assay in conjunction with symptoms or signs compatible with myocardial ischaemia, ECG changes, imaging evidence of new loss of viable myocardium or angiographic identification of intracoronary thrombus [8]. MACE was defined as death (all cause), AMI or coronary revascularization (coronary artery bypass grafting or percutaneous coronary intervention) occurring within 30 days of presentation.

Statistical analysis

We derived HE-MACS by forward stepwise logistic regression. On the basis of previous work in the derivation study [6,9], variables that had been found to predict (P < 0.05) the diagnosis of ACS on univariate analysis and that had good interobserver reliability (κ > 0.6) were considered for inclusion in the model [6]. We explored the relationship between age and the incidence of MACE by stratifying age into deciles and cross-tabulating this with the incidence of MACE. Having noted an approximately linear relationship, we considered age as a continuous variable in the model. Similarly, we explored the relationship between systolic blood pressure and the incidence of MACE. In this instance, we noted a markedly higher incidence of MACE in the bottom decile, which related to a cutoff set at ~100 mmHg. We, therefore, considered systolic blood pressure as a dichotomous variable with a cutoff set at 100 mmHg. Calibration plots were created by stratifying the calculated probability of ACS (using HE-MACS) into deciles. The incidence of MACE was then plotted against the mean probability in each decile. The intercept and slope were calculated by linear regression using GraphPad Prism, version 5.04 (GraphPad Inc., La Jolla, California, USA).

Having derived the HE-MACS model, we then stratified patients into four risk groups, based on the decisions that are likely to be available to clinicians who would apply HE-MACS in practice: ‘very low risk’ (possible immediate ‘rule out’), ‘low risk’ (potentially suitable for care in an ambulatory environment), ‘moderate risk’ (potentially suitable for care in the ED environment) and ‘high risk’ (potentially ‘rule in’ ACS). The thresholds for assigning patients to the different risk groups were set by receiver operating characteristic (ROC) analysis. First, we calibrated the model to achieve 100% sensitivity for ‘very low risk’ versus all other categories. While 100% is not the minimum acceptable sensitivity, we aimed for optimum sensitivity in the derivation phase in the knowledge that sensitivity is likely to be lower on validation. For ‘low risk’ versus all other groups, we calibrated the model to achieve 90% sensitivity (roughly equivalent to a normal initial high sensitivity cardiac troponin T concentration [10]). Finally, we aimed to achieve a specificity of 95% for the high-risk group.

To evaluate diagnostic performance in each individual study, we calculated test characteristics with 95% confidence intervals (CIs) and calculated the area under the ROC curve using the nonparametric method. We then calculated pooled sensitivity, specificity and likelihood ratios using the DerSimonian Laird (random effects) model. We used a χ2-test to test for statistical heterogeneity between studies. The proportions of patients identified as being at ‘very low risk’ in each validation study were aggregated to yield the pooled proportion.

Statistical analyses were undertaken using SPSS, version 23.0 (SPSS Inc., Chicago, Illinois, USA) except for (a) calculation of test characteristics, for which we used MedCalc, version 13.1.2.0 (MedCalc Software, Mariakerke, Belgium); and (b) and diagnostic meta-analysis, for which we used Meta-Disc [11]. To calculate the lower bound of the 95% CI when the negative predictive value (NPV) was 100% we used the epi.stats package in R, version 3.5.0 (University of Auckland, Auckland, New Zealand). As this work is a secondary analysis, we did not undertake a sample size calculation specific to the derivation and validation of HE-MACS. However, the derivation cohort had been powered to derive a decision rule with 15 predictors, assuming a 20% prevalence of ACS with 5% loss to follow-up, requiring a total of 790 participants.

Results

We included 796 patients in the derivation study. A total of 153 (19.2%) patients had prevalent AMI. After 30 days, a total of 118 (14.8%) patients had undergone coronary revascularization, seven (0.9%) had died (all of which had an adjudicated initial diagnosis of AMI) and two (0.3%) had incident AMI. Thus, a total of 179 (22.5%) patients had ACS (either prevalent AMI or at least one incident MACE within 30 days). In validation study 1, we included 474 patients of which 80 (16.9%) had AMI and 93 (19.6%) had ACS. In validation study 2, there were 659 participants including 74 (11.2%) with AMI and 91 (13.8%) with ACS. Follow-up was complete to 30 days in each study. A participant flow diagram is shown in Fig. 1. Baseline characteristics of participants in each study are shown in Table 1.

Fig. 1

Participant flow diagram for the derivation and validation studies. AMI, acute myocardial infarction.

Table 1

Baseline characteristics of included patients

The HE-MACS model we derived incorporated eight variables, as shown in Table 2. This model had an area under the ROC curve of 0.82 (95% CI: 0.78–0.86). We used the ROC analysis to stratify patients into four risk groups, as detailed in the Supplementary Appendix (Supplemental digital content 1, ). The model was well calibrated with a slope of 0.99 and intercept 0.0027. A calibration plot is available in the Supplementary Appendix (Supplemental digital content 1, ). The proportion of patients assigned to each risk group and the number of AMI and ACS in each group are shown in Table 3.

Table 2

Derivation of the HE-MACS model by logistic regression

Table 3

Proportion of patients with ACS and prevalent AMI in the four risk groups for the HE-MACS decision aid in each cohort

The test characteristics of the model in the validation studies are shown in Table 4, together with pooled estimates of sensitivity and specificity. The areas under the ROC curves in the validation studies were 0.78 (95% CI: 0.73–0.83) and 0.73 (95% CI: 0.67–0.78) respectively. Calibration plots are shown in the Supplementary Appendix (Supplemental digital content 1, ). There was no evidence of between-study heterogeneity for the sensitivity of HE-MACS (P = 0.24, I2: 27.6%), but there was evidence of heterogeneity for specificity (P < 0.001, I2: 92.2%). In the validation studies, the total proportion ‘ruled out’ would have been 5.5% in validation study 1 and 12.1% in validation study 2) with one missed ACS. The one patient missed had an adjudicated diagnosis of AMI. This 30-year-old male presented with a typical chest pain and had high-sensitive cardiac troponin T concentrations of 198 and 316 ng/l. Although that patient was assigned an adjudicated diagnosis of AMI, subsequent cardiac magnetic resonance imaging was consistent with a diagnosis of myocarditis, which was the final diagnosis assigned by the patient’s clinical team.

Table 4

Test characteristics of HE-MACS as a ‘rule out’ tool in the validation studies

Discussion

We have successfully derived and externally validated a decision aid that could ‘rule out’ ACS in 9.4% of patients using only the history and ECG with high NPV. The HE-MACS can also be used to effectively risk stratify the remaining patients. To our knowledge, there are currently no similar validated tools that can achieve this without biomarker testing, which requires the availability of laboratory equipment. This special and unique feature, therefore, opens up a number of important opportunities to apply HE-MACS more widely than the original MACS and T-MACS models.

First, HE-MACS could be used at triage when patients first arrive in the ED. Achieving rapid ‘rule-out’ for patients soon after arrival, without biomarker evaluation, could help to alleviate the growing pressures faced by crowded EDs, and accurately assign patients triage category (i.e. time to be seen by a clinician).

Second, the algorithm could be used in the prehospital environment. To date, the requirement to measure biomarker concentrations has been an important barrier to the implementation of rapid ‘rule-out’ strategies in this field. One potential solution would be to implement a point of care biomarker testing. However, the ability to ‘rule out’ ACS without any biomarker measurement would be cheaper, easier to implement and would facilitate more rapid decisions. Finally, the algorithm could be used in settings where blood testing is not available, such as in primary care. This is often the first medical contact for patients with chest pain and a high percentage of those patients are referred to the ED [12]. Thus, HE-MACS could help to avoid unnecessary ED referrals and immediately rule out ACS in those settings.

To our knowledge, there are no comparable triage tools for use in patients with suspected ACS that do not involve the use of biomarkers and that have used adjudicated outcomes with contemporary reference standards. A systematic review from 2016 found no similar triage tools from the era of cardiac troponin testing that have been validated in the emergency setting [13]. Another systematic review, from 2013, which focused on clinical prediction models for use in the prehospital setting, identified five relevant studies but none have been validated to ‘rule out’ ACS without biomarker testing and based on contemporary reference standards [14].

The volume of previous work in this field highlights the clinical demand for a validated tool that could be used to rapidly ‘rule out’ ACS without biomarker testing. It also demonstrates the importance of our current findings. We have derived and externally validated a decision aid that could be rapidly applied either in the prehospital environment or upon patient arrival in the ED. Its use could obviate the need for further investigation in 9.4% of patients. While this may appear to be a modest proportion of the overall total, it is worth noting that chest pain is one of the most common reasons for emergency ambulance calls and the most common reason for emergency hospital admission. The overall impact of reducing unnecessary transport to the hospital or facilitating rapid discharge from ED triage for these patients is therefore likely to be substantial.

It is now possible to measure cardiac biomarkers using portable devices with a turnaround time of 10–15 min. If such a technology is successfully validated in the prehospital environment, for example, alongside a validated risk score [15], and if the effectiveness of such technology is greater than the HE-MACS algorithm derived here, then the algorithm presented here may have limited applicability. However, the advantage of the algorithm presented here is that it does not require any biomarker testing with its associated costs, and could therefore be used even when such technology is not available.

Limitations

It is important to acknowledge that the HE-MACS model has been derived and validated in secondary analyses of prospectively collected data. All of the clinical data, for example, ECG interpretation, were recorded based on the interpretation of emergency physicians. Also, having HE-MACS derived and validated from different populations result in different rule out proportions. In addition, having a wide confidence interval in the validation study 2 (91.8–99.8) for the NPV could indicate that one of 10 patients could be missed. Before clinical implementation, therefore, we must evaluate HE-MACS in the settings in which it is likely to have the most impact, including the prehospital field and ED triage or other. To do this, we must, therefore, determine the feasibility of applying the model when used by paramedics and ED triage nurses, and determine whether diagnostic accuracy is maintained.

We did not include pain severity as a potential predictor in this work, as these data were not collected during the derivation study. However, previous work from validation study 1 has shown that pain severity is poorly predictive of AMI [16].

Conclusion

We have derived and validated the HE-MACS clinical decision aid that could be used to rapidly risk stratify patients with suspected cardiac chest pain without requiring biomarker evaluation. Importantly, this model could allow ACS to be rapidly ‘ruled out’ in 9.4% of patients with high sensitivity and NPV. We must now validate the model when applied by paramedics in the prehospital setting and by nurses at triage in the ED.

Acknowledgements

The Validation of the Early Vascular Markers of Acute Coronary Syndromes study was funded by a research grant from the Royal College of Emergency Medicine (2010), and was sponsored by Stockport NHS Foundation Trust.

The Bedside Evaluation of Sensitive Troponin (BEST) study received funding from Abbott Point of Care, European Union Horizons-2020 (by FABPulous BV) and the Royal College of Emergency Medicine, and was sponsored by Manchester University NHS Foundation Trust.

Conflicts of interest

There are no conflicts of interest.