Accuracy of emergency medical services (EMS) telephone triage in identifying acute coronary syndrome (ACS) for patients with chest pain: a systematic literature review.

OBJECTIVE: To systematically appraise the available evidence to determine the accuracy of decision aids for emergency medical services (EMS) telephone triage of patients with chest pain suspected to be caused by acute coronary syndrome (ACS) or life-threatening conditions.
DESIGN: Systematic review. DATA SOURCES: Electronic searches were performed in Embase 1974, Medline 1946 and CINAHL 1937 databases from 3 March 2020 to 4 March 2020. ELIGIBILITY CRITERIA: The review included all types of original studies that included adult patients (>18 years) who called EMS with a primary complaint of chest pain and evaluated dispatch triage priority by telephone. Outcomes of interest were a final diagnosis of ACS, acute myocardial infarction or other life-threatening conditions. DATA EXTRACTION AND SYNTHESIS: Two authors independently extracted data on study design, population, study period, outcome and all data for assessment of accuracy, including cross-tabulation of triage priority against the outcomes of interest. Risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 assessment tool.
RESULTS: Searches identified 553 papers, of which 3 were eligible for inclusion. Those reports described the evaluation of three different prediction models with variation in the variables used to detect ACS. The overall results showed that dispatch triage tools have good sensitivity to detect ACS and life-threatening conditions, even though they are used to triage signs and symptoms rather than diagnosing the patients. On the other hand, prediction models were built to detect ACS and life-threatening conditions, and therefore, prediction models showed better sensitivity and negative predictive value than dispatch triage tools.
CONCLUSION: We have identified three prediction models for telephone triage of patients with chest pain. While they have been found to have greater accuracy than standard EMS dispatch systems, prospective external validation is essential before clinical use is considered. PROSPERO REGISTRATION NUMBER: This systematic review was pre-registered on the International prospective register of systematic reviews (PROSPERO) database (reference CRD42020171184). © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

To our knowledge, this is the first systematic review looking into the accuracy of prehospital chest pain telephone triage.

It is possible that some relevant papers were missed because researchers only included studies that have adult participants who are >18 years old and excluded papers not published in English language.

Unfortunately, because of the paucity of data for each decision aid, the data were not suitable for meta-analysis.

Also, there was a difference in the definition of life-threatening conditions definition between included studies.

Background

Chest pain, which is one of the main symptoms of acute coronary syndromes (ACS), is one of the most common reasons for ambulance call-outs and presentations to emergency departments (EDs).1 2 However, most patients with chest pain who are transported by ambulance are ultimately diagnosed with self-limiting, non-cardiac disease. Thus, patients with chest pain tend to be systematically overtriaged, increasing ambulance resource utilisation and contributing to ED crowding.2 3

Two emergency medical services (EMS) dispatch systems are currently used in the UK: the Advanced Medical Prioritising Dispatching System (AMPDS) and the National Health Service (999) System.4 Both systems use computerised telephone triage software that identify life-threatening conditions by a series of questions at the beginning of the call. For patients with a primary complaint of chest pain, they also include a specific series of questions to identify ACS and triage the case using the patient’s sign and symptoms.3 5 However, systematic reviews have shown that there is only a low level of evidence to support the use of current triage systems.6 7 There are also insufficient data on the dispatch protocol efficiency.7 8 There is no consensus on the definitions used in Criteria-Based Dispatch, which is the programme used to triage patients by signs and symptoms, and AMPDS.9 10 Also, there is no consensus on the accepted level for overtriage or undertriage for medical emergency dispatch.6 9 Therefore, systematic overtriage for patients with chest pain is used by dispatch systems to avoid potential harm to patients. EMS resource utilisation is increased as a result,6 and there are important differences between patients’ clinical findings when examined and dispatch priority.8 9 Therefore, with the current clinical guidelines for EMS, it is hard to rule in or rule out ACS by telephone triage.11

We aimed systematically to appraise the available evidence to determine the accuracy of decision aids that have been used for EMS telephone triage of patients with chest pain suspected to be caused by ACS or life-threatening conditions.

Methods

The systematic review was conducted in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines and following Cochrane methodology for diagnostic test accuracy reviews.12 A completed PRISMA checklist is provided in online supplemental appendix 1.

Search strategy and eligibility criteria

We searched the Embase 1974, Medline 1946 and CINAHL 1937 databases from 3 March 2020 to 4 March 2020 for the terms ‘chest pain, telephone or dispatch, triage, ACS or AMI, life-threatening conditions, and EMS or prehospital’. Retrospective and prospective cohort studies written in English were eligible. Search strategy table is provided in online supplemental appendix 2.

Studies included

Based on the inclusion criteria, titles and abstracts were independently screened and shortlisted by two reviewers (AAlg and CR). The inclusion criteria were: (1) the study included adult patients (>18 years); (2) the study included patients with a primary complaint of chest pain who called EMS for an ambulance; (3) the studies evaluated dispatch triage priority; (4) the study provided data for a linked final diagnosis of ACS, acute myocardial infarction (AMI) or other life-threatening conditions. Two reviewers independently reviewed and screened potentially relevant full-text papers to identify those fulfilling the inclusion criteria. Any discrepancies between the reviewers were resolved by a third reviewer (AAlo).

Outcome measures

The primary outcome was a diagnosis of ACS on hospital admission, including ST elevation myocardial infarction (STEMI), non-STEMI (NSTEMI) and unstable angina. AMI (incorporating both STEMI and NSTEMI) is defined according to the universal definition of AMI.13 Unstable angina is defined as sudden prolonged chest pain at rest >20 min, new-onset severe angina or previous angina that is increasing in severity of pain, duration of pain and/or frequency.14

The secondary outcome was the diagnosis of a life-threatening condition associated with chest pain. Since there is no universal definition for life-threatening conditions associated with chest pain, we included all relevant data from the studies, regardless of the precise definition used for ‘life-threatening conditions’.

Methodological quality assessment

The quality assessment of eligible articles was independently assessed by two reviewers (AAlo and AAlg), using a modified Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool to suit the purpose of this systematic review.15 Discrepancies between reviewers were solved by discussion.

Data extraction

After selecting the eligible studies, two investigators (AAlo and AAlg) used a standardised data extraction form to extract and collect data as study design, population, study period and outcome. Also, both investigators extracted all data required for diagnostic accuracy assessment including 2×2 tables when available for dispatch priority and prediction models priority for suspected ACS or life-threatening conditions.

Statistical analysis

In line with our protocol, we considered pooling sensitivities, specificities, negative and positive predictive values (NPV and PPV, respectively) by meta-analysis. However, after reviewing the final list of studies meeting inclusion criteria, the reviewers unanimously concluded that meta-analysis would have been inappropriate due to clear evidence of clinical heterogeneity between studies and missing data. We, therefore, undertook a narrative synthesis of the existing evidence.

Results

In total, our literature search identified 553 potentially relevant articles of which 26 were considered potentially eligible for inclusion after screening titles and abstracts. Following independent full-text review, 23 articles were excluded and 3 articles met the inclusion criteria for this review (figure 1). Excluded studies with reasons for exclusion are shown in online supplemental appendix 3.16–18

Figure 1

Flow diagram of study selection.

Study characteristics and quality assessments

General study characteristics are summarised in table 1. Patient characteristics and study weaknesses are summarised in table 2. The number of patients in each study with each component of the outcome measures studied is summarised in online supplemental appendix 4. The details of the prediction models identified, together with the test characteristics (including sensitivity, specificity, NPV and PPV) for the diagnosis of ACS and life-threatening conditions, are shown in tables 3 and 4.

Table 1

Characteristics of included studies

Study ID	Year	Country	Study design	Sites	Study period
Gellerstedt et al16	2006	Sweden	Prospective cohort study	1	3 months
Reuter et al18	2019	France	Follow-up prospective cohort study	1	18 months, May 2010–November 2011
Gellerstedt et al17	2016	Sweden	Retrospective cohort study	1	7 months, 1 May 2009–28 February 2010

Table 2

Study and patient characteristics of all studies included in the systematic review

Study ID	N	Population	Triage criteria	Exclusion criteria	Target condition	Study weaknesses
Gellerstedt et al16	503	Patients who called for an ambulance and who were assessed by the dispatcher as having chest pain	Dispatcher judgement versus prediction model (derived in this cohort)	NA	AMI, life-threatening conditions	Retrospective, no sample size calculation, small simple size, out of date AMI reference standard (based on creatine kinase-MB rather than cardiac troponin)
Reuter et al18	3727	Patients who called for an ambulance complaining of chest pain	Prediction model	Difficult communication, language barrier inability to speak with patient	ACS	Information bias, unclear how AMI was adjudicated
Gellerstedt et al17	1942	Patients who called 112 for an emergency ambulance with a complaint of chest pain	Dispatcher judgement versus prediction model	Lost diagnosis or follow-up.	ACS, life-threatening conditions	95% CIs for test characteristics were not presented. Only internal validation completed, and the full results of that validation were not presented

ACS, acute coronary syndrome; AMI, acute myocardial infraction; NA, not available.

The modified QUADAS-2 methodological quality assessment tool was used to assess all eligible studies as shown in online supplemental appendix 5. A summary of the quality assessment for each eligible study is shown in figure 2. Two out of the three studies were retrospective cohort studies and compared the dispatch triage for ambulance response priority to the prediction model16 17 and one was prospective cohort study that only testing prediction model accuracy.18 ACS was adjudicated by experts in only one study18 while the other two studies used the hospital final diagnosis of ACS, AMI or life-threatening conditions. All eligible studies used the appropriate ACS or AMI definitions at the time of conducting the study.

Figure 2

Quality Assessment of Diagnostic Accuracy Studies 2QUADAS-2 assessment of eligible studies. N/A, not available.

Summary of the existing evidence

Gellerstedt et al16 derived a prediction model to detect AMI and life-threatening conditions patients calling for an emergency ambulance with chest pain. They aimed to effectively triage calls to determine the appropriate level of response (capable of providing advanced life support vs only basic life support). The final model used eight variables (age, gender, strong pain, dyspnoea, cold sweat, nausea, vertigo and syncope) to calculate the probability of AMI or a life-threatening condition. Setting an arbitrary cut-off, the authors found that the model had greater sensitivity than and similar specificity to the judgement of the call handlers, which could potentially improve triage. However, performance of the model was only evaluated in the derivation set, with no validation presented.

In 2016, the same group derived a new prediction model in another study. They aimed to determine whether a computer-based decision support system could have superior accuracy for identifying life-threating conditions or ACS in patients who called for an emergency ambulance with chest pain. Gellerstedt et al17 collected data from patients who called 112 having chest pain using 10 questions related to ACS with yes or no answers. They derived two prediction models based on those answers. The first prediction model (full model) incorporated data from all 10 questions that dispatchers had asked, including: age, gender, central chest pain, high intensity pain, pain duration between 15 min and 24 hours, aggravated pain when breathing or moving, abnormal breathing, cold sweat, diabetes or previous cardiovascular disease, previous ACS and belief that symptoms are heart related. The authors also derived a ‘limited model’, which only included variables that had a significant association with the presence of ACS or a life-threatening condition on bivariate analysis. These included gender, age, central chest pain, high intensity pain, previous ACS and a high priority based on dispatcher judgement. There was no significant difference in performance between the full model and limited model. However, both the limited and full models had superior sensitivity to the priority assigned by dispatchers in practice, both for ACS (table 3) and life-threatening conditions (table 4), with similar specificity.17 The model was derived in 70% of the available cohort and validated in the remaining 30%. The authors state that performance was ‘similar’ in the validation set, although no further details were provided.

Table 3

Diagnostic accuracy of dispatch tools and prediction models for ACS

Study ID	N	Triage criteria	Sensitivity %	Specificity %	NPV %	PPV %	TP	FP	FN	TN
Gellerstedt et al16	503	Dispatch system (standard care)	85.7%	26.9%	87.7	32.02	90	291	15	107
	503	Prediction model	92.4%	28.6%	93.4	25.5	97	284	8	114
Reuter et al18	2363	Male prediction model	NA	NA	NA	NA	NA	NA	NA	NA
	1824	Female prediction model	NA	NA	NA	NA	NA	NA	NA	NA
Gellerstedt et al17	1942	Dispatch system	82.6	39.9	94.3	15.9	194	1026	41	681
		Full prediction model	90.2	40.9	96.8	17.4	212	1008	23	699
		Limited prediction model	91.1	41.1	96.96	17.5	214	1006	22	701

ACS, acute coronary syndrome; FN, false negative; FP, false positive; NA, not available; NPV, negative predictive value; PPV, positive predictive value; TN, true negative; TP, true positive.

Table 4

Diagnostic accuracy of dispatch tools and prediction models for life-threatening conditions

Study ID	N	Triage criteria	Sensitivity %	Specificity %	NPV %	PPV %	TP	FP	FN	TN
Gellerstedt et al16	493	Dispatch system	80.9	37.5	64.1	47.3	178	198	42	75
	493	Prediction model	86.4	31.8	74.4	50.5	190	186	30	87
Gellerstedt et al17	1944	Dispatch system	76.5	39.8	89.9	19.5	238	983	73	650
	1944	Full prediction model	88.4	42.1	95	22.5	275	946	36	687
	1944	Limited prediction model	88.7	42.1	95.1	22.6	276	945	35	688

FN, false negative; FP, false positive; NPV, negative predictive value; PPV, positive predictive value; TN, true negative; TP, true positive.

Reuter et al18 used backward logistic regression to derive two separate prediction models to detect ACS among men and women, respectively. The male prediction model to detect ACS consisted of eight variables, which are: age, smoking, severe pain, permanent pain, breathing non-related pain, retrosternal pain, radiating pain and additional symptoms. The female prediction model to detect ACS consisted of four variables, which are: age >60 years, history of coronary artery disease, breathing non-related pain and radiating pain.

The male prediction model had an area under the curve (AUC) of 0.76 (95% CI 0.73 to 0.79) in the derivation set (70% of the sample, which was randomly selected), which was maintained at 0.76 (95% CI 0.73 to 0.80) in the internal validation set (30% of the cohort). However, while the female prediction model had a similar AUC in the derivation set (0.79, 95% CI 0.75 to 0.83), it had lower accuracy in the validation set (AUC 0.67, 95% CI 0.60 to 0.74). No data were presented to enable calculation of sensitivity, specificity, PPV or NPV, and the output of the model was not compared with standard care or clinical judgement.

Discussion

In this systematic review, we have identified three studies that presented the derivation of clinical prediction models to enhance the telephone triage of patients calling for an emergency ambulance with chest pain. None of the models presented had sufficiently high sensitivity to avoid the need for ambulance dispatch. While two of the prediction models were shown to have greater accuracy than standard dispatch systems, unfortunately the current evidence does not support the use of these prediction models in practice without further validation. However, the evidence to date does suggest that clinical prediction models for this use case could feasibly improve on the accuracy of triage offered by standard EMS dispatch systems.

The original model derived by Gellerstedt et al used a now outdated definition of AMI (using creatine kinase-MB as the reference standard biomarker), meaning that the results could not be applied to modern clinical practice without further validation. The second model derived by Gellerstedt et al underwent internal validation, but full details of validation were not published. Finally, Reuter et al18 derived two prediction models (for men and women, respectively) but did not present sufficient data to compare their performance to standard care. The AUC for the male prediction model was 0.76, showing good overall accuracy. The prediction model for females had a poor AUC (0.67) on internal validation. We identified no prediction models that have undergone external validation.

It is an extremely challenging task for dispatchers to identify ACS over the telephone as there are so many different causes of chest pain,11 19 20 and there is very limited information available at the time of the initial phone call. It has previously been reported that only 1 out of 18 patients who request an ambulance for a complaint of chest pain has ACS.3 While a validated prediction model would have clear benefits, this systematic review has identified no validated tools that could be used in current practice.

It has also been reported that the AMPDS triage system is not an accurate tool for identifying ACS.3 Moreover, in Denmark, Criteria-based Danish index is used to triage emergency calls and it resulted in overtriage for 51% of chest pain calls as they were discharged without a specific diagnosis and AMI was only found in 11% of patients.2 Dispatch triage systems were constructed to prioritise the prehospital response based on a patient’s condition. They were not intended to make diagnoses. Therefore, it is important to highlight that the use of prediction models for telephone triage may not be expected to ‘rule out’ ACS but to identify patients who may safely receive a lower priority response, while correctly identifying the vast majority of those who require an urgent response.

It is clear that chest pain is a main symptom of ACS.2 11 19 21–23 However, previous studies have shown that some patients with ACS have been given lower priority during triage due to not having chest pain.3 24 Deakin et al found that 13% of patients who called for an ambulance with confirmed ACS presented with no chest pain A study was conducted in Australia by Coventry et al25 to identify the sex differences in symptoms related to MI reported to EMS dispatch agrees with Deakin et al3 and our findings that chest pain alone is not a solid predictor of MI. Among female patients who called for an ambulance and were finally diagnosed with MI only 54.4% presented with chest pain, while among male patients 68.7% had chest pain. As a result, priority 1 allocation for women was 67% and for men 81%.25 Also, several studies have identified differences of symptoms reported by patients with ACS. Interestingly, as we found in this systematic review, symptoms of ACS differ between male and female patients.18 19 25 One of the studies included in the search developed two different prediction models for men and women. However, the female prediction model failed to stand up to internal validation and should not be implemented.18 The most common symptom with MI following chest pain is shortness of breath (SOB), which appeared in 28.3% women vs 25.8% men. In patients presenting with no chest pain, SOB was the most common symptom, occurring in 38% women vs 32% men. These findings agree with what we have found as two studies that nausea, SOB and cold sweat were the most common symptoms reported after chest pain.16 17 Women with ACS are less likely to present with chest pain when compared with men. This may explain the relative underperformance of the female prediction model derived by Reuter et al.18

Future research

This systematic review has highlighted two potential areas for future research. First, the three prediction models identified may add value over existing telephone triage algorithms but this would require prospective external validation. Second, the existing evidence suggests that it is feasible to derive a prediction model to enhance telephone triage for patients with suspected ACS. The derivation and validation of a new, contemporary prediction model could reduce the systematic overtriage of patients in future, while ensuring that those with life-threatening conditions consistently receive an urgent response. However, given the risks of delayed responses (eg, ‘time is muscle for patients with STEMI), a higher sensitivity than has been observed in the studies identified is likely to be required.

Patient and public involvement

No patient involved.

Limitations

It is possible that some relevant papers were missed because we only included studies that have adult participants who are >18 years old and we excluded papers not published in English language. Unfortunately, we could not conduct a meta-analysis as there were variations between dispatch system tools and priority types. Also, there was a difference in the definition of life-threatening conditions between included studies.

Conclusion

We have identified three clinical prediction models for telephone triage of patients with chest pain. While the models have been found to have greater accuracy than standard EMS dispatch systems, the level of evidence to support their use is currently low. Prospective external validation is, therefore, required. However, our findings do support the feasibility of deriving and validating clinical prediction models to reduce overtriage while ensuring urgent responses for those who need them most.