Accuracy of telephone triage in primary care patients with chest discomfort: a cross-sectional study.

OBJECTIVE: To assess the accuracy of semi-automatic assisted telephone triage in patients with acute chest discomfort against the diagnosis of acute coronary syndrome (ACS) or other life-threatening events (LTEs).
METHODS: A cross-sectional study was performed of telephone conversations with 2023 patients with acute chest discomfort (pain, pressure, tightness or discomfort) who called out-of-hours services for primary care (OHS-PC) between 2014 and 2016. Sensitivity, specificity, positive and negative predicted values were calculated for a high urgency (patient seen within one hour) against the diagnoses of ACS and other LTEs. Diagnoses were retrieved from the patients' medical records in general practice, including hospital specialists' discharge letters.
RESULTS: Of 2023 patients who called because of chest discomfort, 227 (11.2%) had an ACS (men 14.9%, women 8.2%) and 58 (2.9%) had another LTE (men 3.6%, women 2.3%). The sensitivity and specificity of a high Netherlands Triage System (NTS) urgency allocation against ACS/other LTEs were 0.73 (95% CI 0.68 to 0.78) and 0.43 (95% CI 0.40 to 0.45), respectively. In 13.2% of the calls the triage nurse overruled the NTS urgency, mostly by upscaling (11.0%). The sensitivity and specificity of the final urgency allocation were 0.86 (95% CI 0.81 to 0.90) and 0.34 (95% CI 0.32 to 0.37). The positive and negative predictive values of the final urgency were 0.18 (95% CI 0.17 to 0.19) and 0.94 (95% CI 0.92 to 0.95), respectively.
CONCLUSIONS: The semi-automatic triage NTS tool underestimated the urgency in 27% of patients with ACS/other LTEs. Overruling by triage nurses improved safety, but still 14% of men and women with ACS/other LTEs received too low urgency, while efficiency remained poor. TRIAL REGISTRATION NUMBER: NTR7331. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The accuracy of semi-automatic assisted telephone triage with the Netherlands Triage System (NTS) in out-of-hours primary care (OHS-PC) has never been validated against clinical outcomes.

Two-thirds of all ambulance dispatches from OHS-PC are because of chest discomfort.

Around 10% of patients with chest discomfort referred to the hospital actually have an acute coronary syndrome (ACS).

Primary care semi-automatic assisted telephone triage of patients with chest discomfort is not safe or efficient.

The results were similar for men and women.

Intervening by the triage nurse improved safety.

‘Blind trust’ in decision support systems should be replaced by critical use of it.

Further research into symptom presentation and how to ask for symptoms in the OHS-PC could help improve semi-automatic assisted triage tools for patients who call with chest discomfort.

Introduction

Adequate triage followed by early diagnosis is vital in patients with acute chest discomfort. They might have an acute coronary syndrome (ACS) or other life-threatening event (LTE) such as pulmonary embolism. Telephone triage of such patients as done in out-of-hours services primary care (OHS-PC) and emergency medical services (EMS or ambulance dispatch centres) is, however, challenging because, based on symptoms only, it is difficult to differentiate ACS/other LTEs from other causes of chest pain.1 In the Netherlands, 80% of people with chest discomfort first call the general practitioner (GP) or OHS-PC and 20% directly call EMS (call 112) for an ambulance or are self-referrals to the emergency department (ED).2

In most Western countries, semi-automatic computer triage systems are used to support telephone decision making.3 Triage nurses fill out a standard list of hierarchically ordered questions that pops up after the triage nurse chooses the patient’s main complaint in the computer system, and the computer automatically generates a recommended level of urgency (one out of six urgency levels).3 4 These triage systems have never been validated in the OHS-PC setting against clinical outcomes as the reference.4

Since 2011, most OHS-PC in the Netherlands and half of the EMS use the Netherlands Triage System (NTS).5 In Dutch EDs, both the NTS and the Manchester Triage System (MTS) are used.6 The NTS was developed by an expert panel that integrated knowledge from (1) the MTS, (2) the Dutch national guidelines for general practice (‘NHG triage index’) and (3) the Dutch national standard for ambulance dispatch centres (‘LSMA’).4 The aim of the NTS is similar to other triage systems—that is, to guarantee efficient and safe care.7 The NTS seems safe with 0.006% serious adverse events (SAEs) a year at OHS-PCs (SAE is an unintended or unexpected event resulting in death or a severe harmful event).8 However, most SAEs are of cardiovascular origin (46.2%) and most often missed myocardial infarctions and sudden cardiac death.9 10 More questions are asked about the efficiency of the NTS for patients with chest pain. It is the most common reason for allocating an ambulance within 15 min (U1) to patients calling OHS-PC (60.7% of all dispatches), but only around 10% actually have an ACS.11–13

The aim of our study was to assess the accuracy of telephone triage for patients with chest discomfort who call the OHS-PC with the clinical outcome ACS/other LTE as the reference. Both the accuracy of the NTS tool and the ‘final’ urgency, including overruling by the triage nurses, were evaluated.

Methods

The study design and data collection have been published in our study design paper.14 In short, we performed a cross-sectional study in 2023 patients who contacted one of nine OHS-PCs in the Netherlands with acute chest discomfort (pain, pressure, tightness or discomfort) between 2014 and 2016. We selected calls on the basis of the International Classification of Primary Care (ICPC; a WHO worldwide code system for primary care) codes (K01, K02, K03, K24, K74, K75, K76, K77, K93, L04, P74, R02, R98) and keywords (thoracic pain, chest pain, myocardial infarction, heart attack and their common abbreviations).15 16 We deliberately sampled broadly to capture the entire domain of patients that could be suspected of ACS.

We used electronic health record data of OHS-PC (‘Callmanager’) and re-listened to telephone triage recordings in order to collect information about symptoms, patient and conversation characteristics of patients, and urgency allocation. The final diagnoses were retrieved from the patients’ GP, and these were based on hospital specialist discharge letters or GPs' diagnoses. For the diagnosis of ACS we used the cardiologist’s diagnosis. Patients without raised levels of (high-sensitivity) troponin were classified as unstable angina pectoris (UAP), and those with raised troponin levels as myocardial infarction, divided into STEMI (ST elevation myocardial infarction) and NSTEMI (non-ST elevation myocardial infarction).

Other life-threatening diagnoses were classified by an expert panel of experienced GPs (DZ, FR, RD) based on medical specialist letters (majority of votes).

Context

Triage assessment with the NTS starts with a mandatory ‘ABCD check’ (Airway, Breathing, Circulation and Disability); this check determines whether there are direct life-threatening problems for which an ambulance must be sent straight away.17 When the patient is ‘ABCD-stable’, the triage nurse has to choose one out of 56 main complaints to continue the urgency assessment. Based on the triage nurses’ input, the NTS automatically generates an urgency allocation ranging from U0 (resuscitation) to U5 (telephone self-care advice) (see online supplementary table 1). The triage nurse can overrule this recommendation and up- or downscale the urgency allocation, often after consulting the supervising GP.5 For clarification of the work process in the OHS-PC, see an adapted SEIPS (Systems Engineering Initiative for Patient Safety) framework (online supplementary figure 1).18

In the Netherlands there is a strong primary care and GPs take care of 93% of all healthcare problems for 3% of the total healthcare budget.19 The GP has a gatekeeper’s role to hospital care, although people may call EMS (112) in case they suspect a life-threatening situation.19

NTS and final urgency levels

The NTS urgency level is the automatically NTS-generated level of urgency based on the input of the triage nurse. The final urgency level is the ‘overruled’ level of urgency or the NTS urgency level (when not adjusted). In around 15% the final urgency allocation was unclear because of incomplete registration of overruled urgencies. For these cases, an expert panel of three experienced GPs (DZ, FR, IK) independently decided on the final urgency level blinded for the outcome (majority of votes) (see also online supplementary figure 2).

Data analyses

We dichotomised urgency allocation into high (U1 and U2) and low (U3, U4 and U5) urgency levels and calculated the accuracy of the NTS and final urgency for allocating a high level of urgency to patients with the outcome ACS/other LTEs in terms of sensitivity, specificity, positive and negative predictive values. For patient and call characteristics the χ2 test was used for comparison of dichotomous variables and the independent sample t-test or Mann–Whitney U test for continuous variables. We compared baseline characteristics between patients in whom the eventual diagnosis was retrieved and those in whom the GP did not provide these outcomes, to get an impression on whether selection bias could have occurred. Sample size was calculated using Harrell’s rule of thumb, which can be found in our study design paper.14 All data analyses were performed using SPSS statistical software programme Version 25.0.

Patient and public involvement

No patients were involved in setting the research question or the outcome measures, or in developing plans for design; however, they were involved in the implementation of the study. In addition, they were asked to advise on interpretation and writing up of results. The results will be shared and discussed with the national patient community of cardiovascular diseases (‘Harteraad’).

Results

Baseline characteristics

Of the 2023 patients with acute chest discomfort and a retrieved diagnosis, 227 (11.2%) had an ACS (women 8.2%, men 14.9%) and 57 (2.8%) had another LTE (women 2.3%, men 3.5%) (figure 1). The mean (SD) age was 58.9 (19.5) years and 54.9% were women. The NTS allocated 59.7% (men 57.6%, women 61.5%) to high urgency (U1/U2), and this changed to 68.5% (men 68.2%, women 68.7%) for the final urgency. Triage nurses overruled the NTS level of urgency in 13.2% of patients, mostly upscaling (11.0% of all calls) (figure 2).

Figure 1

Flowchart study population.

Figure 2

Overview of NTS urgency allocation and final urgency allocation after overruling.

Patients allocated to a high urgency level were significantly older (mean (SD) 62.7 (18.2) years vs 53.1 (19.9) years, p<0.001), had a shorter mean duration of calls (6:57 min vs 8:30 min, p<0.001) and more often someone else called on behalf of the patient (58.9% vs 38.9%, p<0.001) than those allocated a low urgency level. Nearly all callers expressed concerns (92.1% vs 86.4%, p=0.003).

In 2.1% of the patients labelled as ‘ABCD-unstable’, the NTS urgency allocation was downgraded by the triage nurse/GP. When the triage nurse chose shortness of breath as the main complaint, relatively more patients were given a high urgency level than when chest pain was the main complaint (73.8% vs 56.5%, p<0.001).

Patients classified as a high urgent level more often had acute shortness of breath (71.0% vs 53.9%, p<0.001), chest pain lasting for less than 12 hours (81.3% vs 62.5%, p<0.001), heavier chest pain (Numeric Rating Scale >7, 46.7% vs 22.1%, p<0.001) and sweating (47.7% vs 31.5%, p<0.001). They also more often recognised their symptoms from a previous cardiac event than those who were classified as a low level of urgency (30.9% vs 21.5%, p=0.002). Patients with stabbing pain (33.8% vs 15.0%, p<0.001) or pain located on either the left or right side of the thorax (53.4% vs 28.6%, p<0.001) were more often classified as of low urgency (table 1).

Table 1

Baseline characteristics of 2023 patients with chest discomfort (chest pain, pressure, tightness, discomfort) calling OHS-PC, divided into high (U1–U2) and low (U3–-5) NTS levels of urgency

	NTS high urgencyn=1208 (59.7%)	NTS low urgencyn=815 (40.3%)	P value
Patient characteristics
Mean (SD) age, years	62.7 (18.2)	53.1 (19.9)	<0.001
Female sex	683 (56.5)	428 (52.5)	0.074
Call characteristics
Mean (SD) total call duration, min	6:57 (3:42)	8:30 (3:48)	<0.001
Mean (SD) patients’ introduction duration, min	0:18 (0:12)	0:21 (0:14)	<0.001
Triage nurse consulted the GP	643 (53.2)	418 (51.3)	0.391
Someone else called on behalf of patient	711 (58.9)	317 (38.9)	<0.001
Caller expressed concerns	500 (92.1)	413 (86.4)	0.003
NTS main complaints
ABCD unstable	93 (7.7)	2 (0.2)	<0.001
Chest pain	867 (71.8)	668 (82.0)	<0.001
Shortness of breath	121 (10.0)	43 (5.3)	<0.001
Palpitations	33 (2.7)	27 (3.3)	0.45
Back pain	19 (1.6)	7 (0.9)	0.162
Dizziness	11 (0.9)	8 (1.0)	0.871
Arm problems	2 (0.2)	8 (1.0)	0.01
Other	62 (5.1)	52 (6.4)	0.233
NTS triage criteria
Chest pain	1088 (93.5)	738 (93.5)	0.954
Shortness of breath	664 (71.0)	337 (53.9)	<0.001
Chest pain duration <12 hours	843 (81.3)	453 (62.5)	<0.001
Pain intensity severe (NRS >7, range 1–10)	224 (46.7)	83 (22.1)	<0.001
Pressing heavy chest pain*	623 (73.4)	309 (47.5)	<0.001
Stabbing chest pain*	127 (15.0)	220 (33.8)	<0.001
Chest pain located retrosternal†	401 (51.2)	196 (30.0)	<0.001
Chest pain located left or right on thorax†	224 (28.6)	349 (53.4)	<0.001
Radiation of chest pain to any location	662 (71.5)	341 (54.3)	<0.001
Radiation to the arm	273 (50.8)	151 (34.5)	<0.001
Radiation to the shoulder blades	206 (43.8)	102 (26.2)	<0.001
Radiation to the jaws	93 (26.1)	22 (7.1)	<0.001
Sweating	363 (47.7)	159 (31.5)	<0.001
Nausea or vomiting	308 (55.3)	137 (39.4)	<0.001
Pallor	187 (39.0)	55 (22.8)	<0.001
Dizziness or (feelings of nearly) fainting	274 (26.6)	113 (14.8)	<0.001
History of coronary artery disease	234 (38.6)	127 (27.1)	<0.001
Diabetes mellitus	98 (23.1)	62 (14.8)	0.002
Other than NTS items, if they were discussed in triage call
Coin size area of chest pain	77 (45.3)	154 (48.0)	0.571
Tingling sensations on chest, hands or elsewhere	112 (86.8)	97 (79.5)	0.121
Patient recognised symptoms from previous cardiac event	152 (30.9)	76 (21.5)	0.002
CVD risk factors other than diabetes
Hypertension	190 (43.0)	116 (28.6)	<0.001
Hypercholesterolemia/statin use	103 (28.0)	89 (22.0)	0.056

Coronary artery disease: history of myocardial infarction, percutaneous coronary intervention, coronary artery bypass graft, stable or unstable angina pectoris (patient reported).

*Pain described by patient. Pressing heavy pain: pressing, heavy or tightening pain versus other types of pain (stabbing, burning, cramping, tearing) Stabbing pain: stabbing versus other types of pain (pressing, heavy, tightening, burning, cramping, tearing).

†P value comparing retrosternal or left/right side thorax versus other locations of pain together (retrosternal, left/right side thorax, back/shoulder, epigastric region).

CVD, cardiovascular disease; NRS, Numeric Rating Scale; NTS, Netherlands Triage System; OHS-PC, out-of-hours services for primary care.

There were no clinically meaningful differences in patient or call characteristics between the 2023 patients with a retrieved diagnosis and the 972 patients without a diagnosis (online supplementary table 2).

Diagnoses

Of the 227 patients with an ACS (91 women, 136 men), 67 (29.5%) patients had a STEMI (women 22.1%, men 33.8%), 93 (40.0%) a NSTEMI (women 46.2%, men 37.5%), 52 (22.9%) UAP (women 20.9%, men 24.3%) and 15 (6.6%) unspecified ACS, including three sudden cardiac deaths in women (women 9.9%, men 4.4%) (table 2). In nearly all cases (97.0%) the ACS diagnosis was from the cardiologist and based on symptom presentation, troponin levels and electrocardiography. Three patients died before arrival of the ambulance (they were classified as acute cardiac death) and one patient died after resuscitation at the ED. Three patients were classified as ACS by the GP, while not referred because of short life expectancy due to cancer. Of the 58 patients with other LTEs (2.9%), 13 had a pulmonary embolism, 5 an acute abdominal aneurysm, 5 a thoracic aortic dissection and 35 other diagnoses (eg, diabetic ketoacidosis, sepsis). The majority of patients had non-urgent medical conditions (85.9%) including: non-urgent cardiovascular diseases (18.7%), non-cardiac chest pain (not further specified) (17.3%), musculoskeletal pain (17.9%), psychogenic disorders (11.2%), gastrointestinal disorders (7.1%), respiratory disorders (5.5%) and other non-urgent diagnoses (8.4%).

Table 2

Diagnosis of 2023 patients who contacted the OHS-PC for chest discomfort (pain, pressure, tightness or discomfort), divided into NTS high urgency (U1–U2) and low urgency (U3–U5)

Diagnosis	NTS high urgencyn=1208 (59.7%)	NTS low urgencyn=815 (40.3%)	P value
ACS	171 (14.2)	56 (5.6)	<0.001
STEMI	51 (29.8)	16 (28.6)	0.858
NSTEMI	71 (41.5)	22 (39.3)	0.768
UAP	38 (22.2)	14 (25.0)	0.668
Non-classified ACS	11 (6.4)	4 (7.1)	0.853
Other life threatening events (LTEs)	38 (3.1)	20 (2.5)	0.361
Pulmonary embolism	6 (15.8)	7 (35.0)	0.095
Acute abdominal aneurysm	5 (13.2)	0 (0.0)	0.151
Thoracic aortic dissection	3 (7.9)	2 (10.0)	0.786
Other*	24 (63.2)	11 (55.0)	0.546
Non-urgent cardiovascular diseases†	257 (21.3)	121 (14.8)	<0.001
Non-cardiac chest pain, not further specified ‡	180 (14.9)	169 (20.7)	0.001
Musculoskeletal pain	173 (14.3)	189 (23.2)	<0.001
Psychogenic disorders	116 (9.6)	110 (13.5)	0.006
Gastrointestinal tract disorders	97 (8.0)	46 (5.6)	0.04
Respiratory tract disorders	66 (5.5)	45 (5.5)	0.955
Other non-urgent diagnoses§	110 (9.1)	59 (7.2)	0.137

*Stroke, severe COPD exacerbation, acute severe heart failure, sepsis, coronary spasm caused by hypokalaemia, diabetic ketoacidosis, epileptic insult, bleeding from oesophageal varices, ovarian torsion, ventricular fibrillation.

†Stable angina pectoris (including atypical chest pain), stable heart failure, arrhythmias, hypertension.

‡Cardiac pathology unlikely after cardiologist’s or GP’s diagnostic work-up, but without differential diagnosis.

§Among others: anaemia, carcinoma, vasovagal collapse, side effects to medication, dermatologic diseases.

ACS, acute coronary syndrome; NSTEMI, non-ST elevation myocardial infarction; NTS, Netherlands Triage System; OHS-PC, out-of-hours services for primary care; STEMI, ST elevation myocardial infarction; UAP, unstable angina pectoris.

Relationship between diagnosis and urgency allocation

Of the patients with an ACS, 75.3% (women 74.7%, men 75.7%) were assigned a NTS high urgency level and 88.5% (women 89.0%, men 88.2%) were assigned a final high urgency level. Patients with another LTE were assigned a NTS high level of urgency in 65.5% (women 76.0%, men 57.6%) and a final high level of urgency in 74.1% (women 84.0%, men 66.7%). Of the 41 ACS/other LTE patients who were assigned a final low urgency level, most were categorised as U3 (87.8%) and a minority as U4 (7.3%) or U5 (4.9%) (table 3).

Table 3

Overview of the NTS and final urgency allocation of 285 patients with acute coronary syndrome or life-threatening events

	Final U1	Final U2	Final U3	Final U4	Final U5	Total
NTS U1	153	14	1	0	0	168
NTS U2	16	24	0	0	1	41
NTS U3	20	9	34	0	0	63
NTS U4	0	0	0	0	0	0
NTS U5	4	4	1	3	1	13
Total	193	51	36	3	2	285

NTS urgency: NTS urgency allocation retrieved from electronic health record.

Final urgency: the overruled urgency retrieved from re-listening to calls.

Black type, the NTS urgency and final urgency were the same.

Blue type, the user (triage nurse in coordination with the GP) has scaled up the urgency.

Red type, the user (triage nurse in coordination with the GP) has scaled down the urgency.

Patients with non-urgent cardiovascular diseases were more often classified as high urgent (21.3% vs 14.8%, p<0.001) and patients with psychogenic disorder as low urgent (9.6% vs 13.5%, p=0.006).

Accuracy of NTS and final urgency levels

The NTS high urgency had a sensitivity of 0.73 (95% CI 0.68 to 0.78), specificity of 0.43 (95% CI 0.40 to 0.45), positive predictive value of 0.17 (95% CI 0.16 to 0.19) and negative predictive value of 0.91 (95% CI 0.89 to 0.92) for the outcome ACS/other LTE (table 4). For the final urgency level, sensitivity was 0.86 (95% CI 0.81 to 0.90), specificity 0.34 (95% CI 0.32 to 0.37), positive predictive value 0.18 (95% CI 0.17 to 0.19) and negative predictive value 0.94 (95% CI 0.92 to 0.95).

Table 4

Accuracy outcomes of the NTS and final high urgency allocation (U1/U2) against ACS or other LTEs (ACS/LTE prevalence 14.1%)

Accuracy outcomes	NTS high urgencyValue (95% CI)	Final high urgencyValue (95% CI)
Sensitivity	0.73 (0.68 to 0.78)	0.86 (0.81 to 0.90)
Specificity	0.43 (0.40 to 0.45)	0.34 (0.32 to 0.37)
Positive predictive value	0.17 (0.16 to 0.19)	0.18 (0.17 to 0.19)
Negative predictive value	0.91 (0.89 to 0.92)	0.94 (0.92 to 0.95)

ACS, acute coronary syndrome; LTE, life-threatening event; NTS, Netherlands Triage System.

Discussion

The safety and efficiency of the Dutch NTS decision support tool were found to be poor for telephone triage at OHS-PCs in patients with acute chest discomfort, with a sensitivity of 0.73 and specificity of 0.43 for high urgency allocation of patients who had an ACS/other LTEs. Safety improved after intervention by the triage nurse and the final high urgency level had a sensitivity of 0.86 and specificity of 0.34.20 Women and men with ACS/other LTEs received equally high urgencies.

At a prevalence of ACS/other LTEs of 14.1% (11.2% and 2.9%, respectively), the positive predictive value for both the NTS and final urgency was low (0.17 and 0.18, respectively) and the negative predictive value was high (0.91 and 0.94, respectively).

Clearly, the triage system is calibrated on the safe side. Most GPs highlight over triage, but in our opinion, in the critical domain of patients with chest discomfort, under triage should receive more attention.5 19 21 It is evident that patients with an ACS and those classified with other LTEs should receive a high urgency allocation, but the NTS classified 27% of the ACS/LTE patients as low urgency, and this was still rather high for the final urgency (14%).

We were able to analyse the original and very first conversation with patients, and these conversations were assessed without knowledge of the diagnosis; the assessment of symptoms was not affected by recall or hindsight bias. Moreover, we were able to include a large sample of patients and missing outcome data did not lead to patient or call selection. We therefore are confident our results are generalizable to Dutch settings and also to similar OHS-PC settings in, for example, the UK and Scandinavian countries.1 Our results may also be generalizable to EMS settings, because the chance of having an ACS among those calling for chest discomfort is similar in the EMS setting and the OHS-PC setting.13 22

A limitation of the study is that, in 15% of calls, the overruled urgency was unclear. In these cases, an expert panel (blinded to the clinical outcome) decided on all the available information to be heard in the calls.

We could not find studies evaluating the accuracy of telephone triage in the OHS-PC among patients with chest discomfort against clinical outcomes.

A systematic review of six studies evaluating the MTS in the ED setting reported that the MTS had allocated a high urgency level in patients with ACS with a sensitivity of 0.70–0.80.23 One of these six studies also reported specificity (0.59) among patients with chest pain referred to a Portuguese ED.24 Thus, the MTS, with the advantage of face-to-face contact, seems to have a similar sensitivity and somewhat higher specificity to that in our study. Interestingly, the prevalence of ACS among the 233 patients with chest pain in the Portuguese study was lower than in our study (9.4% vs 11.2%).24 None of the studies reported positive and negative predictive values, which prevented us from comparing these to ours. The prior risk of ACS and LTEs of 14.1% changed only marginally to 17% and 18% for the NTS and final high urgency, respectively, and to 9% and 6% for the NTS and final low urgency allocation.

To the best of out knowledge, there are no accuracy studies performed in the EMS setting in which urgency allocation with EMS (112 or 911) telephone triage was compared with the presence or absence of ACS. In an observational Swedish study among 14 454 patients calling EMS for chest discomfort, 63.5% received an urgency 1 ambulance and 11.5% turned out to have an ACS (women 9% and men 14%).22 An EMS study in the USA with 3007 calls from patients with acute chest pain to the Medical Priority Dispatch System (MPDS) for decision support reported a prevalence of acute myocardial infarction of 8.7%.25

In a primary care study, the GP’s clinical risk estimate was more accurate in predicting ACS than a prediction rule (c-statistic 0.75 vs 0.66) in patients with acute chest discomfort.20 This is in line with our finding that the sensitivity of the final urgency level (including overruled cases) was higher than that of the NTS level of urgency alone (0.86 vs 0.73).

Previous studies have found that the risk of ACS in women is underestimated, a finding in contrast to our result in which women and men with ACS received similarly high urgency allocations.26 27 One of the former mentioned EMS studies also found that women and men received similarly high urgency allocations and the other EMS study did not report sex-stratified analyses.22 25

Missing an ACS is the most common reason for malpractice claims worldwide.9 28 There are no generally accepted guidelines that define what rate of missed ACS is considered acceptable, and this threshold will likely differ among different healthcare providers. A survey performed among 1029 ED doctors in the USA, New Zealand and Australia showed that they considered on average a rate of 0.1%–1% (range 0%–10%) as acceptable.29 We do not have such information from GPs or lay people, but we suspect that this will be similar in the Netherlands. Alarmingly, this number is not in line with our finding that 27% of patients with ACS/other LTEs received a NTS low urgency allocation and a 14% final low urgency allocation. Indeed, a low urgency allocation does not necessarily mean the ACS is being missed, but that an undesirable risk to patient safety has been taken with the hazard of life-threatening ventricular arrhythmias and sudden death due to severe pumping failure of the heart in those with acute myocardial infarction.30 31 On the other hand, triage nurses and their supervising GPs increased triage safety by overruling the decision support system adequately, although with significantly increasing over triage. Apparently, the ‘human factor’ is conducive to safety in the current telephone triage process.18 32 Nonetheless, potential room for improvement should be studied, both of the decision support tool itself by developing better diagnostic prediction rules as well as by improving its use by triage nurses and management.33

Conclusion

The semi-automatic triage NTS tool underestimated the level of urgency in 27% of patients with ACS/other LTEs. Overruling by triage nurses improved safety, but still 14% of men and women with ACS/other LTEs were allocated too low a level of urgency while efficiency remained poor.