Missed Acute Coronary Syndrome During Telephone Triage at Out-of-Hours Primary Care: Lessons From A Case-Control Study.

OBJECTIVES: Serious adverse events at out-of-hours services in primary care (OHS-PC) are rare, and the most often concern is missed acute coronary syndrome (ACS). Previous studies on serious adverse events mainly concern root cause analyses, which highlighted errors in the telephone triage process but are hampered by hindsight bias. This study compared the recorded triage calls of patients with chest discomfort contacting the OHS-PC in whom an ACS was missed (cases), with triage calls involving matched controls with chest discomfort but without a missed ACS (controls), with the aim to assess the predictors of missed ACS.
METHODS: A case-control study with data from 2013 to 2017 of 9 OHS-PC in the Netherlands. The cases were matched 1:8 with controls based on age and sex. Clinical, patient, and call characteristics were univariably assessed, and general practitioner experts evaluated the triage while blinded to the final diagnosis or the case-control status.
RESULTS: Fifteen missed ACS calls and 120 matched control calls were included. Cases used less cardiovascular medication (38.5% versus 64.1%, P = 0.05) and more often experienced pain other than retrosternal chest pain (63.3% versus 24.7%, P = 0.02) compared with controls. Consultation of the supervising general practitioner (86.7% versus 49.2%, P = 0.02) occurred more often in cases than in controls. Experts rated the triage of cases more often as "poor" (33.3% versus 10.9%, P = 0.001) and "unsafe" (73.3% versus 22.5%, P < 0.001) compared with controls.
CONCLUSIONS: To facilitate learning from serious adverse events in the future, these should also be bundled and carefully assessed without hindsight bias and within the context of "normal" clinical practice.

Ensuring patient safety throughout the care process at out-of-hours services in primary care (OHS-PC) is a challenging task. Telephone triage, the starting point of care at OHS-PC, is considered the most complex and vulnerable part of the out-of-hours care process.[1] During this initial telephone conversation with patients, adequate collection of information is crucial to determine the urgency of the patient’s complaints, which further dictates the next steps in the care process (e.g., diagnosis and treatment).

Similar to other European countries, telephone triage at OHS-PC in the Netherlands is performed by triage nurses, who are supervised by general practitioners (GPs).[2] To support the quality and safety of triage and subsequent decision making, almost all OHS-PC use a decision support tool called the “Netherlands Triage Standard” (NTS).[3]

Inevitably and despite such decision support, however, errors in telephone triage occur, potentially leading to serious adverse events (SAEs). An SAE is defined by the Dutch Healthcare Quality, Complaints and Disputes Act as “an unintended or unexpected event related to the quality of care and resulting in death or a severe harmful event for the patient.”[4] Such SAEs are rare, given the occurrence of just 0.006% of all OHS-PC contacts per year, but have a major impact on both patients and families and on the professionals involved.[5-7] Almost half (46.2%) of the 240 SAEs at Dutch OHS-PC are related to missed acute cardiovascular disease: 30.4% concerned acute myocardial infarction/acute cardiac death; 7.9%, stroke; and 7.9%, ruptured abdominal aortic aneurysm.[5]

To publicly account for and learn from these medical catastrophes, Dutch law requires instant reporting and an elaborate root cause analysis for each SAE. Root cause analysis is aimed at determining how and why the particular SAE could occur based on causal reasoning. However, a known pitfall is hindsight bias because the assessors know the outcome of the case.[8,9] Hindsight bias is the “tendency to exaggerate the extent to which an event can be predicted beforehand if the outcome is known,” which fuels investigators’ expectations that those involved in the SAE could have known what was actually learned by hindsight.[8,10,11] After root cause analysis, recommendations are given to prevent similar future events.[12-14] It is often insufficiently realized that such, mainly defensive measures also may cause harm by burdening the health care system, cause overdiagnosis, and increase the risk of iatrogenic damage.[14,15] Altogether, the effectiveness of this safety management approach based on root cause analysis can be questioned.

For improving the learning from SAEs, it is key to know whether SAEs actually contain clues representing structural flaws in triage or just display regular clinical variance in out-of-hours primary care. For that, SAEs need to be analyzed in comparison to similar cases where no SAE occurred. Therefore, in this study, we compared triage calls of missed acute coronary syndrome (ACS) in patients with chest discomfort to matched controls with chest discomfort but in whom no SAE occurred, with the aim to identify predictors of missed ACS. Furthermore, we hypothesized that experienced GPs might be able to differentiate SAEs from controls based on the safety and quality of the performed triage. Hence, expert GPs were asked to appraise the overall quality and safety of the triage calls.

METHODS

Design and Setting

We conducted a retrospective, matched case-control study among patients who called the OHS-PC with symptoms suggestive of ACS. This study is part of a larger research project on telephone triage at the OHS-PC, of which the design has been published elsewhere.[16] For this case-control study, we used data from a collaboration of 9 OHS-PC locations in the Netherlands providing care to approximately 1.5 million people. These OHS-PC cover both rural and suburban areas and are representative of OHS-PC in the Netherlands. All triage calls with the OHS-PC were routinely recorded and archived for training, quality control, and research purposes.

Patient Selection

Identification of Cases

We retrieved all registered SAEs concerning a missed diagnosis of ACS between 2013 and 2017 from the OHS-PC database. This database includes information on the triage process and urgency allocation. The corresponding triage call recordings were retrieved.

Identification of Controls

We defined controls as triage calls of patients with chest discomfort or other symptoms suggestive of an ACS, and that did not end in SAE. Eight controls per case were matched based on age and sex.[17] The triage recordings of matched controls were collected from an existing database.[16] We retrieved follow-up data on the final diagnosis from the patients’ own GP electronic health records (EHRs). For patients who were referred to the hospital, the final diagnosis of ACS was derived from letters from the emergency or cardiology departments. For more detailed information on the inclusion and exclusion criteria of this larger research project, we refer to our design paper.[16]

Data Collection

Two researchers blinded to the final diagnosis and case-control status relistened to the archived triage calls and retrieved information from the EHR of the OHS-PC using a standardized case record form. Information such as clinical characteristics (i.e., age, sex, symptoms, medical history, and use of medication), call characteristics (i.e., time of calling, call duration, first person calling, and triage nurse’s consultation of a GP), and urgency allocation was extracted. Within the NTS, 6 urgency levels with corresponding response time to medical care can be distinguished (Table 1). The urgency levels U0 to U2 were defined as high because we considered the presence of a GP or ambulance on site within an hour as highly urgent care, and U3 to U5 were defined as low. Items that were not discussed during telephone triage were considered missing.

TABLE 1

NTS Levels of Urgency

NTS Urgency Level	Definition	Response Time	Medical Help
U0: Resuscitation	Loss of vital functions	Immediately	Ambulance
U1: Life-threatening	Unstable vital functions	Within 15 min	Ambulance
U2: Emergent	Vital functions in danger or organ damage	As soon as possible, within 1 h	Home visit by GP or appointment at OHS-PC
U3: Urgent	Possible risk of damage, human reasons	A few hours (<3 h)	Home visit by GP or appointment at OHS-PC
U4: Nonurgent	Marginal risk of damage	24 h	Appointment at OHS-PC or telephone advice
U5: Advice	No risk of damage	Advice, no time related	Telephone advice

Experts’ Assessment of Safety and Quality

Fifteen expert GPs assessed the triage calls. They had at least 5 years of triage consultation experience at the OHS-PC. The experts had at least 5 years of triage consultation experience at the OHS-PC. General practitioner experts were informed that there were cardiovascular SAEs within the sample, but they did not receive further information (e.g., number of SAEs and final diagnoses). Every expert listened individually, and blinded to the case-control status of the patient, to a randomly allocated subset of around 20 triage recordings, and thus, the total set of 135 recordings was relistened to twice. We asked the experts to give an overall appraisal of the triage quality (on a scale of 0–10) and whether the telephone triage process was done safely (yes or no). For quality, we considered a score of 5 or lower as “poor.” A detailed explanation of the expert panel and additional results of their assessments as well as interrater reliability were published elsewhere.[18]

Data Analyses

For the univariable analysis of differences in characteristics between cases and controls, we used conditional logistic regression analysis, which is a standard procedure in matched case-control studies to control for matching factors.[19,20] A P value <0.05 was considered to be statistically significant for all analyses. Statistical analyses were performed using SPSS version 25.0 (SPSS Inc, Chicago, Illinois) and SAS version 9.4 (SAS Institute Inc, Cary, North Carolina).

Ethical Approval

The Medical Ethics Review Committee Utrecht, the Netherlands, approved this study (National Trial Register identification number: NTR7331, reference number WAG/mb/16/003208). In addition, a waiver of informed consent was granted because our study involved a minimal risk to subjects, and this study would not have been practicable without the waiver. All personal and research data were handled and stored according to the European General Data Protection Regulation.

RESULTS

Clinical Characteristics

The flowchart of the study population is displayed in Figure 1. We included 15 missed ACS cases: 5 missed acute myocardial infarctions and 10 acute cardiac deaths. Of the 120 matched controls, 17.5% had an ACS; 21.7%, nonurgent cardiovascular diseases (e.g., stable angina pectoris, stable heart failure, arrhythmias, and hypertension); 19.2%, a musculoskeletal disease; 17.5%, noncardiac chest pain, which was not further specified (cardiac pathology eliminated after diagnostic workup); and 4.2%, gastrointestinal disease; 4.2%, psychiatric disease; 3.3, respiratory disease; and 4.2%, other diagnosis.

FIGURE 1

Flowchart. Flowchart of inclusion of the study population.

Table 2 shows the clinical characteristics of our study population. The median age was 62.0 years (interquartile range, 18.0 years; range, 32–90 years), and 46.7% were men. Cases less often used cardiovascular medication (38.5% versus 64.1%, P = 0.05) and significantly more often reported pain other than retrosternal pain (e.g., epigastric region, upper back, or shoulder pain) compared with controls (63.6% versus 24.7%, P = 0.02).

TABLE 2

Clinical Characteristics of Patients With Symptoms Suggestive of ACS Who Called the OHS-PC, Divided Into Cases and Controls

	Cases (n = 15)	Controls (n = 120)	P
Patient characteristics
Age, median (interquartile range), y	63.0 (18.0)	61.5 (18.0)	0.53
Male sex	7 (46.7)	56 (46.7)	0.99
Cardiovascular history (n = 103)	4 (36.4)	51 (55.4)	0.10
- Diabetes Mellitus (n = 71)	4 (44.4)	8 (12.9)	0.08
- Hypertension (n = 57)	1 (16.7)	22 (43.1)	0.32
- Hypercholesterolemia (n = 73)	2 (25.0)	20 (30.8)	0.68
Cardiovascular medication use (n = 105)	5 (38.5)	59 (64.1)	0.05
Positive family history of cardiovascular disease (n = 25)	2 (40.0)	14 (70.0)	0.44
Caller expresses concern* (n = 117)	9 (90.0)	64 (59.8)	0.11
Symptoms
Shortness of breath (n = 98)	6 (46.2)	57 (67.1)	0.13
Chest discomfort (n = 131)	11 (84.6)	111 (94.1)	0.11
Character: oppressive or heavy feeling (n = 107)	6 (60.0)	68 (70.1)	0.64
Located: retrosternal or left anterior thoracic region (n = 100)	4 (36.4)	67 (75.3)	0.02
Pain onset <12 h (n = 127)	10 (71.4)	96 (85.0)	0.20
Pain duration >15 min (n = 128)	13 (100)	111 (96.5)	0.65
Radiation of pain (n = 115)	9 (90)	67 (63.8)	0.16
Pain scored as severe (VAS > 7) (n = 76)	3 (42.9)	26 (37.7)	0.85
Autonomic nervous system associated symptoms† (n = 133)	11 (78.6)	63 (52.9)	0.09
Symptoms similar to a previous cardiac event (n = 65)	0 (0)	15 (25.0)	0.35
Never experienced similar symptoms before (n = 65)	4 (80.0)	28 (46.7)	0.07

Missing data are presented as (n = x).

*Concerns as expressed verbally by the caller (i.e., the patient, a family member, or the caregiver).

†Occurrence of 1 or more of the following symptoms: nausea, vomiting, sweating, pallor, ashen skin, and (near) fainting.

VAS, visual analog scale.

Call Characteristics

Most calls were in the evening in both groups (46.7% versus 31.7%, P = 0.27), and there was no significant difference in median call duration (Table 3). In SAE cases, consultation of the supervising GP and a GP taking over the triage call were more common than in controls (86.7% versus 49.2% [P = 0.02] and 40.0% versus 10.0% [P = 0.004], respectively). Cases received less often a high urgency (U1 or U2) than did controls (33.3% versus 75.0%, P = 0.003).

TABLE 3

Call Characteristics and Expert Assessments of Patients With Symptoms Suggestive of ACS Who Called the OHS-PC, Divided Into Cases and Controls

	Cases (n = 15)	Controls (n = 120)	P
General call characteristics
Time of calling
- Morning (0600–1200 h)	2 (13.3)	29 (24.2)	0.36
- Afternoon (1200–1800 h)	2 (13.3)	27 (22.5)	0.42
- Evening (1800–0000 h)	7 (46.7)	38 (31.7)	0.27
- Night (0000–0600 h)	4 (26.7)	26 (21.7)	0.65
Call duration, median (interquartile range), min:s	07:28 (03:02)	06:23 (03:32)	0.39
Initial call by someone else than the patient (n = 135)	10 (66.7)	56 (46.7)	0.14
Consultation of the supervising GP by the triage nurse	13 (86.7)	59 (49.2)	0.02
- Supervising GP takes over the call	6 (40.0)	12 (10.0)	0.004
Urgency
High urgency allocation (U1 or U2)	5 (33.3)	90 (75.0)	0.003

*Experts assessed the triage quality on a scale from 1 (worst quality possible) to 10 (excellent); scores of 5 or lower were considered poor triage quality.

All 135 calls were relistened to twice by a panel of 15 GP experts between August and October 2018, with a result of 270 assessments (2 × 15 cases and 2 × 120 controls). The mean (SD) triage quality was 5.8 (2.01) for cases compared with 7.2 (1.5) for controls (P < 0.001). Experts considered the triage quality more often “poor” (score ≤5; 33.3% versus 10.9%, P = 0.001) and “unsafe” (73.3% versus 22.5%, P < 0.001) in cases than in controls. The reasons most often cited by the experts for considering triage quality poor were as follows: too slow pace of triage and insufficient additional questioning when symptoms are still unclear. For unsafe triage, the reasons most often mentioned were insufficient safety check of “ABCD” (i.e., airway, breathing, circulation, disability) and lack of a “safety net” instruction (e.g., “call back if…”).

DISCUSSION

Summary

In this study, we compared recorded triage calls of 15 patients with chest discomfort contacting the OHS-PC in whom an ACS was missed, with 120 triage calls involving matched controls with chest discomfort but without a missed ACS. Cases used less cardiovascular medication (38.5% versus 64.1%) and more often experienced nonretrosternal chest pain (63.3% versus 24.7%) than did controls. Consultation of the supervising GP (86.7% versus 49.2%) occurred more often in cases than in controls. Experts blinded to the case-control status rated the triage of SAE cases more often as unsafe (73.3% versus 22.5%) and of poor quality (33.3% versus 10.9%) than the calls of the controls.

Comparison With Existing Literature

This is the first study that assessed cardiac SAEs at the OHS-PC with matched controls in detail. One previous study analyzed all SAE reports in Dutch OHS-PC settings considering all domains including missed acute cardiovascular diseases.[5] They found that being male, ages of 45 to 74 years, recent previous contact with their own GP for the same symptom(s), multiple contacts with the OHS-PC in a short time frame, and contacts during the night were univariably related to SAEs. However, this study was hampered by a lack of controls from the same domain. Another study compared the malpractice claims (in which an SAE occurred) of a national telephone triage system in Sweden with those of matched controls. However, this study focused specifically on communication patterns and the assessors knew the case-control status of the patients, of which the latter may have biased the results. The researchers reported that triage nurses more often used closed-ended questions in malpractice claimed calls, which resulted in less information on the caller’s symptom presentation and may be considered as a surrogate of poor triage quality.[21]

Our study highlighted a few differences between SAE cases and controls; however, these differences seem to be of limited relevance for daily practice. Compared with controls, the use of cardiovascular medication was reported less in triage calls of cases. With prescription rates of Dutch GPs varying between 12.0% and 27.0%,[22] the remaining majority of the population does not use cardiovascular medication. Thus, considering all patients presenting with chest discomfort and without cardiovascular medication as a possible SAE is rather inefficient. Furthermore, nonretrosternal chest pain was more often reported in cases. It is already well known that among patients with nonretrosternal chest pain a noncardiac cause is more likely,[23,24] and as a result, the diagnosis of ACS may be missed more often. Also, in SAE cases, the GP was involved more often, which mainly suggests that these calls were more troublesome for triage nurses.

Although we did not find differences in characteristics between cases and controls helpful to prevent future adverse outcomes within the domain of patients with chest discomfort, the experts have picked up aspects in the triage calls of cases that led them to score lower on quality and safety. This could be attributable to disturbing factors in the conversation, which we did not score in our study, such as the previously mentioned communication patterns (e.g., open- or closed-ended questions and language barrier) or emotional expressions (e.g., audible anxiety of irritation).[25,26]

Furthermore, the difference in setting between an expert relistening and a triage nurse actually performing the triage call possibly has played a role in our findings concerning the quality and safety of the calls (Supplemental Digital Content Table 1, http://links.lww.com/JPS/A367). It is conceivable that a triage nurse registers information (“in the heat of the moment”) in a different way than an expert who does not perform the triage and only relistens it without direct responsibility. Cognitive overload of triage nurses may easily occur in real practice because they need to multitask under time pressure.[27,28]

Also, disturbing factors such as a hectic work environment, shiftwork, fatigue, stress, or understaffing might contribute to a difference in information registration.[25,26,29,30]

Still, apart from these considerations, it also remains unclear whether the poorer quality and safety in triage calls of cases is either a cause or a consequence in the chain of events up to the SAE.

Implications for Research and/or Practice

Our findings challenge the assumption that SAEs in the domain of cardiovascular disease in OHS-PC are preventable. However, it remains unclear whether structural faults in the triage of this domain exist, which were not included in our study. Searching for true systemic rather than incidental discrepancies between “work processes as imagined” and “work processes as done” (i.e., actual daily practice) more likely will contribute to improving current practice.[31,32] Therefore, to facilitate learning from SAEs in the future, these should be bundled and carefully assessed without hindsight bias within the context of “normal” clinical practice. In addition, further research into disturbing factors and the potential consequences during telephone triage is needed.

Strengths and Limitations

An important strength of our study is the case-control design, which is well suited if outcomes like SAEs are rare.[20] Another strength is the blinding of both the researchers and the experts to the final diagnosis and the case-control status when assessing the calls, which minimized the risk of hindsight bias. A third strength is that relistening to triage recordings enabled us to gather more detailed information on the clinical and call characteristics than would be available, with only considering the information on a patient notified in the EHR of the OHS-PC. An important limitation is the relatively small number of SAEs. Despite using all available data of one of the largest OHS-PC collaborations in the Netherlands and including a study period of 5 years (around 50,000 contacts for chest discomfort), we could only include 15 cardiac SAEs (0.03%).[33] Still, the limitation of our sample size should also be seen in the context of SAEs in OHS-PC settings; with an incidence of 0.006%, it is impossible to conduct a study on SAEs with cohort-like sample sizes. Another limitation is missing data on some determinants, a common problem in research with routine care data. Lastly, we focused on telephone triage, but SAEs may also occur in later steps of the out-of-hours care process. Unfortunately, we had no information on these later steps.

CONCLUSIONS

Differentiating—in a case-control manner and blinded to the outcome—missed ACS cases from others with chest discomfort but without SAEs is difficult with telephone triage at the OHS-PC. The use of cardiovascular medication, pain other than retrosternal, and consultation of the supervising GP occurred more often in missed ACS cases. Nevertheless, these determinants are not helpful to prevent future adverse outcomes because they are very common and not specific enough. Improvement lessons from SAEs should not be drawn without lessons from analysis within the context of normal practice.