Triage tests for identifying atrial fibrillation in primary care: a diagnostic accuracy study comparing single-lead ECG and modified BP monitors.

OBJECTIVE: New electronic devices offer an opportunity within routine primary care settings for improving the detection of atrial fibrillation (AF), which is a common cardiac arrhythmia and a modifiable risk factor for stroke. We aimed to assess the performance of a modified blood pressure (BP) monitor and two single-lead ECG devices, as diagnostic triage tests for the detection of AF.
SETTING: 6 General Practices in the UK. PARTICIPANTS: 1000 ambulatory patients aged 75 years and over. PRIMARY AND SECONDARY OUTCOME MEASURES: Comparative diagnostic accuracy of modified BP monitor and single-lead ECG devices, compared to reference standard of 12-lead ECG, independently interpreted by cardiologists.
RESULTS: A total of 79 participants (7.9%) had AF diagnosed by 12-lead ECG. All three devices had a high sensitivity (93.9-98.7%) and are useful for ruling out AF. WatchBP is a better triage test than Omron autoanalysis because it is more specific-89.7% (95% CI 87.5% to 91.6%) compared to 78.3% (95% CI 73.0% to 82.9%), respectively. This would translate into a lower follow-on ECG rate of 17% to rule in/rule out AF compared to 29.7% with the Omron text message in the study population. The overall specificity of single-lead ECGs analysed by a cardiologist was 94.6% for Omron and 90.1% for Merlin.
CONCLUSIONS: WatchBP performs better as a triage test for identifying AF in primary care than the single-lead ECG monitors as it does not require expertise for interpretation and its diagnostic performance is comparable to single-lead ECG analysis by cardiologists. It could be used opportunistically to screen elderly patients for undiagnosed AF at regular intervals and/or during BP measurement.

Three devices for detecting atrial fibrillation (AF) were tested on an unselected elderly primary care population of 1000 individuals.

The prevalence of AF expected for this setting, which allowed us to determine the operating characteristics of each monitor with precision.

Our population is generalisable to similar primary care settings worldwide.

A reference standard (12-lead ECG) was performed on all patients, and interpreted blindly by cardiologists.

The specificity of one cardiologist was substantially lower than the other three.

Only 12 new cases of AF were detected in this setting, which is as expected for a primary care population such as this.

Introduction

Atrial fibrillation (AF) is the most common cardiac arrhythmia; present in more than 10% of patients aged 75 years or over,1 2 and it significantly increases morbidity and mortality.3 The main significance of AF is as a major independent risk factor for stroke and thromboembolism,4 particularly in older patients. A large evidence base supports the efficacy of oral anticoagulation in reducing AF stroke risk by two-thirds.5 AF may be asymptomatic but can be identified by detecting a characteristic irregularity in pulse rhythm. Despite this, it was found to be undiagnosed in 3.8% of patients aged over 75 years in a large UK screening study.6 Indeed, undiagnosed AF is associated with 3.8–6.1% of all strokes.7 8 Given the importance of identifying patients with risk of stroke, the high prevalence of AF and the effective prevention strategies, there is a strong case to consider screening for AF.

European guidelines for the management of AF recommended opportunistic pulse assessment by a primary care practitioner with a follow-up ECG for an irregular pulse, an approach which is more cost-effective than systematic screening with a 12-lead ECG.6 9 In primary care research studies the sensitivity of nurse pulse assessment is high, varying between studies from 87% to 97% although it has a lower specificity (70–81%) requiring one out of every three or four screened patients to have a follow-up ECG.6 10 11 This is a significant additional workload and could be a barrier to widespread implementation of opportunistic screening. Although pulse examination is a simple screening technique for AF, its detection relies on a subjective assessment in a busy routine care setting, which may partly explain why AF remains undetected in many patients. Indeed the American Heart Association suggests there is a need to develop strategies to detect AF more effectively in individuals and populations.12

Several new electronic devices have the potential to be useful triage tests for AF and initial studies suggest these have higher specificity than pulse palpation, so could significantly reduce the need for confirmatory 12-lead ECGs—but none have been evaluated in a primary care setting.13–15 The National Institute for Health and Care Excellence (NICE) has recently suggested that using a blood pressure (BP) monitor, WatchBP, which is modified to detect irregularity of pulse during BP measurement in the elderly, together with appropriate anticoagulation of those subsequently diagnosed with AF, has the potential to reduce the incidence of stroke.16 As stroke is the third leading cause of global disease burden, this would have substantial economic and patient benefits worldwide.16–18

Methods

Study design

An observational study comparing the accuracy of a modified BP monitor (WatchBP), and two single-lead ECG devices, one with an autoanalysis function (Omron) and one without (Merlin), with the reference standard of blinded cardiologist diagnosis of AF from a 12-lead ECG.

Participants

We recruited participants aged 75 years or over, living at home, from six general practices in the UK between May 2011 and October 2012. Having excluded patients with implanted pacemakers or defibrillators, those unable to give informed consent, or patients in whom the general practitioner (GP) considered participation was inappropriate (eg, terminal illness), all other patients over 75 were invited to take part until the sample size had been achieved (ie, 1000 participants had been recruited). A diagnosis of AF recorded in the patient GP record was not used as part of the selection criteria. Consenting study participants identified from GP records and meeting the eligibility criteria attended a 30 min appointment with a practice nurse for the testing procedure.

Test procedure

All participants received the tests in the same order, which were delivered by nine registered nurses working within the practices.

First, the nurse used WatchBP (Microlife, Switzerland) a modified oscillometric BP monitor which flashes when it detects an irregular pulse during automatic BP measurement.

Second, the nurse then applied an Omron monitor (model HCG-801, Omron Healthcare Europe, the Netherlands) which involved placing one electrode on the bare chest wall 5 cm below the nipple, while the patient held the other electrode with the right index finger. The monitor records a single-lead ECG tracing, and displays a message indicating the presence of possible AF. The device's analysis algorithm includes several cardiac rhythms which could potentially be AF, including fast and irregular, slow and irregular, irregular and those where analysis is impossible. The single-lead recording and text message were recorded and saved for later downloading and analysis.

Third, the nurse applied a Merlin ECG event recorder (Meditech Ltd, Hungary) which resembles a watch, on participants’ left wrist. The participant covered the electrode on the face of the device with the palm of their right hand for 30 s. The recording, with no automated analysis, was saved to a computer for later downloading and analysis. Unlike the Omron, the Merlin monitor does not require removal of any clothing, making it possible for use in public settings, an advantage for participants experiencing an intermittent arrhythmia.

The nurse recorded the results of the WatchBP monitor and the Omron automated text message during the initial examination. Each single-lead ECG trace was sent for interpretation to two independent cardiologists after removing all clinical information and patient identification except for date of birth and the text message (Omron only).

Participants were asked if they had experienced palpitations, racing, pounding, fluttering or irregular heartbeat in the previous 4 weeks.

The reference standard

A 12-lead ECG was performed on all participants at the end of the visit, using standard procedures. ECGs were independently interpreted by a panel of two cardiologists, blind to all patient identifiers, clinical details, results of triage tests, but not blind to study objectives. They were asked to classify the ECG as AF or atrial flutter (yes/no/do not know). A third cardiologist, also blind to any of the other study results, interpreted the ECG readings for which there was no consensus or if both cardiologists were uncertain about the presence of AF. AF is defined as the absence of distinct ‘p’ waves, an absolutely irregular RR interval and an atrial cycle length <200 ms (300 bpm) on a 12-lead ECG.17

To cope with the workload, the task was divided between two separate pairs of cardiologists. All had completed cardiology specialist training of between 5 and 6 years. The third cardiologist, whose role was to resolve uncertainty and moderate disagreement, was a specialist electrophysiologist.

Study power and analysis

We anticipated that the overall prevalence of AF, including both new and prior diagnoses, would be 10%. We therefore estimated that 1000 participants would be needed to provide sufficient precision to estimate the sensitivity of each device with a maximum 95% CI of ±10%.

We compared the accuracy of the WatchBP, Omron (text and tracing) and Merlin monitors, with a 12-lead ECG for detection of AF. All inconclusive results were treated as positive in our analysis (because in the clinical situation these participants could not have AF ruled out and would need to have a 12-lead ECG). We calculated sensitivity, specificity, likelihood ratios and 95% CIs for each test. Analyses for both single-lead ECGs were carried out per cardiologist and the results were subsequently combined using a random effects bivariate model calculating overall sensitivity and specificity.19 We analysed the interobserver agreement of whether the single-lead ECG trace showed AF between the cardiologists within each pair together with the κ value.20

We performed subset analyses of the comparative accuracy in patients with and without existing diagnoses of AF (whether or not treated) recorded in GP records. In addition, we modelled a two-stage screening process for AF by examining patients who were WatchBP flash positive and analysing the sensitivity and specificity of the Omron autoanalysis in this subgroup.

Data analysis was performed using Stata V.11S.

Results

Of 6529 potentially eligible patients, we invited 2673 for a screening visit; no further invitations were sent after the required 1000 sample had been achieved (figure 1). Participants had an average age of 79.7 years (range 75.1–99.8 years), 49.3% were men (table 1). One hundred and ten had AF recorded in the medical summary (11%), of whom 67 (67/110, 61%) had evidence of AF on their 12-lead ECG at the study visit. AF was noted in a further 12 participants based on the 12-lead ECG (1.2%). The three cardiologists could not reach a consensus about whether AF was present in the 12-lead ECG in one participant, who was therefore excluded from further analyses which are therefore based on 999 participants.

Figure 1

Patient recruitment flowchart.

Table 1

Characteristics of participants included in diagnostic accuracy analyses

Total number of participants	999
Age (mean)	79.7 years (range 75.1–99.8)
	(95% CI 79.4 to 79.9)
Gender (%, male)	49.3 (95% CI 46.2 to 52.5)
Comorbidities (from medical summary)
AF	110 (11%)
Heart failure	31 (3.1%)
Hypertension	533 (53.3%)
Diabetes	122 (12.2%)
Stroke	31 (3.1%)
Transient ischaemic attack	65 (6.5%)
Participants with AF on antiarrhythmic medication	87 (8.7%)

AF, atrial fibrillation.

Overall, 50 (5%) participants reported having experienced palpitations, racing, pounding, fluttering or irregular heartbeat during the previous 4 weeks. Of these, 16 were already known, and three were new cases of AF. The interobserver agreement for the 12-lead ECGs was good for both pairs of cardiologists; the κ was 0.69 and 0.96, respectively for pairs 1 and 2.

Accuracy of WatchBP and Omron autoanalysis

Both WatchBP and Omron autoanalysis were highly sensitive (94.9% and 98.7%, respectively; table 2). The specificity of WatchBP was 89.7% (95% CI 87.5% to 91.6%) whereas the Omron text message was lower at 76.2% (95% CI 73.3% to 78.9%; figure 2).

Table 2

Diagnostic performance of WatchBP and Omron autoanalysis for detecting atrial fibrillation

n=999* Prevalence 7.9%	WatchBP with AF indicator flash (95% CI)	Omron autoanalysis* (95% CI)
Sensitivity (%)	94.9 (87.5 to 98.6)	98.7 (93.2 to100)
Specificity (%)	89.7 (87.5 to 91.6)	76.2 (73.3 to 78.9)
Positive likelihood ratio	9.2 (7.6 to 11.2)	4.15 (3.69 to 4.67)
Negative likelihood ratio	0.057 (0.022 to 0.15)	0.017 (0.0024 to 0.12)
Positive predictive value	44.1 (36.5 to 51.9)	26.3 (21.3 to 31.7)
Negative predictive value	99.5 (98.8 to 99.9)	99.9 (99.2 to 100)

*Text message ‘irregular’ or ‘analysis impossible’ counted as positive test.

AF, atrial fibrillation.

Figure 2

Predictive value of WatchBP and Omron autoanalysis for detecting atrial fibrillation (AF).

The higher specificity of WatchBP would translate into a markedly lower follow-on ECG rate of 17% to rule in/rule out AF compared to 29.7% with the Omron text message. The superior sensitivity of the Omron text message means the false negative rate is lower than with Watch BP (0.1% vs 0.4%).

The prevalence of AF was 1.4% (11) in the subset of 889 participants who did not have AF recorded on their medical summary. WatchBP and the Omron autoanalysis provided very similar sensitivity and specificity to the total study population and this translated to a follow-on ECG rate of 11% for WatchBP and 24% for the Omron text message. Of these follow-on ECGs, 12% would be true positives for the WatchBP flash compared to 5% for the Omron text message.

We modelled a two-stage screening process within our total sample, where patients who had a positive result with WatchBP went on to have the Omron autoanalysis. In participants who screened positive using the WatchBP, the sensitivity and specificity of the Omron autoanalysis was 100% (95% CI 95.2% to 100%) and 35.8% (95% CI 26.2% to 46.3%), respectively. This would slightly reduce the follow-on 12-lead ECG requirement from 17% to 13.6%.

Accuracy of Omron and Merlin single-lead ECGs interpreted by a cardiologist

The diagnostic accuracy of the Omron and Merlin single-lead ECGs is shown in figure 3; meta-analysis of the four cardiologist's interpretations gave Omron a summary sensitivity of 94.4% with a summary specificity of 94.6%; Merlin had a summary sensitivity of 93.9% with a summary specificity of 90.1%. Cardiologist variability in single-lead ECG analysis is shown in table 3. There was fair interobserver agreement (κ 0.36 and 0.28 for Omron and Merlin, respectively) between cardiologist pair 1, and good interobserver agreement (κ 0.76 for both devices) between the second pair. The cardiologists were unable to interpret whether the trace showed AF on 0–1.2% of patients using Omron and 0–7.2% of occasions using Merlin.

Figure 3

Forest plot of diagnostic accuracy per cardiologist of Omron and Merlin single-lead ECGs.

Table 3

Diagnostic performance of Omron and Merlin single-lead ECGs for detecting atrial fibrillation

n=999 Prevalence 7.9%	Interpretation	Omron single-lead ECG (95% CI)	Merlin single-lead ECG (95% CI)
Sensitivity (%)	Cardiologist: 1	95 (82 to 99)	97 (85 to 100)
	2	95 (82 to 99)	91 (77 to 98)
	3	90 (77 to 97)	98 (87 to 100)
	4	98 (87 to 100)	88 (74 to 96)
Specificity (%)	Cardiologist: 1	75 (71 to 79)	76 (71 to 80)
	2	98 (96 to 99)	95 (93 to 97)
	3	97 (96 to 99)	92 (89 to 94)
	4	95 (92 to 97)	91 (88 to 94)
Positive likelihood ratio	Cardiologist: 1	3.8 (3.2 to 4.6)	3.99 (3.3 to 4.8)
	2	41.8 (22.5 to 77.5)	18.8 (12.2 to 28.8)
	3	35.8 (20.3 to 63.1)	11.5 (8.5 to 15.6)
	4	18.5 (12.6 to 27.2)	10.1 (7.4 to 13.9)
Negative likelihood ratio	Cardiologist: 1	0.07 (0.02 to 0.28)	0.04 (0.01 to 0.26)
	2	0.05 (0.01 to 0.21)	0.09 (0.03 to 0.27)
	3	0.098 (0.04 to 0.25)	0.03 (0.004 to 0.18)
	4	0.02 (0.004 to 0.17)	0.13 (0.06 to 0.3)
Positive predictive value	Cardiologist: 1	24.1 (17.4 to 31.9)	24.5 (17.6 to 32.5)
	2	77.8 (62.9 to 88.8)	60.4 (46.0 to 73.5)
	3	76.0 (61.8 to 86.9)	50.6 (39.3 to 61.9)
	4	62.1 (49.3 to 73.8)	47.4 (36.0 to 59.1)
Negative predictive value	Cardiologist: 1	99.4 (97.9 to 99.9)	99.7 (98.3 to 100)
	2	99.5 (98.3 to 99.9)	99.3 (97.9 to 99.8)
	3	99.1 (97.8 to 99.8)	99.8 (98.7 to 100)
	4	99.8 (98.8 to 100)	98.9 (97.3 to 99.6)

Discussion

Summary of main results

Our results show that all the methods that we tested have a high sensitivity for detecting AF in an elderly primary care population and thus are useful for ruling out AF. During the study we detected 12 new cases of AF and 7.9% of the study population overall were experiencing AF at the time of the study consultation. The total prevalence in the study sample was 11%, as some patients have paroxysmal AF and were not experiencing arrhythmias at the time of the study.

WatchBP outperformed Omron autoanalysis because it is more specific—89.7% compared to 76.2% and 78.3%, respectively. The specificity of the single-lead ECGs when interpreted by a cardiologist was comparable with WatchBP. Overall Omron produced fewer inconclusive results and achieved slightly better specificity than Merlin. However cardiologist interpretation of the ECG trace was a more powerful factor in determining specificity than the type of monitor used. In this study, operator variability in ECG analysis was partly due to differences in expertise, and partly due to trade-offs made by different cardiologists between sensitivity and specificity.

In our elderly population, only 5% of patients reported experiencing palpitations. Among patients who did report palpitations, prevalence of AF was 36%. Most of these patients already had a diagnosis of AF.

Strengths and weakness

The devices were tested on an unselected elderly primary care population. Although our sample did not include the housebound or patients with dementia, we recruited a large population with a prevalence of AF expected for this setting, which allowed us to determine the operating characteristics of each monitor with precision. We believe our population is generalisable to similar primary care settings worldwide.

The cardiologists analysed ECG recordings in line with recommended practice and all had a minimum of 5 years cardiology specialist training. The specificity of one cardiologist was substantially lower than the other three. Further research with a larger sample of cardiologists would be required to more fully interpret this finding. However, we do not believe this diminishes the reliability of our reference standard since the second pair of blinded cardiologists achieved extremely high concordance, and the moderating consultant cardiologist was a specialist electrophysiologist. Having cardiologists rather than GPs ascertain the presence or absence of AF on the Omron or Merlin single-lead ECGs may have risked over estimating accuracy given the likelihood that cardiologists would have greater skill in ECG interpretation. Finally, given that the devices were used sequentially followed by the 12-lead ECG it is theoretically possible that AF may not have been consistently present or absent during the entire procedure, leading to an increase of false positives and false negatives, however we believe this is a small risk given the extremely short-time frame of approximately 10 min.

Comparison with existing literature

The operating characteristics of WatchBP in this elderly primary care population are comparable to previous studies undertaken in different populations. In a US study of 405 patients attending a cardiology outpatients where 39% of the sample had an abnormal ECG (AF prevalence 23%) it had a sensitivity of 95% (95% CI 93% to 98%) and specificity of 86% (95% CI 84% to 89%) while in a Greek study of a diverse group of 73 individuals (healthy volunteers, outpatients and inpatients) it had similar sensitivity of 93% (95% CI 74% to 99%) and specificity of 89% (95% CI 79% to 96%).

Omron has been evaluated in a German study of 505 patients in a hospital setting where 66% of the sample had an abnormal ECG (mean age 61 years, 66% men, AF prevalence 28%) with all analyses (including the reference standard) being undertaken by a single cardiologist. This study reported higher accuracy than we demonstrated, with a sensitivity of 99% (95% CI 96% to 100%) and specificity of 96% (95% CI 94% to 98%) for AF. We are unaware of any AF validation studies for Merlin.

Clinical implications

Our study supports the recent NICE recommendation that GPs should consider using WatchBP for elderly patients who have a BP measurement in order to opportunistically identify undiagnosed AF. Specificity is comparable with a single-lead ECG analysed by a cardiologist and the result does not require specialist interpretation, which is an advantage over the single-lead ECG monitors and keeps screening straightforward and costs less. It is easily portable for use in housebound patients.

Prior to wider implementation, clinicians need to consider the optimal screening frequency. Assuming a primary care population in which the majority of AF cases had already been identified, the WatchBP would indicate a ‘positive flash’ for 1 in 10 patients who would thus require a follow-up 12-lead ECG. As some patients have their BP measured frequently, there is potential for recurrent false positive results and a high demand for 12-lead ECGs. These issues should be addressed to ensure sustainable implementation in primary care.

Paroxysmal AF presents a particular diagnostic challenge because patients may not be experiencing an arrhythmia when they are being screened with one-off measures. The potential for patients to undertake self-monitoring and to capture an ECG tracing when an arrhythmia is occurring are desirable features of an AF screening method. In our study, only 5% of patients reported experiencing palpitations, which would limit the usefulness of asking patients to record a single-lead ECG at the time of symptoms. Among the patients who did report palpitations, the prevalence of AF was 36%, but most of these patients already had a diagnosis of AF which might have improved their awareness of an arrhythmia.

WatchBP will not identify atrial flutter, which is also a risk factor for stroke. The prevalence of this is much lower than AF; in our study it was 0.4%.

Future research

Screening for AF is not yet routine practice in primary care, therefore determining not only the optimal device, but also the most appropriate and cost-effective process is a priority. This could include an impact analysis using WatchBP as a triage test to evaluate its effectiveness (and cost-effectiveness) for improving the detection of AF and prevention of stroke. Determining the cost-effectiveness of these devices would also have to include doctor time for interpretation of ECGs.

Conclusion

WatchBP performs better than the single-lead ECG monitors as a triage test for identifying AF in primary care as it does not require any expertise for interpretation and their diagnostic performance is comparable. It could be used to opportunistically screen elderly patients for undiagnosed AF at regular intervals and/or during BP measurement. It has advantages over pulse assessment because the result is objective and it also has a higher specificity, keeping follow-up 12-lead ECGs to a minimum.