Assessing Accuracy of Wrist-Worn Wearable Devices in Measurement of Paroxysmal Supraventricular Tachycardia Heart Rate.

BACKGROUND AND OBJECTIVES: Wrist-worn wearable devices provide heart rate (HR) monitoring function via photoplethysmography technology. Recently, these devices have been used by patients to measure the HR when palpitation occurs, but few validation studies of these instruments have been conducted. We assessed the accuracy of these devices for measuring a HR.
METHODS: This study enrolled 51 consecutive patients with a history of paroxysmal supraventricular tachyarrhythmia (SVT) or paroxysmal palpitations who were scheduled to undergo an electrophysiological study (EPS). Three devices were assessed: Apple Watch Series 2 (Apple), Samsung Galaxy Gear S3 (Galaxy), and Fitbit Charge 2 (Fitbit). Patients were randomly assigned to wear 2 different devices. The HR at baseline and induced SVT were measured during the EPS. After successful ablation of SVT, HR measurements was also done during atrial and ventricular pacing study.
RESULTS: The mean patient age was 44.4±16.6 years and 27 patients were male (53%). The accuracy (within ±5 beats per minute [bpm] of an electrocardiogram [ECG] measurement) of the baseline HR measurements was 100%, 100%, and 94%, for Apple, Galaxy, and Fitbit, respectively. The HR during induced SVT ranged from 108 bpm to 228 bpm and the accuracy (within ±10 bpm of an ECG) was 100%, 90%, and 87% for the Apple, Galaxy, and Fitbit, respectively. During pacing study, accuracy of these devices was also acceptable but tended to decrease as the HR increased, and showed differences between the devices.
CONCLUSIONS: Wrist-worn wearable devices accurately measure baseline and induced SVT HR. TRIAL REGISTRATION: Clinical Research Information Service Identifier: KCT0002282.

RCT Entities:   Population Interventions Outcomes

INTRODUCTION

The wrist-worn wearable devices including so-called smart-watches and activity trackers are now very popular and offer heart rate (HR) monitoring technology via photoplethysmography (PPG) technique.1) By measuring the pulsatile signal, each pulse ‘peak’ can be interpreted as an R wave, i.e. one heartbeat, and then translated into the HR.2) Previous studies have shown that the PPG technique using a smartphone camera with specialized smartphone applications allows accurate HR measurements and even can be used to diagnose atrial fibrillation (AF).2)3)4)5) With the emergence of the concept of ‘wearable technology,’ wrist-worn wearable devices are now widely used for HR measurements instead of smartphone applications. However, there are still few published studies evaluating the accuracy of these devices, especially in an arrhythmic situation.6) The objective of our study was to assess the accuracy of wrist-worn wearable devices in the measurement of HR, particularly during tachycardia.

METHODS

Study population

Patients aged from 18 to 80 years with a history of paroxysmal palpitations or documented paroxysmal supraventricular tachyarrhythmia (SVT) and who were scheduled to undergo an electrophysiological study (EPS) were eligible. The detailed exclusion criteria were as follows: 1) a history or suspicion of arterial disease in the upper extremities; 2) a prior coronary artery bypass using a radial artery; 3) receiving dialysis through the arteriovenous fistula of the upper extremity; 4) previous upper limb amputation; 5) skin disorders or ample hair around the wrist; 6) pregnancy; and 7) refusal to participate. The study protocol complied with the Declaration of Helsinki and was approved by the Institutional Review Board of Asan Medical Center. This study was registered with International Clinical Trials Registry Platform (Unique identifier: KCT0002282, registration date: 2017.3.28). All patients gave written informed consent.

Study design

All procedures were done under conditions of a fasting state with light sedation after withdrawal of all antiarrhythmic drugs for at least a period of 5 half-lives. The EPS was performed in accordance with our institutional clinical practice. A 12-lead electrocardiogram (ECG) recording by Prucka Electrophysiology lab (Prucka, GE Medical Systems Information Technology, Milwaukee, WI, USA) was used to provide criterion measures of HR. Three of the most widely used wrist-worn wearable devices were assessed: Apple Watch Series 2 (Apple; Apple Inc, Cupertino, CA, USA), Samsung Galaxy Gear S3 (Galaxy; Samsung Electronics Co., Suwon, Korea), and Fitbit Charge 2 (Fitbit; Fitbit Inc., San Francisco, CA, USA). The patients were randomly assigned to wear two different devices, one on each wrist. The devices were tightly placed above their ulnar styloid process during the study. The HR was measured in the following situations:

1) Baseline HR prior to the EPS

2) Induced SVT HR

3) Atrial pacing study after successful ablation:

A. pacing cycle lengths (PCLs): 600 ms, 500 ms, 430 ms, 380 ms, 330 ms, 300 ms

B. performed only 1: 1 atrioventricular (AV) conduction was maintained

C. at least a 30-second resting period between pacing studies

4) Ventricular pacing study after successful ablation:

A. PCLs: 600 ms, 500 ms, 430 ms, 380 ms, 330 ms, 300 ms

B. at least a 30-second resting period between pacing studies

In each situation, the HR measured by each device was compared with a simultaneous ECG HR. If the devices failed to provide a measurement in initial 30 seconds, a second and third attempt to obtain HR was made. If there was still no result after third attempts or within 3 minutes, then the event was recorded as a failure of the devices.

Definition of accuracy

An accurate measurement was defined when a difference between the device-measured HR and an ECG-recorded HR is 5 beats per minute (bpm) or less. The proportion of measurements which showed a difference of HR within 10 bpm was also evaluated because we thought it was clinically acceptable range. Blood pressures were measured during baseline and induced SVT.

Statistical analysis

Continuous variables were described as a mean±standard deviation and categorical variables were described as number (percentage). The degrees of agreement between HR measurements were assessed by the intraclass correlation coefficient with 95% confidence intervals and the Bland-Altman plot. The comparison between the three devices was tested using the Fisher's exact test. All statistical analyses were performed using the MedCalc software package, version 18.2.1 (MedCalc Software, Mariakerke, Belgium). A p value <0.05 was considered statistically significant.

RESULTS

Patient characteristics

A total of 51 patients were enrolled and 27 patients (53%) were male. The mean age was 44.4±16.6 years. The patient characteristics are summarized in Table 1. All patients had structurally normal hearts. Patients were randomly assigned to wear 2 different devices, one on each wrist. Hence, our study was designed to take 34 measurements per device.

Table 1

Baseline characteristics

Characteristics	Values
Male	27 (53)
Age (years)	44.4 ± 16.6
Height (cm)	163.1 ± 16.4
Weight (kg)	64.5 ± 9.1
Body mass index (kg/m2)	23.7 ± 3.0
Baseline SBP (mmHg)	118.4 ± 19.3
Baseline DBP (mmHg)	73.1 ± 11.7
Baseline HR (bpm)	71.6 ± 13.5
SVT SBP (mmHg)	103.3 ± 21.0
SVT DBP (mmHg)	70.6 ± 18.5
SVT HR (bpm)	169.3 ± 34.4

Values are presented as number (%) or mean±standard deviation.

DBP = diastolic blood pressure; bpm = beats per minute; SBP = systolic blood pressure; SVT = supraventricular tachycardia.

The induced SVTs were as follows: atrioventricular nodal reentrant tachycardia (AVNRT) in 23 patients (45%), atrioventricular reentrant tachycardia (AVRT) in 23 patients (45%), atrial tachycardia (AT) in 3 cases (6%), and typical atrial flutter (AFL) in 1 case (2%). Tachycardia was not induced in one patient. All induced SVTs were successfully ablated without complication.

Measurement of heart rate

The time required to measure the HR was about 20 seconds for the Apple and Galaxy, and 30 to 40 seconds for the Fitbit. In the case of Apple and Fitbit, HR was continuously measured and displayed. With Galaxy, fixed measured HR was displayed only once after the measurement period. A total of 1,063 HR measurements were attempted across all the three devices. Among these measurements, 876 of the HR measurements were within ±5 bpm difference and 902 were within ±10 bpm difference as compared to the ECG measurements. Therefore, the overall accuracy of the wrist-worn wearable devices for the measurement of HR was 82.4% for ±5 bpm difference and 84.9% for ±10 bpm difference.

Measurement of baseline heart rate

The baseline HR in our study patient ranged from 50 to 101 bpm. In one patient, Fitbit showed an error from the beginning. The baseline HR was therefore measured in 34 patients with the Apple and Galaxy, and in 33 patients with Fitbit.

The accuracy of the baseline HR measurement, i.e. within ±5 bpm of the ECG value, was 100%, 100%, and 94% for the Apple, Galaxy, and Fitbit, respectively. There was no statistically significant difference between these devices in terms of accuracy.

Measurement of supraventricular tachyarrhythmia heart rate

Tachycardia was not induced in 1 patient and was not sustained in 4 patients (2 AT, 1 typical AFL, 1 AVRT). Hence, we could not measure the induced SVT HR in these patients. In three patients (2 AVNRT, 1 AVRT), we also could not measure the HR because these patients complained of severe symptoms. As described above, Fitbit showed an error from the beginning in one patient. In addition, one Galaxy measurement and two Fitbit measurements failed. Therefore, an induced SVT HR was measured in 28 patients with Apple, 29 patients with Galaxy, and 30 patients with Fitbit.

The induced SVT HR ranged from 108 to 228 bpm. The accuracy of the HR measurements to within ±5 bpm of the ECG value was 89.3%, 89.7%, and 83.3%, respectively, for the Apple, Galaxy, and Fitbit, which improved to 100%, 89.7%, and 86.7%, respectively, when the criterion was changed to ±10 bpm difference with the ECG measurement. The intraclass correlation coefficients of each device with a Bland-Altman plot of the induced SVT HR measurements are shown in Figure 1. A statistically significant difference was not observed between the three devices (within ±5 bpm difference, p=0.780). The degree of HR did not influence the device accuracy. A summary of the baseline and SVT HR measurements is provided in Table 2.

Figure 1

Bland-Altman plots. This figure shows Bland-Altman plots of wrist-worn wearable device measured versus electrocardiography measured supraventricular tachycardia heart rate. The 95% limits of agreement and ICC were also shown. In all 3 devices, the mean difference was about 0 and most of the measurements were within 95% degree of agreements. The ICCs of devices were also high.

CI = confidence interval; ICC = intraclass correlation coefficient.

Table 2

Accuracy of wearable devices in measuring baseline and induced SVT heart rate

Clinical Situation		Device
		Apple	Galaxy	Fitbit	p value
No. of patients		34	34	33
	Baseline (within ±5 bpm)	34 (100)	34 (100)	31 (94)	*
No. of patients with SVT		28	29	30
	SVT (within ±5 bpm)	25 (89)	26 (90)	25 (83)	0.780
	SVT (within ±10 bpm)	28 (100)	26 (90)	26 (87)	0.568

Values are presented as number of patients (%).

bpm = beats per minute; SVT = supraventricular tachycardia.

*Statistical analysis not applicable.

Measurement of heart rate during atrial and ventricular pacing

During atrial and ventricular pacing study after SVT ablation for inducibility assessments, we attempted to measure the HR at each PCL using the three wearable devices. Although not a real arrhythmia, we hypothesized that the hemodynamic status of the pacing study would be similar to spontaneously occurred tachyarrhythmia. Four patients declined to participate in the pacing study.

In the atrial pacing study, Apple and Galaxy showed more than 90% accuracy up to PCL 380 ms (same as HR 160 bpm). The Fitbit showed a similar accuracy to the other devices at a PCL of 600 ms (same as 100 bpm of HR) but showed a statistically significant lower accuracy than the other devices at higher pacing rates. At a PCL of 330 and 300 ms, HR measurements were attempted in a small number of patients due to a loss of 1:1 antegrade AV conduction and they were excluded from the analysis.

In the ventricular pacing study, the Apple showed a greater than 90% accuracy up to a PCL of 330 ms (same as 180 bpm of HR). The Galaxy showed more than 90% accuracy up to a PCL 430 ms (same as 140 bpm of HR), but only an accuracy of 80% at a PCL of 380 ms (same as 160 bpm of HR) and 70% at 330 and 300 ms (same as 180 and 200 bpm of HR). Similar to the atrial pacing study, the Fitbit showed similar accuracy to other devices at PCL of 600 ms (HR 100 bpm), and statistically significant lower accuracy at higher pacing rate. In summary, the accuracy of the Apple and Galaxy was high and did not significantly change with the pacing rate. However, the Fitbit showed modest accuracy at lower pacing rates which decreased dramatically for rapid pacing rates. These differences between devices in accurate HR measurements during pacing study were statistically significant. The results of the pacing study are summarized in Table 3.

Table 3

Results of the atrial and ventricular pacing study

Pacing cycle length		Pacing site and device
		Atrial pacing				Ventricular pacing
		Apple	Galaxy	Fitbit	p value	Apple	Galaxy	Fitbit	p value
600 ms (100 bpm)
	Attempted	33	30	29		29	25	25
	Within ±5 bpm	33 (100)	29 (97)	27 (93)	0.198	29 (100)	25 (100)	20 (80)	0.007
	Within ±10 bpm	33 (100)	29 (97)	27 (93)	0.198	29 (100)	25 (100)	22 (88)	0.010
500 ms (120 bpm)
	Attempted	32	29	28		32	29	27
	Within ±5 bpm	31 (97)	26 (90)	18 (64)	0.000	32 (100)	27 (93)	20 (74)	0.002
	Within ±10 bpm	31 (97)	28 (97)	20 (71)	0.062	32 (100)	27 (93)	23 (85)	0.025
430 ms (140 bpm)
	Attempted	29	22	24		33	29	29
	Within ±5 bpm	26 (90)	20 (91)	19 (79)	0.005	33 (100)	28 (97)	18 (62)	<0.001
	Within ±10 bpm	26 (90)	20 (91)	21 (88)	0.032	33 (100)	28 (97)	20 (69)	0.005
380 ms (160 bpm)
	Attempted	19	16	17		31	29	27
	Within ±5 bpm	18 (95)	14 (88)	12 (71)	0.016	31 (100)	24 (83)	10 (37)	<0.001
	Within ±10 bpm	18 (95)	14 (88)	12 (71)	0.045	31 (100)	24 (83)	13 (48)	<0.001
330 ms (180 bpm)
	Attempted	9	10	13		31	28	26
	Within ±5 bpm	7 (78)	8 (80)	6 (46)	*	28 (90)	20 (71)	7 (27)	<0.001
	Within ±10 bpm	8 (89)	8 (80)	6 (46)		28 (90)	20 (71)	8 (31)	<0.001
300 ms (200 bpm)
	Attempted	7	7	8		31	27	24
	Within ±5 bpm	5 (71)	3 (43)	3 (38)	*	23 (74)	20 (74)	6 (25)	0.002
	Within ±10 bpm	5 (71)	3 (43)	3 (38)		23 (74)	20 (74)	7 (29)	0.005

Values are presented as number of patients (%).

*Statistical analysis not applicable.

DISCUSSION

In this study, we evaluated the accuracy of three widely used commercial wrist-worn wearable devices, Apple, Galaxy, and Fitbit, in the measurement of HR during SVT and during atrial and ventricular rapid pacing. The results showed that these three devices can accurately measure a baseline and SVT HR and that this accuracy is not affected by the degree of HR. Further validation was attempted during rapid atrial and ventricular pacing to simulate tachycardia. The devices again showed an acceptable accuracy, but this tended to decrease as the HR increased. In the pacing study, the Apple showed highest accuracy and Galaxy was comparable. The Fitbit showed a more than 70% accuracy below PCL 430 ms (same as 140 bpm of HR), but this was less than 50% at higher PCL.

Palpitation is one of the most common symptoms in patients visiting cardiology clinics. However, differentiating benign condition from clinically significant cardiac arrhythmia and making an accurate diagnosis are challenging due to its episodic nature. Holter monitoring and event recorders are currently in use but have availability and accessibility limitations. Accurate HR measurement during palpitations with smart devices that are used by most people may aid in this situation.

Currently, HR measurement technology using smart devices can be divided into 2 groups. The first of these uses PPG technology, an optical method that measures changes in tissue blood volume caused by the pressure pulse. The PPG method has been widely used and is well validated in the medical settings.7) The other group uses a single lead ECG which is achieved using a pair of external electrodes, either attached to the smartphone-case or as an independent additional device that is connected with specialized apps. Cardiio rhythm (Cardiio Inc., Cambridge, MA, USA) and AliveCor Kardia Mobile (AliveCor Inc., Mountain View, CA, USA) are a representative PPG-based application and single lead ECG system, respectively, that can be used with a smartphone. Both systems have been proven to be capable of accurate HR measurement and even to screen for AF in large-scale studies.4)8) Smart wearable devices adopt PPG-based HR monitoring system and prior studies have reported good accuracy of wearable devices in measurement of resting HR. However, some studies also have reported that variable accuracy among devices in activity tracking and exercise HR measurement.6)9)

When assessing HR using photoplethysmography technology, additional factors that affect accuracy of HR measurement should be considered such as body temperature, serum hemoglobin, skin status and blood pressure. For example, blood pressure can decrease during induced SVT.

As described above, HR and ECG monitoring are both possible with smartphone applications using either a PPG technique or single lead ECG system. This is finally evolving into the identification of arrhythmia. Recent study reported that twice-weekly single-lead ECG using an AliveCor device with remote interpretation in ambulatory patients ≥65 years of age at increased risk of stroke was significantly more likely to identify incident AF than routine care over a 12-month period.8) McManus et al.5) reported that pulse-based discrimination of AF, premature atrial contraction, and premature ventricular contraction were possible using a novel app that they developed. Notably, however, Wackel et al.10) attempted to measure HRs during SVT in pediatric patients using smartphone applications. But they found that the accuracy and reliability of smartphone applications was below that needed to confidently detect SVT in children, especially for a HR >200 bpm.

Many patients have now used wrist-worn wearable devices to measure their HR when palpitation occurs instead of smartphone applications but studies to validate these devices especially in arrhythmic situations are scarce. To the best of our knowledge, this is the first study to validate the accuracy of wrist-worn wearable devices in measuring induced SVT and rapidly paced HR.

Despite the accuracy of wearable devices, there is an inherent limitation of arrhythmia detection with HR based monitoring device. Sudden increment of HR might be a clue of SVT in some patients and provide reasonable suspicion of SVT for clinician especially in a situation lacking evidence in other exams. Currently, it is impossible to diagnose arrhythmia and to distinguish arrhythmia type using only pulse-based monitoring system. The accuracy of the device can be predicted, but there is always concern about whether it can improve the accuracy of the diagnosis. To increase the diagnostic value, prospective study is needed to increase the diagnostic rate in patients with symptoms of palpitation, not in the laboratory.

In our pacing study, the accuracy of the wrist-worn wearable devices ranged from 70% to 100% and differences between devices were observed. These results were different from those of actual SVT HR measurement study. Although the wrist-worn devices showed consistent accuracy of HR measurement during induced SVT, the accuracy of the devices was influenced by PCL during atrial stimulation. This result may suggest the accuracy of these devices could be affected by cycle length of tachycardia. And there may be a difference in technology between the devices if the accuracy is different. However, the number of patients who were attempted to measure at higher atrial pacing rates were small (mainly due to loss of 1:1 antegrade atrioventricular conduction). In addition, pacing study is not a real arrhythmic situation. Therefore, to interpret these discrepancies, further studies with a larger number of patients are needed.

On the other hand, we found that wearable devices show acceptable accuracy in HR measurement during rapid ventricular pacing. This suggests that they could possibly be used to measure HR in cases of hemodynamically stable ventricular tachycardia. But differences between devices were still observed.

There are several limitations in our study. First, we enrolled only a small number of patients. Second, as described above, we measured regular SVT HR in very stable patients in motionless state.6)10) Third, we did not consider the skin color or the skin photosensitivity that could affect the accuracy of the PPG technique.11) And all of our patients were Asian. Fourth, blood pressure could not be taken together with all HR measurements. Those measurements which were classified as “failed” or “low accuracy” may have been affected by a low blood pressure. Invasive arterial blood pressure monitoring was also not performed for our current study patients because it is not routine practice for SVT ablation in our hospital.

In conclusion, the accuracy of wrist-worn wearable devices in the measurement of baseline and SVT HR during an EP study is high. These devices also show an acceptable accuracy during atrial and ventricular pacing studies which simulate spontaneous tachycardia. Further validation of these devices using a well-designed, adequately powered study is needed.