Variable agreement between wearable heart rate monitors during exercise in cystic fibrosis.

In people with cystic fibrosis (CF), greater cardiorespiratory fitness is associated with improved survival and quality of life. Wearable activity monitors are a popular method of monitoring exercise, with measures of heart rate used to indicate exercise intensity. We assessed the agreement of heart rate recordings obtained using the Fitbit Charge HR™, Polar® H7 heart rate sensor and Masimo SET® Rad-5v pulse oximeter with the three-lead ECG during continuous and interval exercise. Adults with CF completed two exercise sessions, of 15-min duration per session, on a cycle ergometer while wearing the previously mentioned monitors. Firstly, participants cycled at 30% of estimated peak workload (W peak). Secondly, participants cycled at 1-min intervals at 60% of W peak interspersed with 2 min of unloaded cycling. Heart rate readings on all devices were recorded at minute intervals and their agreement was analysed using the Bland-Altman method. The Polar® H7 heart rate sensor had the best agreement with three-lead ECG, with a bias of 0±1 bpm during both continuous and interval exercise. The Masimo SET® Rad-5v pulse oximeter had good agreement, with a bias of 1±7 bpm and 1±5 bpm during continuous and interval exercise, respectively. The Fitbit Charge HR™ demonstrated less agreement, with a bias of 9±17 bpm and 5±13 bpm during continuous and interval exercise, respectively. The Fitbit Charge HR™ is not recommended for assessing heart rate during exercise in adults with CF. Findings support the use of the Polar® H7 for accurate heart rate monitoring.

Introduction

Cystic fibrosis (CF) is a life-limiting autosomal recessive disease, with respiratory failure being the major cause of mortality [1]. Greater cardiorespiratory fitness, as measured by maximal oxygen consumption, is associated with increased survival [2, 3], slower rate of respiratory decline (measured by forced expiratory volume in 1 s (FEV1)) [4] and improved quality of life [5-7]. For these reasons, optimisation of cardiorespiratory fitness is an important treatment goal. Aerobic exercise is demonstrated to be a safe and effective method to increase cardiorespiratory fitness in people with CF [8]. Current guidelines recommend that people with CF complete 30–60 min of aerobic exercise daily at a moderate intensity (i.e. heart rate increased to 64–76% of maximum heart rate (HRmax)) [9, 10]. This recommendation is consistent with guidelines provided to the healthy population [11].

Wearable activity monitors are an increasingly popular method of monitoring physical activity and exercise. These monitors are small, non-invasive and predominantly inexpensive devices that can be worn on the torso, hip or wrist. Wearable activity monitors are capable of recording pseudo-objective markers of physical activity such as heart rate, step count, distance travelled, elevation climbed and estimated energy expenditure. Heart rate measured by these monitors provides a physiological marker that may be used to indicate exercise intensity, and therefore guide and monitor exercise training. With growing popularity of these wearable activity monitors, the need to validate their accuracy is especially important [12].

Wrist-worn activity monitors, such as the Fitbit®, use photoplethysmography (PPG), an optical sensing technology, to measure heart rate by detecting variations in the blood volumes of microvascular tissue during cardiac contractions [13]. Such monitors have stimulated widespread interest in monitoring physical activity and heart rate in the healthy population, as well as in people with cardiorespiratory conditions [14]. However, the accuracy of heart rate measurements recorded by commercially available heart rate monitors, such as the Fitbit Charge HR™ (Fitbit, San Francisco, CA, USA), is controversial when compared to a gold-standard measurement of heart rate [15-18]. While some studies suggest that the Fitbit Charge HR™ provides more accurate heart rate measurement at rest or during low-intensity exercise [16, 17] other authors have reported that the most accurate heart rate measurement is recorded during higher intensity exercise [18]. To date, there are no published data validating the accuracy of the Fitbit Charge HR™ in people with CF.

Chest strap heart rate monitors, such as the Polar® H7 (Polar Electro, Kempele, Finland), contain electrodes to measure cardiac electrical activity and have been shown to provide an accurate measurement of heart rate in the healthy population [12, 19, 20]. The demonstrated accuracy and low cost have led Polar® heart rate monitors to be commonly used by athletes to monitor and control training load, yet they have not been widely adopted by the general population [12]. Polar® heart rate monitors have been used in research to measure heart rate in people with CF during exercise [8, 21]. However, to the authors' knowledge, the accuracy of Polar® heart rate monitors in comparison to a gold standard (i.e. 12-lead ECG) has not been evaluated in people with CF.

The potential applications of wearable activity monitors in people with chronic disease include the prescription and monitoring of exercise, and enhancing adherence to exercise recommendations through reminders and goal setting [14, 17, 22]. The wide availability and low cost of these monitors makes them ideal for use in healthcare to ensure adequate intensity of exercise. However, their accuracy is paramount for the safety of users, especially among people with a cardiorespiratory condition. Accuracy of heart rate measurements is particularly relevant for people with CF who may be advised to cease vigorous intensity exercise for a period of time during an exacerbation or following the resolution of haemoptysis or pneumothorax [10, 23].

Given the increasing use of wearable activity monitors, and the lack of data in the CF population, the aim of this study was to investigate the accuracy of heart rate recordings obtained from the Fitbit Charge HR™, Polar® H7 heart rate sensor and the Masimo SET® Rad-5v pulse oximeter (Masimo, Irvine, CA, USA) with heart rate from a three-lead ECG during moderate intensity continuous and interval exercise in adults with CF. Interval exercise was included as this is commonly used to reduce the symptom burden associated with exercise in people with CF [24]. The accuracy of heart rate derived from a pulse oximeter was assessed because of the widespread use of oximeters in clinical practice.

Methods

Study design

This was a single group observational study comparing heart rate recordings across multiple heart rate monitors. Heart rate was measured during continuous and interval exercise on an electronically braked cycle ergometer (Orbit OEB2002; Orbit Fitness, Perth, Australia).

This study was approved by the Sir Charles Gairdner and Osborne Park Health Care Group Human Research Ethics Committee and Research and Governance Office. Written informed consent was gained from each participant.

Participants

Recruitment and consent

Potential participants were recruited from the Sir Charles Gairdner Hospital (SCGH) adult CF service. Written informed consent was obtained prior to commencing the study.

Eligibility criteria

Adults with CF admitted to SCGH for treatment of an exacerbation were invited to participate in the study. Participants were excluded if they 1) were clinically unstable (e.g. were febrile or had current or recent haemoptysis); 2) had a comorbid condition(s) thought to adversely affect participation in the study (e.g. cardiac, neurological, musculoskeletal or psychological impairments); 3) had a body mass index <18 kg·m−2; 4) were pregnant; or 5) were unable to communicate in English or provide written informed consent.

Exercise protocol

15 participants were asked to complete two exercise sessions on an electronically braked cycle ergometer lasting ∼20 min each. Both exercise sessions were completed on a single day, separated by a minimum of 2 h. The sessions took place in the physiotherapy department at SCGH and were supervised by two physiotherapists (MG, AS) experienced in the management of people with CF. During exercise, participants were advised to alert the investigator of any symptoms that would limit them from safely continuing the exercise. Participants continued all other usual treatment (excluding exercise) on the day of testing.

Workload

Prior to the first exercise session, participants' anthropometric measures (height and weight) were recorded and used to estimate the peak workload (Wpeak) for each session using the equation of −142.865 + 2.998 × age (years) −19.206 × sex (0=male; 1=female) + 1.328 × height (cm) + 23.362 × FEV1 (L). This Wpeak estimation was based on a model previously developed following the evaluation of 409 cardiopulmonary exercise tests (CPET) in 160 children and adolescents with CF [25]. While this equation has yet to be validated in the adult CF population, Wpeak estimated using the equation was found to be similar to the Wpeak achieved during CPETs undertaken for clinical purposes by a group of young adults (n=10) with CF in the department of pulmonary physiology at SCGH. The estimated workload was piloted in order to determine whether participants achieved the recommended heart rate range for moderate intensity exercise.

Exercise session one (continuous exercise)

Session one was completed in the morning and involved 15 min of continuous cycling on an electronically braked cycle ergometer. Resting heart rate was recorded. Participants were asked to cycle for 2 min at the minimum workload of 10 W as a warm-up and during recovery. In between the warm-up and recovery periods, participants were asked to cycle for 15 min at 30% of their estimated Wpeak, maintaining a cadence of 50–60 revolutions per minute. Due to the potential for overestimation of Wpeak occurring in older participants with the aforementioned equation [25], the workload was set at 15% of the estimated Wpeak for participants aged >30 years who also had severe respiratory disease (i.e. FEV1 <40%). During the last 10 s of each minute, heart rate readings on all monitors were recorded.

Exercise session two (interval exercise)

Session two was completed during the afternoon following session one. Resting heart rate was recorded. Participants were asked to complete interval exercise at 60% of estimated Wpeak, “work” (1 min), interspersed with “rest” periods cycling at 10 W (2 min). These intervals were repeated five times, with a total exercise time of 15 min, in addition to 2-min warm-up and recovery periods at the minimum workload of 10 W. For participants aged >30 years who had severe respiratory disease, the workload for the “work” interval was set at 30% of estimated Wpeak. Recordings of heart rate were made consistent with exercise session one. Following completion of data collection, each participant was asked which (if any) of the device(s) they would prefer to use to monitor their heart rate during unsupervised exercise.

Heart rate monitoring

During both exercise sessions, participants wore the four monitors and were blinded to the heart rate recordings.

Three-lead ECG

A three-lead ECG was used as the criterion value measurement of heart rate [26]. This measure has been demonstrated to be a valid alternative to the gold-standard 12-lead ECG in adults [27]. The lower limb lead was covered with gauze and Fixomull (BSN Medical, Hamburg, Germany) stretch tape to prevent motion artefact during cycling.

Polar® H7 heart rate sensor

The Polar® H7 chest strap sensor was secured with the sensor positioned anteriorly, immediately below the pectoral muscles. The smartphone application Polar Beat® was used to monitor the heart rate readings.

Fitbit Charge HR™

The Fitbit was worn at least two finger widths above the left wrist during exercise. The heart rate readings were monitored using the Fitbit® smartphone application.

Masimo SET® Rad-5v pulse oximeter

The Masimo SET® Rad-5v pulse oximeter with a finger sensor was used to measure heart rate and oxygen saturation of participants during exercise.

Data analysis

Sample size calculations were not undertaken. Our target sample was 12 participants; however, we aimed to recruit 25% more than this to allow for incomplete data collection. Statistical analyses were conducted using SPSS (Statistical Package for Social Sciences, version 22.0; IBM, Armonk, NY, USA). Anthropometric and baseline data were reported using descriptive statistics. Bland–Altman analysis was used to assess agreement between heart rate measured by each heart rate monitor and three-lead ECG. For comparison between each heart rate monitor and three-lead ECG the bias and the 95% limits of agreement were identified, as well as the percentage error. The proportion of missing heart rate data from the monitors, variability of the heart rate measurements (i.e. outside 5 beats per minute (bpm) of three-lead ECG) and patient preference data were reported using descriptive statistics. Data were assessed for normality, and the significance level was set at p<0.05.

Results

All 15 participants completed both exercise sessions (table 1).

TABLE 1

Participant characteristics

Male:female n:n	11:4
Age years	34±11.20 (19–57)
BMI kg·m−2	23.6±3.2 (19.4–31.1)
FEV1 L	2.29±0.87 (0.87–3.83)
FEV1 % predicted	59±22 (32–103)
FVC L	3.57±0.81 (2.18–5.30)
FVC % predicted	77±14 (57–106)
Genotype homozygous F508del %	80

Data are presented as mean±sd (range), unless otherwise stated. n=15. BMI: body mass index; FEV1: forced expiratory volume in 1 s; FVC: forced vital capacity.

Agreement of heart rate measurements

The Polar® H7 heart rate sensor had the best agreement with three-lead ECG during continuous and interval exercise when compared to other heart rate monitors tested (figures 1 and 2). The percentage error of the Polar® H7 heart rate sensor was 0.07% and 0.09% during continuous and interval exercise, respectively. The Fitbit Charge HR™ demonstrated a percentage error of 7.28% during continuous exercise and 3.34% during interval exercise. Figures 1 and 2 suggest that the Fitbit Charge HR™ had a tendency to underestimate heart rate, particularly during continuous exercise. This underestimation is further illustrated by the mean HRmax data from all participants (figure 3). The Masimo SET® Rad-5v pulse oximeter had percentage error of 0.63% and 0.59% during continuous and interval exercise, respectively.

FIGURE 1

Bland–Altman plots depicting heart rate measurements of all participants from each monitor compared to three-lead ECG during continuous exercise. a) Polar® H7 heart rate sensor; b) Fitbit Charge HR™; c) Masimo SET® Rad-5v pulse oximeter. HR: heart rate; bpm: beats per minute; LoA: 95% limits of agreement.

FIGURE 2

Bland–Altman plots depicting heart rate measurements of all participants from each monitor compared to three-lead ECG during interval exercise. a) Polar® H7 heart rate sensor; b) Fitbit Charge HR™; c) Masimo SET® Rad-5v pulse oximeter. HR: heart rate; bpm: beats per minute; LoA: 95% limits of agreement.

FIGURE 3

Mean percentage of maximum heart rate of all participants from each monitor during a) continuous and b) interval exercise. HRmax: maximum heart rate.

Proportion of missing data

Of the 2400 possible heart rate measurements from all monitors, 2332 (97%) were recorded. Missing data were only observed from the Fitbit Charge HR™, with 31 (10%) missing data during continuous exercise and 37 (12%) missing data during interval exercise.

Proportion of heart rate recordings outside 5 bpm of three-lead ECG

During continuous exercise, the number of heart rate measurements by the Polar® H7 heart rate sensor and Masimo SET® Rad-5v pulse oximeter outside of 5 bpm of three-lead ECG readings was four (1%) and 16 (5%), respectively. During interval exercise, this was one (<1%) and seven (2%), respectively. In contrast, the Fitbit Charge HR™ recorded 85 (28%) measurements outside 5 bpm of three-lead ECG during continuous exercise, and 69 (23%) measurements during interval exercise.

Participant preference

Seven (47%) participants reported that they would prefer the Polar® H7 heart rate sensor for heart rate monitoring during unsupervised exercise. The Fitbit Charge HR™ was preferred by six (40%) participants. Two (13%) participants reported that they would not use any heart rate monitor during exercise.

Discussion

The results of this study demonstrate that the Polar® H7 heart rate sensor and Masimo SET® Rad-5v pulse oximeter provide accurate heart rate measurements in adults with CF during exercise. The Polar® H7 heart rate sensor provided the most accurate measurement of heart rate when compared to three-lead ECG during both continuous and interval exercise, with a mean bias of 0 bpm during both continuous and interval exercise. This is in keeping with previous studies that have established electrocardiographic chest strap sensors as an accurate measurement of heart rate in the general population [12, 20]. Our findings raise significant concerns regarding the ability of the wrist-worn Fitbit Charge HR™ to accurately measure heart rate during exercise in this population.

Our results support previous studies demonstrating an underestimation of heart rate measurements recorded by the Fitbit Charge HR™ [12, 15–17, 28]. Several studies have found that the Fitbit Charge HR™ was more accurate during rest or light-to-moderate exercise and had a large tendency to underestimate heart rate at higher intensity exercise [15-17]. One such study investigating healthy adult participants found the Fitbit Charge HR™ recorded a mean bias to ECG of −3.76±21.18 bpm at heart rate recordings <132 bpm and −16.49±31.88 bpm at heart rate recordings >132 bpm [29]. In contrast, other authors have found that the Fitbit Charge HR™ provided more accurate measurement of heart rate at higher exercise intensities [18]. These authors hypothesised that increased level of perfusion during vigorous-intensity exercise may reduce the error associated with PPG technology.

The current study found that the Fitbit Charge HR™ demonstrated poor agreement with three-lead ECG, underestimating heart rate measurements during exercise in this population. In addition, the proportion of data from the Fitbit Charge HR™ outside of 5 bpm of three-lead ECG was greater than any other heart rate monitor (28% during continuous exercise and 23% during interval exercise) and failed to record 11% of all possible heart rate measurements. Missing data were attributable to failure of the monitor to record heart rate despite participants' upper limbs and the heart rate monitor being stationary. Some extrinsic factors that have been suggested to interrupt accurate heart rate detection by PPG-based heart rate monitors include ambient light, sweat, misalignment between the skin and optical sensor and motion artefact [13]. To avoid these problems, all participants completed the study in the same setting and were fitted with each monitor according to the manufacturer's guidelines by the same physiotherapist. It is unclear whether higher levels of sweat chloride in people with CF may have interrupted accurate heart rate detection by the Fitbit Charge HR™. Analysis of our results undertaken considering disease severity did not show a trend towards higher error in participants with more severe lung disease. The authors are not aware of previous studies investigating the accuracy of the Fitbit Charge HR™ in other populations with chronic disease.

To the authors' knowledge, there are no studies investigating the accuracy of heart rate measurements obtained from the Masimo SET® Rad-5v pulse oximeter in people with CF, despite its widespread use in an inpatient setting. Our results demonstrate that this pulse oximeter is an accurate measurement of heart rate in people with CF during exercise. On very few occasions, we observed a transient change in heart rate measurements from the pulse oximeter in comparison to three-lead ECG, seemingly reading every second cardiac contraction; however, this self-resolved within 10 s each time. This was not deemed clinically significant, as this monitor would be used by clinicians with patients in a supervised setting.

This small observational study is the first to investigate the accuracy of wearable heart rate monitors in people with CF. Our results provide clinicians with useful information on the accuracy and acceptability of wearable heart rate monitors in this population. There are some limitations of the study. Firstly, the visual recording of heart rate measurements may have contributed to some error, as opposed to time-stamped data. Continuous heart rate recording, which was not feasible on all heart rate monitors, would enable more detailed comparisons. Secondly, the results only apply to the heart rate monitors tested and constantly developing technology makes it difficult to extrapolate our results to other heart rate monitors; however, updates to such monitors are often aesthetic, with no modification to the technology. In addition, our results may not be representative of those obtained during alternative forms of exercise. Previous studies have shown the extent of agreement between PPG-based heart rate monitors and 12-lead ECG varies with form of exercise [12]. As motion artefacts may interfere with accurate heart rate measurement, exercising on an electronically braked cycle ergometer should have yielded optimal results for each heart rate monitor. Sessions of continuous and interval exercise were not counterbalanced, and the participants' oral intake and activity in between sessions was not controlled for. As people with CF are encouraged to complete ≥30 min of moderate-intensity exercise on ≥5 days per week we chose to investigate the accuracy of heart rate monitors during continuous and interval moderate intensity exercise. Therefore, our results cannot be used to infer the accuracy of these devices during rest or high-intensity exercise. As per figure 3, during continuous exercise participants were largely within the previously specified heart rate range indicative of moderate-intensity exercise. During interval exercise, participants were exercising within the heart rate range indicative of high-intensity exercise (77–95% HRmax); this was not our original aim, but is useful in comparing data from both moderate and high intensity exercise. Further studies would need to validate the accuracy of these monitors in people with CF accounting for different modes of exercise.

As heart rate monitors are commercially available, they can be easily used by people with CF without consultation with clinicians. Although this availability has benefits for accessibility, the variable accuracy of heart rate monitors may lead to exercise being undertaken at an intensity insufficient to improve or maintain cardiorespiratory fitness. Further, the default activity and exercise intensity goals provided by the manufacturers may be inappropriate for people with CF, especially during times of clinical instability. The potential of wearable heart rate monitors, such as the Fitbit Charge HR™, lies in their ability to provide clinicians with real-time monitoring outside of the hospital setting. They can provide objective data for exercise prescription and monitoring, identify those not engaging in regular exercise and be used as a motivational tool to increase adherence. However, the results of this study demonstrate that the Fitbit Charge HR™ has a tendency to underestimate heart rate, particularly during continuous moderate intensity exercise, which has significant implications for patient safety.

Conclusion

Our results demonstrate that the Polar® H7 and Masimo SET® Rad-5v pulse oximeter provide accurate measurements of heart rate in adults with CF in a controlled clinical setting. Heart rate measurements by the Fitbit Charge HR™ were not within an acceptable error range, and therefore this device is not a recommended heart rate monitor for this population. Healthcare professionals and people with CF should be aware of the limitations of the Fitbit Charge HR™. For accurate heart rate monitoring, an electrocardiographic chest strap monitor such as the Polar® H7 should be used.