Comparison of Wearable Activity Tracker with Actigraphy for Sleep Evaluation and Circadian Rest-Activity Rhythm Measurement in Healthy Young Adults.

OBJECTIVE: The purpose of this study was to evaluate the applicability of data obtained from a wearable activity tracker (Fitbit Charge HR) to medical research. This was performed by comparing the wearable activity tracker (Fitbit Charge HR) with actigraphy (Actiwatch 2) for sleep evaluation and circadian rest-activity rhythm measurement.
METHODS: Sixteen healthy young adults (female participants, 62.5%; mean age, 22.8 years) wore the Fitbit Charge HR and the Actiwatch 2 on the same wrist; a sleep log was recorded over a 14-day period. We compared the sleep variables and circadian rest-activity rhythm measures with Wilcoxon signed-rank tests and Spearman's correlations.
RESULTS: The periods and acrophases of the circadian rest-activity rhythms and the sleep start times did not differ and correlated significantly between the Fitbit Charge HR and the Actiwatch 2. The Fitbit Charge HR tended to overestimate the sleep durations compared with the Actiwatch 2. However, the sleep durations showed high correlation between the two devices for all days.
CONCLUSION: We found that the Fitbit Charge HR showed high accuracy in sleep evaluation and circadian rest-activity rhythm measurement when compared with actigraphy for healthy young adults. The results suggest that the Fitbit Charge HR could be applicable on medical research as an alternative tool to actigraphy for sleep evaluation and measurement of the circadian rest-activity rhythm.

INTRODUCTION

The biological clock is endogenous and can be entrained by the light-dark cycle. All physiological and behavioral processes, including the sleep-wake cycle, hormone secretion, eating behaviors, and body temperature adjustments, have circadian rhythms that correspond to the biological clock.123 The sleep-wake cycle and mood are greatly influenced by circadi179an rhythms.4 Circadian gene expression and sufficient light exposure at appropriate times may have effects on circadian rhythms.56

Actigraphy has been widely used over the last 20 years for the study of chronobiology and sleep medicine.78910 Actigraphy, which measures light exposure, sleep, activity, and the sleep-wake cycle,9 has been used to examine changes in the rest-activity cycle, quantify activity, and study the effects of pharmacological manipulations on the sleep-wake cycle and psychomotor activity.911 Further, actigraphy is measured with small wristband-like devices, and has the advantage of being noninvasive.1213

Since 2012, a variety of wearable activity trackers has been introduced, and has been used increasingly by the general population to promote health.14 Currently, their use and availability are increasing, as more devices with various designs and functions are being introduced. Wearable activity trackers can be worn on the wrist, waist, or clothing during daily activities, and are used to monitor various measures, such as sleep and activity patterns, with a smartphone app or a personal computer.15 Compared with actigraphy, these devices have the advantage of increased accessibility due to their low cost.16 In addition, immediate feedback on sleep and activity patterns is easily available to the user.17181920 However, these devices have not been applied in medicine or research because they have not yet been validated for such purposes.2021 Due to their inherent advantages, wearable activity trackers are expected to be used as an alternative to actigraphy, but their reliability or validity still needs to be determined.15

The purpose of this study was to identify the applicability of wearable activity trackers such as Fitbit to medical research as an alternative to actigraphy. We hypothesized that the sleep and activity variables measured by the Fitbit Charge HR would not differ from those measured by actigraphy, and that the sleep-wake rhythms calculated with a Cosinor models would not differ between the two devices.

METHODS

Participants

This study was performed from May to July 2015. The participants were university students who were recruited by an online advertisement on the Korea University Homepage. Twenty-one healthy adults who owned a smartphone, had no psychiatric disorders, and did not use drugs that can affect the sleep and wake states participated in this study. Two psychiatrists (HJL and CHC) who specialize in sleep created the questionnaires and interviewed the participants. The participants were given a detailed explanation of the study, and they provided written informed consents before the study. Five participants were excluded from the final data analysis due to Actiwatch 2 data loss during data acquisition (two participants) or over 10 hours of noncompliance with wearing the device (three participants). Finally, the data of 16 participants were analyzed in the study. The participants consisted of six men and ten women aged 18–26 years (mean±standard deviation, entire group, 22.8±2.8; men, 22.0±2.4; women, 23.3±3.0). This study was approved by the Institutional Review Board of Korea University Hospital.

Actigraphy

The Actiwatch 2 (Royal Philips, Amsterdam, the Netherlands) is a wristband-type actigraphy device measuring 43×23×10 mm in size and weighing 16 g. It has an internal accelerometer and light sensor to detect movement and light exposure, respectively.1622 After it is fully charged, the Actiwatch 2 can be used continuously for 14 days without additional charging. The Actiwatch 2 provides raw data on sleep, rest, activity, and light exposure in 60-s epochs in Actiware, software version 6.0.6.2324 The actigraphy data that were collected with the Actiwatch 2 were compared with the self-reported sleep logs and edited raw data.13 After comparison with the sleep logs, the data were edited and then analyzed with a proportional integral algorithm

Fitbit

The Fitbit is also an accelerometer. The Fitbit Charge HR (Fitbit Inc., San Francisco, CA, USA; https://www.fitbit.com/) that was used in this study has a three-axis acceleration sensor, altimeter, vibration motor, and optical heart rate monitor. The Fitbit Charge HR requires charging after approximately 4–5 days' use, and the collected data, such as the sleep, activity, and heart rate measures, are automatically synchronized with a smartphone app or personal computer using Bluetooth 4.0. Before the start of the study, the settings of the Fitbit Charge HR were set to be the same in all of the devices; this was done by wearing it on the non-dominant hand and setting the sleep measurement sensitivity to normal and the heart rate monitoring to automatic. A Fitbit ID and password were created before the start of the study and provided to the participants for the synchronization of the information during the study period. We obtained raw data on sleep, activity, and heart rate during the study with the Fitbit application programming interface.

Procedures

We used the Fitbit Charge HR, which is a wristband-type wearable activity tracker. The 16 participants wore both the Actiwatch 2 and Fitbit Charge HR at the same time on their non-dominant wrist and recorded sleep logs for 14 days. Because only the Actiwatch 2 has a light sensor,9 the participants wore the Actiwatch 2 and Fitbit Charge HR distally and proximally, respectively. Charging cables were provided so that the participants could charge the Fitbit Charge HR every 4–5 days if the battery level was low. Except for the times when the participants showered and charged the device, the participants maintained their normal daily activities while wearing both devices. After completion of the 14 days, the participants revisited the hospital to return the devices and sleep logs, which the researchers used to obtain the data for analysis.

The measurements of sleep and circadian rest-activity rhythms

For the comparison, analogous variables were extracted from the data obtained with the two devices. For the Actiwatch 2, the sleep variables collected were sleep start time, sleep end time, sleep duration, sleep efficiency, and wake after sleep onset time, and the activity variables collected were activity score, activity amount, and activity intensity. For the Fitbit Charge HR, the sleep variables collected were time in bed, sleep start time, sleep duration, sleep efficiency, awakening count, awakening duration, restless count, and restless duration, and the activity variables collected were activity score, step count, moving distance, calorie consumption, number of steps taken, and active time.

In order to compare the two devices, the sleep variables used were sleep start time and sleep duration and the activity variable compared included circadian rest-activity rhythm. Sleep duration was comparable without correction because it was presented as time in min for both devices. However, the sleep start time and activity score required correction to allow comparison. The sleep start time needed to be converted to a number for the statistical comparison because it was presented as time in both devices. We calculated how much time had passed from 18:00 until the sleep start time and converted that time to min. As the value for the daily activity score was not the same in the two devices, the activity score was divided by the maximum activity score for the day in order to obtain a ratio between 0 and 1.

For all of the participants, the sleep variable data that were outliers according to an outlier boxplot were excluded from the final analysis.25 Two hundred and eight measures were obtained from the participants for the 13 days of sleep. Twelve values of sleep start time and 20 values of sleep duration were outliers and thus excluded.

The activity score was recorded in 60-s epochs. Any missing data for periods when the devices were not worn were given a value of zero. The participants did not wear the devices when they performed certain activities, such as showering and charging of the device. If the non-wearing time was less than 60 min, the missing data were converted to zero and included in the analysis. The Cosinor Periodogram software program (version 3.1, Boise State University, Boise, ID, USA) was used to obtain the circadian rest-activity rhythms for each participant. The Cosinor analysis is a reliable and convenient method that is used to calculate the period, acrophase, mesor, and amplitude of each circadian rest-activity rhythm.26

Statistical analysis

The data were analyzed with SPSS (version 21.0, IBM Corporation, Armonk, NY, USA). The analyzed data included the sleep start time and sleep duration for 16 participants and the circadian rest-activity rhythm for 15 participants. One of the participants was excluded from the analysis of the circadian rest-activity rhythm because a circadian rest-activity rhythm was not produced. Comparisons of the mesor and amplitude were not meaningful because the activity values from the two devices were converted to a ratio between 0 and 1. Therefore, the statistical analyses compared the period and acrophase.

In order to ascertain if the measures collected with the Fitbit Charge HR were associated with those collected with the Actiwatch 2, Spearman's correlation coefficients (r) were calculated. In order to assess if the measures collected with the Fitbit Charge HR were similar to those collected with the Actiwatch 2, the differences between the two devices were assessed with Wilcoxon signed-rank tests.27 The degree of correlation was evaluated by the coefficient of Spearman's correlation (r=0.00–0.30: negligible correlation, r=0.30–0.50: low correlation, r=0.50–0.70: moderate correlation, r=0.70–0.90: high correlation, and r=0.90–1.00: very high correlation). Differences with p values<0.05 were considered statistically significant. Results of the Wilcoxon signed-rank tests that had p values>0.05 indicated no difference between the two devices; therefore, the null hypothesis cannot be eliminated.

The statistical analyses were performed each day independently for 13 days of sleep. Thus, the analyses were performed 13 times. In addition, the data for all of the days were analyzed with the Wilcoxon signed-rank test and Spearman's correlation test.

RESULTS

Table 1 shows the comparison of the sleep start times for the Actiwatch 2 and Fitbit Charge HR. First, the total average sleep start times of all of the participants were 462.3±93.5 min and 460.6±92.2 min for the Actiwatch 2 and Fitbit Charge HR, respectively. The sleep start times for the two devices were significantly correlated (r=0.869, p<0.001, Spearman's correlation), with no significant difference between their values (p>0.05, Wilcoxon signed-rank test). The Spearman's correlation showed a statistically significant correlation coefficient for each day (r=0.581–0.988), and the Wilcoxon signed-rank test was not significant (p>0.05) for 11 of the 13 days. Figure 1 shows the relationship between the Actiwatch 2 and Fitbit Charge HR for the sleep start times of all of the participants.

Table 1

Comparison of sleep start time between Actiwatch 2 and Fitbit Charge HR (N=16)

*p<0.05, **p<0.01, †time had passed from 18:00 until the sleep start time, ‡correlation coefficient (r=0.00–0.30: negligible correlation, r=0.30–0.50: low correlation, r=0.50–0.70: moderate correlation, r=0.70–0.90: high correlation, r=0.90–1.00: very high correlation). N: number of participants

Figure 1

Scatter plot of sleep start time between Actiwatch 2 and Fitbit Charge HR. Sleep start time was compared between the two devices for all participants. Each of sleep start time was assessed from two different devices. For analysis, sleep start time was calculated by time had passed from 18:00, and converted to minutes. Each number of axis x and y indicate the sleep start time, that is total time (minutes) has passed from 18:00 to sleep start time. The black spots are all values of sleep start time between Actiwatch 2 and Fitbit Charge HR. The solid line indicates correlation coefficient of Spearman's correlation (r=0.869, p<0.01).

For sleep duration, the total average sleep durations for all of the participants were 387.3±98.3 min and 409.7±97.6 min for the Actiwatch 2 and Fitbit Charge HR, respectively, which resulted in an approximately 22.4-min difference (Table 2). This difference was significant with the Wilcoxon signed-rank test, but the Spearman's correlation showed a correlation coefficient of 0.918. The Wilcoxon signed-rank test of the daily values was not significant (p>0.05) for only 4 days. However, the Spearman's correlation revealed a relatively high correlation coefficient (r=0.569–0.982), which was statistically significant for all of the days. Figure 2 shows the relationship between the Actiwatch 2 and Fitbit Charge HR for the sleep durations of all of the participants.

Table 2

Comparison of sleep duration between Actiwatch 2 and Fitbit Charge HR (N=16)

*p<0.05, **p<0.01, †correlation coefficient (r=0.00–0.30: negligible correlation, r=0.30–0.50: low correlation, r=0.50–0.70: moderate correlation, r=0.70–0.90: high correlation, r=0.90–1.00: very high correlation). N: number of participants

Figure 2

Scatter plot of sleep duration between Actiwatch 2 and Fitbit Charge HR. Sleep duration was compared between the two devices for all participants. Each of sleep duration was assessed from two different devices. Each number of axis x and y indicate the sleep duration. The black spots are all values of sleep duration between Actiwatch 2 and Fitbit Charge HR. The solid line indicates correlation coefficient of Spearman's correlation (r=0.918, p<0.01).

Spearman's correlation and the Wilcoxon signed-rank test were performed on the period and acrophase values of the circadian rest-activity rhythms from the two devices in 15 participants (Table 3). The periods of the Actiwatch 2 and Fitbit Charge HR were 24.4±1.1 h and 24.3±1.0 h, respectively, which was a difference of approximately 0.1 h. The acrophases were −183.6±107.0 degrees and −181.3±104.4 degrees for the Actiwatch 2 and Fitbit Charge HR, respectively. The differences between the period and acrophase values of the two devices were not significant (p<0.05) with the Wilcoxon signed-rank tests, and Spearman's correlation resulted in correlation coefficients of 0.8 (p<0.001) and 0.980 (p<0.001) for the period and acrophase, respectively.

Table 3

Comparison of circadian rest-activity rhythm using a Cosinor periodogram between Actiwatch 2 and Fitbit Charge HR (N=15)

**p<0.01, †correlation coefficient (r=0.00–0.30: negligible correlation, r=0.30–0.50: low correlation, r=0.50–0.70: moderate correlation, r=0.70–0.90: high correlation, r=0.90–1.00: very high correlation). N: number of participants

DISCUSSION

In this study, the sleep start times determined by the Fitbit Charge HR and Actiwatch 2 did not significantly differ, and showed significant correlation. This result was considered meaningful because the Fitbit Charge HR automatically detects sleep. However, the data were not edited according to the sleep log.

The Wilcoxon signed-rank test revealed no significant differences in sleep durations between the two devices for the duration of 4 days. However, Spearman's correlation resulted in a statistically significant correlation for all of the days. One day showed a moderate correlation, four days showed high correlations, and eight days showed very high correlations. The Fitbit Charge HR consistently tended to overestimate sleep duration for each day compared with the Actiwatch 2. A data review revealed a tendency for the Fitbit Charge HR to overestimate sleep duration by 20–30 min compared with the Actiwatch 2. This was thought to be related to the different mechanisms used by the two devices to detect and calculate sleep duration. Therefore, we hypothesized that this explains the significant differences found with the Wilcoxon signed-rank test for the sleep durations of most days; however, Spearman's correlation was significant for every day.

The two devices showed some differences in the activity scores. The Actiwatch 2 had less zero values and more varied values during activity due to its higher sensitivity compared with the Fitbit Charge HR. In contrast, the Fitbit Charge HR had more zero values and a narrower range of values. We used the activity scores to estimate the circadian rest-activity rhythms with a Cosinor analysis, and compared the periods and acrophases of the circadian rhythms to evaluate the usefulness of the circadian rest-activity rhythm measurements of the Fitbit Charge HR.28 According to the results, the two devices were significantly correlated in the estimations of the circadian rest-activity rhythms, and the recorded period and acrophase values did not significantly differ.

In a previous study, the use of polysomnography (PSG), a Fitbit, and actigraphy in 24 healthy adults were compared for one night. The Fitbit and actigraphy results for recorded total sleep time and sleep efficiency differed significantly between each other and PSG because both the Fitbit and actigraphy overestimated the sleep efficiency and total sleep time. The specificity of actigraphy was higher, except for measuring wake times before sleep onset, and the sensitivity of the Fitbit was higher in all of the sleep stages and during arousals.16 The strength of that study was the comparison of a wearable activity tracker with PSG and actigraphy, but it had the limitation of the short study duration of a single night, which prevented any adjustments for the first-night effects of PSG. In addition, that study only compared sleep variables.

Another study was performed on 42 participants wearing a Fitbit Zip, actigraph, and a Yamax CW700 pedometer for 7 days, and the steps/day were compared among the devices. The authors reported that, although the Fitbit Zip was highly correlated for steps/day with the other two devices, it recorded more steps/day than did actigraph.27 Gusmer et al. compared the use of a Fitbit, actigraph, and metabolic cart for measuring energy expenditure and step count while walking on a treadmill. They found that energy expenditure estimates were moderately correlated between the Fitbit and actigraph accelerometers during slow walking and strongly correlated during brisk walking; the step count was found to be strongly correlated between the accelerometers.18 However, in these two studies, only activity variables such as step count and energy expenditure were compared; there was no comparison for sleep variables.

In summary, previous studies have only performed partial comparisons of sleep or activity variables without comprehensively comparing circadian rhythm variables including the sleep-wake and activity cycles.161827 Furthermore, many previous studies have used clip-type wearable activity trackers instead of the wristband-type, which limits study comparisons. This may be because the differences in the way the device is worn compared with an actigraph may affect the result.1827

Our study used a wristband-type wearable activity tracker and comprehensively analyzed sleep and activity variables over a relatively long period. Our study differed from the previous studies that analyzed only one night of sleep data or less than seven days of activity data. In addition, the current study analyzed the circadian rest-activity rhythm, which is considered the most important factor in chronobiology.29

The current study had some limitations. The major limitations were the small sample size and limited number of variables. The amount of data should ideally take into account the number of participants and accumulated time period of the data.30 In our study, the number of participants was rather small due to unwanted loss of data. However, sufficient data were collected because 13 days of sleep and 14 days of activity data were obtained from each participant. Second, our study did not perform PSG, which is considered the gold standard for sleep measurement.1631 However, because the goal of our study was to verify the applicability of wearable activity trackers as an alternative method to actigraphy, which is considered a reliable method in sleep medicine, it was not necessary to include PSG. Third, we could not compare the data epoch by epoch between the two devices because we were not able to retrieve the data for each epoch from the Fitbit Charge HR. In the future, comparisons of the data for each epoch for the Fitbit Charge HR would be ideal. The last limitation of the present study is the inclusion of only healthy young adults. Therefore, interpretations of the results of this study should be done cautiously because actigraphy is usually performed on subjects with sleep-wake problems. However, the subject homogeneity, which is important when performing controlled experimental studies, is an advantage of the present study. We hope to compare the data for wearable activity trackers with actigraphy in the subjects with sleep-wake problems in the future.

In addition to sleep-wake and activity information, the Fitbit Charge HR collects information on heart rate, which is thought to have a circadian rhythm relative to sleep and the autonomic nervous system.32 In contrast, the Actiwatch 2 collects information on light exposure, which is a very important factor in determining and adjusting the circadian rhythm.133 The development of a wearable activity tracker that measures additional variables, such as light exposure, and the creation of an algorithm that uses various variables to determine circadian rhythms will be useful for medical research and clinical applications, especially the diagnosis, treatment effect monitoring, and prevention of a variety of psychiatric disorders, such as mood disorder and circadian rhythm disorder. In particular, because changes in circadian rhythms can greatly affect the onset and intensity of symptoms of a mental disorder,434353637 examinations of circadian rhythms is expected to help patients with mental disorders monitor and adjust their sleep-wake cycles and circadian rhythms to prevent the onset and aggravation of symptoms.

In conclusion, the findings of this study showed that the Fitbit Charge HR is a valid, reliable, and alternative device to use for sleep evaluations and circadian rest-activity rhythm measurements compared with actigraphy in healthy young adults. However, the sensitivity of the Fitbit Charge HR for accurately identifying activity was lower than actigraphy. Additional studies need to be performed on the validity of wearable activity trackers, ways to improve the low sensitivity of wearable activity trackers for activity measurements, and addition of more functions, such as a light sensor, in order to obtain actigraphic data with wearable activity trackers.