Examination of wrist and hip actigraphy using a novel sleep estimation procedure ☆

OBJECTIVE: Improving and validating sleep scoring algorithms for actigraphs enhances their usefulness in clinical and research applications. The MTI® device (ActiGraph, Pensacola, FL) had not been previously validated for sleep. The aims were to (1) compare the accuracy of sleep metrics obtained via wrist- and hip-mounted MTI® actigraphs with polysomnographic (PSG) recordings in a sample that included both normal sleepers and individuals with presumed sleep disorders; and (2) develop a novel sleep scoring algorithm using spline regression to improve the correspondence between the actigraphs and PSG.
METHODS: Original actigraphy data were amplified and their pattern was estimated using a penalized spline. The magnitude of amplification and the spline were estimated by minimizing the difference in sleep efficiency between wrist- (hip-) actigraphs and PSG recordings. Sleep measures using both the original and spline-modified actigraphy data were compared to PSG using the following: mean sleep summary measures; Spearman rank-order correlations of summary measures; percent of minute-by-minute agreement; sensitivity and specificity; and Bland-Altman plots.
RESULTS: The original wrist actigraphy data showed modest correspondence with PSG, and much less correspondence was found between hip actigraphy and PSG. The spline-modified wrist actigraphy produced better approximations of interclass correlations, sensitivity, and mean sleep summary measures relative to PSG than the original wrist actigraphy data. The spline-modified hip actigraphy provided improved correspondence, but sleep measures were still not representative of PSG. DISCUSSION: The results indicate that with some refinement, the spline regression method has the potential to improve sleep estimates obtained using wrist actigraphy.

Introduction

Physical activity and sleep are both recognized as important health determinants and represent critical targets for chronic disease prevention. A better understanding of the roles that these factors play in health and disease has been facilitated by the development and implementation of ambulatory, accelerometer-based monitoring devices [1-5]. Actigraphy has helped establish linkages between sleep disruption or reduced physical activity and various adverse health outcomes ranging from metabolic syndrome measures (e.g., obesity, hypertension), to increased rates of chronic disease and elevated mortality risk [6-13]. Refinements in actigraphy data processing and analysis may help to improve assessments of sleep and physical activity for use in disease prevention efforts.

Characterization of sleep via wrist actigraphy has gained popularity in clinical and research settings as an alternative to polysomnography (PSG). Though considered the “gold standard” for sleep assessment, PSG can be costly, labor-intensive, and invasive. Also, it typically involves sleeping in a novel environment and only can be reasonably implemented for 1–2 nights at a time [1-5]. Advantages of actigraphy include its low cost, convenience, and an ability to objectively estimate sleep in large populations for periods up to months at a time. However, some differences among actigraphs, including modality of quantifying movement, sampling frequency, and sensitivity of movement detection can influence their accuracy in estimating sleep. In addition, some sleep actigraphy devices lack documentation of their validity relative to PSG. With increasing interest in simultaneous ambulatory monitoring of sleep and physical activity, an unresolved question is whether data collected using hip-mounted actigraphs (typically used solely for physical activity monitoring) also can provide valid estimates of sleep [1-5,29]. If so, this would allow for reduction in cost and subject burden in studies involving both measures.

This investigation compared the accuracy of sleep metrics obtained via wrist- and hip-mounted MTI® actigraphs (Manufacturing Technology, Inc., ActiGraph, Pensacola, FL) with those derived from PSG recordings in a convenience sample of individuals attending a local clinic for sleep evaluation via PSG. The MTI® actigraph has been validated and used to characterize physical activity [14-16]. However, to our knowledge, this monitor had not been validated for sleep. A similar actigraph device has recently been used to assess physical activity and sleep in a nationally representative sample of the United States population [17].

Another study objective was to use spline regression as a novel sleep characterization methodology to improve both wrist- and hip-mounted actigraphic data relative to PSG-defined sleep. Spline regression is a useful mathematical technique for modeling complex nonlinear processes. It has been applied to accelerometer data to estimate energy expenditure and other physical activity measures, although it has not been used previously to characterize sleep [18,19].

Material and methods

The study sample consisted of patients attending a local sleep clinic for various sleep complaints (SleepMed, Columbia, SC) as well as presumed normal sleepers. Participants were given a physical examination and interview by a board-certified sleep physician [20]. All participants had been previously scheduled for an overnight PSG recording session. Clinical staff informed prospective participants of the present study and participants provided written informed consent per University of South Carolina Institutional Review Board (IRB) approval.

A standardized PSG protocol was implemented by clinic staff. Participants arrived at the clinic sixty minutes before their self-reported customary bedtime and were prepared for PSG recording ~30 minutes before initiating their sleep period. Participants completed a single night of PSG recording using Alice 4 PSG instrumentation operated by a certified PSG technician. Demographic and anthropometric information was obtained from clinical records (age, sex, body mass index [BMI=weight(kg)/height(m)2]). Participants also completed the Epworth Sleepiness Scale (ESS) [22].

Each participant was equipped with two actigraphs (MTI® model 7164 accelerometer, ActiGraph, Pensacola, FL); one was worn on the non-dominant wrist, and the other was affixed to the hip. The actigraphs were initialized prior to recording using the same computer that was used for PSG recording so that the actigraphs were synchronized to the internal timing of the computer, thus allowing precise and accurate verification of the beginning and end of each recording period. Moreover, synchronization of the initiation of the PSG and actigraphic recordings also was performed each night with manually activated event markers on the computer and actigraph. At each participant׳s customary bedtime, which varied from 9 pm to 1 am, lights were turned out and participants were asked to attempt to sleep as desired, for up to a maximum of 8 h.

Recordings that included at least 6 h of complete PSG as well as wrist and hip actigraphy data were included in the analysis. PSG data were initially scored using the automated Morpheus® software (Widemed ltd., Tel Aviv, Israel) in 30-s epochs. These data were then manually edited according to standardized criteria by a registered PSG technologist [21,23]. In order to compare PSG scoring with MTI sleep estimates, which are scored in 60-s epochs, PSG recordings from every other 30-sec epoch were synchronized to actigraphy data and scored for the presence of wake or sleep, without consideration of sleep stage. Ancillary analyses based on sensitivity, specificity, and minute-by-minute agreement indicated that differences between this approach and alternative data matching strategies within a given one minute interval were negligible. These strategies and their results are addressed in greater detail in the discussion section. Actigraph epochs were scored for sleep using the Cole–Kripke algorithm [24], which was incorporated into the manufacturer׳s software. The MTI ActiLife5© software used for this purpose applied a combination of regression parameters as follows:where D<1 indicates being asleep, D≥1 denotes being awake, P is a scale factor for the entire equation, W0, W−4, W−3, W−2, W−1, W+1 and W+2 are weighting factors for present (0), previous (−), and subsequent minutes (+), and A0, A−4, A−3, A−2,A−1, A+1, A+2 are activity scores for the corresponding present (0), previous (−), and subsequent minutes (+). For instance, A−4 represents activity scores four time units before the present time and W−4 is the associated weighting factor for A−4. The resulting algorithm in ActiLife5© iswhere activity scores were constrained not to exceed a maximum of 300. This means, for example, that if an epoch had an activity score of 450 a score of 300 was used.

Alice 4 and ActiLife 5 software provided summary PSG and actigraphy sleep statistics, respectively, for each night of recording, including: time-in-bed (TIB, number of minutes from lights out to lights on); sleep onset latency (number of minutes from lights out until the first epoch of recorded sleep); total sleep time (TST, number of minutes scored as sleep during time-in-bed); sleep efficiency ([TST/TIB]×100); wake after sleep onset (WASO, number of minutes scored as wake after sleep onset); and number of awakenings (total number of transitions to wake from sleep).

We attempted to develop a novel method to improve actigraphic sleep estimates by applying a smoothing spline to actigraph activity data using PSG as the reference. The penalized smoothing spline fits a non-linear curve to discretely observed data while maintaining the pattern of the original data. The penalty parameter controls how curvilinear (or smooth) the curve can be. By modeling sleep actigraphy data as continuous rather than discrete time points, our intent was to estimate an actigraphic time series that more closely approximated the pattern of sleep–wakefulness recorded via PSG. This was performed in two stages. First, an overall adjustment to the original activity amplitude was implemented because the magnitude of wrist and hip actigraphy data can be low, which would generate false negative assignment of sleep using the algorithm denoted above. Second, the temporal pattern of the adjusted activity was estimated via a penalized cubic spline, where its smoothness was controlled by a penalty parameter. The inferred non-linear curve was then used to estimate activity levels that were processed using the Cole–Kripke algorithm [24] found in the manufacturer׳s software to score each epoch as awake or asleep. The overall adjusting magnitude and penalty parameter were selected such that their combination minimized the difference in sleep efficiency between the actigraphy and PSG data. Scoring for sleep–wake using the predicted activity values of wrist and hip actigraphy data was conducted in the same manner as the original actigraphy data using the Cole–Kripke algorithm included within the manufacturer׳s software (ActiLife 5).

Statistical comparisons between PSG and actigraphy measures obtained from either the original or spline-modified data were performed using several methods: (1) Sleep summary measures obtained from the wrist and hip actigraphy recordings were compared with mean PSG values using a nonparametric Wilcoxon rank sum test applied to differences. (2) Associations of PSG with actigraphy summary data were further assessed using Spearman correlations. (3) Minute-by-minute agreement of PSG with actigraphy was determined by serially evaluating concordant or discordant epochs of PSG and actigraphy. The total number of concordant epochs were divided by TIB then multiplied by 100 to obtain the percent agreement and then averaged among all subjects. (4) Sensitivity of actigraphy scoring was calculated for each participant as the percent of PSG-scored sleep epochs that also were scored as sleep by the actigraph; specificity was calculated as the percentage of PSG-scored wake epochs that also were scored as “awake” by the MTI software. Individual sensitivity and specificity data were then averaged across all subjects. (5) Bland–Altman plots were used to provide a visual summary of the agreement between PSG and other sleep summary measures. These graphs were generated by plotting the mean of the two sleep measures (e.g., spline-modified sleep efficiency and PSG sleep efficiency) against the difference between the actigraphy and referent (PSG) sleep efficiencies, along with the 95% limit of agreement for the mean differences, and the intraclass correlation coefficient (ICC) (BA.plot function, R MethComp package, http://cran.r-project.org/web/packages/MethComp/MethComp.pdf). All analyses were performed using the R Gui© software program (The R Foundation for Statistical Computing). The statistical significance level was set at α=0.05.

Results

Analyses were performed among the 54 qualified participants who had at least 6 h of PSG sleep and data from both actigraph monitors (54/84; 64%). The average age (± standard deviation) of the study population was 51±13 years, including 29 (54%) women, 20 men (37%) for whom sex data were recorded. Subjects were given a presumptive diagnosis based on their initial assessment and self-reported symptoms which may or may not have been consistent with their final diagnosis. Many patients reported more than one symptom (e.g., insomnia or hypersomnia with sleep apnea). Based on their presumptive diagnosis, the sample consisted of 32 patients with primary symptoms of obstructive sleep apnea, 11 with insomnia, 1 with narcolepsy, and 10 normal or undiagnosed sleepers. Table 1 summarizes demographic characteristics of the participants.

Table 1

Study population descriptive characteristics.

Variable	Minimum	Median	Mean±SD	Maximum
Age	21	51	51±12	74
BMI	21	33	35±9	60
Epworth	0	11	12±7	6
AHI	0.7	12.8	19±18	7

Epworth sleepiness scale score (n=7 missing), AHI – apnea hypopnea index (n=6 missing). SD – Standard deviation. BMI – body mass index (kg/m2).

Fig. 1 presents an example of the spline-modified wrist actigraphy data (lower panel) relative to the original (middle panel) and PSG-defined sleep and wake (upper panel) for a participant׳s single night sleep record (An example of hip actigraphy data is presented in the supplemental appendix Fig. S1.) The estimated overall adjusting magnitude and penalty parameter that minimized the sleep efficiency between actigraphy and PSG for the wrist data was 300 and 0.00025, respectively, and for the hip data 400 and 0.00025, respectively. Note that in this example, wakefulness (white background) for the spline-modified sleep scoring procedure coincides more consistently with PSG sleep–wake scores relative to the original actigraphy data, and that sleep efficiency using spline-modified values more closely approximates PSG sleep efficiency.

Fig. 1

Comparison of PSG sleep relative to original and spline-modified wrist actigraphy. Top panel presents polysomnographic sleep scores (gray: asleep, white: awake). Middle panel presents sleep scores overlaid with original activity scores from wrist actigraphy (black line). Bottom panel presents spline-modified sleep scores overlaid with predicted wrist activity values from inflation and penalized spline (black line).

Sleep summary measures for PSG, original wrist and hip actigraphy, and spline-modified wrist and hip actigraphy data are presented in Table 2. Compared to the statistics using spline-modified data, summary statistics based on original wrist actigraphy data were farther from the statistics for PSG. The spline-modified wrist actigraphy means for sleep efficiency, TST, WASO, and number of awakenings were not statistically different from PSG data. The spline-modified latency was statistically lower (p<0.001) than the PSG data, but it is still closer to the PSG value than if not modified. The mean spline-modified hip actigraphy data corresponded more closely with PSG than the original hip data, although the summary measures were statistically different from the PSG measures (all p<0.01, Table 2).

Table 2

Mean sleep summary measures by data collection method (n=54).

Variable	PSG	Original wrist	Spline-modified wrist	Original hip	Spline-modified hip
TST (min)	334±69	396±67⁎	353±76	452±46⁎	410±56⁎
Sleep efficiency (%)	73±14	86±1⁎	77±14	99±2⁎	89±8⁎
WASO (min)	94±52	58±43⁎	98±56	6±7⁎	48±35⁎
Latency (min)	30±33	4±7⁎	8±15⁎	0±1⁎	1±2⁎
No. awakenings	21±7	13±7⁎	21±8	3±3⁎	16±7⁎

PSG: Polysomnography, TST: total sleep time, WASO: wake after sleep onset.

p-Value≤0.01; the tests compared PSG to other data collection methods using Wilcoxon rank-sum tests.

Spearman rank-order correlations between PSG and actigraphy measures are presented in Table 3. Positive correlations were noted between PSG and both wrist actigraphy sleep summary measures (original: r=0.24–0.53, all p<0.05 and spline-modified: r=0.29–0.56, all p<0.05). In general, the correlations with PSG measures were improved or comparable when using spline-modified data except for WASO and number of awakenings. Across all epochs, the minute-by-minute agreement between PSG sleep scores and those obtained using the original wrist actigraphy data was 81% (± standard deviation: 10%). The spine-modified wrist actigraphy data produced a similar agreement with PSG (81%±9%). For the hip actigraphy data, spline-modified hip actigraphy recordings produced higher agreement (78%±12%) than the original data (74%±14%).

Table 3

Correlation of actigraphic sleep summary measures with PSG (n=54)a.

Variable	Original wrist	Spline-modified wrist	Original hip	Spline-modified hip
TST	0.53 (<0.001)	0.56 (<0.001)	0.52 (<0.001)	0.45 (<0.001)
Sleep efficiency	0.41 (0.002)	0.42 (0.002)	0.22 (0.105)	0.24 (0.079)
WASO	0.39 (0.004)	0.29 (0.031)	0.05 (0.699)	0.02 (0.875)
Latency	0.24 (0.077)	0.31 (0.023)	0.20 (0.148)	0.26 (0.057)
No. awakenings	0.45 (0.001)	0.32 (0.017)	0.03 (0.828)	0.23 (0.091)

PSG: Polysomnography, TST: total sleep time, WASO: wake after sleep onset.

Spearman rank correlation coefficient (p-value).

Across all epochs, the minute-by-minute sensitivity and specificity of the original wrist actigraphy data to detect PSG-defined sleep or wake was 96% (±5%) and 41% (±23%), respectively. The spline-modified wrist actigraphy data generated a much improved specificity (59%±23%) to detect wakefulness, while the sensitivity was still reasonably high (89%±9%). For hip actigraphy, the sensitivity and specificity of the original values relative to PSG were both <0.01% and the corresponding values for spline-modified hip actigraphy were both improved, 0.21% (±5%) and 0.05% (±19%), respectively.

Bland–Altman plots of PSG sleep relative to original and spline-modified wrist actigraphy are presented in Fig. 2 along with the corresponding ICCs. Similarities between PSG and the sleep scores are observed if there are small differences between means that cluster near the horizontal line, indicating small differences, and by moderate to strong ICCs. Dissimilar results produce larger differences that are typically outside the 95% limit of agreement. Relative to the original wrist actigraphy data, the spline-modified wrist data generated sleep efficiencies that were closer to those obtained using PSG. The original wrist data (Fig. 2 top panel) tended to have a less symmetric distribution around zero with more positive differences, whereas the spline-modified wrist actigraphy measures tended to have a more symmetric distribution around zero (Fig. 2 bottom panel). The ICC for sleep efficiency between PSG and the predicted wrist data (0.47) also was higher than the corresponding ICC between PSG and the original wrist data (0.29).

Fig. 2

Bland–Altman plots of mean spline-modified wrist sleep efficiency versus the difference between the original (top) or spline-modified (bottom) wrist actigraphy and polysomnographic (PSG) sleep efficiencies. (—) mean of the differences, (⋯) horizontal marker at zero, (.) point difference original PSG sleep efficiency and wrist actigraphy sleep efficiency, (- - -) UAL: Upper 95% agreement limit and LAL: Lower 95% agreement limit.

Bland–Altman plots for both the original (Fig. S2 top panel) and spline-modified hip actigraphy (Fig. S2 bottom panel) data had positive differences and similar averages, indicating that both measures had estimated sleep efficiencies greater than PSG and that they were missing assignment of wakefulness epochs captured by PSG. Hip actigraphy from the original data had a distinct negative linear relationship, indicating that differences between PSG and original hip data decreased as mean sleep efficiency increased. Sleep efficiencies from both the original and spline-modified hip actigraphy data had low ICCs in relation to PSG (0.01 and 0.09, respectively).

Discussion

Characterization of sleep via wrist actigraphy has gained popularity in clinical and research settings, and has helped advance the understanding of how sleep disruption can affect the incidence or mortality of various diseases including depression, obesity, hypertension, cardiovascular disease and cancer [6-13]. Actigraphy is more cost-efficient and can be used to collect data over many consecutive nights while also being less disruptive to natural sleep than is PSG. Actigraphy has been used in some cases to help establish diagnoses of insomnia [25] or circadian rhythm sleep disorders [26]; however, clinical consensus is that a full-night PSG exam is required for establishing a sleep disorder diagnosis.

In this study, the original wrist actigraphy data for this previously non-validated monitor had only modest correspondence with PSG, and the correspondence of hip actigraphy data with PSG was unsatisfactory. Several of the average sleep measures were statistically different between PSG and actigraphy; correlations between these data were low; and minute-by-minute agreement was modest. These correspondences were lower than other published comparisons. For example, minute-to-minute agreement between wrist actigraphy and PSG was 86–95% in studies of other actigraphy devices [24,27,28], whereas it was 80% in the present study. For most sleep summary measure comparisons, the spline-modified sleep measures obtained using wrist-mounted actigraphs produced better agreement with PSG-defined sleep than the data summarized using the standard method for the original wrist-mounted data.

Despite its practical appeal, a key issue is how well actigraphic data approximates patterns of PSG-defined sleep, and how this agreement can be maximized. By modeling sleep using spline regression, this study sought to more closely approximate patterns of sleep–wakefulness recorded via PSG. Spline regression combined with inflation proved useful for these purposes because it enhanced both the amplitude and duration of activity during sleep, which provided a better opportunity to capture bouts of wakefulness. Compared with the original summary measures, the spline-modified data produced fewer statistically significant differences between PSG and wrist-derived sleep summary measures, a higher ICC, and improved specificity to detect PSG-defined wakefulness. Moreover, the Bland–Altman plot comparing spline-modified wrist actigraphy with PSG indicated that this approach generated sleep efficiencies that matched PSG more closely than the original wrist actigraphy data. On the other hand, spline-modified wrist actigraphy measures did not differ appreciably from the original data when minute-to-minute agreement, sensitivity, or correlations with WASO or number of awakenings were considered.

There were several noteworthy limitations or uncertainties in this study. Reductions in sample size that occurred due to missing data may have reduced statistical power and representativeness of data used in the analyses. The sleep scoring algorithm implemented in the MTI software was originally developed by Cole et al. [24] for a different actigraphy device and was not calibrated to maximize its correspondence with PSG measures, which may have contributed to its reduced correspondence with PSG in the present study.

Another potential limitation was that PSG readings were measured every 30 s while actigraphic data were measured every minute, and the analyses were based on the use of every other PSG epoch in order to match the data collection scheme to the actigraphy readings. To assess the possibility that this influenced the results, supplementary analyses were conducted using three different scenarios to evaluate whether alternative methods for selection of PSG epochs within a one-minute time frame would alter correspondence between actigraphic and PSG data. First, the PSG sleep score between two consecutive 30-s epochs was randomly chosen. In the second scenario, the one-minute interval was scored as “awake” if one of the 30-s PSG epochs was scored as awake. In the third scenario, the one-minute interval was scored for sleep if one of the 30-s PSG epochs was scored as “asleep”. For each scenario, the average minute-by-minute agreement, sensitivity, and specificity were compared, and differences in average agreement among these scenarios were negligible (<1%) relative to the ‘every other PSG׳ method used in the analysis, which indicates that the use of every other PSG epoch did not bias the analysis.

In the present study, hip actigraphy data corresponded poorly with PSG by all measures evaluated. This was likely due to less hip movement compared with the wrist, which is consistent with recent findings by Hjorth et al. [29], who reported low specificity and overestimation of sleep when waist-worn actigraphy data were examined. On the other hand, Enomoto et al. [28] and Paavonen et al. [27] found that hip actigraphs produced statistically similar results to PSG or wrist actigraphy measures. Similar to our objective, Enomoto et al. [28] aimed to create an algorithm that improved the sleep scores for wrist actigraphy (Lifecorder PLUS, LC; Suzuken Co. Ltd., Nagoya, Japan), whereas Paavonen et al. [27] applied wrist and hip mounted Mini-MotionLogger® actigraphs (Mini-MotionLogger, Ambulatory Monitoring, Inc., Ardsley, NY) to assess which location could best describe the sleep habits of children. Inconsistencies between results obtained for hip measures in the present study and in Hjorth et al. [29] and those reported previously in Enomoto et al. [28] and Paavonen et al. [27] could be due, in part, to differences among actigraphic devices, or among the populations studied.

In previous studies, correspondence between actigraphy and PSG tended to be better among participants who were normal sleepers relative to those with sleep disorders or other medical conditions [4,30,31]. Spline-modified wrist actigraphy data in the present study modestly improved the ability to detect wakefulness, a key element of actigraphic sleep characterization, yielding a specificity of 59% relative to a specificity of 41% for the original wrist actigraphy data. The study population was comprised of individuals attending a sleep clinic, most of whom had presumed sleep disorders. Our results suggest that spline-modified sleep scores may help improve the use of wrist actigraphy for sleep characterization among those with clinically referable sleep disruption.

Further refinement of the spline regression sleep estimation procedures used in this study may provide more favorable agreement with PSG scores. Examination of the original actigraphy data indicated that peak activity did not necessarily correspond with median wakefulness episodes recorded via PSG. Once inflation and spline regression were applied, the bouts of wakefulness for the actigraphy were slightly shifted compared to those from PSG. An additional factor that would adjust for these shifts when implementing spline regression may enhance the ability of this method to evaluate actigraphy data from different devices and manufacturers. Implementation of spline regression separately among patients with different types of sleep disorders may provide some additional benefit.