Exploring Accelerometer Versus Self-Report Sleep Assessment in Youth With Concussion.

This study examines accelerometer-based and self-report assessment of sleep disturbance from a larger prospective cohort of youth 5 to 18 years of age with postconcussive injury. Twenty-one participants with self-reported sleep disturbance were evaluated using accelerometers. Participants completed the Pittsburgh Sleep Quality Index (PSQI) every 48 hours and also measured sleep via accelerometry. Correlations were conducted matching PSQI scores to accelerometry assessment. PSQI scores were significantly correlated only with "average number of awakenings" (r = -0.21; P = .049). Accelerometer-measured mean (standard deviation) sleep efficiency was 79.9% (5.20%), with normal sleep defined as >85%. The mean (standard deviation) PSQI global score was 10.5 (3.78) out of 21, where scores of >5 indicate subjective insomnia. Results suggest the PSQI and accelerometers may be measuring different attributes of sleep. Both may be needed as actual sleep is important but so is perception of good sleep. These findings call for further validity testing of objective sleep assessment measures and commonly used self-report tools.

Introduction

Poor sleep quality and its effects on cognitive function are a concern for adolescents.[1,2] Sleep disturbance in early adolescence is associated with compromised emotional processing during this critical period of socioemotional development.[3] Sleepiness, defined as difficulty in maintaining a desired level of wakefulness,[4] can negatively affect academic performance. Youth who report sleep disturbances have been shown to perform poorly in school.[5] Studies have confirmed this by indicating that students with more regular sleep-wake patterns reported higher academic standing, and students with lower grades reported increased daytime sleepiness.[6,7] There are indications that learning and attention skills,[8,9] daytime functioning,[10] and concentration and memory[11] are compromised by insufficient sleep. These youth might be criticized and mistaken for being lazy by their peers.[11] Given the significant effect lack of sleep has on adolescents without injury, it is likely to have an even more profound effect on youth dealing with a concussive injury.

Concussions, also known as mild traumatic brain injuries, are common in youth. A population-based study of pediatric emergency department and office visits for concussions in Ontario between 2003 and 2010 reported a total of 88 688 concussions in youth 3 to 18 years of age.[12] In addition, the same study found that the rate per 100 000 doubled from 340.5 in 2003 to 601.3 in 2010.[12] These numbers are concerning because of the number of studies that have shown that sustaining a concussion can interfere with an individual’s performance at home, school, and community activities.[13,14] A significant number of individuals report postconcussion symptoms for months or even years after their injury.[15,16] These symptoms can be categorized into 4 main areas: somatic, cognitive, mood, and sleep-related symptoms.[17] These symptoms include complaints of headache, dizziness, fatigue, depressed or anxious mood, sleep disturbance, light sensitivity, forgetfulness, and concentration difficulties.[18]

Sleep disturbance is more common in individuals with concussion than noninjured controls or those with severe traumatic brain injuries.[19,20] Sleep disturbances following a concussion affect 30% to 70% of people.[21] In a randomized controlled trial, Towns et al found that 92% of 18- to 55-year-old individuals with a concussion reported poor sleep quality.[22] A prospective cohort study investigating subjective and objective sleep in adolescents found that sleep disturbances could be experienced up to a year following concussion.[23] A previous cross-sectional study associated poor sleep quality in adolescents with a concussion and impairments in visual memory, visual motor speed, and reaction time.[24] Moreover, in adults with a concussion and other adolescents without a concussion, sleep disturbances and poor sleep hygiene have been reported to predict depression and other psychiatric disorders.[20,25]

Unfortunately, it can be difficult to measure self-perceived sleep quality, as there is no generally accepted gold standard.[26] Polysomnography records various functions during sleep, such as brain waves, blood oxygen levels, heart rate, breathing, and eye and leg movements. Laboratory-based polysomnography is the gold standard for objectively measured sleep; however, it is not feasible in most studies, since it cannot be done at home and cannot be used long-term.[27] More realistically, a questionnaire that incorporates the recommendations from the American Psychological Association pertaining to clinical sleep dysfunction evaluation should be used to evaluate sleep.[28] This ideal screening instrument would be able to distinguish between “good” and “poor” sleepers.[29] The Pittsburgh Sleep Quality Index (PSQI)[30] is able to make this distinction and is the most commonly used generic measure in clinical and research settings.[29] It has been reported that the PSQI demonstrates good convergent and divergent validity, moderate reliability, and adequate psychometric properties for use in clinical trials involving young adults.[31] A systematic review by Mollayeva et al[29] suggests that the PSQI requires further evaluation as a predictive and evaluative tool of sleep quality. Moreover, the PSQI has been previously used to assess sleep quality in individuals with a concussion.[19,22]

It is difficult to provide accurate estimates of specific sleep problems in youth with self-reported measures.[32] Researchers have now started using wrist-worn accelerometers for the measurement of sleep in young adults[33] and adolescents with mild head injuries.[34] Increasingly, there have been studies exploring these objective tools as a supplement to self-reported outcomes in individuals with concussions.[35] The use of accelerometers to record movement is both noninvasive and objective.[36] The use of accelerometry in youth with concussive injury is promising because of its potential to measure sleep and physical activity with a single device, thereby efficiently obtaining information about well-being and health behaviors.[36]

As the dilemma of comparing sleep assessment modalities to measure sleep disturbance in youth postconcussive injury remains, this study aimed to answer the main research question: In youth diagnosed with a concussion, how does accelerometer-based sleep data compare to self-reported sleep disturbance outcomes measured with the PSQI? We hypothesized that accelerometer data will reflect a sleep disturbance if youth with a concussion are reporting sleep disturbance on the PSQI. This study specifically compares accelerometer-based measures with the validated PSQI measure in a sample of youth with concussion and self-reported sleep disturbance. We will also explore the demographic differences between study participants who did and did not report a sleep disturbance in order to better understand our sample.

Methods

Participants

The sample was part of the ongoing Safely Returning Children and Youth to Activity after Concussion study. This larger prospective cohort included youth aged 5 to 18 years who had experienced a concussion within the previous month and who were still symptomatic at the time of study enrolment. Inclusion criteria for data analysis in the sleep disturbance (SD) group were the following: (1) youth were aged 5 to 18 years, (2) reported sleep disturbances on the Post-Concussion Symptom Scale (PCSS), (3) had complete PSQI data available, and (4) had accelerometer data available for 2 consecutive days prior to each PSQI score.

Updated data for 105 participants were extracted from the larger sample in March 2017. Participants who dropped out or did not have complete PCSS or PSQI data were excluded. The participants were then divided into 2 groups, those who reported sleep disturbance and those who did not, no sleep disturbance (ND). This was determined based on their response to “Complaints of sleep problems” on the PCSS, a component of the study’s 48-hour survey. Participants were automatically prompted to fill out the PSQI every 48 hours if they indicate a number higher than 0 on a scale from 0 to 6 when rating their sleep problems on the PCSS. We used the child-reported PCSS instead of the parent-report since previous research suggests that parents significantly underestimate sleep disturbance in their children.[37]

From the SD group, participants were excluded for not wearing the accelerometer or for incomplete PSQI, which made it impossible to calculate a global score. The final number in the SD group was n = 29, with n = 47 in the ND group. Of the 29 participants in the SD sample, only 21 had both complete PSQI and corresponding 48-hour accelerometer data. This reduced the sample size in the data analysis to n = 21 who had a total of 86 PSQI scores, with each of the 21 participants contributing between 1 and 22 eligible scores. Figure 1 illustrates the inclusion and exclusion criteria in this sample.

Figure 1.

The inclusion and exclusion criteria in this sample for the sleep disturbance (SD) and no sleep disturbance (ND) groups, as well as the group for whom Pittsburgh Sleep Quality Index (PSQI) and matched accelerometer data were available in the final data analysis (n = 21).

Procedures

Pittsburgh Sleep Quality Index

The PSQI is a 19-item self-reported questionnaire that assesses sleep quality for the preceding month, and it is the only standardized clinical instrument that broadly covers relevant sleep quality indicators.[29,30] It was developed using adults to (1) provide a reliable, valid, standardized measure of sleep quality; (2) discriminate “good” and “poor” sleepers; (3) provide an easy index for patients to complete and for clinicians and researchers to interpret.[30] This measure has been validated in adolescents.[31] The majority of studies in younger and older adults report Cronbach α values of 0.70 to 0.83, which is above the 0.70 cutoff for within- and between-group comparisons.[29] It can be noted that the PSQI has also been used to track sleep disturbance in adolescents,[38] and it has demonstrated validity in assessing sleep disturbance after concussion in individuals 16 to 78 years old.[39] For the purposes of this study, the questionnaire was modified to ask about sleep quality for the preceding 48 hours. Other studies have also previously altered the PSQI to measure shorter time periods,[40,41] and researches have justified this by noting that more recent events might be easier to remember and less impacted by recall biases.[42,43]

Responses to PSQI questions are grouped into 7 equally weighted component scores, on a 0 to 3 scale, that are summed to yield a global PSQI score. The global score ranges from 0 to 21, and the higher the score, the worse the sleep quality. The 7 components are subjective sleep quality, sleep latency, sleep duration, habitual sleep efficiency, sleep disturbances, use of sleep medications, and day time dysfunction.[30] A global score of >5 indicates subjective insomnia,[30] and it is a sensitive and specific measure of poor sleep quality.[44] This cutoff has been used to investigate clinically significant sleep problems in adolescents and young adults after concussion.[45]

Accelerometry

ActiGraph GT3X wrist monitors (ActiGraph LLC, Pensacola, FL) have been previously determined to be both valid and reliable for measuring sleep/wake periods in 60-second epochs (time unit) in young adults and in children.[36,46] This monitor records accelerations in 3 planes of motion (vertical, anteroposterior, and mediolateral), which are used to determine periods of sleep or wakefulness. The Sadeh algorithm, which has been previously validated in a young population of 10 to 25 year olds,[47] was used to quantify sleep and wakefulness. Moreover, accelerometer variables were normalized by either accounting for the actual amount of time spent in bed, or spent asleep, according to the ActiLife software. The study focused on the following variables: (1) counts per hour of time in bed; (2) efficiency, defined as the number of hours slept over the number of hours spent in bed multiplied by 100 (%); (3) average awake time (minutes) per hour of sleep; and (4) number of awakenings per hour of sleep. Counts are based on the frequency and intensity of the acceleration at each 60-second epoch and are calculated by the ActiLife software. One individual was responsible for cleaning and scoring all sleep data for the selected participants.

Analyses

Descriptive statistics were calculated for the SD and ND groups, in order to ensure that the 2 groups were similar in gender, age, previous number of concussions, mechanism of current injury, type of sport, prior diagnosis of sleep disorder, and symptoms of concussion in the last 48 hours. The groups were compared using χ2 and t test comparisons depending on the variable.

Pearson correlations were conducted between selected PSQI global scores and each of the 4 key accelerometer variables. It had been previously determined that 4 days is sufficient to obtain acceptable estimates.[46] However, the objective of this study was to directly compare the 48 hours of self-reported sleep on the PSQI with the same 48 hours of accelerometer data. Therefore, the accelerometer data on 2 consecutive days were averaged and subsequently correlated to the PSQI global score. IBM SPSS Statistics Version 20 (IBM Corp, Armonk, NY) was used for all analyses. All data are reported as mean (standard deviation) unless otherwise specified. Statistical significance was set at P ≤ .05.

Results

Descriptive Analysis

The SD and ND groups were similar on all key demographic variables, except for gender (P = .034), age (P = .018), and symptoms of concussion in the last 48 hours (P = .001), as seen in Table 1.

Table 1.

Descriptive Comparison Results Between No Sleep Disturbance (ND) and Sleep Disturbance (SD) Groups.

Variable	ND Group (n = 47)	SD Group (n = 29)	P
Gender, n (%)			.034*
Male	28 (59.6)	10 (34.5)
Female	19 (40.4)	19 (65.5)
Age (years), mean (standard deviation)	12.4 (3.0)	14.0 (2.3)	.018*
Median number of previous concussions[a]	0; 37 (78.7%)	0; 19 (65.5%)	.508
Most common mechanism of injury	Sports or recreational play; 38 (80.9%)	Sports or recreational play; 20 (69.0%)	.259
Most common type of sport associated with injury	Hockey; 15 (39.5%)	Recreational play (gym, recess); 6 (30.0%)	.472
Prior diagnosis of sleep disorder	1 (2.1%)	0 (0.0%)	.429
Mean (standard deviation) % of “yes” answers on “symptoms of concussion in the last 48 hours”[b]	31.6% (32.5)	59.3% (34.1)	.001*

Abbreviation: PSQI, Pittsburgh Sleep Quality Index.

This variable is reported as a median instead of a mean in order to reflect the proportions used in the χ[2] analysis and to account for outliers.

Although 31.6% in the ND group reported having symptoms, these participants did not fill out a PSQI and were ineligible for analyses.

P < .05.

Descriptive statistics for the 21 SD participants that remained with complete PSQI and accelerometer data can be found in Table 2. From the 21 participants, there were a total of 86 PSQI scores obtained from 48-hour surveys that had 2 consecutive days of accelerometry data. There are 86 PSQI scores because each of the selected participants contributed 1 to 22 PSQI scores, with a mean of 4.1 PSQI scores each. Out of the 86 PSQI scores, 86.0% (n = 74) were above the >5 cutoff, indicating subjective insomnia. Only 14.3% (n = 3) of the 21 participants in the SD group never reported a PSQI score greater than 5. The mean scores and standard deviations from accelerometry were the following: counts/h of time in bed, 7293 (3413); sleep efficiency, 89.9% (5.20%); average awake time, 15.1 (4.84) min/h of sleep; and number of awakenings, 3.49 (0.81) per hour of sleep. In terms of sleep efficiency, it can be noted that “normal” sleep efficiency is defined as >85% according to the DSM-IV-TR (Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition–Text Revision).[28]

Table 2.

Descriptive Sleep Data From 21 Participants for Whom Pittsburgh Sleep Quality Index (PSQI) and Matched Accelerometer Data Were Available.

	Mean (Standard Deviation)
Gender (%)	Female (76); male (24)
Age (years)	14.0 (2.60)
PSQI score (out of 21)	10.5 (3.78)
Counts/h of time in bed[a]	7293 (3413)
Sleep efficiency (%)[b]	79.9 (5.20)
Average awake time (min/h of sleep)	15.1 (4.84)
Number of awakenings/h of sleep	3.49 (0.81)

Counts are based on the frequency and intensity of the acceleration at each 60-second epoch and are calculated by the ActiLife software.

Efficiency (%) = (Number of hours slept/number of hours spent in bed) × 100 = Habitual sleep efficiency (%).

Correlations

The results of the correlation analysis using the mean score for each of the 4 selected and normalized accelerometer variables on 2 matched days versus each of their respective PSQI global scores can be found in Figure 2. There were no statistically significant correlations between each PSQI global score and the following variables: counts per hour of time in bed, r = −0.08 (P = .451); efficiency, r = 0.17 (P = .123); and average awake time in minutes per hour of sleep, r = −0.16 (P = .151). The only statistically significant correlation was for number of awakenings per hour of sleep, r = −0.21 (P = .049).

Figure 2.

Correlation analysis using the average score of 4 selected and normalized accelerometer variables versus the respective PSQI global score. N = 21 (8 participants excluded due to missing data).

Discussion

This study compared self-reported sleep disturbance (PSQI) with matched accelerometer recording of sleep in youth postconcussive injury. Overall, 44 out of 91 (48.4%) participants reported a sleep disturbance at some point after their concussion (see Figure 1). There were proportionally more females (65.5%) in the SD group than in the ND group (40.4%), and this result was statistically significant (P = .034). A possible explanation for this is that young adult females report insufficient sleep more frequently than males.[48] The mean age for the ND group was 12.4 (3.0) years, and the mean age for the SD group was 14.0 (2.3) years. The difference in means was statistically significant (P = .018). Older children postconcussion may actually experience more sleep disturbances, or simply perceive that they do. This is supported by a cross-sectional study that found older college-age athletes postconcussion reported more sleep symptoms than younger high school athletes postconcussion.[49] There was no significant difference with regard to previous number of concussions, or mechanism of injury between the SD and ND groups. The most common mechanism of injury was a sports or recreational play injury. Compromised sleep has been associated with an increased risk of general sports injuries in adolescent athletes.[50] This is particularly concerning, since sports-related concussions are the most common mechanism of concussive injury in youth,[51] and this suggests that youth who are experiencing postconcussion sleep symptoms might be more susceptible to sustaining another debilitating concussion. Only one individual reported a prior diagnosis of sleep disorder, and interestingly, this individual was in the ND group, perhaps indicating that the concussion did not exacerbate a preexisting sleep problem. There was a statistically significant difference between the SD and ND groups for the “symptoms of concussion in the last 48 hours” variable (P = .001). This suggests that there might be a particular pattern or constellation of symptoms that are associated with sleep disturbance postconcussion in this sample. However, the large standard deviations in this sample for this variable should be noted.

A statistically significant but weak (r = −0.21) correlation between PSQI and the number of awakenings per hour of sleep was found. This negative correlation was actually in the opposite direction than hypothesized. The data suggest that self-reported poor sleep with a higher PSQI score was associated with fewer awakenings per hour of sleep according to accelerometry. This could indicate that the PSQI and the accelerometer are simply measuring 2 different things. For the other accelerometer variables, the correlations were minimal and not significant. A prospective controlled trial reported that self-assessments in patients experiencing sleep-wake disturbances are underestimated and unreliable.[52] Previous studies have also concluded zero to limited correlations between the PSQI with accelerometry data, but in samples of older black and white women, and a nonclinical sample of younger and older adults, both without concussion.[53,54] These studies note that depressive symptoms might be reflected in the PSQI rather than sleep complaints. Although these studies focus on different populations, this point is important when evaluating sleep in youth with concussion who have been reported to experience depressive symptoms postconcussion.[18]

It can be noted that the sixth component on the PSQI asks the participant about their use of sleep medication. Out of the 21 participants, 62% (n = 13) indicated using sleep medicine either prescribed or over the counter to help them sleep at least once in the last week. Taking sleep medication could have created a bias in how the participants answered the PSQI, since this may have influenced them into thinking they were sleeping better. This could be a reason for the lack of a significant correlation between the PSQI and actual sleep measured by accelerometry. Therefore, assessing these differences specifically with regard to sleep medication is warranted.

There are limitations in this study that are important to consider. First of all, it is difficult to obtain accurate estimates of specific sleep problems in youth.[32] It is possible that the PSQI as a self-report instrument may not be sensitive to sleep difficulties in younger people.[45] However, if parents are the ones responsible for reporting sleep symptoms, they tend to overestimate the sleeping hours of their children.[55] A child filling out their own surveys is able to provide more information that would not otherwise be captured on a parent report.[56] Despite this, child reports might still be insufficient in capturing sleep duration.[57] Also, there were a total of 291 sleep complaints on the PCSS with a score greater than 0 on a scale of 0 to 6. Of these 291 complaints, numbers indicate that some participants who reported sleep disturbance on their PCSS did not go on to complete the PSQI. This means that participants from our larger cohort were omitted from the study since they did not have any PSQI data available. Thus, our final sample may not be representative of our larger cohort. Additionally, it is possible that the PSQI global score might be too broad and is therefore not associated with the individual accelerometer-based variables. An analysis using the 7 individual component scores on the PSQI could be promising for the future. Another suggestion is to breakdown the ActiGraph days based on weekdays versus weekends. A systematic review found that there is a strong correlation between sleep time questionnaires and accelerometers for weeknights, but only a moderate correlation for weekend nights.[58] Investigating these ActiGraph days individually might strengthen the findings of this study. Understanding sleep disturbance has the potential to help address concussion-related symptoms in order to improve outcomes in school and daily life. Finally, the resulting small sample size of 21 participants is a limiting factor in generalizing our conclusions from this study.

Notwithstanding these limitations, this study is among the first to specifically use and match the validated, self-reported PSQI against accelerometer-based sleep data in a sample of youth with a variety of mechanisms of concussive injury between the ages of 5 and 18 years. A recent study examining sleep difficulties in children after a concussion shows that providing support to facilitate sleep has the potential to improve long-term functioning.[59] Therefore, determining how to identify sleep difficulties post-concussion is of particular importance.

Overall, the results suggest that the PSQI and accelerometry may be measuring different attributes of sleep. The literature has reported the poor ability of accelerometry to detect wakefulness,[60] thus lying in bed awake but motionless is likely to be coded as sleep.[27] Therefore, perceived sleep might simply be different than recorded sleep. The idea of perception of sleep is important, because it might have more clinical relevance in terms of daytime function compared with objectively measured sleep outcomes. A literature review by Sadeh et al concluded that due to the limitations of actigraphy, it is recommended to use complementary objective and subjective assessment methods when possible.[60] These findings call for further assessment of objective sleep measures like accelerometry against commonly used self-report tools so as to decide which measure has more clinical utility and significance.