Accuracy of Accelerometers for Measuring Physical Activity and Levels of Sedentary Behavior in Children: A Systematic Review.

Objectives: This systematic review evaluated the accuracy of triaxial and omnidirectional accelerometers for measuring physical activity and sedentary behavior in children. Design: Systematic review of the literature.
Methods: We comprehensively searched several databases for studies published from January 1996 through June 2018 that reported diagnostic accuracy measures in children and adolescents (age 3-18 years) and compared accelerometers with energy expenditure using indirect calorimetry.
Results: We included 11 studies that enrolled 570 participants. All studies used indirect calorimetry as the reference standard. Across the studies, median sensitivity ranged from 46% to 96% and median specificity ranged from 71% to 96%. Median area under the curve ranged from 69% to 98%. Conclusions: Accuracy measures were greatest when detecting sedentary behavior and lowest when detecting light physical activity. Accuracy was higher when the accelerometer was placed on the hip compared with the wrist. The current evidence suggests that triaxial and omnidirectional accelerometers are accurate in measuring sedentary behavior and physical activity levels in children.

Introduction

Physical activity is necessary to promote optimal physical and emotional health in children. The Centers for Disease Control and Prevention recommend policies and social-physical environments that promote lifelong physical activity.[1] The National Association for Sport and Physical Education recommends at least 60 minutes of daily moderate to vigorous physical activity.[2] Sedentary behavior, defined by postural position (lying, reclining, or sitting down) and energy expenditure ≤1.5 metabolic equivalents (METs), is a unique behavior construct that can affect health outcomes.[3-5] Thus, it is important to have consistent, reliable, and cost-effective methods of measuring sedentary behavior and physical activity levels in observational and experimental studies.

In the past 20 years, the ability to quantify activity levels has markedly improved by using accelerometers. Historically, physical activity was measured by questionnaires or interviews that were prone to recall bias.[6,7] Indirect calorimetry is a criterion standard method of measuring energy expenditure.[6,8] Energy expenditure can be expressed as oxygen consumption, carbon dioxide production, or METs. A MET is defined as the amount of oxygen consumed while at rest and is equal to 3.5 mL O2 per kg body weight × minutes.[9] MET value cutoffs are different in children as compared to adults because of known physiologic/developmental differences.[10] The cutoff levels for sedentary (<1.5 METs), light (1.5-3 METs), moderate (3-6 METs), and vigorous (>6 METs) physical activity is one approach used to categorize activity levels in children, although MET thresholds are not consistently defined across studies.[11]

Many wearable devices can be used to monitor physical activity and sedentariness.[12] Those containing triaxial accelerometers, such as GENEA, GENEActive, Actigraph, and RT3, measure acceleration in 3 orthogonal axes whereas omnidirectional accelerometers such as MiniMitter and Actical assess acceleration in multiple directions but are most sensitive to movement in the vertical plane.[13] Modern accelerometers generally incorporate microelectromechanical systems (MEMS) accelerometer chips. Although MEMS-based accelerometers are used in efficient and cost-effective wearable devices for measuring daily physical activity and sedentary behavior,[14] there is substantial variability in how accelerometers are used, where they are positioned on the body, the type of accelerometer used, and the setting for use (eg, school vs sport).

The systematic reviews that initially demonstrated effectiveness were published before 2010 and studied uniaxial and biaxial accelerometers that are not used in contemporary studies. It is important to understand whether modern accelerometers accurately measure physical activity levels and sedentary behavior because a recent review showed that most of the daily physical activity of children is not captured accurately with accelerometers.[15] The objective of this study was to summarize the available evidence in a systematic review to determine the accuracy of triaxial and omnidirectional accelerometers in measuring physical activity and sedentary behavior in children.

Methods

This systematic review was reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement.[16]

We included studies that targeted children and adolescents (age ≤18 years) and compared triaxial and omnidirectional accelerometers with energy expenditure determined through indirect calorimetry and reported validity or accuracy measures with sensitivity and specificity. We excluded studies that used a correlation analysis because of variation in the techniques used and the inability to pool data results. We excluded studies of specific populations with limited physical activity (eg, children with asthma, motor disability, cerebral palsy). Studies enrolling overweight and obese children were eligible.

We conducted a comprehensive, English-language search of the following databases from January 1996 through June 2018: Medline In-Process and Other Non-Indexed Citations, MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, and Scopus. The search strategy was designed and conducted by a medical reference librarian with input from the study authors. We used a controlled vocabulary, supplemented with keywords, to search for the concepts of accelerometer usage and childhood obesity. Online Appendix A shows the full search strategy.

We used an online reference management system (DistillerSR; Evidence Partners, Inc) to conduct the review. Two authors (B.L., T.K., S.K., or N.G.) independently reviewed titles and abstracts for eligibility and further screened the full text of the included articles in duplicate. Disagreements were resolved by a third author (B.L. or J.L).

We extracted the following variables from each study: primary author, year of publication, country of study, study design, baseline characteristics of the study population, location of the accelerometer on the body, comparison test, and outcomes reported.

To assess the risk of bias and applicability of diagnostic accuracy, we used the Quality Assessment of Diagnostic Accuracy Studies tool.[17] We assessed the following domains: patient selection, index test, reference standard, and flow and timing. Each item was investigated via yes/no signaling questions. For example, in the assessment of patient selection (“Could the selection of patients have introduced bias?”) we answered the following questions: Was a consecutive or random sample of patients enrolled? Was a case-control design avoided? Did the study avoid inappropriate exclusions? If the answers were “yes,” the item of patient selection was categorized as having low risk of bias. If any were answered “no,” the item was categorized as having high risk of bias. The “unclear” category was used only when insufficient data were reported to permit a judgment.

We summarized sensitivity, specificity, and area under the curve (AUC) values across studies with the median and interquartile range of diagnostic accuracy measures. We were unable to conduct a meta-analysis with a bivariate model because the included studies did not report sufficient data for analysis (a 2 × 2 diagnostic table).

Results

We identified 921 studies through the database search and other resources. After screening titles and abstracts, 37 were deemed eligible for full-text retrieval. We excluded 26 studies (some for multiple reasons) that did not use triaxial or omnidirectional accelerometers (n = 2), did not compare findings with indirect calorimetry (n = 10), did not use defined activity level cutoffs (n = 4), or did not report the accuracy results with sensitivity or specificity (n = 20). Eleven studies were included in the final analysis (Figure 1), and the studies are summarized in Table 1.[13,18-27]

Figure 1.

Study selection flow diagram.

Table 1.

Characteristics of Included Studies.

Age and Gender of Participants	Type of Accelerometer[a]	Study Description[b]	Comparison	Study (Country)
Wrist placement
Age, 4-5 y8 girls, 13 boys	GENEActive (dominant and nondominant wrist)	5 min lying supine, 5 min playing with Lego, 4 min at 2.5 km/h, 4 min at 3.4 km/h, 4 min at 4.3 km/h, 4 min at 5.4 km/h	Indirect calorimetry	Roscoe et al, 2017[18] (United Kingdom)
Age, 5-8 y6 girls, 9 boys	GENEActive (nondominant wrist)	5 min per activity: lying, sitting, playing with toys, slow walk, medium walk, fast walk, medium run	Gas calorimetry	Duncan et al, 2016[25] (United Kingdom)
Age, 6-11 y13 girls, 11 boys	GENEActive (wrist)	6 min rest, coloring, playing with LEGOS, Nintendo Wii tennis, Nintendo Wii boxing, walking at 2 speeds, jogging, running	Indirect calorimetry	Schaefer et al, 2014[27] (United States)
Age, 8-15 y84 girls, 97 boys	Actigraph GT3X or GT3X+ (dominant wrist)	2 days (minimum of 24 h and maximum of 2 wk between the 2 days)Day 1, determined resting metabolic rate; day 2, monitored 1 activity routineActivities (each lasted 8 min): Nintendo Wii, slow walking, sweeping, Dance Dance Revolution, television, self-paced running	Gas calorimetry[c]	Crouter et al, 2015[20] (United States)
Hip or waist placement
Age, 10-15 y39 girls, 40 boys	Actigraph GT3X, RT3, Actical (all waist)	20 minutes of rest, then 5 min of activityActivities: watching a video, writing, playing a videogame, standing, light walk, moderate walk, volleyball, running, soccer, basketball, jump rope	Gas calorimetry (portable metabolic system)	Romanzini et al, 2012[21] (Brazil)
Age, 12-16 y12 girls, 19 boys	Actigraph GT3X (right hip)	10 min per activity, interspersed with 5-min breaks: resting, treadmill walking or running at 3, 5, 7, and 9 km/h, repeated sit-stands (30/min)	Gas calorimetry[d] Vector magnitude activity counts	Santos-Lozano et al, 2013[22] (Spain)
Age, 3-5 y11 girls, 7 boys	MiniMitter Actical (right hip)	Watching a movie, walking at 3 speeds, 20 min of free play	Gas calorimetry[c]	Pfeiffer et al, 2006[24] (United States)
Age, 7-18 y18 girls, 14 boys	MiniMitter Actiwatch (right hip), MiniMitter Actical (right hip)	Basal metabolic rate, 20 min playing Nintendo, 20 min using a computer, 10 min cleaning, 12 min aerobics, 10 min basketball, 7 min slow walk, 7 min fast walk, 7 min jog	Room respiration calorimetry	Puyau et al, 2004[13] (United States)
Age, 8-12 y17 girls, 18 boys	RT3 (right hip)	20 min rest, 10 min sit, 10 min stand, 1 min at 2 km/h, 1 min at 4 km/h, 1 min at 6 km/h, 1 min at 8 km/h, increase speed until exhaustion	Gas calorimetry[e]	Chu et al, 2007[26] (China)
Ankle placement
Age, 10-14 y24 girls	Actical (random ankle)	Energy expenditure measured in 10 prescribed activities	Gas calorimetry[c]	Hager et al, 2015[19] (United States)
Multiple site placement
Age, 8-14 y26 girls, 18 boys	GENEA (3 units): 1 on each wrist and 1 on the right hip	10 min lying, then 3 min of each activity: sitting, active Nintendo Wii games, slow walk, brisk walk, slow run, medium run	Gas calorimetry	Phillips et al, 2013[23] (United States)

Manufacturer information is as follows: Actical and Actiwatch, Philips Respironics; GENEA and GENEActive, Activinsights Ltd; GT3X, Actigraph; RT3, Stayhealthy Inc.

All studies reported outcomes of energy expenditure, sensitivity, and specificity.

Cosmed portable metabolic system.

Oxycon Pro Metabolic cart.

Oxycon Mobile.

All studies used indirect calorimetry as the reference standard, with intensity level cutoffs as follows: sedentary, <1.5 METs; light activity, 1.5 to 3.0 METs; moderate activity, 3.1 to 6.0 METs; and vigorous activity, >6.0 METs. Hager et al[19] combined levels of moderate-vigorous at 6.1 METs. Chu et al[26] and Roscoe et al[18] set sedentary and light activity levels at <2 METs and 2 to 3 METs, respectively).

The accelerometers’ sensitivity, specificity, and AUC in detecting sedentary, light, moderate, and vigorous activity by attachment site are shown in Table 2. The AUC was reported in only 8 studies.[18-23,25,27] Median sensitivity ranged from 46% to 98%, median specificity ranged from 71% to 96%, and median AUC ranged from 69% to 98%. Diagnostic accuracy measures were highest for sedentary and lowest for light activity.

Table 2.

Diagnostic Accuracy Measures, Based on Activity Intensity and Accelerometer Site.

Validity Measure	Level of Activity	Accelerometer Site	Median	IQR[a]
Sensitivity	Sedentary	Hip	0.98	0.96-0.98
		Wrist	0.95	0.92-0.96
	Light	Hip	0.83	0.80-0.86
		Wrist	0.46	0.28-0.63
	Moderate	Hip	0.92	0.89-0.97
		Wrist	0.88	0.84-0.90
		Ankle	0.91	—
	Vigorous	Hip	0.93	0.97-0.98
		Wrist	0.91	0.83-0.96
		Ankle	0.91	—
Specificity	Sedentary	Hip	0.96	0.94-0.98
		Wrist	0.93	0.88-0.96
	Light	Hip	0.71	0.31-0.83
		Wrist	0.71	0.63-0.78
	Moderate	Hip	0.88	0.79-0.92
		Wrist	0.83	0.73-0.84
		Ankle	0.84	—
	Vigorous	Hip	0.72	0.51-0.85
		Wrist	0.85	0.81-0.87
		Ankle	0.84	—
AUC	Sedentary	Hip	0.98	0.97-0.99
		Wrist	0.97	0.96-0.97
	Light	Hip	0.80	—
		Wrist	0.69	0.66-0.73
	Moderate	Hip	0.98	0.83-0.99
		Wrist	0.91	0.86-0.93
		Ankle	0.87	—
	Vigorous	Hip	0.92	0.67-0.94
		Wrist	0.94	0.94-0.96
		Ankle	0.87	—

Abbreviations: AUC, area under the curve; IQR, interquartile range.

IQR is not reported when only 1 study was completed.

Diagnostic accuracy measures were slightly lower with wrist placement and higher with hip placement. Limited data were available to perform subgroup analysis by participant age or accelerometer type.

The overall risk of bias of the 11 studies was high (online Appendix B). For applicability to clinical practice, most studies had a high risk of bias in terms of patient selection but were considered to have a low risk of bias in terms of the reference standard (criterion standard) and index test (here, the different levels of activity). For example, 5 studies did not describe their sampling process, which may introduce bias when interpreting the results of experiments.[13,22,23,25,27]

Discussion

This systematic review shows that triaxial and omnidirectional accelerometers accurately categorize sedentary behavior and physical activity in children when using energy expenditure, as measured by indirect calorimetry, as the reference standard. Our review was unique in that it focused on triaxial and omnidirectional accelerometers, used indirect calorimetry as the reference standard, and used outcomes that determined the sensitivity and specificity for predicting activity intensity levels. Accelerometers were most accurate when placed on the hip (compared with wrist placement), were most accurate for detecting sedentary behavior, and were least accurate for detecting light activity.

Multiple reviews published before 2010 concluded that accelerometer use was reasonably feasible, gave reproducible findings, and provided valid measures of physical activity levels in children, plus no brand of accelerometer was shown to be strongly superior to others.[28-31] A recent review evaluated the validity of accelerometers to estimate energy expenditure against doubly-labeled water and noted greater inter- and intradevice variability than we report[15]; in that review, the correlation coefficient (between the accelerometers and physical activity energy expenditure) ranged from 0.17 to 0.85 (median, 0.37) and the study included uniaxial and biaxial accelerometers.[15] Their review implies that most daily physical activity of children is not captured by accelerometers. Our results differed in that we found that accelerometers were reasonably accurate in detecting physical activity–level categories.

The findings of this review were consistent with previous systematic reviews that showed that accelerometers can accurately detect sedentary behavior in children and adolescents.[32] A review of 5 studies reported that when accelerometers were validated against direct observation, metabolic monitoring, and energy expenditure by calorimetry, accelerometers had greater than 80% sensitivity and specificity for detecting sedentary behavior.[32] Our study showed even higher sensitivity and specificity (>95% for hip and wrist locations) for detecting sedentary behavior. There are multiple potential reasons why our study showed improved accuracy for detecting sedentary behavior, including the use of a standard definition of sedentary behavior (≤1.5 METs[4]), comparing only to indirect calorimetry, and using a consistent outcome measurement (sensitivity and specificity). Accelerometers offer the benefit of quantifying sedentary behavior in free-living conditions but most, especially with wrist placement, cannot differentiate well between various sedentary behaviors such as lying, reclining, or sitting.[33] A number of factors should be considered when choosing an accelerometer, including type, placement site, and data-gathering rate. Although most accelerometers used to measure physical activity levels are based on MEMS chips, they differ in performance and locations of attachment.[14,34] Low-amplitude movement (nonexercise activity) is ideally detected with an accelerometer with a low g-force setting (~1-2 g).[34] Most movement amplitudes can be detected with a g-force setting up to 6,[35] but high-intensity movements can be detected only with a high g setting (up to 10 g).[34,36] Raw accelerometer g-force data can be converted to activity counts, but studies vary in the cutoff points used to define different levels of physical activity.[37,38] Another factor that impedes comparison of accelerometers is the lack of standardized methods for cleaning, processing, analyzing, and describing accelerometer data.[28]

As described by Dadlani et al,[34] and Ziebart et al,[36] the rate at which the accelerometer gathers data is an important consideration. For example, a brief burst of activity (eg, a short sprint) may not be detected by a device that records movement data only every 10 seconds. However, when data are collected frequently, noise and interference signals become concerns because data acquisition must occur rapidly and the volume of data also will be high. Furthermore, such a device would require high data storage capacity, which increases costs.[34]

Although the current proliferation of wearable activity-tracking devices suggests that monitors can be placed on the wrist or ankle, our study and others do not confirm the reliability when devices are placed in those locations. A study by Tudor-Locke et al[39] found that the hip attachment site outperformed the wrist site for visually counted steps. In addition, Rosenberger et al[40] showed greater sensitivity and specificity with hip placement compared with wrist placement for detecting movement. One study included in our systematic review directly compared hip versus wrist location and found that the hip-mounted monitor had higher criterion and concurrent validity compared with the wrist.[23] However, some evidence suggests that children may be more compliant with wrist placement than hip placement.[41,42] Our systematic review did not assess compliance, but it should be evaluated in future studies.

There appears to be a paradox in the use of accelerometers that is unique to children. In adults, triaxial accelerometers are reliable because most adult movement is based on walking and output correlates with velocity. However, in children, active movement is more difficult to measure. When a child is sedentary, such as when watching television, their movement levels are so low that a triaxial accelerometer can accurately and precisely detect the lack of activity; but when children move (eg, during play or going to school), their body movements are more sporadic, volatile, and disorganized compared with those of adults. Thus, measurements of movement in children are less valid during periods of activity than when they are sedentary.

We acknowledge many limitations associated with reviewing literature on accelerometer use in children. First, we included only studies that reported sensitivity and specificity and excluded those that used regression analysis; this design limited the number of studies included in the systematic review but facilitated comparison of diagnostic measures. Because of the small number of included studies, we were unable to compare different brands of accelerometers or different age groups such as preschool versus adolescent. Second, energy expenditure was measured by different methods of indirect calorimetry, and this inconsistency may have resulted in variability in outcomes. Third, we were unable to obtain raw data to pool for a meta-analysis and instead completed a systematic review. Use of a standard repository for accelerometer data would mitigate these limitations in future studies and allow for pooled analyses.[43] Fourth, we evaluated the validity of accelerometers in laboratory settings, which may overestimate energy expenditure compared with free-living settings.[44,45] However, children in our included studies participated in activities that are currently prevalent and popular, including playing video games, watching movies, and playing team sports. Last, most of the studies included in this review had a high risk of bias, methodologic limitations, and considerable heterogeneity.

This review and the prior studies raise a number of considerations that can improve our knowledge regarding the accuracy of modern accelerometers in detecting physical activity and sedentary behavior. First, the development of a shared data repository would improve our ability to evaluate different types of accelerometers, assess different age groups, and complete meta-analyses with pooled data. Second, it is important that standard comparison outcomes, such as MET levels using indirect calorimetry, are used consistently in validation studies. Third, evaluation of compliance, durability, and usability should be included in future studies.

Conclusion

The accuracy of triaxial and omnidirectional accelerometers appears to be greatest when detecting sedentary behavior and least when detecting light physical activity; this distinction is important because most of a child’s daily activity is within this range. Hip accelerometer placement is more accurate than wrist placement for measuring activity. Triaxial and omnidirectional accelerometers accurately measure physical activity and sedentary behavior and can be used in observational and experimental studies.

Click here for additional data file.

Supplemental material, Appendix_A for Accuracy of Accelerometers for Measuring Physical Activity and Levels of Sedentary Behavior in Children: A Systematic Review by Brian A. Lynch, Tara K. Kaufman, Tamim I. Rajjo, K. Mohammed, Seema Kumar, M. Hassan Murad, Natalie E. Gentile, Gabriel A. Koepp, Shelly K. McCrady-Spitzer and James A. Levine in Journal of Primary Care & Community Health

Click here for additional data file.

Supplemental material, Appendix_B for Accuracy of Accelerometers for Measuring Physical Activity and Levels of Sedentary Behavior in Children: A Systematic Review by Brian A. Lynch, Tara K. Kaufman, Tamim I. Rajjo, K. Mohammed, Seema Kumar, M. Hassan Murad, Natalie E. Gentile, Gabriel A. Koepp, Shelly K. McCrady-Spitzer and James A. Levine in Journal of Primary Care & Community Health