Objective measurement of physical activity outcomes in lifestyle interventions among adults: A systematic review.

Valid, reliable, and direct measures of physical activity (PA) are critical to assessing the impact of lifestyle PA interventions. However, little is known about the extent to which objective measures have been used to assess the outcomes of lifestyle PA interventions. This systematic review had two aims: 1) evaluate the extent to which PA is measured objectively in lifestyle PA interventions targeting adults and 2) explore and summarize what objective measures have been used and what PA dimensions and metrics have been reported. Pubmed, Cochrane Central Register, and PsychInfo were searched for lifestyle PA interventions conducted between 2006 and 2016. Of the 342 articles that met the inclusion criteria, 239 studies measured PA via subjective measures and 103 studies measured PA via objective measures. The proportion of studies using objective measures increased from 4.4% to 70.6% from 2006 to 2016. All studies measuring PA objectively utilized wearable devices; half (50.5%) used pedometers only and 40.8% used accelerometers only. A majority of the 103 studies reported steps (73.8%) as their PA metric. Incorporating objective measures of PA should continue to be a priority in PA research. More work is needed to address the challenges of comprehensive and consistent collecting, reporting, and analyzing of PA metrics.

Introduction

Physical inactivity is responsible for 1 out of 10 premature deaths worldwide and is a risk factor for numerous chronic diseases including obesity, type 2 diabetes, cardiovascular disease, and some types of cancer (Lee et al., 2012). The World Health Organization recommends that adults engage in at least 150 min of moderate-intensity physical activity per week in order to receive the well-documented benefits of regular physical activity such as weight control, improved mental health and mood, and a reduced risk of chronic disease and all-cause mortality (Global recommendations on physical activity for health, 2016; Blair et al., 1996; Medicine ACoS, 2017). Unfortunately, about 1 in 4 adults worldwide are insufficiently active, and physical inactivity is more common in high-income countries compared to low-income ones (Hallal et al., 2012). As of 2012, the prevalence of inactivity was 43.3% in the Americas versus 27.5% and 17% in Africa and Southeast Asia, respectively (Hallal et al., 2012). Thus, development of strategies to increase the physical activity levels of adults is critical to reduce the global burden of chronic disease.

Given the high rates of physical inactivity, attention has focused on developing and testing lifestyle interventions that promote leisure-time physical activity and increase the number of adults meeting the public health recommendations for physical activity (Kahn et al., 2002; Dunn et al., 1998). These interventions often take into account individual, cultural and environmental factors influencing health behaviors and allow participants to individualize their physical activity programs to best fit their own life circumstances (Dunn et al., 1998). The ability to reliably measure the impact of these interventions on physical activity is critical for progress in this area of public health promotion (Welk, 2002). Thus, valid, reliable, and direct measures of physical activity are needed to understand the impact of lifestyle physical activity interventions (Welk, 2002; Kelly et al., 2016).

A review published in 2001 estimated there to be more than a dozen methods to measure physical activity, often categorized into subjective and objective measures (Tudor-Locke and Myers, 2001). Comprehensive summaries of these methods have been published elsewhere, (Welk, 2002; Strath et al., 2013) and are briefly outlined in a supplemental file (File S1). The most common measurement types are subjective (or self-report) measures, which include tools such as physical activity diaries and recall questionnaires, and are considered practical, versatile, low cost, and easy to use (Welk, 2002; Tudor-Locke and Myers, 2001; Ainsworth et al., 2015; Blair, 1984). However, subjective measures present limitations in capturing physical activity due to poor reliability and validity, participant recall bias and interpretation of questions, and floor effects created by instruments failing to capture the lower end of the physical activity spectrum such as spontaneous or light activities (e.g., household chores, family care) (Welk, 2002; Tudor-Locke and Myers, 2001; Prince et al., 2008). Objective methods include measures that directly assess one or more dimensions of physical activity (e.g., frequency, intensity, time, type), and have the ability to capture a variety of metrics such as number of steps, minutes of activity, intensity of activity, and bouts of activity (Strath et al., 2013). Although it has been argued that there is no “gold standard” for objective physical activity measurement, (Kelly et al., 2016; Ridgers and Fairclough, 2011; Aparicio-Ugarriza et al., 2015) commonly used tools include: wearable monitors (e.g. accelerometers, pedometers, and heart rate monitors) as well as indirect calorimetry and direct observation. Physical activity is a multifaceted and complex behavior, and research has shown that these objective measures are more precise compared to subjective measures, (Welk, 2002; Freedson and Miller, 2000) better capture the intricacies of physical activity dimensions, (Kelly et al., 2016; Prince et al., 2008) and provide a more continuous evaluation of free-living activity (Yang and Hsu, 2010).

Given these advantages, epidemiological and observational studies have begun to utilize objective measures (e.g. accelerometers) of physical activity to describe physical activity patterns across diverse population subgroups (e.g., healthy adults and children, and adults with diabetes, chronic obstructive pulmonary disease [COPD], and arthritis) (Troiano et al., 2014; Troiano, 2005; Loprinzi et al., 2014; Lee et al., 2013; Colley et al., 2011). However, while guidelines have been outlined for selecting physical activity measurement tools for use in lifestyle physical activity interventions, (Strath et al., 2013; Freedson et al., 2012; Bowles, 2012) little is known about the extent to which objective compared to subjective measures have been actually incorporated into these interventions. To advance the field of physical activity measurement in the context of lifestyle physical activity interventions, a summary of the use of recent methods is needed. Therefore, the purpose of this systematic review was to 1) evaluate the extent to which physical activity is measured objectively in lifestyle interventions targeting physical activity in adults and 2) explore and summarize what objective measures techniques have been used as well as physical activity dimensions and metrics that were reported.

Methods

Search procedure

The search strategy for this review was developed by a trained research librarian with experience in conducting systematic reviews. A computerized search was conducted in March 2016 for peer-reviewed original research published in English after January 1, 2006. The following databases were searched: Pubmed, Cochrane Central Register, and PsychInfo. The keywords in the search included (“physical activity” OR “physical activities” OR “exercise” OR “leisure time physical activity” OR “leisure time physical activities”) AND (“intervention” OR “interventions” OR “randomized controlled trial” OR “comparative study” OR “clinical trial”).

Selection criteria

Studies were included if they were randomized controlled trials or quasi-experimental interventions focused on increasing lifestyle physical activity among adults (≥18 years of age). Articles also needed to be published in English, peer-reviewed, and published between January 1, 2006 and March 30, 2016.

The focus of this paper was to examine lifestyle interventions centered on increasing physical activity among free-living adults. Thus, studies were excluded if they targeted inpatient populations, were not interventions containing at least two groups (i.e., observational, cross-sectional, secondary analyses), or if they were structured, supervised exercise interventions conducted in lab-based settings. Studies were also excluded if weight, diet, fitness, or other metabolic outcomes (e.g. glucose, cholesterol) were considered the primary focus and/or outcome of the intervention. Finally, studies were excluded if the intervention addressed multiple lifestyle behavior changes or general health behavior change. An example of this would be a study that was framed around diabetes self-management versus focused specifically on increasing physical activity.

Data extraction and synthesis

Five reviewers (VS, CH, ALC, CM, MS) screened titles and abstracts of the studies to identify potentially relevant articles. Reviewers were paired together so that each title and abstract was screened independently by two reviewers, and discussed discrepancies in eligibility until a consensus of inclusion or exclusion was determined. Interrater agreement (IRA) for titles and abstracts was 99.6% and 89.7% agreement, respectively. After this screening process, remaining eligible articles were selected for full text reviews.

A standardized data abstraction form was utilized for full text review. The data abstraction form was drafted by one author (VS) with input from the research team and was then piloted by reviewers with a set of five randomly selected articles prior to beginning full text data extraction. All the reviewers met to discuss discrepancies with the form which was edited and finalized prior to the full text review. A finalized document with the agreed upon coding procedures was created in the data management tool REDcap and used by a total of six authors (VS, CH, ALC, DJS, CM, MS) during the full text review process. During the data abstraction phase, each article was reviewed separately by two reviewers. Disagreements in abstracted data and article eligibility were discussed by the authors until consensus was reached. IRA between the reviewers was calculated for each abstracted variable and values are reported below.

Aim 1

The first step in data synthesis was to determine the proportion of lifestyle physical activity interventions that utilized objective measures of physical activity (e.g., pedometer, accelerometer) versus subjective, self-report measures only (IRA = 75.7%). For articles that only measured physical activity outcomes via subjective measures, full text reviews ceased after the citation information and the name of the self-report measure(s) were extracted. The percentage of lifestyle interventions that utilized objective measures was calculated by taking the proportion of studies using objective measures out of all included articles. The proportion of all included articles that utilized objective measures per year of publication from 2006 to 2016 was also calculated.

Aim 2

Articles included in Aim 2 were reviewed for the components of the objective physical activity measures that were utilized in the study. Type of Measure. Reviewers selected from a list of all measurement types including both subjective and objective measures: self-report, direct observation, pedometers, heart rate monitors, accelerometers, multi-sensor devices, indirect calorimetry, and double-labeled water (IRA = 92.2%). Reviewers could select more than one measurement type, where applicable. Name of Measure. For included articles, reviewers recorded the name, make, and model of the measurement type in a textbox (IRA = 90.3%). Physical Activity Metrics. Reviewers selected the physical activity metrics that were collected and reported by choosing from a list including: energy expenditure (kcal, METS), minutes of activity, days of activity, hours of activity, counts of activity (e.g. accelerometer epochs), and number of steps, (IRA = 86.4%). Reviewers could select more than one type of physical activity metric where applicable. Finally, data on whether the measurement tool was downloaded by the researchers or recorded by participants (e.g., participants were asked to write down the steps from a pedometer in a log) were categorized in the following manner: data was recorded by participants, data was downloaded and/or recorded by researchers, or the study did not specify how the data was extracted (IRA = 73.8%).

Additional study characteristics were extracted from the articles included in Aim 2. Sample Characteristics. Coders recorded the target population (IRA = 91.1%), baseline sample size (IRA = 93.2%), mean age (IRA = 92.1%), gender (IRA = 90.9%) and race/ethnicity distributions (IRA = 95.0%). Study Location. Coders recorded the city, state, and country where the study was conducted (IRA = 91.0%). Study Design. The study design was coded as either a quasi-experimental or randomized control trial (IRA = 97.1%). Intervention Setting. The type of setting where the study was conducted was coded as one or more of the following: health care, community organization, worksite, church, college, academic/research facility, fitness facility, neighborhood, or other (e.g. home-based) (IRA = 80.6%). Intervention Delivery. The format of the intervention was first coded as group versus individual (IRA = 83.5%), then coded for delivery methods: in-person, telephone, mail, Internet, social media, mobile app, physical activity device, mobile phone, or other (IRA = 66.0%).

Results

Study selection

The search yielded a total of 13,718 unique articles after removal of duplicates (Fig. 1). Of those, 10,903 were excluded based on title review. Following abstract review of the remaining 2815 articles, 1498 articles were excluded, leaving 1317 articles that were assessed for eligibility through a full text review. A total of 342 articles met eligibility criteria and were included in the review. The references of included trials are included in supplemental files (S2 and S3).

Fig. 1

Consort diagram of physical activity lifestyle interventions.

Of the 342 articles included in the review, 239 (69.9%) articles measured physical activity outcomes via subjective measures and 103 (30.1%) measured physical activity via objective measures (Fig. 1). The proportion of studies using objective measures also changed over time. In 2006, 4.4% of all studies used objective measures compared to in 70.6% of all studies in 2016 (Fig. 2).

Fig. 2

Percentage of included studies that used objective measures of physical activity, by year of publication (N = 342).

Study characteristics

The characteristics of studies included in Aim 2 are outlined in Table 1. Of the 103 studies included in Aim 2, almost half (47.6%) were conducted in the United States, and one-third were (33.0%) conducted in Europe. Most lifestyle interventions took place in an academic or research setting (57.3%) and one-fifth (21.4%) in a healthcare-based setting. Over two-thirds (64.1%) of the interventions included in-person sessions as a component of the intervention delivery, and most of studies (71.8%) also incorporated a physical activity device (e.g., pedometer) as part of the intervention delivery. The mean sample size of the studies that objectively measured physical activity was n = 123 (SD = 179.9), ranging from 16 to 1240 participants. Almost all the studies (90.3%) were randomized controlled trials and a few (9.7%) were quasi-experimental trials. Finally, 23.3% targeted their sample by older age, 21.4% by female gender, and 26.2% by disease status such as cancer, COPD, or type 2 diabetes.

Table 1

Characteristics of lifestyle interventions that used objective measures of physical activity (N = 103).

Participant sample size, mean (SD)	123 (179.9)
Study location*	n (%)
United States	49 (47.6)
Canada	8 (7.8)
Europe	34 (33.0)
Australia/New Zealand	7 (6.8)
Asia	5 (4.8)
Study setting*
Academic/research	59 (57.3)
Health care	22 (21.4)
Worksite	11 (10.7)
Community organization	7 (6.8)
Neighborhood/geographic	3 (2.9)
Church	2 (1.9)
College	3 (2.9)
Home-based	4 (3.9)
Study design
Randomized controlled trial	93 (90.3)
Quasi-experimental trial	10 (9.7)
Primary intervention delivery channel*
In-person	66 (64.1)
Telephone	22 (21.4)
Mail or print	31 (30.1)
mHealth or eHealth (e.g., Internet, mobile phone)	40 (38.8)
Environmental	4 (3.9)
Used physical activity device in intervention	74 (71.8)
Participants targeted by*
Age
Young & Middle-Aged adults (18–50 years)	10 (9.7)
Older adults (≥50 years)	24 (23.3)
Gender
Female	22 (21.4)
Male	1 (1.0)
Disease status (e.g., Arthritis, Type 2 Diabetes, Cancer)	27 (26.2)
Overweight/obesity status	13 (12.6)
Place of residence, employment, or schooling	22 (21.4)
General inactive population	11 (10.7)
Race/ethnicity	9 (8.7)
Dog ownership	2 (1.9)
*Categories are not mutually exclusive

Objective physical activity measurement

Table 2 presents the types of measures used by the 103 studies that used objective measurements of physical activity. All the studies utilized wearable devices (pedometers, accelerometers, multi-sensor devices), and no studies used non-wearable objective measurement tools such as direct observation. A majority reported steps (73.8%) as their physical activity outcome metric. Over one-third reported minutes (34%), 1.9% reported energy expenditure, and 10.7% reported accelerometer counts. Further, 34% of studies categorized their outcomes based on intensity level; for example, by reporting minutes of light intensity activity and minutes of moderate intensity activity or by using at least moderate intensity activity as a cutoff for data inclusion. The physical activity metrics stratified by measurement type are presented in Fig. 3. Finally, a majority (58.3%) of the studies reported that the data from these measures were downloaded and/or recorded by researchers, 22.3% reported that the data was recorded by the participants, and 19.4% of studies did not specify how the data was extracted or recorded.

Table 2

Types of physical activity measurements in lifestyle interventions using objective measures of physical activity (N = 103).

Measurement type	Number of studies (%)
Pedometer only	52 (50.5)
Accelerometer only	42 (40.8)
Accelerometer + pedometer	5 (4.9)
Multi-sensor device	4 (3.9)
Non-wearable assessment	0 (0.0)

Fig. 3

Physical activity data reported by lifestyle interventions using objective measures of physical activity (N = 103)*.

*Studies may have reported more than one data type.

Among studies utilizing pedometers (n = 57), the brands used most often were Yamax (49.1%) and Omron (31.6%), followed by New Lifestyles (8.8%). Among studies utilizing accelerometers (n = 47), the most frequently used brand was ActiGraph (53.2%) followed by Lifecorder Ex (8.5%), Actical (6.4%) ActivPAL (4.3%) and GENEActiv (4.3%). Finally, three of the four studies using multi-sensor devices used SenseWear/BodyMedia armbands (75.0%) and one study used the StepWatch Activity Monitor (25.0%).

Discussion

To our knowledge, this is the first review to summarize the extent to which objective compared to subjective measures of physical activity outcomes have been used in recent lifestyle physical activity interventions. While a majority of studies in this paper (70.1%) used subjective measures to assess physical activity outcomes, this finding is over the course of a 10-year period, and the trends in objective measures increased dramatically from 2006 to 2016. This review also provides a comprehensive overview of the types of objective measures being used, finding that almost all studies (96.1%) used pedometers, accelerometers, or both. Additionally, this review categorized the types of physical activity metrics reported by studies using objective measures, and found that, although steps were most frequently reported, there was a wide variety of outcomes metrics reported in lifestyle physical activity interventions.

Of note in this review is the considerable increase in the use of objective compared to subjective measures from 2006 to 2016, suggesting that researchers have begun to prioritize objective measures (in the form of wearable monitors) to assess outcomes in lifestyle physical activity interventions. First, given the recent advances in technology, the wearable activity monitors utilized in the studies in this review are now more readily available at a lower cost to researchers (Piwek et al., 2016). Second, wearable devices have had a stronger mainstream presence due to the emergence of consumer-based monitors (e.g. FitBit, Apple Watch) (Piwek et al., 2016; Smith, 2016). While these consumer products are often being used for intervention delivery as opposed to measurement, (O'Reilly and Spruijt-Metz, 2013) they also have demonstrated reliability and validity for physical activity measurement (Ferguson et al., 2015; Mammen et al., 2012; Lee et al., 2014). As a result, researchers and participants may perceive these wearable devices to be both valuable and acceptable and our results suggest that there remains room for growth in this area to enhance the evaluation of interventions primarily focused on physical activity (Rosenberg et al., n.d.; Perry et al., 2010).

This review demonstrated great variability in terms of the types of wearable devices being used as well as the reported physical activity metrics. Different types of objective measures capture different physical activity metrics. For example, pedometers primarily capture number of steps while accelerometers can capture information on the intensity, duration, and frequency of physical activity as well as number of steps (Strath et al., 2013; Ridgers and Fairclough, 2011). However, very few studies utilizing accelerometers in this review provided a comprehensive report of physical activity outcomes. This is critical as the ultimate goal of assessing physical activity outcomes within lifestyle interventions is to understand the extent to which participants are meeting physical activity guidelines as well as characterize changes in physical activity in response to interventions, and an increase in health-enhancing physical activity can occur in any of the physical activity dimensions (Strath et al., 2013). For instance, one participant may meet the physical activity guidelines (Committee, 2008) by engaging in 75 min of vigorous-intensity activity per week, while another participant may fail to meet guidelines by engaging in 75 min of light-intensity activity per week. However, if one only reports total duration of physical activity as opposed to categorizing physical activity by intensity level, these two participants would be rendered equal. This lack of uniformity makes it difficult to compare and summarize outcomes across physical activity interventions. More comprehensive reporting of physical activity metrics is warranted to better understand the changes in physical activity across lifestyle interventions.

Despite the important advantages of utilizing objective measures to measure physical activity outcomes in lifestyle interventions, research has documented several limitations of these measures including inability to capture all types of activity (e.g., swimming, cycling, strength training), limitations in technological advancement and device capabilities, burden of wear time for participants, and the potential for high reactivity (i.e. the impact of wearing a pedometer on daily steps) (Strath et al., 2013; Motl et al., 2012; Baumann et al., 2018). First, although the advances in technology have allowed for a rapid increase in the use of wearables devices, there remains heterogeneity in terms of their capabilities and data analytic processes (Ainsworth et al., 2015). Particularly for accelerometers, data processing algorithms (e.g. epochs) can increase the complexity of utilizing these devices and present challenges for researchers when trying to make methodological decisions regarding device placement, data collection protocols, data processing, and cut-points (Migueles et al., 2017). Despite advances in protocol and data processing recommendations, (Freedson et al., 2012; Troiano et al., 2008) researchers are still left with several choices; (Migueles et al., 2017) potentially exacerbating issues with comparability across studies. Thus, a step toward uniformity may include technological advances that allow for common metrics across devices and consistent data processing techniques. Second, in the present study, about one-quarter of the studies had the participants record the data from the physical activity device, and almost all of these studies (92.3%) utilized pedometers. A documented limitation to pedometers is that some brands lack sufficient memory storage and therefore require the daily step count to be written down, which increases the potential for measurement error (Welk, 2002; Strath et al., 2013; Van Camp and Hayes, 2012). One way to address this issue is to conduct a ‘blinded assessment’, where participants receive a device that is sealed shut and only to be opened by researchers during or following the measurement phase (Welk, 2002). Second, a majority of studies also incorporated a physical activity device into the physical activity intervention. While it has become common to use monitors such as pedometers and other consumer devices to promote physical activity, studies are often using the activity monitor to both motivate and measure physical activity (Bravata et al., 2007). Participants may have a tendency to be more active as a result of knowledge that they are wearing the device (Bravata et al., 2007). Ways to reduce this potential for reactivity include delaying the start time of the intervention following baseline measurement or increasing the measurement period (e.g., from one week to two weeks) to facilitate device familiarization (Kahn et al., 2002; Clemes and Deans, 2012).

This review has several limitations. First, although a protocol was developed according to the Cochrane systematic review guidelines, (Higgins and Green, 2011) the search was limited to publications available in electronic bibliographic databases and conference proceedings or unpublished interventions were not searched. Further, despite requesting the articles through online library sources and prompts to the corresponding authors, there remained a small number of articles (N = 2) for which the electronic full text could not be located. Nonetheless, studies included in this review represent the core work in this area of lifestyle physical activity interventions. Second, this review was limited to English language articles and, given the volume of studies conducted in other countries; studies published in other languages may have been missed, reducing the global generalizability of our findings. Lastly, studies in which physical activity was not the primary outcome were excluded from this review. This meant that interventions that targeted physical activity were primarily focused on broader health outcomes such as weight loss or type 2 diabetes prevention were excluded, regardless of how they measured physical activity outcomes. Although it was not the purpose of this review, it may be worthwhile to investigate the measurement tools used in studies that target physical activity as a secondary outcome to broadly understand how objective measures of physical activity are being utilized in other health behavior and chronic disease interventions.

Conclusion

Despite the advances in technology and wearable physical activity monitors over the past several years, objective measures of physical activity remain underutilized compared to self-report measures across physical activity lifestyle interventions. Further, among studies that used objective measures, there was great variability in the types of measures and physical activity metrics being reported. Obtaining accurate measurements of physical activity is critical to understanding the impact of lifestyle interventions on all physical activity dimensions and metrics. Thus, future research should prioritize not only incorporating objective measures of physical activity, but including a more comprehensive report of physical activity metrics. Further, as technology continues to advance the field, there is an increased need for prioritization of consistent collecting, reporting, and analyzing data across devices. This would improve our ability to make comparisons between studies and help us to gain a better understanding of the extent to which lifestyle physical activity interventions are stimulating meaningful changes in physical activity.

The following are the supplementary data related to this article.

Supplemental File S1

Summary of the types of physical activity assessment methods.

Supplemental File S2

References of Studies that Measured Physical Activity using Subjective Measures (N = 239)1–239.

Supplemental File S3

References of Studies that Measured Physical Activity using Objective Measures (N = 103)1–103.